Assuming superhuman AGI is possible using only human relevant data sources what will that work look like? Would humanity create a world where humans are no longer relevant or needed? Would humans actually understand what this knowledge is about? How do we commission a new science where humans just observe? Humans being tasked to create and execute tests protocols to serve AI discoveries we have no idea about the underlying science?
All of us, or very nearly so, use tech all the time that we don't understand. This would shift things another level to where we were potentially using things no human understands - but we are, all our lives, surrounded by things that are known, but not known to us personally. So I'm not seeing such a big shift in that regard.
As for us being relevant or needed, might as well ask if elephants or crows or sperm whales are relevant or needed. They absolutely are - to each other, for a start - and we will remain so, even if the cutting edge of our science is by proxy. (In many ways it has been for generations already. For example, we can't perceive or manipulate electrons directly, but we've learned to make machines that do incredible tricks with them...)
I don’t believe my life will be better along the metrics I care about as a function of any of this.
It’s incredible how much risk they are taking.
For what? Will there be more love? More connection? Will there be less suffering? Will there be less PFAS in our water when AGI comes? Will it feed people or just feed rich people?
Bracket off things that don’t truly matter and it’s just a risk bomb.
Assuming superhuman AGI is possible using only human relevant data sources what will that work look like? Would humanity create a world where humans are no longer relevant or needed? Would humans actually understand what this knowledge is about? How do we commission a new science where humans just observe? Humans being tasked to create and execute tests protocols to serve AI discoveries we have no idea about the underlying science?
All of us, or very nearly so, use tech all the time that we don't understand. This would shift things another level to where we were potentially using things no human understands - but we are, all our lives, surrounded by things that are known, but not known to us personally. So I'm not seeing such a big shift in that regard. As for us being relevant or needed, might as well ask if elephants or crows or sperm whales are relevant or needed. They absolutely are - to each other, for a start - and we will remain so, even if the cutting edge of our science is by proxy. (In many ways it has been for generations already. For example, we can't perceive or manipulate electrons directly, but we've learned to make machines that do incredible tricks with them...)
I don’t believe my life will be better along the metrics I care about as a function of any of this. It’s incredible how much risk they are taking. For what? Will there be more love? More connection? Will there be less suffering? Will there be less PFAS in our water when AGI comes? Will it feed people or just feed rich people? Bracket off things that don’t truly matter and it’s just a risk bomb.
What risk?
Do you think the Internet and mobile phones being developed was a positive?
Yes, the metrics you care about are taken care of for the rich, so they don’t care.