Superintelligence, Bostrom
Either build AGI and destroy all of humanity or build AGI and colonise $4\times 10^{20}$ stars.
This book was really, really good. I’ve been interested in the idea of AI safety for a while now after reading LessWrong and this book was just full of so much good information and new perspectives.
Flashcards
Past developments and present capabilities
Growth models and big history
- A few hundred thousand years ago, it took on the order of a million years to increase civilisations ability to support an extra one million citizens living at subsistence level.
- Then it took 2 centuries.
- Now it takes 90 minutes.
- Another revolution on the scale of the industrial revolution could mean the world economy doubling every 2 weeks.
Great expectations
- Sometimes a problem that initially looks hopelessly complicated (creating superintelligence) turns out to have a surprisingly simple solutions (but most of the time, this isn’t the case).
Seasons of hope and despair
No highlights.
State of the art
- Donald Knuth: “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’”.
- An AI-Complete problem: a problem whose difficulty is roughly equivalent to creating a generally intelligent AI.
- Copernican Revolution – astronomy seemed hopelessly complicated when we thought we were at the center of the earth, until a paradigm shift occurred. Could be be tackling AI in the wrong way?
- 2010 Flash Crash: a trillion dollars were wiped off the market due to the complex and unexpected interactions of individually simple trading components.
Opinions about the future of machine intelligence
No highlights.
Paths to superintelligence
Artificial intelligence
- We know that blind evolutionary processes can produce human-level general intelligence, since it’s happened at least once.
- But we don’t have to rely on it being “blind”, a specially crafted approach could get us there a lot quicker.
- This would be computationally unfeasible to carry out, but it does provide us with an upper bound on the difficulty.
Whole brain emulation
- We learnt to fly after ditching nature and running careful experiments in wind tubes.
- We learnt combustion after observing naturally occurring fires.
- What will the path to superintelligence be? Artificial or natural?
- The WBE approach looks to create superintelligence by simulating human brains.
Biological cognition
- This approach is about augmenting our existing biological cognition, like if there was a magical smart drug that everyone could take and it would dramatically boost their IQ.
- This is not revolutionary, the author drank coffee and chewed nicotine gum to help write the book.
- There are already low-hanging fruit with biological cognition, such as fixing iodine deficiency in pregnant woman and infants: Iodine and Adult IQ Meta-analysis.
- It’s not likely that there will be a magical smart drug however due to how delicately balanced and complex the human brain is. It’s much more likely that a careful approach will need to be taken rather than flooding the brain with a certain type of chemical.
- Another approach is embryo selection.
- Embryo selection is already used in practice to select against genetic diseases that the parents of the embryos have, such as Huntington’s disease.
- In this new approach, embryos would be selected according to genetic markers of intelligence.
- 10 generations of selecting 1 out of 10 embryos could lead to an IQ boost of 130 points.
- Techniques could be used to get sperm and egg cells directly from the embryos instead which would prevent the need for the selected embryos to grow up.
- This is ignoring the ethical objections, though perception could change. 18% of Americans said they wouldn’t use IVF and then 53% they would once the first IVF baby was born.
- Other ideas: vats full of cortical tissue, large animal brains enhanced with human genes.
Brain-computer interfaces
- Brain-computer interfaces would be complex to set up, and to get them to a level that had seamless integration of information in conciousness would likely be an AI-complete problem.
- It’s not about how fast information