Better Know AGI

AGI's dancing in virtual reality (2042) or The Morning Stars (Les Etoiles du Matin) (1887)

I spun up this site on a smartphone on a lunch break, an impossible feat in 1997, and yet, while computer engineering has dramatically improved in speed and memory capacity, the quality of artificial general intelligence (AGI) programs hasn't gotten anywhere. But you might say, DeepMind! OpenAI! No. Once you grok the connections of epistemology to computer science, you will see those programs as very helpful tools, but not anything close to a real AGI mind.

If you do not understand the ideas below, which stem from Karl Popper and later from David Deutsch, you will be lost. And most crucially, if you do not understand the ideas that were first established and published in 1997 by David Deutsch, you will not make any progress in understanding AGI, let alone building it.

This post is more of a catalog of the history of ideas of AGI and links out to the actual arguments rather than actually making the case here. Feel free to skim the post and follow your curiosity to whatever seems interesting to you.

Unless you reinvent the ideas below, you are building on sand. As the picture below shows, dominating over you in the sea, looming over your sandcastles, stands a powerful forgotten force — epistemology. David Deutsch connected epistemology, or the theory of knowledge, to the theory of computation unlike anyone before him. Bold words, but Karl Popper can't know everything 😉.

Everything is applied epistemology, meaning that all the fields of knowledge, computer science, physics, mathematics, etc., must all be in accordance with our best theory of epistemology.

The simplest way that Deutsch puts it is:

However, the problem is, to understand how powerful that claim is, you have to see how he gets there. Deutsch is a physicist who wrote the first quantum program in 1985, which pioneered the field of quantum computation, so he knows a thing or two about computer science. To disregard his work is a mistake, which is why if you are interested in AGI, you need to read the list below.

These are the prerequisites, in chronological order, starting with the astounding and amazing book, The Fabric of Reality by David Deutsch, written in 1997, which cemented what the problems in epistemology are, their solutions, and how they connect with the theory of computation, building from the work of Alan Turing. This is the groundwork one must do to begin to understand true AGI.

The Fabric of Reality, Chapter 5 - Virtual Reality, (1997)

  • What makes reality comprehensible?
  • The Turing principle.
  • The universality of computation.

The Beginning of Infinity: ch 5, ch 6, ch 7, (2011)

The Beginning of Infinity: official book website

These chapters are key to computer science and AGI:

  • Ch 5, The Reality of Abstractions
  • Ch 6, The Jump to Universality
  • Ch 7, Artificial Creativity:

Brett Hall's excellent breakdown of The Beginning of Infinity:

Why has AGI not been created yet? (2012)


How close are we to creating artificial intelligence? | Aeon Essays
The very laws of physics imply that artificial intelligence must be possible. What’s holding us up?


Beyond Reward and Punishment (2019)

AGIs are people, universal explainers, as well as universal computers.


Full Book:

Possible Minds: Twenty-Five Ways of Looking at AI, Brockman, John, eBook -
Possible Minds: Twenty-Five Ways of Looking at AI - Kindle edition by Brockman, John. Download it once and read it on your Kindle device, PC, phones or tablets. Use features like bookmarks, note taking and highlighting while reading Possible Minds: Twenty-Five Ways of Looking at AI.


Bonus content:

Knowledge Creation and its Risks

Talk at the Centre for the Future of Intelligence (CFI), Trinity College, Cambridge.

Dennis Hackethal

Artificial Creativity Podcast

Dennis Hackethal’s Blog | Podcasts – Artificial Creativity

A Window on Intelligence A Window on Intelligence: The Philosophy of People, Software, and Evolution – and Its Implications eBook : Hackethal, Dennis: Kindle Store A Window on Intelligence: The Philosophy of People, Software, and Evolution – and Its Implications eBook : Hackethal, Dennis: Kindle Store

Carlos De la Guardia

Making Mind & Making Progress | Substack

3: Minds on my Mind
How I came to be interested in studying minds

Increments Podcast

#43 - Artificial General Intelligence and the AI Safety debate
Is advanced AI going to kill everyone? How close are we to building AGI? Is current AI creative? Put aside your philosophy textbooks, because we have the answers.
More Compute Power Doesn’t Produce AGI
The artificial general intelligence crew gets it completely wrong, too: “Just add more compute power and you’ll get intelligence,” when we don’t know what it is underneath that makes us creative and allows us to come up with good explanations. People talk a lot about GPT-3, the text-matching engin…
If You Can’t Program It, You Haven’t Understood It
These are all uncertain hypotheses, but we also have to keep in mind that there’s so much about evolution by natural selection that we don’t know. David Deutsch has this little quip, “If you can’t program it, you haven’t understood it.” In the case of AGI, this means we can’t program it because we…

Infinite Days

A relational database of Karl Popper, David Deutsch, Richard Feynman, and more.

Roam Research – A note taking tool for networked thought.
As easy to use as a word document or bulleted list, and as powerful for finding, collecting, and connecting related ideas as a graph database. Collaborate with others in real time, or store all your data locally.

updated nov 2022 to add the increments podcast

Jesse Ray Nichols

Jesse Ray Nichols