On the Bounds of Artificial Intelligence
Understanding the mechanics of artificial intelligence is tricky.
Unsurprisingly, artificial intelligence has become a hot topic for discourse on social media, with a lot of people expressing both intense hatred and excessive hype for the technology. Much of this is rooted in the fact that the mechanics of AI are difficult to understand — from “it just copies and pastes” to “it’s a machine god that can divine the secrets of anything.”
Most modern AI relies on complex linear algebra, modeling approximations of the world through mathematical equations — no matter the subject or technique. These equations are so intricately complex that they would make the brightest of academic mathematicians’ head spin trying to apply them. However, they can be calculated relatively quickly using modern GPUs, which are particularly well-suited for processing geometric information efficiently.
For the purposes of this discussion, we will simplify everything down to something most people did in school, graphing straight lines in two dimensions: y = mx + b. It is important to note that artificial intelligence thinks in far, far more dimensions and variables than these examples, conceiving of even basic information geometrically in ways that we would simply not.
Say you want to train a model about the concept of a dog. You provide some training data, which includes x,y points: (0, 2) (2, 4) (4, 6). “Aha!” the model goes, “dogs can be understood with the equation y = x + 2.” If you ask it to output a dog, it could now output (856723, 856725) — something entirely novel while conforming to its understanding of “dog.” Through such processes, AI can interpolate from what it knows to create entirely new versions of things we have not seen before.
Say we want to train it about the concept of a cat. Once again, you provide it training data, and it comes to the conclusion that cats can be represented by y = 50 - x. Now we decide we want to ask it for an image that looks like it could be either a cat or a dog. The AI simply looks for the intersection of these lines. Since we simplified things so much for the purposes of this discussion, we can do this easily ourselves.
So, based on the model’s understanding of the world, the point (24,26) represents something that is either plausibly a cat or a dog. Despite not having seen any sort of cat-dog hybrid before, it can generate something that closely conforms to its expectations of both.
Understanding this makes it easy to see why AI does not simply “copy and paste” — not to mention it simply does not keep the training data around to copy from in the first place! But, contrary to what some of the biggest AI hype bros think, transcendentally intelligent AI will prove a lot more challenging to create than an AI that is “merely” on the level of very smart humans.
Say we want to cure all diseases — something that will certainly be accelerated by artificial intelligence. Some AI boosters claim that we can do this within ten years by the current systems simply continuing to improve. Undoubtedly, AI will help a lot with the research process, providing rapid feedback about information and ideas based on the combined total of everything we already know. But there are limits to this with current techniques.
Let’s say the cure for cancer exists at (69, 420) in our overly simplistic two-dimensional representation of all knowledge. To cure cancer, we need to somehow arrive at that point and know what it represents. There are an infinite number of different lines that can be drawn — even in two-dimensional space. If we just draw random lines, we don’t know what they mean. There is not one singular “cure for cancer line” we can determine either — we have to arrive here by finding the intersection of numerous different lines. We may figure out that cancer is likely to be cured by a combination of concepts that can be represented by equations like:
And only once we have discovered and determined all of these lines through a lot of trial and error grounded in real world data can we actually conclusively arrive at (69, 420) as the cure for cancer.
There is much discussion about AGI (artificial general intelligence) vs. ASI (artificial superintelligence), the former as smart as the best humans and the latter on a transcendental level, capable of solving problems we cannot. By modeling all of the lines representing all of what we collectively know as a species well, it is quite likely that current methods of developing AI can attain an AGI level of intelligence.
However, to be ASI, the AI would need to somehow come up with entirely novel lines that do not map to existing human knowledge at all. While it is likely that AI will continue to get smarter, nothing that we are seeing currently is getting us closer to it being able to pull that y=x+351 out of thin air! It is just doing a better and better job at quite literally “connecting the dots” between the mountains of data points that we do have.
But there are an infinite number of possible lines, and useful information only exists along a tiny sliver of them. You cannot brute force exploration of infinity. Much like in literal space, unimaginably massive voids of nothingness separate the interesting and useful parts.
Once you understand this dynamic, it is obvious that AI as it exists now is both capable of being immensely creative, coming up with things unlike anything seen in its training data, while also being lightyears away from being some sort of god-like being that can magically solve all of our problems overnight once it simply has a bit more data and compute. There is likely to be a long period where AI certainly enables us to automate and accelerate so much of what we do in revolutionary ways — while still falling incredibly short of being some omniscient, omnipotent entity.