AI (and Robotics) From Scratch in 2026

We will learn modern AI has a generic mean or 'smoothing' effect on everything it generates. The human user has to provide the sparse logical jumps to escape the annoying politeness and homogenizing of scientific thought that current day AI agents default to.

Compositionality is another reason why deep learning works which means a complex thing is constructed of many small reusable basic things and these are (as described in the prologue) mathematically implied to be sparse for the simple reason that we can compute them efficiently.

Genericity we'll see is another trait of modern AI and means a learning function being defined by any set of coordinates is invariant to transformations and defining it in different spaces where you shift the function doesn't really matter. This means it's possible to use gradient-based optimization.

We also learn what's missing from modern AI.

LLMs have to replay every computation while your biological system can reduce reaction times through practice. Your brain learns function composition as a single operator short cut but LLMs can't do this unless you build one that can compress the 'hidden layer' computational structure and avoid all the scaffolding.

Modern AI has no persistent internal state, no sense of continuity, frozen dynamics and insufficient modularity. There is no physical constraints therefore AI models keep getting more ridiculously large with ever growing parameters and skyrocketing complexity while your own brain can't get bigger so reuses all it's modules for everything from reasoning to vision.

LLMs can't simulate interventions without acting they can only imitate the description of causal reasoning. Intelligence requires climbing the causal ladder and even the most impressive AI agent in Sep 2026 cannot do minimal causal reasoning. If you want an AI doctor or claim you've reached some kind of superintelligence you're going to need this.

There is many researchers at Anthropic and OpenAI (such as the ai-2027 site I posted above) claiming if governments don't establish a total monopoly and do regulations where only certain companies are allowed to develop AI then it's all over. This is just a scam to eliminate competition from Zuckerberg and China. They fear the ZUCC. He is the last big tech founder who hasn't cashed out yet and still runs a company and he's actually producing some interesting open source AI. Llama models are likely the future of big money enterprise AI where companies pay for some 'Meta certified' infrastructure/hosting and save millions per year than paying Anthropic or OpenAI for token outputs. In addition the ZUCC is the second biggest datacenter operator so now he owns all the data access too and this just prints money as all the big general models need to pay him to access his now 160+ datacenters. Finally he has the money to keep disrupting without needing to beg AWS or Nvidia for investments and anyone can buy Meta stock instead of waiting for the OpenAI IPO which may never come.

Such a person who has persisted all these decades causes AI CEOs to panic. I never thought I'd hear myself say this but if you want the best job in 2026 then Meta is where you should go the work as the work they're doing is very interesting and Llama open source models is how you would get in the door.

The best agents right now can't even do basic physics they know the high dimensional map and have to infer all this for our 3D world they can't simply look at the territory. There's no superintelligence do not believe the hype and marketing that we are all doomed unless we 'slow down' (let China catch up and delay IPOs) or push for government oversight (enforced monopoly). Everytime I read in the headlines that some AI researcher has quit to sound the alarm and demand a slow down I imagine that guy being hired in Shanghai the next day for $10M/year.

Does there exist a unified theory of modern AI yet or something close to it?

• These 30 Lectures were recently put together by MIT's most cited Neuroscience researcher. He used the assistance of a research paper Agentic Investigator. We will go through this entirely.

If you want to be a 'LLM Engineer' or you want to hack around some LLM open source model project this is all you need to start:

• 10-202 Intro to Modern AI is taught by Zico Kolter who besides being a CMU prof he also sits on the board of Open AI. This is mostly self-contained and you can of course use some AI Agent like ChatGPT to teach you whatever it is you don't understand. It uses Colab/Google online notebooks for assignments. You can do this on a phone if needed.
• These notes from MIT's 6.390 Intro to ML are also excellent.
• An optional free book Understanding Deep Learning
  • Zico Kolter also has a Pytorch course where you build your own clone and we may do it if there's time

As per Tomaso Poggio in his 30 lectures on Deep Learning almost every Real Number is uncomputable and undecidable so numerical approximations in the discrete world have to be done. Almost all linear algebra learned in undergrad is not used as now you live in the world of factoring (compact SVD), matrices have to be randomized (jack polynomials), eigenvalues are approximated, and preconditioning is needed as gradient descent is too slow because it 'zig-zags'.

This math is mostly for implementing current AI architecture or 'AI Engineering'. The following is very similar to CMU's Math Foundations for Robotics course as well:

• Matrix calculus
• MIT Numerical Methods and large scale linear algebra.
• Modern Regression and Data Analysis because in applied ML you often find yourself trying to interpret data.
• 6.8200 Computational Sensorimoto Learning has full lecture notes for the math model of control learning. There is many other math models for robotics but an AI will eventually discard them for it's own interpretation.

Besides coming up with a new architecture that fixes all the problems we'll discover there's other fields of interest here wide open for anyone to jump in.

Alignment is a marketing term meaning AI must align with our 'human goals and values'. These goals and values are defined by tech billionaires and likely don't resemble your goals and values at all. They try and align AI by post-training where the outputs to prompts are ranked and this ranking is used as additional reinforcement training for the model with rewards. Almost all of the alignment strategies fall into this category of wishful thinking where we can reward the AI for good behavior.

Another alignment method is red teaming where you jail break using prompts to fool the AI into giving you dangerous results like step-by-step bioweapon instructions. All these methods rely on the creativity of the red team h4xxors or some lesser AI trying to jail break a more sophisticated one and seems like a gimmick to me.

Another method falls into the interpretability bucket meaning you try and figure out exactly how an AI came to a specific decision. It's probably impossible with today's complex agents though this has the most promise as it also leads to higher quality training data. This better training eliminates the probability distribution drift where some model encounters something it doesn't really know and in this blind spot you can jail break.

They have names for many other methods but really they just fall into those 3 categories where it's either trying to interpret the execution, trying to jail break or the wishful thinking category.

We will stop after chapter 1 and catch up on everything written here so we have a concrete understanding how associative memories work and attention in modern transformer architecture.

We begin with cognition or intelligence modeled as an associative memory. A radial basis function (RBF) network you can ask AI about the only important point is that it measured the distance from the input to one of these 'templates' and activated those neurons/templates like a nearest neighbor strategy. Association by similarity. Ask AI what a kernel function is it's a trick to perform high dimensional mathematics without actually moving all the data into higher dimensions and gaining all that extra complexity. Speaking of complexity we are told these modern attention mechanism are really just a random-access Turing machine which is taught in every complexity course in undergrad as a model of computation and you can look up on YouTube exactly how that model works.

Ask whatever AI you use to give you a summary of the Hyper-RBF paper by Poggio. They simply replaced Euclidean distance with a measure of how far a datapoint is from the mean of a distribution. This new space is a transformation where it's easier to calculate the Euclidean distance because you adjust for things and it works like a standard z-score in multiple dimensions. Look up all these terms, z-score, Euclidean distance, outliers, standard deviation, and Mahalanobis distance.

Page 7 gives us an informal operational schema and a Turing machine example for how every LLM loops. Create a new query from the current state and jump to associated memory that matches the contents. Then perform PointMLP a point cloud analysis you can ask AI about which all it does is make nearest neighbor searching faster. This gets updated into the local state to be found more easily. Finally as the linear approximation goes on through the hidden layers the network learns a controller that selects which attention heads to activate and now this is a complex calculation all being performed in distributed computing based on data.

Finally because deterministic Turing machine models have been proved to have only polynomial computing time there's a cost estimate here where for every content addressable read there is a local update. If the depth of this neural network for each layer is n length of nodes that's n² because you loop an n-sized neural network layer, and do a local update. How many local updates? N-sized many. That's n * n or n² worst case scenario where every node has to update the local state but here it says due to sparsity this is really O(n) or linear with only O(n) or linear memory needed for the key-value cache. That is unless YOU come up with a more efficient read or a compression where you don't have to always read every node and a short cut can be learned.

A few spelling mistakes in 1.2 I guess the LLM paper assistant used for this hallucinated. You can ask YouTube or AI what a diffusion model is helps solve a fluid dynamics model. Imagine all the pixels for an image are so dense they are some kind of fluid now solve that model to identify what the pixels are showing in the noise and you will understand all the image and video generators out there right now. Maybe there is a better model you can invent here that accomplishes the same thing faster.

The equation in 1.3 are to show attention heads are identical to a normalized HyperBF network or a kernel trick where associative memories all existing in higher dimension can be calculated nearest neighbor without actually going into those dimensions and performing hard calculations. That's how attention works it's inferring a high-dimensional mapping to grab all the nearest data points.

Chapter 1.6 with figure 1.2 explains why AI is consistent in results. Ask some AI how manifold geometry in transformers works and this is yet another optimization they came up with to prevent 'drifting' where the tokens are constrained to follow a tangent of the data. Any deviation from this is the LLM halluncinations we've all seen so it's not a perfect solution but explains how almost every prompt results in the same answer. A 'stochastic' deviation just means random.

There's a differential equation in 1.6.2 which we'll learn after this but these are how how you would step through dynamics to move forward or backwards in a simulation. Here it is claimed that consistency in prompt answers is an emergent property of manifold-constrained associative recall. We will have to teach ourselves some autoregressive dynamics and see why they self-stabilize.

In 1.7 this is the interesting part, what's the difference between our own brains and AI. So far AI is doing convoluted one-step approximations in a high-dimensional associative recall that our brains do not do. They claim our brains do an optimized, multi-layer variant of Kanerva's associative architecture where Kanverva's is a single-layer map of the higher dimensions that is flat. Look up what episodic context means. They are saying here that our brains are basically transformers but have deep functional composition instead of simple maps to higher dimensional space and a local neural plasticity instead of backpropagation which is propagating the correct results back to the neural net. Our brains reorganize circuits to encode new skills and adapt behavior.

Now that we know what we need to know we have to go back and do this one more time at the math model level.

TODO

---

Home