AI (and Robotics) From Scratch in 2026

If you want to be a 'LLM Engineer' or go out into the world and sign up companies to AI (and get paid), or you want to hack around some LLM open source model project this is all you need:

• 10-202 Intro to Modern AI is taught by Zico Kolter who besides being a CMU prof he also sits on the board of Open AI. This is mostly self-contained and you can of course use some AI Agent like ChatGPT to teach you whatever it is you don't understand. It uses Colab/Google online notebooks for assignments. You can do this on a phone if needed.
• These notes from MIT's 6.390 Intro to ML are also excellent.
• An optional free book Understanding Deep Learning

Of course there is vast research into LLM tuning but this is a good enough start and he has interviews on YouTube saying how the majority of AI code is incredibly simple the only thing that's changed is the data available to train it is now massive.

• 10-714 DL Algorithms & Implementation is again taught by Zico Kolter and shows how to build your own PyTorch stack from scratch. You can do the assignments in Python (it too uses Colab/Google online notebooks) to really learn how PyTorch works or instead of doing the assignments you could roll your own modest PyTorch clone in any language you want which is what I plan to do.

Does there exist a unified theory of modern AI yet or something close to it?

• These 30 Lectures were recently put together by MIT's leading Neuroscience researcher by that I mean he's the world's most cited Neuroscience researcher. He used the assistance of a research paper Agentic Investigator. Give it a research topic and it gives you back an entire paper on that topic.

These lectures show current AI provides a generic mean or 'smoothing' effect on everything they generate. The human user has to provide the sparse logical jumps to escape the annoying politeness and homogenizing of scientific thought that AI defaults to. They also show compositionality which means a complex thing is constructed of many small reusable basic things and these are (as described in the prologue) mathematically implied to be sparse for the simple reason that we can compute them efficiently. Genericity is another trait of modern AI and means a learning function being defined by any set of coordinates is invariant to transformations and defining it in different spaces where you 'shift' the function doesn't really matter. There will be many lectures about this in detail and we will take them. AI also can't be sentient (yet) no matter what AGI marketing these companies claim. This is another theme of these lectures.

Things we have to take to understand these lectures:

• The mathematical model of Machine Learning which hasn't changed.
  • All lectures are taught by the author of the (free) book and there's undergrad notes (MIT).
    • This book is over simplified to the point where sometimes things can seem 'incorrect' but teaches the mathematical justification of modern machine learning and is good enough for us in the beginning.
• Some optional short lectures from the perspective of theoretical neuroscience.
• All the Calculus/Algebra content below
• Maybe the Zhang book on statistical learning theory but depends what I can find, what I want is Poggio's MIT 9.520 'draft book' that isn't available anywhere but we can always watch old lectures and take our own notes then go on a hunt for new papers.

As per Tomaso Poggio in his 30 lectures on Deep Learning the vast majority of Real Numbers are uncomputable and undecidable so numerical approximations in the discrete world have to be done. The inputs are simply too gigantic, a tiny 3D model is 100 x 100 x 100 with 10⁶ unknowns, so almost all linear algebra learned in undergrad is not used as now you live in the world of factoring (SVD), matrices have to be randomized (jack polynomials), eigenvalues are approximated, and preconditioning is needed as gradient descent is too slow because it 'zig-zags'.

The following is very similar to CMU's Math Foundations for Robotics course as well:

• Matrix calculus
• MIT Numerical Methods and large scale linear algebra.
• Modern Regression and Data Analysis because in applied and research ML you often find yourself trying to interpret data. We came this far why not be advanced students in the art of data analysis too (think quant finance).

• 6.8200 Computational Sensorimoto Learning this guy is the leading researcher for "control learning" so we will learn the math model. There is many other math models for robotics but an AI will eventually discard them for it's own interpretation hence I chose this course just to get a taste of robotic learning. We are interested in that link to the full lecture notes.

You can use ChatGPT or the Agentic Investigator to help you research these fields by yourself.

Modern machine learning is theoretically done with Banach spaces where you get a vector space with measurement. This is to promote sparsity (weights that are exact zeros) while classical machine learning relies on Hilbert spaces (inner product norms). Then you have to convert all this to the real world of computer hardware using numerical methods.

We should just skip all of vector calculus and proceed in a direct path to Banach spaces. There's a free book for this "TBB-Dripped" or Elementary Real Analysis by Thomson-Bruckner the dripped version or 'Dump the Riemann Integral Project' version. Many concepts here will come up in the Tomaso Poggio Deep Learning lectures and for any probability material. It looks like a large book but half way through any undergrad math text you pick up speed as the problems become too easy to solve. However I will go even beyond that and straight into functional analysis.

There's a concept of groups and inside those are an abelian group with the ability to perform 'actions' over a ring structure that has a special case of being a field. This is the space of vectors and linear maps. A book exists to teach all this from scratch by Paolo Aluffi called Algebra: Chapter 0. There are many other great undergrad linear algebra texts but what is the point in taking them if we are only using numerical analysis (SVD) and matrix calculus thus throwing out most of their content. We may as well learn how it all works at the big picture level plus group theory structures and symmetry in ML is another emerging field right now.

I will create my own Linear Algebra course

• NJ Wilberger geometric demonstrations of transformations
• The SVD 'compact version' to avoid block matrices and Edelman's 18.06 MIT course
• The entire field described in category theory so you understand what's going on at the overview level and now everything makes sense

Home