I Found a Tiny AI That Learns From Scratch. So I Built My Own.

Most AI projects begin with something enormous.

A pretrained model. Billions of parameters. A cloud API. A GPU cluster somewhere far away.

mini-AGI begins almost at the opposite end.

I came across the project by volotat on GitHub. It describes itself modestly as a toy-level experiment: a small byte-level language model designed to learn continuously from the text it encounters.

There is no pretrained intelligence hiding underneath. No enormous model waiting behind an API call. The interesting part is precisely that it starts small.

And that idea stayed with me.

What if, instead of downloading another increasingly capable model, I tried to create something that knew almost nothing at the beginning?

Something that could live on my own computer, preserve what it had learned, and change gradually as I interacted with it.

So I tried.

I called it Tama.

Tama is not mini-AGI, and it isn't an attempt to reproduce the project exactly. mini-AGI was the spark. I took the part of the idea that fascinated me most — a tiny model beginning from scratch and developing locally over time — and started experimenting with it on my Mac.

That changes the experience in a surprisingly important way.

With ChatGPT, Claude or Gemini, you meet an intelligence that arrives fully formed. You ask something and immediately encounter an enormous amount of accumulated capability.

With a tiny model trained from scratch, the relationship is almost reversed.

At first, there is essentially nothing there.

Characters, fragments, patterns. Then slightly less random fragments. Small regularities begin to appear. Progress can be painfully slow, and sometimes what looks like progress disappears again.

The interesting question stops being “How intelligent is this model?”

Instead it becomes: “What is changing?”

That is a very different way of experiencing machine learning.

There is also something appealing about the scale of it. The experiment lives on an ordinary personal computer. Its state can be saved. Training can stop and resume. You can inspect what is happening without pretending that the machine has suddenly become sentient.

It is small enough to remain understandable as an experiment.

And perhaps that is why I found mini-AGI so interesting in the first place.

Not because it promises AGI. It doesn't.

It suggests something quieter: that experimenting with artificial intelligence doesn't always have to mean accessing a larger model.

Sometimes it can mean watching a very small one begin.