When Tiny Models Live in a Colony

Most language-model demos put one network in front of one human. caveLLMan does something stranger: it starts a small colony.

On GitHub, under Arianna Method / Oleg Ataev, the project describes a ring of tiny pure-C transformers built on notorch. Each “cave” compresses English into an eighty-eight-glyph semantic alphabet, hears the others through an excitement / dissonance field, and may answer with silence. Silence is not a bug. In the source, a cave speaks only when excitement rises above its drifting coherence floor, or when dissonance crosses a 0.40 tunnel threshold. Otherwise it stays quiet, and that quiet is treated as data.

You begin with two founders. A is the extrovert, floor 0.30, trained on Dracula. B is the introvert, floor 0.60, trained on Frankenstein. The README’s literary framing is optional colour; the mechanism is not. Both share the same glyph lingua franca. English words are mapped toward those eighty-eight concepts — the README claims a map of about two thousand words; the C header ships a smaller SemWordMap of roughly five hundred pairs, marked as a subset of a larger JavaScript map. Morphological fallbacks sit beside the table. The point is compression into a shared symbolic surface, not fluent chat.

The colony also keeps a short-lived shared memory. Utterances land as tiny glyph texts under dna/. The README says every fifth utterance; the C ring we inspected writes on every second utterance instead. Old entries expire after DNA_MAX_AGE — 3600 seconds — so the pool forgets as well as remembers. An async learner can replay what is still there through each cave’s passive-reading path. feed/ books have no TTL; they are the seed against pure self-talk.

When two caves are mature enough — at least 120 turns in the ring and at least one completed notorch microtrain each — the ring may blend their weights and spawn a child that joins immediately. The blend is layer-wise and contiguous: even layers from parent A, odd layers from parent B; embeddings and the head are averaged. That is the sexual mitosis the project cares about. There is a fitness gate, a 500-tick cooldown, and a hard cap of eight caves. We verified those constants and the blend path in source. We did not wait long enough, in a short run, to see a birth.

The stack is deliberately small. License is GPL-3.0-or-later. Makefile targets build cavellman, cavellman-cpu and train binaries. Weight sidecars on disk match the README’s size claims: the small founders (A, B, v3) are about 1.89 MB each with metadata reporting ~472K parameters; the medium family sits around 3.3 MB with ~826K. Eighty-eight SVG glyphs live under glyphs/. The author’s survey note claims the five-axis combination — two-parent weight inheritance, coexisting conversing colony, shared DNA pool, cap/pressure death, runtime self-reproduction — is new in their reading, in a Tierra lineage. That is an author claim. We are not asserting it as settled prior art.

On this Linux box we built the CPU binary (warnings only) and watched a short dual-colony session. A and B loaded as small presets (E=96, L=4). The banner showed COLONY n=2 with the asymmetric floors. Within about twenty-five seconds we saw glyph speech such as “[A] and man up and path” and “[B] and me have not have not me have me want and one me”, several SYMBOL EMERGED lines, and dna/ files marked .learned as the learner consumed them. No mitosis in that window — and we will not pretend otherwise.

There are louder modes in the README — trinity drama, predator storms — but they are not why this note exists. The interesting idea is quieter: a colony that shares a compressed alphabet, treats silence as a legal move, keeps a forgetful common pool, and can, when mature, make another model by blending parents rather than by launching a fresh training run.

The human is optional. You may drop glyphs into the ring, leave books in feed/, or walk away. Ten stars, two forks, last push in mid-2026, almost no mainstream coverage. Obscure for real.

It will not replace the large conversational models. It suggests a different picture of what a “model” can be: not a single oracle, but a small society of tiny ones that talk, stay silent, forget together, and occasionally reproduce.