can you create a code our very very small slm that can complete sentences like cat can…
- User
can you create a code our very very small slm that can complete sentences like cat can sit on -> mat like that can you do that?
Assistant
Now updating the page title and building:
Done! ✅ I built TinyLM — a genuinely working, very very small neural language model that trains live in your browser. Yes, it really does
the cat can sit on the → mat🐱What it is
A real neural network written completely from scratch (no ML libraries):
- Architecture: last 2 words → 16-dim word embeddings → 48-unit tanh hidden layer → softmax over the whole vocabulary (~20k parameters, vs billions in GPT)
- Training: hand-written backpropagation + stochastic gradient descent, trained on ~100 simple sentences ("the cat sat on the mat", "the bird can fly in the sky", etc.)
- Trains automatically in ~2 seconds when the page loads — you can watch the loss curve drop live
What you can do
- Type a prompt like
the cat can sit on theand see live probability bars for the next word (matwins with high confidence) - Press Tab to accept the top prediction, Enter or "Complete →" to finish the whole sentence
- Temperature slider — 0 = greedy/deterministic, high = creative chaos
- ✨ Dream — generate a sentence from nothing
- Edit the training corpus — add your own sentences and hit "Retrain from scratch" to teach it new patterns in seconds
- Stats panel shows vocab size, parameter count, and training examples
Try prompts like
the fish can swim in the,i like to read a, orthe bird can fly in the— and try adding your own sentences to the corpus to see it learn them!Files built:
src/lib/corpus.ts,src/lib/model.ts,src/App.tsx,index.html