Language Model Builder with Felix Rieseberg

On Hanselminutes, Rieseberg talks with Scott Hanselman about Language Model Builder, his free Mac app that teaches how a language model works and then has you train a small one yourself. Hanselman compares it to Ben Eater’s build-a-computer-from-scratch videos. Rieseberg sized the app for a friend with one afternoon: a 10,000-token tokenizer, a model you can pretrain in about a day, and fine-tuning that needs only around 800 steps. One overnight model told him “the sun is a moon at the bottom of the earth.” He lists the parts he likes best: the model blueprint, an “x-ray view” of the tokens the model didn’t pick at each position, and a fine-tuning option that turns a group chat into a mimic of it. He came to computing from poetry. Looking at a model, he says, he gets the feeling he gets at a planetarium, as if the universe were staring back. Understanding the mechanics, he argues, is the way into the bigger questions, from moral patienthood to why researchers say they grow a model rather than build one.