Wanderer above the Sea of Fog by Caspar David Friedrich

Language Model Builder

Learn how to build AI from scratch (for free!)

Download for Mac
Free download. No account, no subscription. Apple Silicon, macOS 15 or later.

Train your own language model

One of the best ways to understand technology, especially when it seems like magic, is to build a small version yourself. Language Model Builder explains the foundations behind mainstream AI tools and lets you build models from scratch. Choose a tokenizer and training data, watch the loss fall during pre-training, teach the model to answer questions, and end up chatting with something that you made.

Some assembly required, manual included.

Language Model Builder has two parts: It offers both a textbook-style introduction to the foundations of how language models are built as well as beautifully designed tools for actually building a model.

Interactive textbook

A short textbook with interactive playgrounds

I often find myself explaining how language models are built and trained to friends and family. I finally wrote the introduction I always wanted: It’s friendly to non-programmers, contains many interactive examples, and covers all the fundamentals without treating the reader like a child or a machine learning expert.

  • Interactive from the first page

    Run the experiments yourself: build a tokenizer, move through embedding space, inspect attention, and watch a tiny model learn.

  • Go as deep as you like

    No coding, machine learning, or math experience required. Where useful, I’ve added opportunities for those curious to dive a little deeper.

  • No artificial additives

    The educational content is only as long as it needs to be, without generated filler content.

Native training workbench
MLX

Train a GPT on your Mac.

Run the entire model creation pipeline directly on your Mac, with hardware-accelerated MLX, without installing any additional tools. Every major step comes with beautifully crafted tools that you’ll love.

  • The whole training pipeline

    Pre-train with a data set of your choice, sample the base model, run supervised fine-tuning and direct preference optimization, then chat with the model you made.

  • See the run as it happens

    Follow loss, throughput, validation, saved samples, and checkpoint history without stitching together scripts. It’s really fun to watch your model become more and more capable.

  • Fully local & free

    Resume from a saved checkpoint tomorrow. Your models, datasets, and training history stay on your Mac. No accounts, no cloud, no bill.

Part I · A friendly introduction

Understand language models in about 90 minutes

I tried very hard to make the introduction as tight as possible: It covers all the major components, yet doesn’t ask you to read hundreds of pages. The app is a living project, I’ll add more content based on feedback and the (usually quick!) evolution of the landscape.

Contents

  1. Language modelsnext-word prediction, n-grams, RNNs
  2. Tokenizationhow text becomes numbers
  3. Embeddingshow numbers acquire meaning
  4. Attentionhow words take in their context
  5. The transformerthe architecture inside every modern model
  6. Training dataweb crawls, filtering, deduplication
  7. Loss & gradient descenthow a model actually learns
  8. Fine-tuningteaching by example (SFT), then by preference (DPO)
ToolsAccelerated training with Apple’s MLX

Build a model with beautiful tools

Language Model Builder offers well-crafted tools you’ll love. Dataset selection, pre-training, sampling, supervised fine-tuning and direct preference optimization; they all come with approachable and powerful tools. Everything runs locally, in-process, on your Mac.

01 · Training

A training run you can watch.

Pre-training, supervised fine-tuning, direct preference optimization, and sampling — all in-process, on your Mac. Watch the loss curve fall, teach the model your taste and preferences, then compare different model settings.

02 · Data

A catalog of good data.

Curated datasets for both pre-training and fine-tuning, each explained and one click away: bigger corpora for pre-training; friendly chats, human-written instructions, and math problems for fine-tuning. Also, obviously, full support for your own data sets.

03 · Chat

Talk to your creation

If you’ve never had the pleasure, it’s hard to describe just how fun it is to talk to a model that you’ve made. Language Model Builder allows you to chat in a pretty environment that also allows deep inspection - like token probabilities and tokens not chosen, for instance.

Questions & answers

Frequently asked questions

What kind of model can I actually build with this?

Using the default settings, you’ll get a model that writes coherent, grammatical multi-paragraph text in as little as a day. On a MacBook Pro M5 Max you could train a GPT-2-small-class model (~100–150M parameters on a few billion tokens) in about a week. It might be obvious, but to avoid disappointment: you will not train a Claude Fable 5 or ChatGPT 5.6 Sol in your garage.

Can I stop training and come back later?

Yes. Training runs save checkpoints as they go, so you can stop at any point and resume from a saved checkpoint tomorrow — or keep an earlier checkpoint around to compare against.

Does anything leave my Mac?

No. Your models, datasets, and training history stay on your Mac. There is no cloud component and no account — the model you chat with at the end exists nowhere but your own machine.

Can I share the models I made?

Yes! Language Model Builder creates files in the standardized “safetensors” format. Your friends can either open it in an app of their choice or directly with Language Model Builder.

Why is this app free?

Because I like building things and this app was fun to build. I am fortunate enough that I don’t need to sell it. In any case, I hear there’s very little money to be made by building educational software anyway. If you want to repay me, just send me a quick thanks, it’ll make my day.

When will you release a version for Windows, Linux, or Intel Macs?

Language Model Builder is a native macOS app that heavily relies on Apple’s machine learning framework MLX, which requires an Apple Silicon chip. I do not expect this app to be very popular, but if it somehow does, I might make ports to other platforms.

Go on then, build a language model.

Download for Mac
Free download. No account, no subscription.