metacareerguide.com
Understanding & Using LLMs · Module 02

Tokens & embeddings

A model does not read words. It reads numbers. Understanding the translation explains most of the strange behaviour you will run into — why it miscounts letters, why some words cost more than others, and why it seems to grasp meaning at all.

In module 01 we said a language model predicts what comes next. That was true but incomplete, because it skipped a step. Before any prediction happens, your sentence has to become something a machine can do arithmetic on.

There are two translations, and they do different jobs. The first chops your text into pieces and gives each piece a number. The second turns each number into a position in space — and that second one is where something like meaning appears.

The first translation: tokens

A token is a chunk of text. Not quite a word, not quite a syllable. Common words are usually one token. Rarer words get split into pieces. Punctuation and spaces count too.

The model has a fixed vocabulary — typically somewhere around 100,000 tokens — and every piece of text you send has to be expressed using only those. Think of it as a very large set of alphabet blocks. Anything you want to say must be spelled out with the blocks that exist.

Figure 1 — watch a sentence become tokens
3 words · press the button

Each block is one token. The small number beneath is its ID — the actual value the model receives.

Notice what happened with the longer or stranger words. “Unbelievable” is not in the vocabulary as a single block, so it comes apart into pieces that are. This is why a model can handle a word it has never seen: it spells it out of smaller parts.

This explains the strawberry problem

Ask a model how many R’s are in “strawberry” and it often gets it wrong. Not because it cannot count — because it never saw the letters. It saw two or three tokens. Asking it to count letters is like asking you to count the brushstrokes in a word someone spoke aloud.

It also explains cost. Providers bill per token, not per word. English runs roughly 0.75 words per token, so 1,000 tokens is about 750 words. Code, unusual names, and languages that do not use the Latin alphabet all use more tokens for the same content — sometimes several times more.

The second translation: embeddings

Token IDs are just labels. Token 3007 is not larger or more important than token 12. The number carries no meaning at all — it is a name, not a value.

So the model does something more interesting. It turns each token into a long list of numbers — hundreds or thousands of them — called an embedding. And you can think of that list as coordinates. A position in space.

Two dimensions are easy to picture: a point on a map. Three, a point in a room. An embedding might have 1,536 dimensions, which nobody can picture — but the principle survives. Things near each other are similar. Things far apart are not.

Figure 2 — meaning as distance
stage 0 of 3

Flattened to two dimensions so it fits on a screen. The real thing has hundreds — but the idea is exactly this.

Nobody hand-placed those words. The model arrived at the arrangement by reading enormous amounts of text and noticing that some words keep similar company. “Doctor” and “nurse” appear in the same kinds of sentences. “Doctor” and “pavement” do not.

That is the whole trick. Meaning, to a language model, is a claim about which company a word keeps.

Why this matters practically

Embeddings are what make search-by-meaning possible. Ask a normal search engine for “how do I stop my code crashing” and it looks for those words. Ask an embedding-based search and it can return a document about “exception handling” that shares not one word with your question. You will meet this again in module 06, where it becomes the engine behind retrieval.

What this does not mean

It is tempting to conclude the model understands. Be careful with that word.

The arrangement is real and it is genuinely useful. But it was built entirely from statistical regularities in text. The model has never seen a doctor, felt an illness, or been to a hospital. It knows the shape of how people write about those things, which is a much narrower thing than understanding — and it is why a model can produce a fluent, well-structured paragraph that is confidently wrong.

Fluency and accuracy come from different places. Tokens and embeddings buy you fluency. Accuracy has to come from somewhere else, and most of this course is about where.

Try it — 10 minutes

Two short experiments. Both are free, using an AI assistant you may already have access to.

1. Open any AI assistant you have access to — Claude, ChatGPT, or Gemini. Ask: “How many times does the letter r appear in the word strawberry?” Note the answer.
2. Now ask the same question but write the word with spaces: “How many r’s in s t r a w b e r r y?” Compare. If the second is more reliable, you have just watched tokenisation in action — the spaces forced each letter into its own token.
3. Ask it to list five words that are “close in meaning” to bank. Then ask again, specifying you mean the river kind. Watch how much the neighbourhood changes.
4. Optional, and worth it: search for “OpenAI tokenizer” and use the free web tool. Paste in a paragraph of your own writing, then the same idea written in another language, and compare token counts. That difference is a real cost difference.

What to take away

You do not need to know how embeddings are computed. You need to know that text becomes chunks, chunks become positions, and nearness in that space is the model’s entire notion of similarity. Everything from search to hallucination traces back to it.

Check yourself

1. A colleague says an LLM is bad at spelling because it was trained on messy internet text. What is the better explanation?

The tokenisation answer is right. The first option is tempting because training data is messy — but a model can spell perfectly well when the word happens to be split into individual-letter tokens, which is the giveaway. The problem is the granularity of what it receives, not the quality of what it read.

2. You are estimating API cost for a document-processing tool. Your test documents are in English; production will include Japanese and Arabic. What should you expect?

Higher, often substantially. The third option is the trap — Japanese is visually compact, so it feels like it should cost less. But tokenizers are built primarily on English text, so other scripts get chopped into more, smaller pieces. Budget for it, and test with real production text rather than English samples.

3. Which claim about embeddings is accurate?

Distance reflects usage. The other two both assume something deliberate — a definition being stored, or a human encoding relationships — and that intuition is the thing to unlearn. Nobody placed those points. The arrangement emerged from statistical patterns across billions of sentences, which is why it captures relationships nobody thought to specify, and occasionally captures biases nobody wanted.
← Module 01 Course outline Module 03 →