The Architecture and Evolution of Large Language Models — From Transformer to ChatGPT, with Interactive Diagrams

Introduction

Since the release of ChatGPT, large language models (LLMs) have come into wide use. They write code, summarize papers, and answer complex questions, and all of this comes from repeating one simple operation: predicting the next token.

This article traces the path from Word2Vec to GPT-4, then explains how text is split into tokens, how the Transformer and the self-attention mechanism at its center compute a probability distribution over the next token, how one token is chosen from that distribution, and how training proceeds from pre-training to RLHF.

The LLM Genealogy — A Decade of Evolution

The Word2Vec Era (2013)

Mikolov et al. (2013) proposed Word2Vec, which converts words into fixed-dimensional vectors (embeddings). In the famous example king - man + woman ≈ queen, relationships between word meanings show up as vector arithmetic. However, each word gets the same vector regardless of context (a static embedding).

The Arrival of the Transformer (2017)

The Transformer, proposed by Vaswani et al. (2017) in "Attention Is All You Need," replaced the sequential processing of RNNs with self-attention, which lets all positions in a sequence be computed in parallel. This architecture is the foundation of today's LLMs.

The Pre-training Era (2018–)

Three architecture families grew out of the Transformer:

Encoder-Only (BERT Family)

BERT (2018) is pre-trained with Masked Language Modeling (MLM), predicting masked tokens. Because it captures bidirectional context, it does well on understanding tasks like classification, named-entity recognition, and similarity computation.

Encoder-Decoder (T5 Family)

T5 (2019) treats every task, including translation, summarization, and question answering, in the same text-to-text format: text in, text out.

Decoder-Only (GPT Family)

The GPT series uses autoregressive language models, generating one token at a time from left to right. Performance improved substantially as parameter counts grew, and GPT-3 (175B parameters) demonstrated few-shot learning: performing a new task from a few examples given in the prompt. Today's ChatGPT, Claude, and Gemini all belong to this decoder-only, autoregressive family, though they are separate product and model lineages.

Scaling Laws

Kaplan et al. (2020) discovered that language model loss follows a power law with respect to parameters $N$, data $D$, and compute $C$:

Here $x$ is one of $N$, $D$, or $C$, and the law holds when the other two are not the bottleneck. The exponent $\alpha$ differs for each quantity.

Hoffmann et al. (2022) (the Chinchilla paper) showed that compute-optimal dense language models, which use all their parameters for every token, need far more training tokens than many earlier large models used — roughly 20 training tokens per parameter in their setup. A 70B-parameter model, for example, corresponds to about 1.4 trillion training tokens.

Tokenization — Converting Text to Numbers

LLMs split text into units called tokens and process it as a sequence of integer token IDs.

BPE (Byte Pair Encoding)

Many LLMs, including the GPT series and LLaMA, use BPE (Sennrich et al., 2016) for tokenization. BPE builds its vocabulary as follows:

1. Split text into characters (or bytes)
2. Find the pair that appears next to each other most often in the corpus
3. Merge that pair into a new token
4. Repeat steps 2–3 until the vocabulary reaches the target size

# Simplified BPE pseudocode
vocab = set(all_characters)
for i in range(num_merges):
    # Find the most frequent pair
    pair = most_frequent_pair(corpus)
    # Merge into new token
    new_token = pair[0] + pair[1]
    vocab.add(new_token)
    corpus = merge_in_corpus(corpus, pair, new_token)

GPT-4 has a vocabulary of approximately 100,000 tokens (cl100k_base in tiktoken). In languages with many distinct characters, such as Japanese, byte-level BPE often splits a less frequent character into more than one token.

Why Subwords?

A word-level vocabulary is large and cannot represent words unseen during training (out-of-vocabulary words). A character-level vocabulary is small and can represent any word, but sequences become long. Subwords sit between the two: the vocabulary stays at a target size, and unseen words can still be represented as combinations of subwords. The same BPE tokenizer can also handle text in multiple languages.

Transformer Architecture

Most LLMs process the sequence of token IDs with a decoder-only Transformer.

A decoder-only Transformer consists of input embeddings, a stack of N Transformer blocks (self-attention, residual connections with normalization, and an FFN), and an output layer.

1. Input Embedding + Positional Encoding

The embedding layer converts token IDs into high-dimensional vectors (e.g., 4096 dimensions). An RNN reads its input one token at a time, so order is built into the computation. Self-attention processes all tokens in parallel and by itself does not distinguish their order, so positional information must be added explicitly via positional encoding.

Sinusoidal Positional Encoding (Original)

The Transformer of Vaswani et al. (2017) used sinusoidal positional encoding, where $pos$ is the position, $i$ is the dimension index, and $d$ is the embedding dimension:

In the heatmap below, lower dimensions (left) oscillate quickly with position (high frequency), while higher dimensions (right) change slowly (low frequency).

With sinusoids, $\text{PE}(pos + k)$ can be expressed as a linear function of $\text{PE}(pos)$ for any fixed offset $k$. The original paper hypothesized that this property would make it easy for the model to learn relative positions.

RoPE (Rotary Position Embedding)

Many current LLMs, including LLaMA, Gemma, and Mistral, use RoPE. RoPE groups the dimensions of the query and key vectors into pairs and rotates each pair in its 2D plane by an angle that depends on the position.

RoPE's key property is that the dot product of the rotated query $q_m$ and key $k_n$ depends on position only through the relative position $(m - n)$. Each token is rotated according to its absolute position, but the attention score reflects only the distance between the two tokens.

In the formula, $\theta$ is the rotation frequency, which takes a different value for each pair of dimensions. When RoPE is applied unchanged to sequences longer than those seen in training, performance tends to drop, but adjusting these frequencies lets the model handle longer sequences. Context-extension methods such as NTK-aware scaling and YaRN are based on this adjustment.

2. Multi-Head Self-Attention

The attention computation itself is covered later, in the "Self-Attention" section.

A single attention head computes only one set of attention weights for each token. With multiple heads, different heads can learn different relationships in parallel, such as subject-verb agreement, adjective-noun modification, and coreference.

where $\text{head}_i = \text{Attention}(Q W_Q^{(i)}, K W_K^{(i)}, V W_V^{(i)})$. GPT-3 uses 96 heads; LLaMA 2 70B uses 64 heads.

MQA / GQA — Shrinking the KV Cache

In standard multi-head attention (MHA), each head has its own K and V projections. The KV cache stored during inference (described later) therefore grows with the number of heads. Multi-query attention (MQA) shrinks this cache by sharing a single K and V across all Q heads. Grouped-query attention (GQA) divides the Q heads into groups of $g$ and shares K and V within each group. According to Ainslie et al. (2023), MQA speeds up inference substantially but can lower quality, whereas GQA achieves quality close to MHA at a speed close to MQA's.

LLaMA 2 uses GQA in its 34B and 70B models. In the paper's comparison of 30B models, GQA performed comparably to MHA on most evaluation tasks and, because it could run larger batches, achieved higher inference throughput.

3. Residual Connections & Layer Normalization

In each sublayer (self-attention and FFN), the input is added to the output (a residual connection) and combined with normalization. Where the normalization goes differs between Post-LN and Pre-LN, described below.

Why Residual Connections Are Essential

Without residual connections, backpropagated gradients tend to decay (or explode) exponentially as they pass through $N$ layers. A residual connection adds each layer's input directly to its output, so gradients from the output side can flow back through this shortcut straight to the layers near the input.

This makes it possible to train networks approaching 100 layers, such as GPT-3 (96 layers) and LLaMA 2 70B (80 layers).

Post-LN vs Pre-LN

RMSNorm (Root Mean Square Layer Normalization) is a simplified LayerNorm that skips mean subtraction:

Skipping the mean makes it cheaper to compute, and the original paper's experiments showed performance comparable to LayerNorm, so it has become standard from LLaMA onward.

4. Feed-Forward Network (FFN)

The FFN is a fully connected network applied to each token's vector independently. Attention captures relationships between tokens; the FFN transforms the features of each token. In the standard configuration, where the intermediate layer is four times as wide as the input, the FFN has twice as many weights as self-attention's four projection matrices combined, so FFN parameters make up roughly 2/3 of a Transformer's parameters (excluding embeddings). The FFN is thought to store much of the knowledge the model learns.

Standard FFN

The original Transformer (Vaswani et al., 2017) used ReLU; GPT and BERT switched to GELU:

The intermediate dimension $d_{\text{ff}}$ is typically $4 \times d_{\text{model}}$ (e.g., $d_{\text{ff}} = 16384$ when $d_{\text{model}} = 4096$).

SwiGLU — The Modern Standard

Shazeer (2020) proposed SwiGLU, which combines a Gated Linear Unit with Swish activation. LLaMA and PaLM, among others, use it.

In Shazeer's experiments, models with SwiGLU in place of the standard FFN achieved higher quality, and two properties may explain why. First, $x W_3$ acts as a gate that controls, depending on the input, how much of each component of $\text{Swish}(x W_1)$ passes through. Second, $\text{Swish}(x) = x \cdot \sigma(x)$ is smooth, unlike ReLU, and returns small nonzero values for negative inputs, so gradients are less likely to vanish.

Because SwiGLU uses three matrices ($W_1, W_2, W_3$), the intermediate dimension is set to $d_{\text{ff}} = \frac{8}{3} d_{\text{model}}$ to keep the parameter count equal to a standard FFN (which uses $4 \times d_{\text{model}}$).

5. Output Layer

The output of the last Transformer block is normalized (with RMSNorm in LLaMA-family models) and projected to vocabulary size $|V|$ by a linear transformation (unembedding). Softmax then converts the result to a probability distribution over the next token.

Some models, such as GPT-2 and Gemma, use weight tying: $W_U = W_E^T$, sharing the embedding matrix $W_E$ with the output weight $W_U$. This removes $|V| \times d$ parameters and makes the input and output use the same semantic space.

Causal Mask in Decoders

In decoder-only models like GPT, a causal mask blocks attention to future tokens. The token at position $i$ can attend only to positions $1, 2, \ldots, i$.

In practice, the entries above the diagonal of the score matrix are set to $-\infty$, which become zero after softmax. Because each position sees only itself and earlier tokens during training as well, training matches the conditions of autoregressive generation, which produces one token at a time from left to right.

Full Transformer Block Data Flow

The demo below steps through the data flow of a single Transformer block, one component at a time.

Self-Attention — The Core of Transformers

Self-attention computes, from the input, how relevant each token is to every other token, and uses those scores to build context-aware representations.

Formula

• $Q$ (Query): What each token is "looking for"
• $K$ (Key): What each token "contains" (how it wants to be found)
• $V$ (Value): The actual information each token passes along
• $\sqrt{d_k}$: Scaling factor, where $d_k$ is the dimension of the key vectors; it prevents dot products from growing too large, which would saturate softmax

Interactive Demo

Advantages of Self-Attention over RNNs

1. Long-range dependencies: In an RNN, information from distant tokens fades; self-attention accesses every token directly
2. Parallel computation: An RNN processes tokens one after another; self-attention is a set of matrix operations that GPUs compute in parallel
3. Interpretability: Visualizing attention weights shows which tokens each token weighted most heavily

Computational Complexity

Self-attention computes a score for every pair of tokens, so its cost is $O(n^2)$ in the sequence length $n$, which becomes expensive for long contexts. The techniques below speed up this computation, reduce its memory use, or split it across devices.

Text Generation — Next-Token Prediction

Given a context (an input token sequence), an LLM outputs a probability distribution over the next token. It picks one token from this distribution, appends it to the context, and repeats the process to generate text. This is autoregressive generation.

Generation Strategies

Greedy decoding: Always pick the highest-probability token. The output is deterministic but tends to be monotonous.

Temperature scaling: Divide the logits $z_i$ (the scores that softmax converts into probabilities) by temperature $T$ before applying softmax.

• $T < 1$: Distribution sharpens → more deterministic (good for factual tasks)
• $T > 1$: Distribution flattens → more diverse/creative (good for creative writing)
• $T \to 0$: Converges to greedy decoding

Top-k sampling: Keep only the top $k$ tokens by probability, zero out the rest, and renormalize.

Top-p (Nucleus) sampling: Keep the most probable tokens, in order, until their cumulative probability exceeds $p$ (e.g., 0.95). A sharp distribution leaves few candidates and a flat one leaves many, which makes top-p more flexible than top-k with its fixed candidate count.

Interactive Demo

Training Pipeline — From Raw Model to Conversational AI

Stage 1: Pre-training

To acquire general language knowledge, the model learns to predict the next token on a corpus of trillions of tokens, including web text, books, and code. The loss sums, over all positions, the negative log of the probability the model assigned to the correct next token $x_t$.

At this stage the model can continue a piece of text, but it has not been trained to answer questions or follow instructions.

Stage 2: Supervised Fine-Tuning (SFT)

To teach the model to follow instructions, SFT fine-tunes it on a dataset of high-quality (instruction, response) pairs (about 13,000 in InstructGPT). From this it learns the format of answering a user's question.

[Instruction] What is the capital of Japan?
[Response] The capital of Japan is Tokyo.

Stage 3: Reinforcement Learning from Human Feedback (RLHF)

A model after SFT can still produce answers not grounded in fact, or harmful output. RLHF uses human judgments of which answer is better to align the model's output with human preferences and reduce such output.

1. Train a reward model: Human annotators rank several answers to the same prompt, and that data is used to train a reward model that scores how good an answer is
2. PPO (Proximal Policy Optimization): Update the policy (LLM) to maximize the reward model's score, with a KL penalty that keeps it from drifting too far from the original model

Here $R(x, y)$ is the reward model's score for answer $y$ to prompt $x$, $\pi_\theta$ is the policy being updated, $\pi_{\text{ref}}$ is the original (post-SFT) model, and $\beta$ is a coefficient that sets the strength of the KL penalty.

DPO (Direct Preference Optimization) simplifies RLHF by optimizing the policy directly from preference data without explicitly training a reward model. LLaMA 2 used PPO-based RLHF, but DPO has since been widely adopted in open models such as Zephyr and LLaMA 3.

Modern LLM Techniques

KV Cache

In autoregressive generation, the K and V of earlier tokens do not change as generation proceeds. The K and V computed once are therefore stored in a KV cache and reused at later steps. Each step then only has to compute the new token, which speeds up inference considerably, but the cache's memory use grows linearly with sequence length.

Quantization

Quantization converts LLM parameters from 32/16-bit floating point to 4/8-bit integers to reduce memory and compute costs. Methods such as GPTQ and AWQ keep the accuracy loss small on many workloads. As a result, a 70B model, whose weights take about 35 GB at 4 bits, becomes practical to run on a single high-memory GPU. Whether it works in a given setting depends on the KV-cache size for the context length and on latency requirements.

MoE (Mixture of Experts)

MoE splits the FFN layers into multiple "experts," and a router selects a few of them for each token (2 of 8 in Mixtral). The total parameter count is large, but each token uses only part of the parameters at inference time, so the model can grow without a matching growth in compute.

Context Length Extension

Early Transformer-based models such as BERT had context lengths of about 512 tokens; some models today, such as Gemini 1.5, can process over 1 million tokens. Techniques used to extend context length include the RoPE frequency adjustments described earlier (NTK-aware scaling and YaRN) and Ring Attention.

Summary

The main points of each section:

1. History: Word2Vec → Transformer → BERT/GPT → scaling laws driving larger models
2. Tokenization: BPE subword splitting converts arbitrary text to numerical sequences
3. Transformer: N stacked layers of self-attention + FFN + residual connections
4. Self-attention: $\text{softmax}(QK^T/\sqrt{d_k})V$ computes relevance across all tokens from the input
5. Text generation: Temperature, top-k, top-p sampling for stochastic autoregressive generation
6. Training pipeline: Pre-training → SFT → RLHF/DPO turns a "next-token predictor" into a "conversational AI"

At every stage, what the model outputs is a probability distribution over the next token. Scaled up far enough, this simple-looking mechanism makes possible the code writing and question answering mentioned at the start.

References

• A. Vaswani et al. Attention Is All You Need. NeurIPS 2017.
• J. Devlin et al. BERT: Pre-training of Deep Bidirectional Transformers. NAACL 2019.
• T. Brown et al. Language Models are Few-Shot Learners. NeurIPS 2020 (GPT-3).
• J. Kaplan et al. Scaling Laws for Neural Language Models. 2020.
• J. Hoffmann et al. Training Compute-Optimal Large Language Models. NeurIPS 2022 (Chinchilla).
• L. Ouyang et al. Training language models to follow instructions with human feedback. NeurIPS 2022 (InstructGPT / RLHF).
• R. Rafailov et al. Direct Preference Optimization. NeurIPS 2023.