Inside Bytecode VMs — From Interpreters to JIT Compilers

You've probably seen programming languages classified as either "compiled" or "interpreted." But most major languages today — Python, Java, C#, JavaScript, Ruby, Lua — actually sit somewhere in between, using a bytecode VM (virtual machine) architecture.

In this model, source code is first compiled to an intermediate form called bytecode, and the VM's interpreter executes it one instruction at a time. For frequently executed parts, a JIT compiler translates the bytecode to native code while the program runs, removing the interpreter's overhead.

Execution Models — Compiled, Interpreted, and VM-Based

Program execution models fall into three broad groups, depending on what the source code is translated into before it runs.

Native Compilation

Languages like C, C++, Rust, and Go compile source code directly into machine code (native code). The resulting binary is executed directly by the CPU, avoiding VM dispatch overhead and usually delivering very high performance, though profile-guided JITs can sometimes beat ahead-of-time native code on specific workloads.

Source code (.c) → Compiler → Machine code (.exe) → CPU executes directly

The drawback is that the binary is platform-specific: running it on another OS or architecture requires recompiling.

Pure Interpretation

Early BASIC and shell scripts work by reading and executing source code line by line.

Source code → Interpreter reads and executes one line at a time

This is platform-independent and convenient, but repeatedly parsing and interpreting source tends to be slower than bytecode or native execution.

Bytecode VM

This is the model used by Python, Java, C#, and many others. Source code is first compiled to an intermediate representation (bytecode), which is then executed by a virtual machine (VM).

Source code → Compiler → Bytecode → VM executes

This approach offers several advantages:

• Portability: Bytecode is platform-independent. It runs anywhere the VM exists (Java's "Write once, run anywhere")
• Faster than pure interpretation: Source code is parsed only once. Bytecode is a compact, efficient format
• Room for optimization: A JIT compiler can achieve near-native speed at runtime

The Compilation Pipeline — From Source Code to Bytecode

Source code becomes bytecode in four stages: lexical analysis, parsing, semantic analysis, and code generation. The examples here use Python.

Lexical Analysis (Lexing / Tokenization)

The source code string is converted into a sequence of tokens (lexemes).

# Source code
x = 1 + 2 * 3

Token sequence:
  NAME("x") → EQUAL("=") → NUMBER("1") → PLUS("+")
  → NUMBER("2") → STAR("*") → NUMBER("3") → NEWLINE

The lexer skips whitespace and comments while splitting the source into the smallest meaningful units.

Parsing

The token sequence is transformed into an Abstract Syntax Tree (AST) — a tree structure representing the program's structure.

       Assign
       /    \
    Name   BinOp(+)
     "x"   /      \
        Num(1)   BinOp(*)
                 /      \
              Num(2)   Num(3)

The parser follows the language's grammar rules, correctly reflecting operator precedence (* before +) in the tree structure.

Semantic Analysis

The compiler checks the AST and attaches information that later stages use:

• Variable scope resolution (local, global, or closure capture)
• Type checking (for statically typed languages)
• Simple optimizations like constant folding

In Python, symbol table construction happens at this stage, determining which scope each variable belongs to.

Code Generation

The AST (or its intermediate representation) is traversed to generate bytecode instructions.

# Examining actual bytecode in Python
import dis
 
def example():
    x = 1 + 2 * 3
 
dis.dis(example)

  2           RESUME                   0
 
  3           LOAD_CONST               1 (7)
              STORE_FAST               0 (x)
              RETURN_CONST             0 (None)

The output shows three things:

1. Constant folding: 1 + 2 * 3 has been precomputed to 7. CPython's compiler evaluates constant expressions at the AST stage
2. LOAD_CONST / STORE_FAST: The bytecode is based on a stack machine model (more on this later)
3. RETURN_CONST: Python functions implicitly return None when there's no explicit return

A function with a loop compiles to:

def loop_example():
    total = 0
    for i in range(10):
        total += i
    return total
 
dis.dis(loop_example)

  2           RESUME                   0
 
  3           LOAD_CONST               1 (0)
              STORE_FAST               0 (total)
 
  4           LOAD_GLOBAL              1 (range)
              LOAD_CONST               2 (10)
              CALL                     1
              GET_ITER
        >>    FOR_ITER                 7 (to 36)
              STORE_FAST               1 (i)
 
  5           LOAD_FAST                0 (total)
              LOAD_FAST                1 (i)
              BINARY_OP               13 (+=)
              STORE_FAST               0 (total)
              JUMP_BACKWARD            9 (to 18)
 
        >>    END_FOR
              POP_TOP
 
  6           LOAD_FAST                0 (total)
              RETURN_VALUE

The for loop becomes the sequence GET_ITER → FOR_ITER → JUMP_BACKWARD.

The demo below steps through bytecode one instruction at a time and shows how the instruction pointer (IP), operand stack, and local variables change.

Bytecode Structure

Bytecode is essentially a byte sequence of instruction (opcode) and operand pairs.

Opcodes and Operands

Bytecode instruction:
  [opcode] [operand]
  1 byte   0 to several bytes
 
Example (CPython 3.12+, base instruction = 1 word = 2 bytes; some have inline cache entries):
  LOAD_CONST    1     → Push value at index 1 of constant table onto stack
  STORE_FAST    0     → Store stack top into local variable slot 0
  BINARY_OP     0     → Pop 2 values from stack, add, push result

Since CPython 3.6, bytecode uses a word code format: each code unit is 2 bytes (1 byte opcode + 1 byte oparg). Some opcodes also reserve inline-cache code units (2 bytes each) after the instruction. When an operand exceeds 255, one or more EXTENDED_ARG instructions are prepended to extend the operand.

Code Objects

In Python, compiled bytecode is stored as a code object.

def add(a, b):
    return a + b
 
code = add.__code__
print(code.co_code)        # Raw byte sequence
print(code.co_consts)      # Constant table: (None,)
print(code.co_varnames)    # Local variable names: ('a', 'b')
print(code.co_stacksize)   # Required stack depth: 2
print(code.co_nlocals)     # Number of local variables: 2

Besides the instruction sequence, a code object holds the metadata the VM uses to run it: constant tables, variable name tables, the required stack depth, and more.

.pyc Files — Bytecode Caching

Python caches compiled bytecode as .pyc files when importing modules.

__pycache__/
  module.cpython-312.pyc

.pyc file format:

On subsequent imports, if the .pyc is up to date, Python skips recompilation and loads the bytecode directly, reducing startup time.

Stack Machines vs Register Machines

VM architectures fall into two major categories: stack machines and register machines.

Stack Machines

This is the model used by CPython, JVM, and CLR. All operations go through an operand stack.

Expression: a + b * c bytecode (stack machine)
 
Instruction       Stack state (left = bottom)
────────────────────────────────────────
LOAD a           [a]
LOAD b           [a, b]
LOAD c           [a, b, c]
MUL              [a, (b*c)]
ADD              [(a+b*c)]

Pros:

• Compact instructions (no need to specify register numbers in operands)
• Easy code generation (direct post-order AST traversal)
• Simple instruction set makes VM implementation straightforward

Cons:

• More instructions needed for the same computation (frequent LOAD/STORE)
• Stack reads/writes become memory accesses with associated overhead

Register Machines

This model is used by Lua (5.0+), Dalvik VM (Android), and some JS engines. Operations use an array of virtual registers.

Expression: a + b * c bytecode (register machine)
 
Instruction                Register state
────────────────────────────────────────
MUL  R1, R_b, R_c         R1 = b * c
ADD  R0, R_a, R1           R0 = a + b * c

Pros:

• Fewer instructions (one instruction can specify multiple operands)
• Less data movement, reducing interpreter dispatch overhead

Cons:

• Larger instructions (register numbers require extra bits in operands)
• More complex code generation (register allocation needed)

Real-World VM Choices

Shi et al. ("Virtual Machine Showdown: Stack Versus Registers", VEE 2005) translated Java VM stack bytecode into register code and report that the number of executed VM instructions, and thus of interpreter dispatches, fell by more than 47% on average, while the bytecode grew 25% larger. The extended journal version (ACM TACO 2008) reports more than 46% and 26%.

The demo below executes the same expression a + b × c on a stack machine and a register machine side by side, so you can compare their instruction counts.

Interpreter Implementation Patterns

Interpreter implementations — the core of the VM that executes bytecode — differ mainly in how they do dispatch: transferring control to the handler for the next instruction.

Switch-based Interpreter

The simplest implementation. A giant switch statement handles each opcode.

// Simplified VM loop (C language)
void eval(uint8_t *bytecode, Value *stack) {
    uint8_t *ip = bytecode;   // Instruction Pointer
    Value *sp = stack;         // Stack Pointer
 
    for (;;) {
        uint8_t opcode = *ip++;
        switch (opcode) {
            case OP_LOAD_CONST: {
                uint8_t idx = *ip++;
                *sp++ = constants[idx];
                break;
            }
            case OP_ADD: {
                Value b = *--sp;
                Value a = *--sp;
                *sp++ = value_add(a, b);
                break;
            }
            case OP_STORE_FAST: {
                uint8_t slot = *ip++;
                locals[slot] = *--sp;
                break;
            }
            case OP_RETURN: {
                return;
            }
            // ... hundreds of opcodes
        }
    }
}

Pros: Simple to implement and understand. The historical core of CPython.

Cons: Poor CPU branch prediction. All opcodes share the same switch branch point, so the branch predictor can't predict "which opcode comes next," causing frequent pipeline stalls.

Threaded Interpreter (computed goto)

Uses GCC's computed goto extension so that each handler jumps directly to the handler of the next instruction. In the example below, the bytecode keeps opcode numbers, and each dispatch looks up the jump target in dispatch_table. In the classification of the next section, this is token threading.

void eval_threaded(uint8_t *bytecode, Value *stack) {
    // Address table for each handler
    static void *dispatch_table[] = {
        [OP_LOAD_CONST] = &&op_load_const,
        [OP_ADD]        = &&op_add,
        [OP_STORE_FAST] = &&op_store_fast,
        [OP_RETURN]     = &&op_return,
        // ...
    };
 
    uint8_t *ip = bytecode;
    Value *sp = stack;
 
    #define DISPATCH() goto *dispatch_table[*ip++]
 
    DISPATCH();  // Jump to first instruction
 
op_load_const: {
        uint8_t idx = *ip++;
        *sp++ = constants[idx];
        DISPATCH();
    }
op_add: {
        Value b = *--sp;
        Value a = *--sp;
        *sp++ = value_add(a, b);
        DISPATCH();
    }
op_store_fast: {
        uint8_t slot = *ip++;
        locals[slot] = *--sp;
        DISPATCH();
    }
op_return:
        return;
}

Pros: Since each handler ends with a different indirect jump, the CPU branch predictor can maintain independent entries per handler. A comment in CPython's source (Python/ceval_macros.h) states that, at the time it was written, this version was up to 15–20% faster than the switch version, depending on the compiler and CPU architecture.

Cons: Depends on GCC/Clang extensions (not standard C). Not supported by MSVC.

CPython 3.x detects at compile time whether computed goto is available and uses this approach when it is.

Direct Threaded / Subroutine Threaded

Besides looking up a table at dispatch time as above (token threading), there are other threading approaches:

• Direct Threaded: Translates the bytecode into a sequence of handler addresses before execution and jumps without a table lookup, as in goto **ip++. Each dispatch needs one fewer memory read, but every instruction takes a pointer's worth of space, so the code is larger than with token threading
• Subroutine Threaded: Implements each opcode handler as a function and converts the bytecode sequence into an array of function pointers. Uses call/ret instructions, which benefit from the CPU's Return Stack Buffer (RSB) for branch prediction

Specializing Adaptive Interpreter

An optimization introduced in CPython 3.11 (PEP 659). It specializes bytecode instructions for the data type patterns observed at runtime.

Generic instruction:     BINARY_OP  ADD
                           ↓ Runtime detects "int + int" pattern
Specialized instruction: BINARY_OP_ADD_INT

How it works:

1. Each instruction gets an adaptive counter
2. After an instruction executes a certain number of times, the VM analyzes recent operand type patterns
3. If the pattern is stable, the generic instruction is rewritten to a more efficient specialized instruction
4. If an unexpected type appears, deoptimization reverts to the generic instruction

Key specialized instructions (CPython 3.12+):

According to What's New in Python 3.11, specialization, together with other 3.11 optimizations such as lighter-weight frames (covered below), made CPython 3.11 on average 25% faster than 3.10 on the pyperformance benchmark suite (measured with GCC builds on Ubuntu Linux). The gain came from interpreter improvements alone, without a JIT.

Frames and the Call Stack

Frame Objects

Each time a function is called, the VM creates a frame (stack frame / execution frame).

┌─────────────────────────────────┐
│           Frame                 │
├─────────────────────────────────┤
│  Code object (code)             │  → Bytecode instructions
│  Instruction pointer (IP)       │  → Current execution position
│  Local variable array (locals)  │  → [a, b, result, ...]
│  Operand stack                  │  → Temporary stack for operations
│  Link to previous frame         │  → Caller's frame
│  Exception handler stack        │  → try/except info
│  Builtins namespace reference   │  → builtins
│  Global namespace reference     │  → globals
└─────────────────────────────────┘

Call Stack Growth

def outer():
    x = 10
    return inner(x)
 
def inner(n):
    return n * 2

Runtime call stack:
 
inner(10) frame  ← Currently executing
  code: inner.__code__
  locals: [n=10]
  stack: []
──────────────────
outer() frame
  code: outer.__code__
  locals: [x=10]
  stack: []
──────────────────
<module> frame
  code: <module>.__code__
  locals: []
  stack: []

When inner executes return, its frame is discarded and control returns to outer's frame.

CPython caps the frame stack depth with a recursion limit (default 1000, readable via sys.getrecursionlimit()). This is a safety valve to prevent segmentation faults from stack overflow.

Frame Optimizations

Creating and destroying a frame costs time on every call, so VMs reduce that cost in several ways:

• CPython 3.11+: Inlined frames (_PyInterpreterFrame). Avoids eagerly allocating PyFrameObject instances by storing interpreter frames in an internal frame stack; frame objects are materialized only when needed
• JVM: Frames are placed on the native stack. The JIT compiler may completely eliminate function calls through inlining
• V8: The JIT compiler merges frames via inlining and optimizes stack frame layout for deoptimization metadata

Object Representation and Tagged Values

A VM for a dynamically typed language has to represent each value together with its type. That representation decides whether handling a single integer requires a heap allocation, so it directly affects performance.

Boxing

In CPython, every value is represented as a pointer to a PyObject struct, and integers are heap objects too. Only the small integers from -5 to 256 reuse shared objects created in advance.

// CPython's PyObject struct (simplified)
typedef struct {
    Py_ssize_t ob_refcnt;    // Reference count
    PyTypeObject *ob_type;   // Pointer to type
} PyObject;
 
// Python integers are arbitrary precision, so they use a variable-length digit array
typedef struct {
    PyObject_VAR_HEAD            // ob_refcnt + ob_type + ob_size
    digit ob_digit[1];           // Arbitrary-precision digit array (30 bits/digit)
} PyLongObject;
// Note: CPython 3.12+ changed PyLong internals so compact values are cheaper
//   to inspect, but integers remain PyObject-based values

Python's int is an arbitrary-precision integer that can represent values of any size. Even a small integer like 42 uses 28 bytes on 64-bit CPython: reference count (8 bytes) + type pointer (8 bytes) + size (8 bytes) + one 4-byte digit (this is the value sys.getsizeof(42) returns). That is seven times the 4 bytes of a C int, and the memory allocator's rounding adds more on top.

NaN Boxing

Some VMs exploit the IEEE 754 NaN (Not a Number) bit pattern to store both a type tag and value within a 64-bit double.

IEEE 754 double (64 bits):
  sign(1) | exponent(11) | mantissa(52)
 
NaN bit pattern:
  x | 11111111111 | 1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
                     ↑ quiet NaN bit
 
Value storage:
  - Floating point: stored directly as IEEE 754 bit pattern
  - Integers, pointers, booleans: stored in NaN mantissa with tag bits + value

NaN Boxing layout example:
  [0][11111111111][1][TTT][PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP]
                       ↑ tag (3 bits)  ↑ payload (48 bits)
 
Tags:
  000 = pointer (object)
  001 = 32-bit integer
  010 = boolean
  011 = nil/null
  ...

LuaJIT, JavaScriptCore (Safari), and SpiderMonkey (Firefox) use this technique. In SpiderMonkey, NaN boxing is the primary value representation used throughout the engine. Since values fit in a 64-bit word on the stack, floats and small integers can be handled without heap allocation or pointer indirection.

Tagged Pointers

On 64-bit architectures, alignment ensures that the lower few bits of pointers are always zero. These spare bits can be used as type tags.

Normal pointer (8-byte alignment):
  PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP000
                                                                ↑ always 0 (lower 3 bits)
 
Tagged pointer:
  PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP|TTT|
  ↑ pointer portion (upper 61 bits)                             ↑ tag (lower 3 bits)

V8 (Google's JavaScript engine) uses tagged pointers to distinguish Smi (Small Integer) from HeapObject:

• Least significant bit is 0 → Smi (small integer; commonly 31-bit with pointer compression)
• Least significant bit is 1 → HeapObject pointer (mask off tag bit to dereference)

CRuby (MRI) uses a similar technique, representing Fixnum (small integers) without heap allocation.

JIT Compilers — Runtime Native Code Generation

A JIT (Just-In-Time) compiler compiles bytecode to native machine code during program execution and then runs that native code directly.

Why JIT Is Needed

When interpreting bytecode, dozens to hundreds of native instructions execute for each bytecode instruction.

Bytecode: BINARY_OP_ADD_INT
 
What the interpreter actually does (outline):
  1. Fetch opcode (memory read)
  2. Get handler address from dispatch table
  3. Jump to handler
  4. Pop operand B from stack
  5. Pop operand A from stack
  6. Type check A (is it int?)
  7. Type check B (is it int?)
  8. Overflow check
  9. Perform addition
  10. Push result onto stack
  11. Advance to next opcode
 
When JIT compiles to native code:
  add rax, rbx    ← Just 1 instruction (or a few)

For one integer addition, the interpreter executes tens of machine instructions; after JIT compilation, it takes one or a few. The JIT removes that difference.

JIT Compilation Flow

  Source code
      ↓
  Compile to bytecode
      ↓
  Start executing in interpreter
      ↓
  Profiling (collect type info, execution frequency)
      ↓
  Hot spot detection (frequently executed code)
      ↓
  JIT compile (bytecode → IR → optimization → native code)
      ↓
  Execute native code (bypass interpreter on subsequent runs)
      ↓
  If assumptions break, deoptimize (native code → back to interpreter)

Method JIT vs Tracing JIT

JIT compilers fall into two kinds, depending on the unit they compile.

Method / Function JIT

Compiles at the function or method level. JVM (HotSpot C2), V8, and .NET's RyuJIT use this broad approach.

Function calculate() was called 1000 times
  → Identified as "hot method"
  → Compile entire function to native code
  → Execute native code on subsequent calls

Pros: Clear compilation units make optimization straightforward. Pairs well with inlining. Cons: Compiling large functions takes time. Even if only part of a function is hot, the entire function is compiled.

Tracing JIT

Traces (records) execution paths and compiles the actual executed bytecode sequence. LuaJIT uses this approach, as did Mozilla's former TraceMonkey.

Record loop execution path:
  LOAD x → LOAD y → ADD → COMPARE → BRANCH(true) → STORE → JUMP_BACK
 
Compile this "trace" to native code:
  - Assume branch is always true (verified by guard instructions)
  - Assume types match recorded types (verified by guard instructions)
  - Fall back to interpreter if a guard fails

Pros: Extremely effective for loop optimization. Only the actually executed path is compiled, eliminating dead code. Cross-function inlining happens naturally. Cons: Weak against complex branching patterns (frequent deoptimization from guard failures). Trace recording has overhead.

Type Specialization

The most important JIT optimization for dynamically typed languages is type specialization.

def add(a, b):
    return a + b

This function can accept int, float, str, list, or any other type. But if profiling reveals "this function is always called with int," the JIT can specialize:

Generic version (interpreter):
  1. Get type of a
  2. Get type of b
  3. Look up __add__ method for the type
  4. Call __add__
  5. Box result and return
 
Specialized version (JIT, assuming int):
  ; Guard: verify a is int
  test [rdi + type_offset], INT_TAG
  jne deoptimize
  ; Guard: verify b is int
  test [rsi + type_offset], INT_TAG
  jne deoptimize
  ; Direct addition
  mov rax, [rdi + value_offset]
  add rax, [rsi + value_offset]
  ; Overflow check
  jo  deoptimize
  ret

Type checking overhead is minimized through guard instructions. If the guard succeeds, the optimized path runs; if it fails, deoptimization returns to the interpreter.

Inlining

Inlining replaces a function call with the body of the called function. It removes the cost of the call and also lets the JIT apply further optimizations to the expanded code, so it has a large effect on what a JIT can achieve.

def square(x):
    return x * x
 
def sum_of_squares(a, b):
    return square(a) + square(b)

The JIT compiler transforms this into:

sum_of_squares native code (after inlining):
  ; Inline square(a)
  mov rax, [rdi]      ; value of a
  imul rax, rax       ; a * a
  mov rcx, rax        ; save result
 
  ; Inline square(b)
  mov rax, [rsi]      ; value of b
  imul rax, rax       ; b * b
 
  ; Addition
  add rax, rcx        ; a*a + b*b
  ret

Benefits of inlining:

• Function call/return overhead is eliminated
• Further optimizations (constant propagation, dead code elimination) can be applied to the inlined code
• Optimizations can use the caller's context (type information)

Deoptimization

When assumptions made by the JIT are violated during execution, native code execution must be suspended and control returned to the interpreter. This is deoptimization.

def process(x):
    return x + 1
 
# First 10,000 calls with int → JIT generates int version
for i in range(10000):
    process(i)
 
# Suddenly called with float → guard fails, deoptimization occurs
process(3.14)

The deoptimization process:

1. Guard instruction detects failure
2. Capture current machine register state
3. Reconstruct interpreter frame (stack, local variables, instruction pointer)
4. Return control to interpreter
5. Update profile information and set recompilation trigger

Deoptimization is expensive, but if it is rare, the program still ends up faster overall. If it happens often, the compile → deoptimize → recompile cycle can make things slower.

On-Stack Replacement (OSR)

Normally, JIT compilation takes effect on a function's next invocation. But if a long loop runs inside a single call, waiting for the next call means the whole loop keeps running in the interpreter.

OSR is a technique that replaces the currently executing interpreter frame with JIT-compiled native code mid-execution.

Running loop:
  for i in range(1000000):  ← During loop iteration...
      ...
                             ★ JIT compilation complete!
                             ★ OSR: Switch from interpreter to native code
                             ★ Remaining loop iterations run in native code

What OSR requires:

1. Capture interpreter frame state (local variables, stack, loop counter)
2. Map to native code frame layout
3. Jump to the appropriate position in the native code loop body

JVM's HotSpot, V8, and SpiderMonkey all support OSR.

Multi-Tier JIT Compilation

JIT compilation trades compile time against optimization: the more a compiler optimizes, the longer it takes. Most JITs handle this with tiered compilation, which compiles code quickly with light optimization first and later recompiles only the code that keeps running, this time with heavier optimization.

The demo below shows functions moving from the interpreter to a baseline JIT and then to an optimizing JIT, along with the deoptimization and recompilation cycle.

JVM HotSpot's Tiered Compilation

JVM's HotSpot has 5 tiers; the most common path is Tier 0 → Tier 3 → Tier 4 (interpreter → C1 with full profiling → C2):

Tier 0: Interpreter (collects profiling data)
  ↓ Method call count exceeds threshold (this is typically the first compilation)
Tier 3: C1, full profiling
  ↓ Sufficient profile data accumulated
Tier 4: C2 (Server compiler), aggressive optimization
 
Tiers used in special cases:
Tier 1: C1, no profiling (for trivial methods)
Tier 2: C1, limited profiling (fallback when C2 queue is full)

C1 (Client compiler) performs quick, basic optimizations for applications where short startup time matters. C2 (Server compiler) spends more time on aggressive optimizations for long-running server applications.

Key C2 optimizations:

• Loop unrolling
• Escape analysis for allocation elimination
• Lock elision
• Bounds check elimination
• Dead code elimination
• Vectorization (SIMD instructions)

V8's Tiered Compilation

Google's V8 JavaScript engine also uses tiered compilation.

Ignition (interpreter)
  ↓ Function becomes hot
Sparkplug (baseline compiler, V8 v9.1+)
  ↓ Further profiling
Maglev (mid-tier compiler, Chrome 117+ / V8 v11.7+)
  ↓ Sufficient profile data
TurboFan (optimizing compiler)

• Ignition: Register-based bytecode interpreter. All code starts here
• Sparkplug: Non-optimizing compiler that generates native code directly from bytecode. Extremely fast compilation (a single pass over the bytecode, with no intermediate representation)
• Maglev (Chrome 117+ / V8 v11.7+): Mid-tier compiler using SSA-based intermediate representation with type-feedback-based optimization
• TurboFan: Top-tier optimizing compiler using Sea of Nodes IR. Performs inlining, loop optimization, escape analysis, and more

GraalVM's Graal JIT

Oracle's GraalVM ships with a JIT compiler (Graal) that is implemented in Java itself.

Main features:

• Partial Escape Analysis: When an object escapes only on certain paths, allocation happens only on those paths — on other paths, it's kept as scalar values
• Polyglot support (Truffle framework): Write an AST interpreter using Truffle's API (@Specialization, @NodeChild, etc.), and Graal JIT automatically applies optimization. Supports Python, Ruby, JavaScript, R, LLVM bitcode, and more
• Native Image: AOT (Ahead-Of-Time) compilation produces standalone binaries with near-instant startup time (on the order of milliseconds)

Major VM Comparison

The sections below describe five major bytecode VMs: CPython, the JVM, the CLR, LuaJIT, and V8.

CPython (Python)

Features:
  - Stack-based bytecode VM
  - GIL (Global Interpreter Lock) limits multi-threading
  - Reference counting + cycle detection GC
  - CPython 3.11: Specializing Adaptive Interpreter
  - CPython 3.13: Experimental JIT (copy-and-patch JIT)
  - CPython 3.13: Free-threaded mode (GIL disabled, experimental)
  - CPython 3.14: Free-threaded build officially supported (PEP 779)
  - CPython 3.14: Tail-call interpreter (opt-in)

CPython 3.13's copy-and-patch JIT is an ultra-lightweight JIT that copies pre-prepared native code templates (stencils) and patches operands. It minimizes compilation time while eliminating interpreter dispatch overhead.

The copy-and-patch JIT does not compile bytecodes directly. Instead, the Tier 2 optimizer first converts bytecodes to a micro-operation (μop) intermediate representation, and the JIT is applied to that μop sequence.

Copy-and-patch flow:
  1. At build time, generate native code templates (stencils) for each μop
  2. At runtime, the Tier 2 optimizer converts bytecodes to μop sequences
  3. Copy templates for hot μop sequences into memory
  4. Patch operands (constant values, variable slot numbers)
  5. Concatenate templates to form complete native code sequence
  6. Execute the concatenated template sequence directly

As of Python 3.14 this JIT is still experimental, but it now ships inside the official macOS and Windows binaries and can be tried by setting PYTHON_JIT=1. What's New in Python 3.14 says the typical performance impact of enabling it ranges from 10% slower to 20% faster depending on workload, and does not recommend it for production. At runtime, sys._jit.is_available() tells you whether the executable supports JIT compilation, and sys._jit.is_enabled() whether it is enabled for the current process.

Python 3.14 also added an opt-in interpreter of a different kind: instead of one enormous C switch statement, it uses small per-opcode C functions chained by tail calls (--with-tail-call-interp, currently Clang 19+ on x86-64 and AArch64 only). The same document reports a geometric-mean speedup of roughly 3–5% on the pyperformance suite over a Clang 19 build without the new interpreter, depending on platform and architecture. The gain came from changing only how dispatch is written, with no change to the language or the bytecode; as discussed in "Interpreter Implementation Patterns," the way dispatch is implemented affects performance.

On the concurrency side, the free-threaded build graduated to official support (phase II) in Python 3.14 under PEP 779. According to What's New in Python 3.14, the specializing adaptive interpreter (PEP 659), originally disabled in free-threaded builds, is now enabled there too, bringing the single-threaded penalty down to roughly 5–10% depending on the platform and C compiler. Free-threaded builds and the JIT cannot currently be combined.

JVM (Java, Kotlin)

Features:
  - Stack-based bytecode VM
  - Tiered JIT (C1 + C2, or Graal)
  - Advanced GC (G1, ZGC, Shenandoah)
  - Bytecode verification for security guarantees
  - Dynamic class loading via ClassLoaders

The JVM bytecode verifier checks before execution:

• No stack overflow/underflow
• Local variable access is within bounds
• Type consistency is maintained
• finally blocks are properly connected

This prevents execution of malformed bytecode and guarantees VM memory safety.

CLR (C#, F#)

Features:
  - Stack-based CIL (Common Intermediate Language)
  - RyuJIT (JIT compiler)
  - Value types (struct) — avoids boxing
  - Span<T> / stackalloc for zero-allocation
  - NativeAOT for ahead-of-time compilation
  - Generational mark-and-sweep GC

A distinctive feature of the CLR is value type support. A local struct variable lives on the stack, and a struct field is stored inline inside its containing object. Neither needs a separate heap allocation, which reduces allocations and GC pressure.

// Value type — placed on stack
struct Point { public int X, Y; }
 
// Reference type — heap allocated
class PointClass { public int X, Y; }
 
void Example() {
    Point p = new(1, 2);       // On the stack. No GC needed
    PointClass pc = new(1, 2); // Heap allocated. GC managed
}

LuaJIT

Features:
  - Register-based bytecode VM
  - Tracing JIT
  - NaN Boxing for value representation
  - Extremely fast (among the fastest dynamic language VMs)
  - Fast FFI (Foreign Function Interface)

LuaJIT is a tracing JIT developed by Mike Pall. According to its official site, LuaJIT is widely considered one of the fastest dynamic language implementations. The site also says that greatly reducing the overhead of dynamic languages lets it reach the performance range traditionally reserved for offline static-language compilers. The codebase is small: about 120K lines of C and assembly (DynASM) in the v2.1 branch as of November 2026, counting all CPU architectures.

How LuaJIT's tracing JIT works:

1. Hot loop detection: Counts loop iterations and function calls; a loop or function that reaches a threshold count is marked as "hot"
2. Trace recording: Records the hot loop's execution path as a "trace." Function calls are unfolded into the trace
3. SSA conversion: While recording, translates each bytecode into SSA (Static Single Assignment) form IR instructions
4. Optimization: Constant folding, algebraic simplification, CSE (Common Subexpression Elimination), DCE (Dead Code Elimination), hoisting of loop-invariant instructions via loop unrolling and copy-substitution, register allocation
5. Code generation: Generates native code directly from optimized IR (the official site lists x86, x64, ARM, ARM64, PPC, MIPS32, and MIPS64)

V8 (JavaScript)

Features:
  - Bytecode (Ignition) + tiered JIT
  - Hidden Classes for efficient object representation
  - Inline Caches (IC) for fast property access
  - Generational GC (Scavenger + Mark-Compact)
  - Tagged pointers

V8's Hidden Class (Map) system infers fixed layouts — similar to C++ classes — for dynamically typed objects.

// These two objects share the same Hidden Class
let a = { x: 1, y: 2 };
let b = { x: 3, y: 4 };
 
// This triggers a transition to a new Hidden Class
a.z = 5;

Objects with properties added in the same order share the same Hidden Class, allowing the JIT compiler to optimize property access based on it.

Concrete Example: Python Bytecode Execution Trace

Here is how CPython executes the bytecode of a recursive function, one instruction at a time.

def fibonacci(n):
    if n <= 1:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

import dis
dis.dis(fibonacci)

  2           RESUME                   0
 
  3           LOAD_FAST                0 (n)
              LOAD_CONST               1 (1)
              COMPARE_OP              26 (<=)
              POP_JUMP_IF_FALSE        1 (to 14)
 
  4           LOAD_FAST                0 (n)
              RETURN_VALUE
 
  5     >>    LOAD_GLOBAL              1 (fibonacci)
              LOAD_FAST                0 (n)
              LOAD_CONST               1 (1)
              BINARY_OP               10 (-)
              CALL                     1
              LOAD_GLOBAL              1 (fibonacci)
              LOAD_FAST                0 (n)
              LOAD_CONST               2 (2)
              BINARY_OP               10 (-)
              CALL                     1
              BINARY_OP                0 (+)
              RETURN_VALUE

Here is the execution of fibonacci(3), with the stack at each step:

fibonacci(3) execution trace:
 
Instruction             Stack              Notes
─────────────────────────────────────────────────────
RESUME                  []                 Function start (internal checks only)
LOAD_FAST 0 (n)         [3]                Push n=3
LOAD_CONST 1 (1)        [3, 1]             Push constant 1
COMPARE_OP <= (26)      [False]            3 <= 1 → False
POP_JUMP_IF_FALSE       []                 False, so jump
LOAD_GLOBAL 1           [NULL, fibonacci]  Push NULL + function object
LOAD_FAST 0 (n)         [NULL, fibonacci, 3]  n=3
LOAD_CONST 1 (1)        [NULL, fibonacci, 3, 1]  Constant 1
BINARY_OP - (10)        [NULL, fibonacci, 2]  3 - 1 = 2
CALL 1                  [→ calls fibonacci(2)]
                        ... recursive call ...
                        [result_1]          fibonacci(2) result = 1
LOAD_GLOBAL 1           [1, NULL, fibonacci]
LOAD_FAST 0 (n)         [1, NULL, fibonacci, 3]
LOAD_CONST 2 (2)        [1, NULL, fibonacci, 3, 2]
BINARY_OP - (10)        [1, NULL, fibonacci, 1]  3 - 2 = 1
CALL 1                  [→ calls fibonacci(1)]
                        ... recursive call ...
                        [1, result_2]       fibonacci(1) result = 1
BINARY_OP + (0)         [2]                1 + 1 = 2
RETURN_VALUE            → returns 2

Inline Caching — Speeding Up Dynamic Dispatch

In dynamically typed languages, every property access or method call requires checking the object's type and looking up the property's location. Inline Caching (IC) caches these lookup results for reuse.

IC State Transitions

Uninitialized
  ↓ First access
Monomorphic
  → Only one type observed. Highest cache hit rate
  ↓ Different type observed
Polymorphic
  → 2-4 types observed. Linear search through multiple cache entries
  ↓ More types observed
Megamorphic
  → Too many types observed. Cache abandoned, falls back to generic path

Concrete Example

function getX(obj) {
    return obj.x;  // ← IC is set up at this call site
}
 
// Always called with same-shape objects → Monomorphic (fastest)
getX({ x: 1, y: 2 });
getX({ x: 3, y: 4 });
 
// Mixed object shapes → Polymorphic → Megamorphic
getX({ x: 1 });
getX({ x: 1, y: 2, z: 3 });
getX({ a: 0, x: 1 });

A monomorphic IC can speed up property access to one comparison + one memory read. JIT compilers use monomorphic IC information to generate direct field access code with type guards.

Profiling Mechanics

A JIT compiler decides "which code to compile" and "which types to specialize for" based on profiling data collected while the program runs.

Profiling Approaches Compared

HotSpot decays its counters so that code that ran often during startup, but rarely in steady state, is not treated as "hot" forever.

GC–JIT Coordination — Safepoints

JIT-compiled native code must cooperate with the garbage collector. For the GC to scan and relocate heap objects, all threads must stop at a safe state (safepoint).

Safepoint Placement

JIT compilers insert safepoint polls into generated native code.

JIT-generated code (outline):
  ...
  add rax, rbx          ; computation
  test [safepoint_page], eax  ; ★ safepoint poll
  ...                    ; when safepoint_page is made unreadable,
                         ; a page fault occurs → control transfers to GC

In JVM HotSpot, safepoints are placed at:

• Just before method return
• Loop backedges (jumps from end to start of loop)
• JNI / native method call transitions

For counted loops (loops with an int counter and a known bound), backedge safepoints may be omitted, which can cause unexpectedly long GC pauses (Time-To-Safepoint issues). Loop strip mining, introduced in JDK 10 and enabled by default with collectors such as G1, mitigates this by breaking counted loops into chunks with safepoint polls between them.

JIT Security — The W^X Problem

A JIT writes native code into memory and then executes that memory. Keeping the same region both writable (W) and executable (X) violates the W^X (Write XOR Execute) security policy.

Addressing the W^X Problem

On iOS, Apple prohibits JIT for third-party apps. Safari and WKWebView (including third-party browsers like Chrome and Firefox) have JIT enabled, but apps that embed JavaScriptCore directly run with only the interpreter (LLInt) and no JIT tiers. As a result, JavaScript can run more slowly in those apps.

Conclusion

A bytecode VM compiles source code to platform-independent bytecode and uses a JIT to turn only the frequently executed parts into native code, which gives it both portability and execution efficiency.

Source Code
  ↓ Lexing → Parsing → Semantic Analysis → Code Generation
Bytecode (portable intermediate representation)
  ↓ VM executes
Interpreter (executes all code first)
  ↓ Profiling
JIT Compiler (converts hot spots to native code)
  ↓ Type specialization → Inlining → Various optimizations
Native Code (CPU executes directly)

Each VM makes different tradeoffs:

• CPython: Prioritizes simplicity and ecosystem stability. An experimental JIT arrived in 3.13
• JVM: A heavyweight JIT pipeline optimized for long-running servers
• V8: Multi-tier compilation balancing short script startup with server-side optimization
• LuaJIT: A tracing JIT that gets high performance from a small codebase

CPython has continued its performance work through the 2020s, with the Specializing Adaptive Interpreter in 3.11, the experimental copy-and-patch JIT in 3.13, and the tail-call interpreter in 3.14. With WebAssembly (WASM), bytecode VM design is also spreading beyond the browser into server-side and edge computing.

References

• Aho, Lam, Sethi, Ullman. Compilers: Principles, Techniques, and Tools (2nd Edition), 2006
• Smith & Nair. Virtual Machines: Versatile Platforms for Systems and Processes, 2005
• Yunhe Shi, David Gregg, Andrew Beatty, M. Anton Ertl. "Virtual Machine Showdown: Stack Versus Registers", VEE 2005. https://doi.org/10.1145/1064979.1065001
• Yunhe Shi, Kevin Casey, M. Anton Ertl, David Gregg. "Virtual Machine Showdown: Stack Versus Registers", ACM Transactions on Architecture and Code Optimization 4(4), 2008. https://doi.org/10.1145/1328195.1328197
• CPython Developer's Guide. https://devguide.python.org/internals/
• Mark Shannon. "Faster CPython" (PEP 659 — Specializing Adaptive Interpreter)
• Mike Pall. "LuaJIT 2.0 intellectual property disclosure and research opportunities", 2009
• "Life of a V8 JavaScript object", V8 Blog. https://v8.dev/blog
• Würthinger et al. "One VM to Rule Them All" (Truffle/Graal), ACM DL, 2013
• Bryce Adelstein Lelbach. "A Deep Introduction to JIT Compilers: JITs are not very Just-in-Time", CppCon 2021
• Hölzle, Chambers, Ungar. "Optimizing Dynamically-Typed Object-Oriented Languages With Polymorphic Inline Caches", ECOOP, 1991
• Agesen. "Design and Implementation of PEP, a Java Just-In-Time Translator", Sun Microsystems, 1997