Follow the detour

The function’s first instructions become a redirect

After compilation, do_work has an address in the process’s memory. A call transfers execution to that address. Normally the CPU would immediately start executing the function’s original instructions.

Gum changes the instructions at that entry point. They now transfer execution to Gum’s interception code. That code invokes a callback: a function supplied by bpftime to handle the event. In our case, bpftime’s callback runs the entry probe.

The application’s call site can keep calling the same function address. What changes is the code it reaches there.

Before the hook:
    call do_work → original instructions → result

After the hook:
    call do_work → redirect → Gum → bpftime entry probe
                                 ↓
                   original instructions → result

This changes the running process’s memory. It does not rewrite the executable file on disk. That distinction will matter when we inspect the function with tools.

The original instructions still have to run

The overwritten instructions may be important: they might reserve stack space or save an argument. Executing the rest of the function without them would break the function.

Gum therefore preserves those instructions in another small piece of executable memory. This code runs the displaced instructions and transfers execution back to the rest of the original function. That small bridge is called a trampoline.

Moving an instruction can require rewriting it. For example, an instruction that calculates an address relative to its own position needs adjustment when its position changes. Gum includes instruction relocation code to do that work.

A hook can fail if the function is too short or its instructions cannot be handled by the selected hooking method. That is why our first lab uses a separate, non-inlined function and turns optimization off.

+00  d10083ff  sub sp, sp, #0x20
+04  b9000fe0  str w0, [sp, #12]
+08  b9400fe1  ldr w1, [sp, #12]
+12  2a0103e0  mov w0, w1
+16  531f7800  lsl w0, w0, #1

+00  58000050  ldr x16, .+8
+04  d61f0200  br x16
+08  9db29000  // low half of a 64-bit destination address
+12  0000ffff  // high half of that address
+16  531f7800  lsl w0, w0, #1

What bpftime runs in the callback

Our BPF program needs an execution engine. One option is an interpreter, software that reads BPF instructions and performs each operation. Another is to translate the program into the CPU’s own instructions before using it. Those CPU instructions are called native code.

That second option is just-in-time compilation, usually shortened to JIT. In this workflow, bpftime prepares the program when the agent sets up the requested probes. On an event, it calls the prepared code; it does not need to compile the BPF program again for every function call.

Our probe then needs to find its counter. bpf_map_lookup_elem is a helper: an operation exposed to BPF programs and implemented by the runtime. It returns a pointer to a map value, or no value if the lookup fails.

__u64 *value = bpf_map_lookup_elem(&counts, &key);
if (value)
    __sync_fetch_and_add(value, 1);

__u64 means an unsigned 64-bit integer. key is 0 for entries or 1 for returns. The last line increments the selected counter atomically, so concurrent updates do not overwrite each other. The complete map declaration and probe programs are in probe.bpf.c.

The helper runs inside the target too. A call to a helper is not automatically a system call into the kernel. A particular helper could make a system call, but that depends on what its implementation does.

What can the probe learn about the call?

Besides running the callback, Gum captures the CPU state around the function call. That state includes registers, the CPU’s small named storage locations, which calling conventions use for arguments and results.

bpftime converts the saved register values into a structure that a BPF probe can inspect. This is the probe’s context; in this implementation the register structure is called pt_regs. Our counter ignores it. Reading an argument would require using the appropriate context layout for the target architecture.

Follow one complete call

The entry and return probes borrow the application thread while they run. A slow probe therefore delays the call. A helper that blocks can block that thread. Calling into another language’s runtime also requires respecting its rules for stacks and scheduling; sharing a process is not enough to establish a safe call.

A return probe needs more than an entry redirect: Gum must retain enough information about the invocation to handle its return and resume the original caller. Our lab uses ordinary C returns. Exceptions, nonlocal jumps, and recursion need additional reasoning before you rely on entry/return probes to acquire and release resources.

When you need a native function of your own

The map helper was already supplied by bpftime. Suppose your research probe needs an operation that is not built in—for example, calling a function in a native library loaded into the target.

That requires an explicit connection between BPF and the native function. bpftime has a user-function registry, often called ufunc. Native code registers a function there; BPF calls a dispatcher helper with the function’s identifier and arguments.

You need the implementation and registration inside the target’s runtime, ready before the probe uses them. Adding a function to the separate loader is not enough. The first lab uses only built-in helpers, so you can run it without changing the agent.

Another kind of hook: intercept a syscall

This section explains a second interception mechanism. The function-hook lab does not depend on it; you can continue to the lab and return here later.

A program sometimes needs the kernel to do something, such as write bytes to a file. It makes a system call, or syscall. On Linux x86-64, the program puts the operation’s number in a register called rax and executes the syscall instruction.

zpoline is a method for redirecting those syscall instructions to a userspace hook. Its trick depends on two instructions having the same size. The syscall instruction occupies two bytes; so does an indirect call to the address stored in rax.

# Linux x86-64, AT&T assembly syntax
mov $1, %eax        # operation 1: write
syscall            # bytes: 0f 05

# After rewriting that instruction:
mov $1, %eax
call *%rax         # bytes: ff d0; call the address stored in rax

The register contains the number 1, so the replacement calls virtual address 1. Ordinarily that would be invalid. zpoline first prepares executable memory near address zero so that small syscall numbers become valid entry addresses.

That region starts with NOPs—instructions that do nothing and advance to the next instruction. A call landing at address 1 runs through the NOPs into a common trampoline, which transfers execution to the syscall hook.

The hook receives the requested operation and arguments. It can run a probe, change the handling, or forward the request through a real syscall instruction. If it forwards a write, the kernel still performs the write; we have intercepted the request before it enters the kernel.

bpftime includes syscall interception using this approach on x86-64. Its transformer library rewrites the executable memory, and bpftime start -s selects that startup route. Mapping address zero can be restricted by the host OS. The ARM64 function-hook lab uses Gum instead and does not need this address-zero setup.

Both mechanisms redirect execution, but at different points: Gum intercepts our chosen function; the syscall rewrite intercepts a syscall instruction. In each case, setup must put code and routing into the target before the event reaches them.