How eBPF Programs Run: Hooks, Events, and the Lifecycle

An eBPF program doesn't just 'run.' It attaches to a hook in the kernel and executes when a specific event fires (a packet arrives, a syscall is made, a function is called). Understanding this event-driven model is the key to everything.

---

The Concept Explained

You want to know why one service's calls to Postgres occasionally take 400 milliseconds when the database says every query finished in under 5. The delay is somewhere between the two, and "somewhere between" spans a socket, a policy check, a NAT translation, a virtual interface, an encapsulation header, and a physical NIC. You already know eBPF can answer this. The question that decides whether you get a useful answer or a useless one is where you attach.

That question has a specific technical meaning, because an eBPF program is not a process. It has no thread, no address space, no entry in the scheduler, and nothing to run when the system is idle. It is a callback registered against a point in the kernel, and the kernel invokes it synchronously, on whichever CPU is handling the event, with the event's data as its only argument. Until your program returns, the packet is not forwarded, the syscall does not complete, and that CPU does nothing else. There is no queue and no worker pool between the event and your code.

Everything about the model follows from that. Programs cannot sleep, because sleeping in that context would block a softirq or a syscall path — a narrow exception exists for explicitly sleepable programs on a few hook types, which is the exception that proves the shape. Programs must terminate, because nothing will preempt them. And the cost of your program is not what it does per second but what it does per event, multiplied by an event rate you did not choose.

The attach point also decides what you are allowed to know. A program on the driver receive path gets a raw buffer and nothing else: no socket, no process, no cgroup, because none of those have been determined yet for this packet. A program on connect() runs in the calling task's context, so the PID, the cgroup, and therefore the pod are all trivially available — but there is no packet yet to inspect. A kprobe on a kernel function gets whatever that function was called with. The hook is not a deployment detail. It selects your entire universe of available facts, and no amount of clever code recovers information the hook never had.

---

How It Works

Two of those steps can fail and they fail differently. A verifier rejection is loud, immediate, and comes with hundreds of lines of explanation. An attach failure is quiet: the program loaded fine, the process is running, and the only symptom is that the events you expected never arrive. Most eBPF outages in production are attach problems wearing the costume of "no data."

Program type is fixed at load time, not at attach time. It determines the layout of the context pointer, which helpers you may call, and which return values are legal, and the verifier enforces all three. A program compiled for tc cannot be attached to XDP even though both handle packets, because struct __sk_buff and struct xdp_md are different objects with different lifetimes. The section name is what carries that decision from source to loader.

SEC("xdp")
int drop_bad_src(struct xdp_md *ctx) { return XDP_DROP; }

SEC("tc")
int mark_egress(struct __sk_buff *skb) { return TC_ACT_OK; }

SEC("tracepoint/syscalls/sys_enter_execve")
int on_exec(void *ctx) { return 0; }

Three programs, three context types, three unrelated sets of return values. TC_ACT_OK is 0 and XDP_ABORTED is also 0, so returning the tc constant from an XDP program compiles, verifies, loads, and then drops every packet while firing an error tracepoint.

Where The Hooks Actually Live

Walk down the stack and the tradeoff is always the same: the earlier you attach, the faster you are and the less you know.

XDP runs in the driver's receive routine before the kernel allocates a socket buffer, which is why it can drop packets at rates the rest of the stack cannot survive. It has no socket, no PID, no conntrack entry. Native XDP requires driver support; without it you get generic XDP, which runs after allocation and gives up most of the advantage.

tc, or sched_cls, attaches to the clsact qdisc on ingress and egress of any interface and receives a fully formed sk_buff, with metadata, VLAN handling, and the ability to redirect to another interface. This is where most of Cilium's datapath lives, and on newer kernels the tcx attach type replaces the old shared-filter arrangement with proper multi-program ordering.

Socket and cgroup hooks run in process context. A cgroup/connect4 program sees a connect() before any packet exists and can rewrite the destination address in place, which is how a service IP becomes a backend IP with no NAT translation on the packet path at all. sockops and sk_msg operate on established connections and can splice two sockets on the same node together.

Tracing hooks cover everything else. Tracepoints are static, named, and stable across kernel versions; kprobes attach to nearly any kernel symbol but depend on that symbol existing under that name in that kernel; fentry/fexit do the same job through BPF trampolines with meaningfully less overhead. Uprobes reach into user-space functions at the cost of a trap per hit. LSM hooks return a verdict rather than a report, which is the difference between observing an action and refusing it — and the basis of Tetragon's kernel-level enforcement.

$ bpftool net show dev eth0
xdp:
eth0(2) driver id 118

tc:
eth0(2) clsact/ingress cil_from_netdev id 247
eth0(2) clsact/egress  cil_to_netdev   id 251

What Keeps A Program Alive

Nothing in the lifecycle guarantees persistence. A loaded program is refcounted, and it is freed the moment the last reference disappears. References come from three places: an open file descriptor in the loading process, a BPF link, or a pinned path in the bpffs filesystem, conventionally mounted at /sys/fs/bpf.

$ ls /sys/fs/bpf/tc/globals/
cilium_ct4_global   cilium_lb4_services_v2   cilium_policy_00437
cilium_ipcache      cilium_lb4_backends_v3   cilium_metrics

This is why Cilium's datapath keeps forwarding while the agent is restarting or upgrading: the programs and maps are pinned, so they outlive the process that created them. It is also why a bpftrace one-liner stops collecting the instant you press Ctrl-C.

---

Practical Application

Learn what is attached, not just what is loaded. bpftool prog show lists programs that exist; it says nothing about whether they run. bpftool net show covers XDP and tc attachments, bpftool link list covers link-based ones, bpftool cgroup tree covers cgroup hooks, and ip -d link show dev eth0 tells you whether XDP is running in native or generic mode. You need all of them to answer "what is on this interface."

$ bpftool link list
14: cgroup  prog 302  cgroup_id 1  attach_type cgroup_inet4_connect
15: tracing prog 318  prog_type tracing  attach_type trace_fentry
    target tcp_sendmsg

Pick the hook by the facts you need in scope. If the answer must name a pod, attach where a task and a cgroup exist, or accept that you will be correlating IP addresses to pods after the fact and losing the race against pod churn. If the answer must be fast enough to survive a flood, attach before the socket buffer exists.

Prefer the stable attach point when there is one. A tracepoint is a kernel ABI. A kprobe on an internal function is a name that upstream may inline, rename, or delete without warning, and static functions frequently acquire compiler suffixes like .isra.0 between builds.

Treat attach failure as an alertable event. A loader that logs "could not attach" once at startup and then runs happily forever is worse than a crash, because the metric it feeds goes flat and flat looks like healthy.

---

Tradeoffs and Decision Framework

Hook	Fires on	Sees	Can change	Relative cost
XDP	Packet arrival in the driver	Raw frame only	Drop, pass, transmit, redirect	Lowest per packet
tc / sched_cls	Ingress and egress of an interface	Full sk_buff plus metadata	Rewrite, redirect, drop	Low
cgroup/connect4, sockops	Socket operations in task context	Socket, PID, cgroup	Destination address and port	Once per connection
Tracepoint	A named static kernel event	Fixed, documented arguments	Nothing, read only	Low, stable
kprobe / fentry	Entry to a kernel function	That function's arguments	Nothing, read only	Low, but per call
uprobe	A user-space function	Application arguments	Nothing, read only	High, trap per hit
LSM	A security decision point	The object being acted on	Allow or deny the operation	Once per operation

Four questions pick the hook. Does the answer need a process or pod identity, which only exists in task context? Does it need packet bytes, which only exist below the socket layer? Do you need to change the outcome, or only record it, because that difference eliminates most of the table immediately? And how often does the event fire, since a hook on a function called a million times a second has a very different budget than one on process execution?

Default to the highest hook that still has the facts you need — tracepoints over kprobes, socket hooks over tc, tc over XDP — because the higher hooks are more stable, better documented, and easier to reason about. Drop lower only when you have a concrete reason: raw packet access, a rate the stack cannot absorb, or a decision that has to be made before the kernel commits work.

---

Common Mistakes

Assuming loaded means running. A program with no attachment is inert and still appears in bpftool prog show with a healthy-looking entry.

Attaching a kprobe to an internal symbol and calling it done. Symbols get inlined and renamed across kernel versions, and the failure surfaces as silence rather than an error.

Treating flat metrics as good news. In an event-driven system, zero events and zero listeners look identical downstream unless the agent reports its own attach state.

Expecting pod identity at the wrong layer. An XDP or early tc program has no task, no cgroup, and no PID, so anything pod-aware has to come from a map populated elsewhere.

Forgetting that programs die with their loader. Without a pin or a link, the program is freed when the process exits, which surprises people the first time a DaemonSet restarts.

Leaving stale pins behind. A partially removed agent leaves maps in /sys/fs/bpf, and the next version inherits them along with whatever layout they had.

Ignoring native versus generic XDP. Generic XDP attaches on almost anything and quietly gives up the performance that was the entire reason to use XDP.

Underestimating per-event cost. Ten cheap instructions on a function called a million times a second is real CPU, and it lands in softirq where nobody is looking for it.

---