Linux Qdisc Selection Playbook (fq vs fq_codel vs CAKE)

Date: 2026-03-22
Category: knowledge
Scope: Practical operator guide for choosing and tuning Linux qdiscs to control latency, fairness, and throughput at bottlenecks.

1) Why this matters

Many “random latency spikes” in production are simply queueing policy mistakes:

• pacing expected, but no pacing-aware scheduler;
• fairness expected, but one elephant flow dominates;
• shaping expected, but only AQM is configured;
• AQM tuned, but hidden lower-layer queues still bloat.

In practice, qdisc choice is a first-order SLO lever for p95/p99 network delay.

2) Mental model: what each qdisc is best at

fq (Fair Queue) — best for host egress pacing

Use when:

• traffic is mostly locally generated by this host,
• you want TCP/app pacing (SO_MAX_PACING_RATE, EDT-aware stack behavior),
• you are not trying to do full link shaping policy.

Key traits (from tc-fq(8)):

• per-flow separation,
• non-work-conserving pacing scheduler,
• strong fit for sender pacing and burst smoothing.

fq_codel — best general low-latency fairness without heavy shaping policy

Use when:

• you need fair sharing across mixed flows,
• you need AQM to control persistent queue delay,
• you want robust defaults with low tuning burden.

Key traits (from tc-fq_codel(8) and RFC 8290):

• DRR-like flow queueing + CoDel AQM per queue,
• defaults: target 5ms, interval 100ms, flows 1024, ECN enabled,
• strong anti-bufferbloat baseline for many edge/gateway cases.

cake — best when you need shaping + fairness + practical ISP/link compensation

Use when:

• the bottleneck rate is known/estimable and must be shaped,
• overhead compensation matters (DSL/cable/encapsulation realities),
• you want host/flow fairness + DiffServ tins in one qdisc.

Key traits (from tc-cake(8)):

• shaping-capable qdisc (deficit-mode shaper),
• AQM via COBALT (CoDel + BLUE) + flow isolation,
• built-in overhead compensation keywords and host isolation modes,
• defaults include diffserv3, triple-isolate, rtt 100ms profile.

3) Fast decision map

1. Single server, locally generated egress, pacing quality is priority
   → start with fq.

2. General gateway/edge, mixed traffic, want low latency + fairness with minimal policy
   → start with fq_codel.

3. You must enforce a bandwidth ceiling at the bottleneck and account for link overhead
   → start with cake bandwidth ... (plus proper overhead mode).

4. Datacenter ultra-low RTT microbursts
   → be careful with default Internet-scale constants; validate AQM constants against real RTT regime before rollout.

4) Baseline configs (safe starting points)

4.1 Host pacing baseline (fq)

tc qdisc replace dev eth0 root fq

When to add knobs:

• cap abusive senders: maxrate;
• DCTCP-style shallow marking experiments: ce_threshold;
• collision pressure/high orphan traffic: buckets, orphan_mask review.

4.2 Fairness + AQM baseline (fq_codel)

tc qdisc replace dev eth0 root fq_codel

If your bottleneck RTT is far from “internet-ish”, tune conservatively:

tc qdisc replace dev eth0 root fq_codel target 5ms interval 100ms ecn

(Defaults shown explicitly for auditability.)

4.3 Shaping baseline (cake)

tc qdisc replace dev eth0 root cake bandwidth 500Mbit diffserv3 triple-isolate

If Ethernet framing overhead matters at the true bottleneck:

tc qdisc replace dev eth0 root cake bandwidth 500Mbit ethernet diffserv3 triple-isolate

If NAT fairness is needed at this host:

tc qdisc replace dev eth0 root cake bandwidth 500Mbit diffserv3 triple-isolate nat

5) Observability: what to watch in tc -s qdisc

Run:

tc -s qdisc show dev eth0

For fq

• throttled, flows, flows_plimit, ce_mark, dropped.
• Watch for chronic drops with low throughput: often pacing cap/queue limits mismatch.

For fq_codel

• dropped, drop_overlimit, ecn_mark, new_flow_count, backlog.
• Rising backlog + low ECN/drop response = congestion control or RTT constant mismatch.

For cake

• overall dropped, overlimits, memory used, capacity estimate.
• per-tin: pk_delay, av_delay, sp_delay, drops, marks, way_inds/miss/cols.
• host fairness stress appears as skewed tin/flow stats and persistent delay in one class.

Operational rule: trend these metrics together with app p95/p99 latency and retransmit/ECN counters. Qdisc counters alone can look “healthy” while app tails burn.

6) Common failure modes (and fixes)

Failure mode A: “We enabled AQM but latency still spikes under load”

Likely causes:

• bottleneck is actually elsewhere (driver/NIC/ISP queue),
• no real shaping at true choke point,
• offload/aggregation effects hiding queue dynamics.

Fix:

• identify real bottleneck hop first,
• if you own bottleneck rate, shape slightly below it,
• verify lower-layer queue behavior and offload posture during tests.

Failure mode B: “Throughput dropped after aggressive RTT profile”

Likely causes:

• AQM constants too aggressive for actual RTT,
• congestion signaled too early, sparse-flow bias over-amplified.

Fix:

• return to conservative defaults,
• tune RTT/interval with controlled sweep + throughput/latency Pareto chart.

Failure mode C: “One host still monopolizes link”

Likely causes:

• pure flow fairness not equal to host fairness,
• many-flow host outcompetes few-flow host.

Fix:

• use CAKE host-aware isolation mode (dual-*/triple-isolate) where appropriate,
• validate with synthetic many-flow bully tests.

Failure mode D: “DiffServ policy looks right but user experience is wrong”

Likely causes:

• inbound DSCP is untrusted/mis-marked,
• tins configured but traffic classification not aligned.

Fix:

• start with simpler policy (besteffort or controlled diffserv3),
• use wash on untrusted domains,
• only add class complexity when measurements justify it.

7) Rollout pattern (low risk)

1. Canary interface/host with mirrored workload profile.
2. Baseline capture: app latency, retransmits, tc -s, CPU.
3. One change at a time (qdisc type first, then params).
4. Guardrail thresholds: rollback if p99 latency or drop/mark ratio breaches limits.
5. Document known-good profile per link type (DC LAN, metro WAN, consumer ISP edge).

8) Practical defaults (starting point, not dogma)

• Default server egress pacing: fq.
• Default general fairness/AQM: fq_codel.
• Default bottleneck shaping with real-world overhead quirks: cake bandwidth ... with explicit overhead mode.

If you can’t explain where the true bottleneck queue lives, tuning qdisc knobs is mostly placebo.

9) References

• Linux tc-fq(8) manual
  https://man7.org/linux/man-pages/man8/tc-fq.8.html

• Linux tc-fq_codel(8) manual
  https://man7.org/linux/man-pages/man8/tc-fq_codel.8.html

• Linux tc-cake(8) manual
  https://man7.org/linux/man-pages/man8/tc-cake.8.html

• RFC 8290 — The Flow Queue CoDel Packet Scheduler and Active Queue Management Algorithm
  https://datatracker.ietf.org/doc/html/rfc8290

• RFC 8289 — Controlled Delay Active Queue Management
  https://datatracker.ietf.org/doc/html/rfc8289