# cpu_release: a blind spot, an answer, and a knob that changed nothing

The interactive tail on an idle-ish desktop is consistently worse than during a
full compile -- 0.29% past 1 ms against 0.15% -- which is backwards, and the
one number that tracked it was `cpu_release`: about 130 a second on the desktop
against 3 during a compile.

## The instrument was blind to it

`scx_bpf_reenqueue_local()` throws the local DSQ back through `ops.enqueue`
with `SCX_ENQ_REENQ`. `ops.runnable` is not part of that path, so `wake_at` is
never stamped, and `ops.running` records a latency sample only when `wake_at`
is set.

So every task bounced by a higher sched class spent that trip invisible. The
tail that every decision on this branch was measured against excluded exactly
the population under suspicion. `bounce_hist` was added to close that, stamped
at `SCX_ENQ_REENQ` in `ops.enqueue` and read at `ops.running`, recorded
alongside the wake sample rather than instead of it.

## What takes the CPU

`ops.cpu_release` now splits by `enum scx_cpu_preempt_reason`.

| | releases | rt | stop | dl |
| --- | --- | --- | --- | --- |
| compile, 6 min | 662 | 307 | 355 | 0 |
| desktop, 5 min | 38916 | 38775 | 141 | 0 |

Desktop is **99.6% RT**, at roughly 130 releases a second, each bouncing 0.98
tasks. No deadline tasks at all.

## Bounced tasks wait about twice as long

| | wake->run p99 | bounce->run p99 | bounce share |
| --- | --- | --- | --- |
| compile | 655 us | 786 us | 0.1% |
| desktop | 655 us | 1.3 ms | 1.8% |

Real, and too small to matter. At 1.8% of samples with a p99 of 1.3 ms,
bounces contribute roughly 18 events past 1 ms per interval against 193 from
the wake population -- about 9% more tail, not the doubling that needed
explaining.

## So the answer is the boring one

The desktop's *non-bounced* population is already worse: `wake->run` p99.9 of
1.45 ms against 1.00 ms during a compile. RT preemption at 130 a second does
not mainly hurt the task it bounces, it steals CPU from everyone and raises
queueing delay across the board.

That is RT working as designed. The scheduler has no move here.

## The knob, and its null

If the RT burst is short -- an audio callback is tens of microseconds -- then
re-enqueueing is waste: the task pays a full trip through `ops.enqueue`, level
recomputed and DSQ reinserted, to avoid a wait it would have outlasted.
`--reenqueue-on-release=false` tests that.

| | intervals | cpu_release/iv | enq_reenq | tail >1 ms | p99 | p99.9 | max_wait |
| --- | --- | --- | --- | --- | --- | --- | --- |
| true | 14 | 1138 | 15580 | 0.368% | 655 us | 1.60 ms | 32 j |
| false | 30 | 897 | 0 | 0.410% | 655 us | 1.60 ms | 22 j |

Paired: `false` better in 4 intervals, `true` in 10, sign test p = 0.18.

The knob works -- `enq_reenq` and the bounce histogram both go to zero -- and
changes nothing measurable. p99 and p99.9 are bit-identical. This was predicted
before the run, on the arithmetic above, and the prediction is recorded here
because on this branch it is the third or fourth hypothesis to be tested and
the first to survive.

The comparison is weak on its own terms: the two runs differ by 27% in release
rate and by more than double in length, because desktop activity is not
reproducible the way a compile is. It is not worth strengthening. The prior and
the data agree, and they agree on "no effect".

## Default

`reenqueue_on_release = true`. It is the conventional behaviour, it is what the
paired direction leans toward, and it is the safe side of a risk this test did
not exercise: with it off, tasks in the local DSQ of a CPU taken by a *long*
RT or DL task cannot migrate away, and `starve_ns` walks the shared level DSQs,
not local ones. Short audio bursts are fine. An RT hog would be a 5 s watchdog
ejection.

`bounce_hist` and the reason split stay regardless. They cost nothing and they
closed a hole in the instrument that had been open the whole time.