docs

2026-10-31 19:08:03 +03:00 · 2026-09-04 23:53:20 +03:00
1 parent b64e40d88d
commit 3a89080495
@@ -0,0 +1,92 @@
 # `--clear-bitmap`: measured worse, default off
 `lvl_bitmap` advertises which levels hold tasks. Bits were originally set-only
 -- never cleared -- because clearing one needs a store/load barrier that LFBMQ
 gets wrong at `alt_core.c:4260`, and getting it wrong strands tasks forever.
 The cost of not clearing is that `ops.dispatch` walks every level ever used on
 every call. Measured before any change: a median of **55 DSQ probes to place
 one task**, peaking at 279, against 33 levels. Higher than the level count
 because most scanning happens on dispatch calls that find nothing and never
 increment `dispatch` -- a CPU going idle was touching 25 DSQs to learn it had
 no work.
 So clearing was implemented, correctly, with a repair sweep as a safety net.
 It works: probes per dispatch fall from ~19.5 to 0.53 under compile load.
 **And it is off by default, because it costs interactive latency.**
 ## The measurement
 Same binary, same instrument, same source tree, one flag. Two rounds run back
 to back, with the order reversed between them so session drift could not align
 with the result.
 | round | order | n | clear=false better | clear=true better | sign test | median ratio |
 | --- | --- | --- | --- | --- | --- | --- |
 | 1 | true then false | 35 | 23 | 11 | p = 0.058 | 0.84 |
 | 2 | false then true | 35 | 24 | 10 | p = 0.024 | 0.81 |
 | **pooled** | | **70** | **46** | **23** | **p = 0.008** | **0.84** |
 Round 1 is the more trustworthy of the two: `false` ran second there, so drift
 worked against the result and it still won. Round 2 has order and result
 aligned, so it likely overstates. The pooled figure is the honest middle.
 Corroborating and drift-independent, the p99 bucket median:
 | | p99 | p99.9 |
 | --- | --- | --- |
 | clear=true (both rounds) | 524 us | 1.00 ms |
 | clear=false (both rounds) | 459 us | 0.92 / 1.00 ms |
 One bucket, consistent across rounds, while absolute tail percentages moved
 between 0.098 and 0.160.
 ## Why
 The likely mechanism, unproven: set-only left `lvl_bitmap` effectively
 read-only after warmup, and the scan touched 25 different DSQ cache lines with
 unlocked reads every CPU can hold shared. Clearing turns one u64 into a
 write-hot line -- roughly 600k atomic RMWs on top of the ~1.1M the enqueue side
 already does -- that four threads must take exclusive turns owning.
 Twenty-five shared reads are cheaper than one contended line. That is a
 statement about a 2c/4t Ivy Bridge and may not survive a wider machine.
 ## The asymmetry that decides it
 Clearing buys 37x fewer probes per dispatch. That scan never had a *measured*
 latency cost: `consume_dispatch_q()` tests `list_empty()` unlocked and returns
 before taking `dsq->lock` (`ext.c:2517`), so a miss is a cache read and a kfunc
 call. Paying a measured cost for an unmeasured saving is the wrong trade.
 The knob stays. On a machine with more cores, more cache, or a longer level
 scan, the arithmetic could invert.
 ## What clearing did not buy
 It looked like clearing was also required for anti-starvation to work:
 `starve_serve` was 27 with bits set-only against 244 with them cleared, because
 the probe was gated on `lvl_bitmap` and most probes landed on levels that had
 been empty for minutes.
 That was an argument for not asking the bitmap, not for clearing it. The probe
 now calls `scx_bpf_dsq_nr_queued()` directly -- one unlocked read, and only
 after the free timestamp compare has already shown the level is past
 `starve_ns`. With bits set-only and the probe decoupled, `starve_serve` is 268:
 the protection works without paying for the clear.
 ## Open
 `p99` tracks `starve_serve` rather than `clear_bitmap` across all five runs:
 459 us when starvation is quiet (27, 32 serves), 524 us when it is active (244,
 268, 283). That suggests clearing and active anti-starvation are two
 independent causes of the same one-bucket loss, which would also explain why
 disabling anti-starvation entirely while clearing was still on changed nothing
 (17 of 35 intervals, p = 1.0) -- the bucket was already spent.
 If that holds, a working stall protection costs about one p99 bucket, and that
 is a policy price worth paying rather than a bug. It is built on cross-run
 comparisons, which this project has already been burned by once, so it is
 recorded as a hypothesis and not a result. Deciding it needs a knob for the
 probe gating and another back-to-back pair.
@@ -0,0 +1,104 @@
 # `--boost-on-wake`: the typo wins
 BMQ's `sched_task_ttwu()` is meant to boost a waking task in proportion to how
 long it slept. That is what makes sleeping for 1 us worth nothing and closes
 the sleep-gaming hole. The line reads
 ```c
 s64 delta = this_rq()->clock_task > p->last_ran;
 ```
 A comparison, not a subtraction. `delta` is 0 or 1, the shift takes it to 0,
 and the function has been dead code in every shipped BMQ build. `soryu` made it
 a knob rather than silently picking a side: `--boost-on-wake=true` is what BMQ
 meant to do, `false` is what BMQ actually does.
 **Result: off is better, and it is not close.**
 ## What was run
 The scheduler compiling itself, twice per configuration, on the Ivy Bridge
 target (i5-3210M, 2c/4t, kernel 7.1.6). Each run logged one report every 10 s
 with `--report-sec 10`, giving ~34 intervals per run. Rounds 1 and 2 were run
 in separate working trees.
 Every number below is per-interval, not cumulative. That matters: before the
 histograms were differenced, a run printed one figure blended across every
 phase the build went through, which described none of them.
 ## Latency
 Interactive (`boosted` class) wake->run, median across intervals:
 | | p99 | p99.9 | tail >1 ms |
 | --- | --- | --- | --- |
 | on, round 1 | 524 us | 1.30 ms | 0.152% |
 | on, round 2 | 524 us | 1.80 ms | 0.168% |
 | **off, round 1** | **459 us** | **0.92 ms** | **0.075%** |
 | **off, round 2** | **459 us** | **0.92 ms** | **0.077%** |
 The two `off` runs agree to the last digit on all three columns.
 Pairing the runs interval by interval on the >1 ms tail:
 | | intervals | off better | on better | sign test | median ratio |
 | --- | --- | --- | --- | --- | --- |
 | round 1 | 34 | 31 | 3 | p = 7.7e-07 | 0.53 |
 | round 2 | 34 | 32 | 2 | p = 6.9e-08 | 0.45 |
 | pooled | 68 | 63 | 5 | p = 7.7e-14 | 0.47 |
 Worst single interval, which is the number a stutter would live in:
 | | worst tail >1 ms | worst p99.9 | worst `max_wait` |
 | --- | --- | --- | --- |
 | on, round 1 | 0.299% | 4.2 ms | 21 j |
 | on, round 2 | 1.787% | 21.0 ms | 38 j |
 | off, round 1 | 0.656% | 7.3 ms | 22 j |
 | off, round 2 | 0.129% | 0.92 ms | 21 j |
 Each configuration produced one bad round, and `on`'s was worse: two
 consecutive intervals at 21 ms p99.9.
 ## Throughput
 | | round 1 | round 2 |
 | --- | --- | --- |
 | on | 5 m 34 s | 5 m 45 s |
 | off | 5 m 39 s | 5 m 32 s |
 The ordering flips between rounds. With n = 2 per configuration and a ~10 s
 spread already known from earlier runs, there is no throughput difference to
 report in either direction. The latency result stands on 68 paired intervals;
 the timing result stands on nothing, and is not claimed.
 ## Health
 `insert_fail = 0`, `starve_serve` 22-26 across all four runs, `max_wait` 21-22
 jiffies except the one 38 j interval noted above. No stalls, no ejections.
 ## Why the default flipped
 Two independent arguments point the same way. The measurement says off. And
 off is BMQ as actually shipped -- the behaviour a decade of field use already
 ran, rather than a resurrection nobody has tested. The knob stays so the
 question can be reopened on other hardware.
 ## Open question
 The mechanism is not understood. The wake boost grants up to 24 steps at once,
 so a waking task is slammed to the floor rather than climbing at +1 per sleep,
 and the guess is that this makes a task's level change often enough that
 `cpu_level[]` is stale when `try_preempt()` reads it. That is a guess. It would
 be tested with a per-task counter of how often the level computed at enqueue
 differs from the one seen at `ops.running`.
 Note this is not the boost distribution collapsing. Under compile load both
 configurations sit bimodal at roughly 54% on the -12 floor and 39% on the +12
 ceiling, with about 5% spread across the twenty-three values between, and
 `preempt_none` stays near 1.2% in both.
 That bimodality was expected to disable preemption -- if everything sits at one
 level, nothing outranks anything and `try_preempt()` finds nobody worth an IPI.
 It does not, because the range saturates at *both* ends rather than one: the
 crowd on the floor always has a ceiling hog to displace. Saturation at one end
 would have killed preemption; saturation at both preserves it.
@@ -0,0 +1,135 @@
 # cgroup weights, and priority donation down the wake chain
 Two additions, both knobs, both default off. One closes a functional gap, the
 other tested an idea and came back null.
 ## cgroup cpu.weight
 Until now the scheduler was completely deaf to it. `ops.cgroup_set_weight` was
 not implemented, so systemd's `CPUWeight=`, and with it the whole
 `system.slice` / `user.slice` / `app.slice` arrangement every desktop already
 has, did nothing. nice worked; cgroups did not.
 `ops.cgroup_init`, `ops.cgroup_set_weight` and `ops.cgroup_move` are now
 implemented, with weight held in `BPF_MAP_TYPE_CGRP_STORAGE` keyed by the
 cgroup pointer.
 `cpu.weight` is `[1, 10000]` with 100 the default and it is geometric: one nice
 step is a factor of 1.25, so an octave of weight is `log(2)/log(1.25)` = 3.1
 nice steps. Rounded down to 3, which errs toward giving cgroups slightly less
 authority than nice rather than more.
 The awkward part is caching. `ops.cgroup_set_weight` affects every task in a
 cgroup and there is no way to walk them, so a generation counter is bumped on
 any weight change; each task compares its own `cgrp_seen` against it and
 refreshes once, in `ops.enqueue`. `task_level()` only reads the cached value,
 so the hot paths never pay for `scx_bpf_task_cgroup()`'s acquire/release.
 `--cgrp-weight-levels` caps the movement in each direction. 0 is the default.
 It works, and what it does is not what `cpu.weight` means anywhere else.
 ### Measured
 `tools/cgroup-share.sh`: two cgroups at `cpu.weight` 1000 and 10, N spinners in
 each, on a 4-CPU machine. CPU time each group actually got over 30 s.
 | spinners per group | ratio hi:lo |
 | --- | --- |
 | knob off, 4 | **1.00 : 1** |
 | 2 (half the CPUs) | 1.36 : 1 |
 | 4 (exactly the CPUs) | 3.83 : 1 |
 | 8 (twice the CPUs) | **497 : 1** |
 The first line is the gap this closed: `cpu.weight` had precisely no effect
 before.
 The rest is the shape of the thing. At 4 spinners per group the ratio looks
 almost reasonable, and it is an artifact of the test: a hi task whose slice
 expires is not yet back in its DSQ when that CPU calls `ops.dispatch`, and the
 other three are on CPUs, so its level is momentarily empty and dispatch falls
 through to lo. Nothing is violated -- hi genuinely was not queued.
 Oversubscribe the group and that window closes. At 8 per group, lo got 200 ms
 out of 30 s. The scheduler log says why: `enq_local` 0 throughout, so no idle
 fast path, and `starve_serve` at up to 30/s. That 200 ms *is* the
 anti-starvation crumb and nothing else.
 ### It is a switch, not a dial
 BMQ is strict priority. Once two groups land on different levels, the better
 one wins whenever it has anything queued -- and the size of the weight
 difference does not enter into it, only whether it crossed a level boundary.
 Lowering `--cgrp-weight-levels` does not soften that. It only raises the weight
 ratio needed to cross a boundary. 1000-against-100 at `levels 2` is still one
 level apart and still 497:1.
 CFS gives a 100:1 weight ratio a 100:1 share. Here a 100:1 weight ratio means
 the lighter group is off.
 ### The number that decides the default
 `max_wait` reached **1067 jiffies -- 1.07 seconds** -- during the 8-spinner run,
 with `l25` (the lo group) as the level waiting. The watchdog ejects at 5 s.
 So this knob, at 6, on an oversubscribed machine, runs at a fifth of the stall
 threshold, and the only thing between it and an ejection is the anti-starvation
 mechanism. That is a real dependency and it is worth stating: `starve_ns = 0`
 together with `cgrp_weight_levels > 0` is a combination that will eject.
 Default stays 0. Anyone enabling it should know they are getting a priority
 band, not a share. Proportional behaviour is not reachable from a level
 adjustment at all -- it would need a second mechanism underneath the levels,
 which is a different scheduler.
 ## Wake-chain donation: null
 XNU propagates priority through blocking chains with turnstiles; Linux does the
 same for `rt_mutex` but only inside the RT class. From BPF the lock graph is
 invisible, but the wake graph is not, and it is a decent proxy: if A wakes B, B
 is usually about to do something for A.
 `ops.runnable` runs in the waker's context for a synchronous wakeup, so
 `bpf_get_current_task_btf()` is the waker. If the waker's boost is better, the
 wakee is pulled toward it, capped by the gap so it never overtakes -- without
 that cap a chain of wakeups ratchets everyone to the floor, which is the
 saturation boost already suffers from.
 This was deliberately the opposite kind of idea from the ULE ratio. ULE graded
 more finely and the tail did not move, which said the problem is not priority
 resolution. Donation does not grade; it changes *who* is urgent.
 `--wake-donate 4` against 0, back to back on the same compile:
 | | intervals | wake_donate | tail >1 ms | p99 | p99.9 | boost floor |
 | --- | --- | --- | --- | --- | --- | --- |
 | 4 | 37 | 5035 | 0.155% | 655 us | 1.00 ms | 50.9% |
 | 0 | 38 | 0 | 0.142% | 655 us | 1.00 ms | 50.8% |
 Paired: donate better in 14 intervals, off in 22, sign test p = 0.24. p99 and
 p99.9 bit-identical.
 ## Why it did nothing, which is the useful part
 Donation fired 5035 times, roughly 0.6% of wakeups, and the boost floor came
 out at 50.9% against 50.8% -- it did not move the population at all.
 The condition needs the waker to be strictly better than the wakee, and half
 the population is already pinned at -12. Waker at the floor, wakee at the
 floor, no gap, no donation.
 So saturation kills both ideas from opposite ends: ULE produces a gradient and
 grading turns out not to help, while donation needs a gradient and boost does
 not have one. Feeding ULE's gradient into donation is the obvious next chain,
 and it is built on the half of the pair that was already measured not to
 matter, so it was not attempted.
 A larger dose is the one cheap variant left. Moving 0.6% of the population
 further is not going to move a p99 computed over all of it, so the expectation
 is another null.
 ## Defaults
 Both 0. The cgroup path is a gap worth having closed and awaits a share
 measurement; donation is a tested null kept for the knob and the record.
@@ -0,0 +1,89 @@
 # cpu_release: a blind spot, an answer, and a knob that changed nothing
 The interactive tail on an idle-ish desktop is consistently worse than during a
 full compile -- 0.29% past 1 ms against 0.15% -- which is backwards, and the
 one number that tracked it was `cpu_release`: about 130 a second on the desktop
 against 3 during a compile.
 ## The instrument was blind to it
 `scx_bpf_reenqueue_local()` throws the local DSQ back through `ops.enqueue`
 with `SCX_ENQ_REENQ`. `ops.runnable` is not part of that path, so `wake_at` is
 never stamped, and `ops.running` records a latency sample only when `wake_at`
 is set.
 So every task bounced by a higher sched class spent that trip invisible. The
 tail that every decision on this branch was measured against excluded exactly
 the population under suspicion. `bounce_hist` was added to close that, stamped
 at `SCX_ENQ_REENQ` in `ops.enqueue` and read at `ops.running`, recorded
 alongside the wake sample rather than instead of it.
 ## What takes the CPU
 `ops.cpu_release` now splits by `enum scx_cpu_preempt_reason`.
 | | releases | rt | stop | dl |
 | --- | --- | --- | --- | --- |
 | compile, 6 min | 662 | 307 | 355 | 0 |
 | desktop, 5 min | 38916 | 38775 | 141 | 0 |
 Desktop is **99.6% RT**, at roughly 130 releases a second, each bouncing 0.98
 tasks. No deadline tasks at all.
 ## Bounced tasks wait about twice as long
 | | wake->run p99 | bounce->run p99 | bounce share |
 | --- | --- | --- | --- |
 | compile | 655 us | 786 us | 0.1% |
 | desktop | 655 us | 1.3 ms | 1.8% |
 Real, and too small to matter. At 1.8% of samples with a p99 of 1.3 ms,
 bounces contribute roughly 18 events past 1 ms per interval against 193 from
 the wake population -- about 9% more tail, not the doubling that needed
 explaining.
 ## So the answer is the boring one
 The desktop's *non-bounced* population is already worse: `wake->run` p99.9 of
 1.45 ms against 1.00 ms during a compile. RT preemption at 130 a second does
 not mainly hurt the task it bounces, it steals CPU from everyone and raises
 queueing delay across the board.
 That is RT working as designed. The scheduler has no move here.
 ## The knob, and its null
 If the RT burst is short -- an audio callback is tens of microseconds -- then
 re-enqueueing is waste: the task pays a full trip through `ops.enqueue`, level
 recomputed and DSQ reinserted, to avoid a wait it would have outlasted.
 `--reenqueue-on-release=false` tests that.
 | | intervals | cpu_release/iv | enq_reenq | tail >1 ms | p99 | p99.9 | max_wait |
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | true | 14 | 1138 | 15580 | 0.368% | 655 us | 1.60 ms | 32 j |
 | false | 30 | 897 | 0 | 0.410% | 655 us | 1.60 ms | 22 j |
 Paired: `false` better in 4 intervals, `true` in 10, sign test p = 0.18.
 The knob works -- `enq_reenq` and the bounce histogram both go to zero -- and
 changes nothing measurable. p99 and p99.9 are bit-identical. This was predicted
 before the run, on the arithmetic above, and the prediction is recorded here
 because on this branch it is the third or fourth hypothesis to be tested and
 the first to survive.
 The comparison is weak on its own terms: the two runs differ by 27% in release
 rate and by more than double in length, because desktop activity is not
 reproducible the way a compile is. It is not worth strengthening. The prior and
 the data agree, and they agree on "no effect".
 ## Default
 `reenqueue_on_release = true`. It is the conventional behaviour, it is what the
 paired direction leans toward, and it is the safe side of a risk this test did
 not exercise: with it off, tasks in the local DSQ of a CPU taken by a *long*
 RT or DL task cannot migrate away, and `starve_ns` walks the shared level DSQs,
 not local ones. Short audio bursts are fine. An RT hog would be a 5 s watchdog
 ejection.
 `bounce_hist` and the reason split stay regardless. They cost nothing and they
 closed a hole in the instrument that had been open the whole time.
@@ -23,8 +23,9 @@ services.scx-soryu = {
 };
 ```
-that's it. the defaults are measured values: `sliceUs = 1000`
+that's it. the defaults are measured values: `sliceUs = 1000` (the same
-([`slice.md`](slice.md)), `starveMs = 128` ([`stage6-starve.md`](stage6-starve.md)),
+compile took 4m49s at 4194 and 4m39s at both 1000 and 500), `starveMs = 128`,
 `boostOnWake = false` ([`boost-on-wake.md`](boost-on-wake.md)),
 `timeoutMs = 5000`, `reportSec = 60`.
 to turn it off: `services.scx-soryu.enable = false;` or
@@ -75,7 +76,8 @@ sudo scx_soryu --monitor 3
 ## verified in a VM
 `nixosConfigurations.servicevm` — the same test VM, but running the
-scheduler as a service instead of by hand. `tools/servicevm.sh`:
+scheduler as a service instead of by hand. the checks that were run against
 it:
 | check | result |
 | --- | --- |
@@ -0,0 +1,87 @@
 # ULE's interactivity ratio: it separates, and separating does not help
 BMQ grades a task with a saturating counter: one step better per sleep, one
 step worse per burnt slice. Measured under compile load, that counter is not a
 grade -- it is a one-bit classifier. The distribution sits bimodal at roughly
 50% on the -12 floor and 44% on the +12 ceiling, with about 6% spread across
 the twenty-three values between.
 FreeBSD ULE asks a different question: not how many times you slept, but what
 fraction of your recent life was sleep, over a decaying window. That is a
 continuum by construction. The hypothesis was that it would place the tasks
 BMQ cannot -- a render thread, a compositor, a decoder, things that both sleep
 a lot and burn a lot and so oscillate around the middle under a counter.
 **The hypothesis was half right, and the useful half was the wrong half.**
 ## The score does separate
 `ule_weight = 0`, so the score was computed and histogrammed without being
 allowed to touch a scheduling decision. Split by what BMQ's boost thought of
 the same task at the same moment:
 | class | share of events | ULE mean | off both extremes |
 | --- | --- | --- | --- |
 | boosted (boost < 0) | 52.8% | 4.7 | 18.5% |
 | neutral (boost == 0) | 0.7% | 58.7 | 72.5% |
 | deboosted (boost > 0) | 46.4% | 94.8 | 2.7% |
 On the hog end the two metrics agree completely: 95.3% of the deboosted class
 scores 95-99. ULE adds nothing there and is exactly as saturated as boost.
 On the interactive end it adds a lot. Boost puts 50.4% of all events at
 precisely -12, which is about 95% of the boosted class on a single value. ULE
 splits that same population:
 ```
  0-4    67.9%
  5-9    14.6%
 10-14   16.5%
 ```
 Three populations where boost sees one.
 ## Letting it decide changes nothing
 `--ule-weight 12`, the largest setting that keeps ULE's authority comparable to
 nice's, run back to back against `--ule-weight 0` on the same tree.
 | | intervals | tail >1 ms | p50 | p99 | p99.9 | distinct levels | max_wait |
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | W=12 | 34 | 0.108% | 16.4 us | 393 us | 1.00 ms | 26 (span 0-28) | 21 j |
 | W=0 | 34 | 0.106% | 16.4 us | 459 us | 1.00 ms | 15 (span 0-22) | 20 j |
 Paired: W=12 better in 16 intervals, W=0 in 17. **Sign test p = 1.0.**
 The mechanism engaged -- distinct levels in use went from 15 to 26 and the
 span from 0-22 to 0-28, so the score really did reach the level arithmetic and
 stretch the distribution. It simply did not move the interactive tail.
 p99 came out one bucket better, on one pair. This project has already been
 burned once by trusting a single-run bucket comparison, so that is recorded as
 a hint and not a result.
 No harm either: `insert_fail` 0, `max_wait` 21 j against 20 j, and the deeper
 level span was absorbed by anti-starvation (392 serves against 281).
 ## What this rules out
 The interactive tail is not limited by priority resolution. Grading more
 finely, carrying the grade into the level, and roughly doubling the width of
 the priority distribution all failed to move it.
 That is worth knowing because it was the premise behind the whole ULE detour.
 ## Default
 `ule_weight = 0`. The score is still computed and reported, because measuring
 it is free and it may answer a different question later.
 ## Caveat on the mapping
 `adj = (score - 50) * weight / 50`, and the level is `(... + adj) / 2`, so the
 0-14 band that holds 99% of the interactive population maps to only `0.28 *
 weight` units and then halves again. At W=12 that is three distinct adjustments
 (-11, -10, -9) and one or two distinct levels. A mapping fitted to the observed
 distribution would extract more separation -- and would be fitted to one
 workload on one machine, which is why it was not done.
@@ -0,0 +1,78 @@
 # vtime ordering inside a level: null, because the target was already gone
 Every mature sched_ext scheduler orders its DSQs by virtual time --
 `scx_bpf_dsq_insert_vtime` is used by bpfland, flash, lavd, p2dq, cake,
 mitosis, beerland and forge. soryu inserted plain FIFO, and was the outlier.
 It was worth trying here for a specific reason. The ULE ratio had already shown
 that grading more finely does nothing (p = 1.0), so priority *resolution* is
 not the problem. vtime is a different axis: it changes ordering *within* a
 band, which nothing had touched.
 And FIFO's worst case had the shape of the tail that was left. Arrive at a
 level behind k tasks and wait k slices; at 1 ms slices, k = 5 is 5 ms, which
 was the >4.2 ms band. Roughly half of all events sit on the boost floor, so
 that queue is the deep one and FIFO is at its worst there.
 ## Design
 Unweighted, on purpose. Tasks sharing a level share a `(nice + boost)` by
 construction, so priority already lives in the level and vtime inside one means
 only "who has used least CPU goes first". BMQ's strict priority between levels
 is untouched.
 - `ops.stopping` charges `dsq_vtime += ran`, actual runtime
 - `ops.running` advances `vtime_now`, the reference clock
 - `ops.enqueue` clamps a sleeper's credit to one slice, then inserts by vtime
 - `init_task` starts a task at `vtime_now`; starting at 0 would read as an
  enormous credit and hand every new task the head of its level
 `time_before()` rather than `<`, so the comparison survives `vtime_now` passing
 2^63. A DSQ must be used consistently as FIFO or vtime and never both; the knob
 is read-only after load and the other paths use built-in DSQs, so the level
 DSQs stay consistent either way.
 ## Measured
 Back to back, same tree, `cargo clean` before each build:
 | | intervals | >1 ms | >4.2 ms | >16.8 ms | p50 | p99 | p99.9 |
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | vtime | 40 | 0.170% | 0.002% | 0.0000% | 16.4 us | 655 us | 1.00 ms |
 | FIFO | 41 | 0.161% | 0.000% | 0.0000% | 16.4 us | 655 us | 1.00 ms |
 Paired: >1 ms, 22 to 18, p = 0.64. >4.2 ms, 9 to 14 with 17 ties, p = 0.40.
 >16.8 ms, 33 of 40 intervals tied at zero. p50, p99 and p99.9 bit-identical.
 ## Why it is null
 The band it was written to fix is empty. Across the day:
 | | >1 ms | >4.2 ms |
 | --- | --- | --- |
 | boost_on_wake on (original) | 0.156% | 0.057% |
 | boost_on_wake off | 0.075% | 0.004% |
 | vtime on | 0.170% | 0.002% |
 | vtime off | 0.161% | 0.000% |
 Flipping `boost_on_wake` took the far tail from 0.057% to 0.004%, and it now
 sits at 0.000-0.002%, which is the measurement floor. FIFO depth is not
 producing a far tail because there is no far tail left.
 The prediction was not wrong so much as unemployed. There was no way to know
 without measuring: that the >4.2 ms band had bottomed out only became visible
 with every run of the day lined up side by side.
 The >1 ms tail that remains, ~0.16% and identical in both configurations, is
 most likely irreducible -- four saturated threads and real work queued ahead.
 vtime never claimed to change average wait, only worst-case wait, and the
 worst case is gone.
 ## Kept, default off
 The mechanism is sound and the rest of the ecosystem relies on it. What failed
 was not the idea but its applicability to a 4-thread machine with a single LLC,
 where level queues do not get deep enough for ordering within one to matter.
 On the 6c/12t targets the queues are deeper and this is worth re-running before
 concluding anything. The knob is there for that.