Game</th>	Narrator</th></tr></thead>
Civilization IV</td>	Leonard Nimoy</td></tr>
Civilization V</td>	W. Morgan Sheppard</td></tr>
Civilization VI</td>	Sean Bean</td></tr>
Civilization VII</td>	Gwendoline Christie</td></tr> </tbody></table> Sean Bean is Civ VI</strong>. Civ V — the one I was sure about — was W. Morgan Sheppard, the English character actor with the gravel in his voice, who did the tech quotes and the intro and died in 2019. Civ VII, in 2025, went to Gwendoline Christie, the first woman to narrate the series and the second Game of Thrones alumnus in a row, which is a hiring pattern I respect.</p> And the Karlton quote is in none of them. As far as I can tell it has never appeared in a Civilization game. What Sean Bean actually reads you when you finish Computers</strong> in Civ VI is:</p> To err is human, but to really foul things up you need a computer. — Paul R. Ehrlich</em></p> </blockquote> The good thing about computers is that they do what you tell them to do. The bad news is that they do what you tell them to do. — Ted Nelson</em></p> </blockquote> Same register. Same cadence. Same wry-aphorism-about-machines slot in the sentence. If you were going to falsely remember a programming joke as having been narrated to you over a tech-tree fanfare, these are exactly the neighbours that would make it feel true.</p> Why the misremembering is on-topic#</span></a> </h2> Two decades of Civilization trained a generation of people who work with computers to receive pithy aphorisms in a serious British voice with strings under it. The Karlton line fits that slot so precisely — right length, right irony, right subject — that my brain filed it there and, when asked for a source, returned the one it had lying around.</p> Which is, and I'm aware of how neat this is, a cache invalidation bug. The aphorism was cached. Its provenance was evicted. Nothing signalled that the entry had gone stale, so the lookup succeeded and returned confident garbage: Sean Bean, Civ 5</em>. No error. No miss. Just a plausible value served warm from a bad line.</p> Human memory is a cache with an excellent hit rate, no TTL, and no invalidation protocol whatsoever. It will hand you a wrong answer with exactly the same subjective confidence as a right one, which is the whole reason the original observation is any good: the hard problems aren't the ones that announce themselves by crashing.</p> So, corrected and with numbering intact:</p> There are only two hard things in Computer Science: cache invalidation and naming things. — Phil Karlton</strong>, out loud, sometime between 1970 and 1997, to somebody who was paying attention</p> </blockquote> Nobody read it to you. It doesn't need the voice.</p> Sources:</strong> Martin Fowler, Two Hard Things</em></a> · David Karlton, Naming things is hard</em></a> · karlton.org</a> · Phil Karlton's Netscape home page</a> · Kellan Elliott-McCrea (2005)</a> · PC Gamer on Sean Bean and Civ 6</a> · PC Gamer on Gwendoline Christie and Civ 7</a> · Civilization Wiki: Computers (Civ6)</a></p> I Put Claude Code in a Cgroup Jail Because It Kept Eating My Workstation Unknown — Fri, 31 Jul 2026 00:00:00 +0000 For two weeks, heracles — my main box, 128 GB of RAM, the machine I drive the whole fleet from — kept hanging solid and rebooting itself out from under me.</p> Not a crash you get to watch. A wedge</em>. One second I'm working across a dozen terminals; the next the pointer won't move, the fans spin up like a jet on the apron, and every input I give the machine vanishes into a tarpit. Thirty seconds of that and the hardware watchdog gives up, yanks the reset line, and I'm looking at a login greeter with every tmux session, every browser tab, every train of thought I had going — gone</em>.</p> It happened on the 13th. It happened again on the 26th. It happened on the 29th while I sat there and watched it die. Losing a day's context to a silent reboot is a special kind of rage, the kind that makes you want to physically fight a computer. So I stopped being angry at the symptom and went to find out what was actually killing my machine.</p> This is that story. It has a villain, and the villain is JavaScript. Obviously it's JavaScript.</p> The autopsy#</span></a> </h2> Here's the thing about a machine that wedges hard: its own logs are useless, because the journal never gets flushed. heracles' on-disk journal just ends</em>, mid-line, at the moment of death. So years ago I set up netconsole — the kernel screaming its dying words over UDP to a different box on the network before it loses the ability to do anything at all. ares catches heracles' last breath.</p> And heracles' last words, every single time, were the same:</p> oom-kill:constraint=CONSTRAINT_NONE,...,global_oom</span></span></code></pre> A global out-of-memory. The kernel ran out of RAM, invoked the OOM killer, and the OOM killer — in its infinite wisdom — started executing whatever</em>. Not the thing eating the memory. Just... things. wireplumber. My emacs daemon. The accessibility bus. Tiny 30-megabyte desktop services, shot one after another, each death freeing nothing, while the actual gluttons sat there untouched and the box spiraled into a reclaim stall until the watchdog put it out of its misery.</p> So: what filled 128 gigabytes? I ranked the processes from the OOM dump. The entire top of the list was one thing, over and over:</p>7.36 GB .claude-unwrapped</span></span> 7.35 GB .claude-unwrapped</span></span> 7.35 GB .claude-unwrapped</span></span> ... (twenty of them, five to seven gigs each)</span></span></code></pre> Twenty-odd Claude Code sessions, north of 100 GB combined. My AI assistant had eaten my workstation.</p> "Bun Pool 5 invoked oom-killer"#</span></a> </h2> Now, a detail that made me tilt my head. The thread that actually tripped the final allocation was named Bun Pool 5</code>. Bun. As in the JavaScript runtime.</p> I want to be precise about what I learned next, because it reframes everything. Claude Code is not a Node script. It's not a little CLI that shells out. Claude Code, as shipped, is a 257-megabyte single-file executable</strong> — the entire Bun runtime (v1.4.0) with JavaScriptCore baked in, plus the whole application bundle, bun build --compile</code>'d into one enormous ELF binary. Bun Pool 5</code> wasn't some side process. It was Claude Code itself. Those worker threads are</em> the program.</p> Which means every one of those 7-gigabyte lines was a single chat session's JavaScript heap.</p> Seven gigabytes. For a terminal program. That mostly sits there waiting for me to type.</p> The ratchet#</span></a> </h2> It gets worse, and this is the part that made me put my head on the desk.</p> A few days later — after I'd started building the fix — I was mid-session, sitting at a prompt Claude had drawn for me, deciding how to answer. Hadn't typed anything in a while. And the session got killed. So I went and looked at what it had been holding at the moment of death. systemd logs it right there on the way out:</p> claude-1654491.scope: Consumed 25min CPU over 3h 18min wall clock time, 21.5G memory peak.</span></span></code></pre> Twenty-one and a half gigabytes.</strong> Sitting idle. At a menu. For three hours. That one session was 83% of all the Claude memory on the box. A fresh session starts at about 400 megabytes. This thing had ratcheted to fifty times</em> that and then just... held it, doing nothing, while I stared at a multiple-choice question.</p> And "ratcheted" is exactly the right word, because this is the specific technical sin, and I want to name it precisely instead of just yelling "JavaScript bad."</p> JavaScriptCore — like V8, like every one of these runtimes — allocates its heap in a big arena. When garbage collection runs, it frees your dead objects inside</em> that arena. It reuses the space. What it does not</strong> do, essentially ever, is give the pages back to the operating system. So the arena is a one-way ratchet. The instant a session briefly needs 21 GB — you had it read a big file, or it captured a huge build log into its context, or you resumed a long conversation and it deserialized the whole transcript back into memory — the arena grows to fit. And then it stays that big. Forever. For the entire life of the process. The memory is "free" as far as the JS heap is concerned, and completely unavailable as far as the rest of the box is concerned.</p> This is not a bug in Claude Code, exactly. It's the water the entire ecosystem swims in. It's what you get when an industry decides that the correct runtime for a systems tool — a thing that runs on my machine, in my resource budget, next to my real work — is a browser engine with a garbage collector that treats "give memory back" as somebody else's problem. Ship it as a 257 MB binary, let the heap grow without bound, and if the user's workstation falls over under ten concurrent copies, well, buy more RAM. I have</em> 128 gigabytes. It wasn't enough, because the software was written by people for whom "it works on my machine" is the entire quality bar.</p> I'm not going to fix their memory management. I can't; it's compiled into a quarter-gigabyte blob and there isn't even a heap-limit flag exposed. But I run NixOS, and I don't have to trust</em> software. I can put it in a cage.</p> Building the cage#</span></a> </h2> Here's the good news, and the reason I'll die on the NixOS hill: my entire fleet is one declarative repo. Every machine, every service, every dotfile, and — as of this saga — the exact resource jail every claude</code> process runs inside. I don't SSH into a box and tweak things. I change the source and rebuild.</p> The first thing I tried was wrong</strong>, and I'm including it because pretending you got it right on the first go is how blogs lie to you. I added zram — compressed swap in RAM — figuring a swap cushion would let pressure build gracefully instead of slamming into a wall. Reader, it made the next crash worse</em>. When your problem is a hot working set of live JavaScript heaps, compressing them into a different part of the same RAM buys you nothing and costs you CPU. At the next death, 56 GB of physical memory was consumed by zram's own compressed pages, on top of</em> 60 GB of resident heaps, and the box thrashed itself to death faster than before. I ripped it back out. Wrong tool. Moving on.</p> The fix that actually worked has three parts, and they're all just cgroups — the Linux kernel's resource-accounting jails — wired up declaratively.</p> One: every Claude session runs in its own systemd scope with a hard memory cap.</strong> The wrapper launches claude</code> inside a transient --user</code> scope with MemoryMax=12G</code>. A single session that ratchets past 12 gigs now hits the wall of its own</em> cgroup and gets OOM-killed inside its own jail</em> — the kernel kills that one process tree and nothing else. The bloated session executes itself. Twelve gigs is still obscene for a chat client, but it's a number that lets ten of them coexist on this box, and a session that blows through it was going to hurt me anyway.</p> Two: all of those scopes live under one parent claude.slice</code> with an aggregate cap</strong> — MemoryMax</code> at 66% of RAM. This is the one that actually addresses my real failure mode, which was never one</em> runaway. It was twenty reasonable-looking sessions summing past 128 GB with no single one looking guilty. Now, all Claudes combined</em> are bounded. When they collectively push the ceiling, a kill fires inside the slice</em> — a Claude dies, never the compositor.</p> Three: I un-inverted the OOM priorities.</strong> This was an embarrassing bug of my own making. I'd biased claude</code> toward being the preferred kill victim... but I'd set the bias too low. It turns out logind session services (wireplumber, emacs, the a11y bus) carry a higher OOM-priority by default than the timid value I'd picked, so the kernel would rather shoot a 30 MB audio daemon than a 7 GB Claude. That's exactly backwards</em>, and it's why the desktop kept dying instead of the thing at fault. Fixed the numbers so a fat Claude is always first against the wall.</p> Belt and suspenders on top: systemd-oomd watching memory-pressure (PSI) on the user slice and killing the worst offender from userspace</em> before the kernel's blunt global killer ever wakes up, plus earlyoom as a dumb backstop.</p> Did it work?#</span></a> </h2> The night I finished it, that idle 21.5 GB session I told you about got killed. And here is the entire point of two weeks of work, in the logs:</p> kernel global-OOM events this boot: 0</span></span> systemd-oomd kill actions this boot: 1 -> claude-1654491.scope</span></span> claude sessions still alive: 8</span></span> box reboots: 0</span></span></code></pre> One</strong> session died. The one holding 21.5 gigabytes for no reason. The other eight kept running. My desktop didn't even flicker. The machine that had hard-rebooted three times in two weeks took the exact same kind of memory pressure and handled it as a routine event — killed one recoverable chat session, which I brought right back with claude --resume</code>, and stayed up. Uptime held. Nothing got kicked to a greeter. I didn't lose a day.</p> That's the difference between "the OS is a suggestion" and "the OS is in charge."</p> The part where I make it a graph#</span></a> </h2> The last thing that bugged me: I only found the 21.5 GB session after</em> it died, in a scope-exit line. I was blind to the ratchet while it was happening. So I wired Claude's cgroups into the Telegraf → InfluxDB → Grafana stack that already runs my fleet — every session's live memory, its high-water mark, its throttling events, sampled every 15 seconds, graphed next to everything else. Plus a little journal-scraper that turns every session teardown into an event so I can see how</em> each one ended — clean exit, self-OOM'd on its own cap, or reaped by oomd — and backfills the peak history from those same scope-exit lines.</p> Now I can watch the ratchet climb in real time. When some future Claude session starts marching toward 12 gigs while sitting at a prompt, I'll see the line go up and to the right, and I'll know exactly which JavaScript heap to be furious at.</p> The lesson#</span></a> </h2> You are not going to fix the memory management of software you didn't write, especially not when it's shipped as a quarter-gigabyte compiled blob by an ecosystem that considers releasing memory back to the OS a quaint premodern concern. That fight is unwinnable and it's not yours.</p> But you don't have to win it. The kernel has had the tools to contain a greedy process since forever — cgroups, memory limits, OOM scoring, PSI. The tragedy is how few people wire them up, because doing it by hand is fiddly and doesn't survive a reboot. That's the whole reason I build my machines the way I do: declaratively, in one repo, where "every claude</code> runs in a 12-gig jail under a slice capped at two-thirds of RAM, and here's the exact OOM priority of every service" is fifteen lines of Nix that are true on every boot forever.</em></p> I can't make the JavaScript behave. But I can build a cage strong enough that it doesn't matter, and I can prove the cage holds, and I can graph the animal throwing itself against the bars.</p> Now if you'll excuse me, I have a chat session at 11 gigs and climbing.</p> Booting Back Out of a Bad Timeshift Snapshot (Pop!_OS + BTRFS) Unknown — Fri, 26 Jun 2026 00:00:00 +0000 I upgraded a Pop!_OS box from 22.04 LTS to 24.04 LTS, hit a snag, and reached for Timeshift to "restore" an older snapshot. It booted — into the old</em> system — and then every subsequent boot dropped me into the systemd emergency shell</strong>. systemd-modules-load.service</code> was failing with "Failed to look up module alias … Function not implemented"</em> (ENOSYS), vfat</code> wouldn't load, and so /boot/efi</code> couldn't mount. One illness, several symptoms.</p> The root cause: Timeshift's BTRFS restore doesn't roll back in place — it swaps subvolumes</em>, so the old snapshot became my live root (@</code>). The booted system's /lib/modules</code> no longer matched the running kernel, which is why module loading died and the FAT EFI partition couldn't be mounted.</p> Below is the full recovery, in fish-shell syntax</strong>, from the emergency shell back to a clean 24.04 boot.</p> Shell note:</strong> the systemd emergency shell is usually bash</code>/sh</code>. To use these exact commands, start fish first with fish</code> (if the binary is present). The only fish-specific bits are variable handling (set NAME value</code> instead of NAME=value</code>, and command substitution with (...)</code> instead of $(...)</code>); everything else is identical to bash.</p> </blockquote> How BTRFS "chooses" what to boot#</span></a> </h2> BTRFS doesn't choose — the bootloader and mount options do:</p> The GRUB kernel line carries root=UUID=<fs-uuid> ... rootflags=subvol=@</code>. That subvol=@</code> selects which subvolume becomes /</code>.</li> With no subvol=</code>, BTRFS falls back to the filesystem's default subvolume</strong> (btrfs subvolume get-default</code>, normally subvolid 5 = the top level).</li> After root mounts, the /etc/fstab</code> inside that subvolume</em> re-asserts subvol=@</code>.</li> </ol> On Ubuntu/Pop the live system is the subvolume named @</code></strong> (home is @home</code>). The fix is to put your good 24.04 system back at @</code> so the existing subvol=@</code> boot entry lands on the right system. Timeshift snapshots are read-only</strong>, so you can't boot one directly — you create a fresh writable</em> @</code> from the snapshot.</p> Step 1 — Identify the BTRFS device#</span></a> </h2> btrfs</span> filesystem show</span></span> lsblk</span> -f</span></span></code></pre> Note the device for your root filesystem (e.g. /dev/nvme0n1p3</code>). If it's LUKS-encrypted, unlock it first:</p> cryptsetup</span> luksOpen /dev/nvme0n1pX cryptroot </span># then use /dev/mapper/cryptroot</span></span></code></pre>Step 2 — Mount the top-level subvolume (subvolid=5)#</span></a> </h2> This exposes every subvolume, not just the currently-booted one.</p> mkdir</span> -p /mnt/top</span></span> mount</span> -o subvolid=5 /dev/nvme0n1p3 /mnt/top</span></span> ls</span> -la /mnt/top</span></span></code></pre>Step 3 — List all subvolumes#</span></a> </h2> btrfs</span> subvolume list /mnt/top</span></span></code></pre> You'll see your live @</code> and @home</code>, plus Timeshift snapshots under timeshift-btrfs/snapshots/<timestamp>/@</code> and .../@home</code>.</p> Step 4 — Pick the snapshot and VERIFY it's the 24.04 system#</span></a> </h2> Don't trust names — check the contents before changing anything. In fish, set the variable with set</code> and use (...)</code> for any command substitution:</p> ls</span> /mnt/top/timeshift-btrfs/snapshots/</span></span> set</span> SNAP /mnt/top/timeshift-btrfs/snapshots/2026-06-19_XX-XX-XX </span># your real timestamp</span></span> </span> grep</span> VERSION= $SNAP/@/etc/os-release </span># expect VERSION="24.04..."</span></span> ls</span> $SNAP/@/lib/modules </span># expect a 24.x kernel directory</span></span></code></pre> Or loop over every candidate subvolume at once (fish for ... end</code>):</p> for</span> s </span>in</span> /mnt/top/@</span></span></span> echo</span> "=== </span>$s</span> ==="</span></span> grep</span> VERSION= $s/etc/os-release</span></span> ls</span> $s/lib/modules</span></span> end</span></span></code></pre> Proceed only once you've confirmed which subvolume is the 24.04 system.</p> Step 5 — Move the broken subvolumes aside (rename, never delete)#</span></a> </h2> cd</span> /mnt/top</span></span> mv</span> @ @broken-2204</span></span> mv</span> @home @home-broken-2204</span></span></code></pre> Keeping them (renamed) makes this whole operation reversible.</p> Step 6 — Create new WRITABLE subvolumes from the snapshot#</span></a> </h2> btrfs</span> subvolume snapshot $SNAP/@ /mnt/top/@</span></span> btrfs</span> subvolume snapshot $SNAP/@home /mnt/top/@home</span></span></code></pre> No -r</code> flag → these are read-write, which is what a root filesystem requires. The original snapshots stay untouched as backups.</p> Note on @home</code>:</strong> only restore it if the snapshot's home is the version you want. If you've created nothing in /home</code> since the breakage, restoring it is fine. If unsure, skip the @home</code> lines and keep your current @home</code> — the system will still boot.</p> </blockquote> Step 7 — Sanity-check the mount options#</span></a> </h2> The new @</code> came from a system that already used subvol=@</code>, so its /etc/fstab</code> and the GRUB entry should already agree. Quick confirm:</p> grep</span> -E</span> 'subvol'</span> /mnt/top/@/etc/fstab</span></span></code></pre>Step 8 — Reboot#</span></a> </h2> cd</span> /</span></span> umount</span> /mnt/top</span></span> reboot</span></span></code></pre> GRUB still points at subvol=@</code>, but @</code> is now your real 24.04 root with a kernel that matches its /lib/modules</code>. Module loading works → vfat</code> loads → /boot/efi</code> mounts → boot completes.</p> Step 9 — Confirm success and clean up#</span></a> </h2> After a clean graphical/login boot:</p> grep</span> VERSION /etc/os-release </span># 24.04</span></span> systemctl</span> --failed</span> # should be empty</span></span> findmnt</span> /boot/efi </span># mounted</span></span> uname</span> -r; </span>ls</span> /lib/modules </span># kernel matches a modules dir</span></span></code></pre> Once you're confident, delete the leftovers:</p> sudo</span> btrfs subvolume delete /@broken-2204</span></span> sudo</span> btrfs subvolume delete /@home-broken-2204</span></span></code></pre> (Mount subvolid=5 again if you need the top-level path to reach them.)</p> Troubleshooting#</span></a> </h2> Drops right back to emergency:</strong> a unit is still failing. Run systemctl --failed</code> and journalctl -b -p err</code>. A failed .mount</code> usually means a stale UUID/PARTUUID in /etc/fstab</code> — compare against blkid</code>. Add nofail</code> to non-critical mounts (like /boot/efi</code>, /recovery</code>) so a missing mount can't abort the whole boot.</li> "Function not implemented" on modprobe:</strong> confirms a kernel/modules mismatch — i.e. you're still on the wrong subvolume. Re-check Step 4.</li> GRUB itself is broken after the swap:</strong> boot a live USB, mount the top-level, mount --bind</code> /dev /proc /sys, chroot into @</code>, then update-grub && grub-install</code>.</li> Never run btrfs check --repair</code> first</strong> — use btrfs check --readonly</code> to diagnose; --repair</code> can worsen damage.</li> </ul> Recovery complete — you're back on 24.04.</em></p> Two Family Trees: systemd Targets and Slices Unknown — Thu, 25 Jun 2026 00:00:00 +0000 I went looking for a way to group</em> a handful of my own systemd units — my sync timers, a mail fetcher, a couple of personal daemons — so I could see them and operate on them as a set instead of squinting at the full systemctl</code> output. Two answers kept coming up: targets</strong> and slices</strong>. They both "group units." They both show up in the same listings. And for an embarrassing minute I couldn't have told you the difference between them.</p> So I went and read the history. It turns out the reason they feel like overlapping tools is an illusion of proximity: they live next to each other in systemctl</code> today, but they were born in different decades, at different companies, to solve completely unrelated problems.</em> Targets are the descendants of init runlevels</strong>. Slices are the descendants of control groups</strong>. They only look like siblings because systemd dresses every unit type in the same uniform.</p> Here's the one sentence to hold onto before we dig in:</p> A target is a node in a dependency graph. A slice is a node in a cgroup tree.</strong></p> </blockquote> One answers "what should be running, and in what order?"</em> The other answers "whose resources are these, and what are the limits?"</em> Those are orthogonal questions, and once you see that, the whole model snaps into focus.</p> Let me trace each family tree.</p> Family tree #1: runlevels → targets#</span></a> </h2> The 1980s: a runlevel is a state</em>#</span></a> </h3> The story starts with SysV init</strong>, the init system that came out of AT&T's UNIX System III in the early 1980s and carried into System V (hence "SysV"). Its central concept was the runlevel</strong>: a named configuration of the machine — a state</em> you transition the system into</em>, in which only a selected group of processes is allowed to run. The mapping of "what runs at which level" lived in /etc/inittab</code>, and init's entire job was to drag the machine into the requested state and keep the right processes alive.</p> There were eight of them, 0–6 plus S</code>, and the ones at the edges were universal:</p> 0 halt / poweroff</span></span> 1 single-user (maintenance)</span></span> S single-user (≈ 1)</span></span> 6 reboot</span></span></code></pre> The interesting part is the middle, 2–5</strong>, because that's where the distros quietly disagreed — a fact that bites people to this day:</p> Red Hat / Fedora:</strong> runlevel 3</strong> was full multi-user with networking but a text</em> console; runlevel 5</strong> added the graphical X11 login. The famous "boot to a GUI vs boot to a console" toggle was literally 3</code> vs 5</code>.</li> Debian / Ubuntu:</strong> runlevels 2 through 5 were all identical</strong> — full multi-user mode, no text-vs-graphical distinction at all.</li> </ul> The mental model that matters: a runlevel is a state machine position</strong>. You are at</em> a runlevel; you switch to</em> another. It's about the global condition of the box, and which bundle of services that condition implies.</p> The init wars (2005–2010)#</span></a> </h3> SysV init had an Achilles' heel: it ran service scripts sequentially</strong>, in a numbered order, each one blocking the next. On a machine with dozens of services, boot was a slow conga line. Two projects attacked this before systemd existed:</p> launchd</strong> (Apple, 2005, shipped in Mac OS X 10.4 "Tiger"). One daemon to replace init, cron, inetd, and friends — and crucially, it started services on demand via socket activation</strong> rather than eagerly in a fixed order. Apple relicensed it under Apache 2.0 in 2006 to encourage adoption elsewhere.</li> Upstart</strong> (Canonical, 2006, by Scott James Remnant, first in Ubuntu 6.10). An event-based</strong> init: services reacted to emitted events ("a disk appeared", "a filesystem mounted") instead of marching through runlevel scripts.</li> </ul> Both were saying the same thing in different dialects: the rigid, sequential runlevel model is the wrong abstraction.</em></p> April 30, 2010: "Rethinking PID 1"#</span></a> </h3> Then Lennart Poettering (at Red Hat), with design work from Kay Sievers (then at Novell), published an essay titled Rethinking PID 1</strong></a> and announced systemd. The pitch: aggressive parallelization</strong> of boot, socket activation</strong> pushed to its limit (create all the listening sockets first, hand them to services, let everything start at once without hand-written ordering), D-Bus activation</strong>, cgroups to track processes</strong> (more on that in the second family tree), and — the relevant bit here — targets to replace runlevels.</strong></p> A target</strong> is systemd's answer to "what is a runlevel, really?" Stripped of the state-machine framing, a runlevel was just a name for a set of services that should be up together.</em> So systemd made that explicit. From the systemd.target(5)</code> man page:</p> Target units are used to group units</strong> and to set synchronization points</strong> for ordering dependencies with other unit files. […] Target units do not offer any additional functionality on top of the generic functionality provided by units. They merely group units. […] Target units provide a more flexible replacement for SysV runlevels in the classic SysV init system.</p> </blockquote> Read that twice, because it's the whole thing: a target has no processes of its own</strong>, does nothing on its own</strong>, and exists purely as a named node in the dependency graph</em> that other units can point at. The old runlevels are still there as compatibility aliases — and the mapping is hard-coded right in systemd's build files (units/meson.build</code>):</p> runlevel0.target → poweroff.target</span></span> runlevel1.target → rescue.target</span></span> runlevel2/3/4.target → multi-user.target</span></span> runlevel5.target → graphical.target</span></span> runlevel6.target → reboot.target</span></span></code></pre> systemd's own FAQ doesn't mince words: "The concept of runlevels is obsolete. A set of target units are exposed that carry similar semantics, e.g. runlevel 5 -> graphical.target."</em></p> Targets also do the job runlevels never could: they're synchronization points</strong> in a real dependency graph. Boot is a chain of them — sysinit.target</code> → basic.target</code> → multi-user.target</code> → graphical.target</code> — and units hook themselves in with WantedBy=</code>, Wants=</code>, Requires=</code>, After=</code>, Before=</code>. That last point matters for later: a single unit can be WantedBy</code> many targets at once.</strong> Membership in the dependency graph is many-to-many.</p> You can see the graph directly:</p> systemctl</span> list-units --type=target</span> # what's active</span></span> systemctl</span> list-dependencies graphical.target</span> # the tree under a target</span></span></code></pre> So: runlevels → targets.</strong> The lineage is init, boot, state, ordering.</em> Nothing about resources.</p> Family tree #2: cgroups → slices#</span></a> </h2> Now rewind to a completely different room.</p> 2006–2008: Google needs to box in processes#</span></a> </h3> Around 2006, engineers at Google — Paul Menage</strong> and Rohit Seth</strong> — were wrestling with enormous shared Linux clusters and built a kernel feature they called "process containers."</strong> The goal had nothing to do with booting: it was about hierarchically grouping processes</strong> so you could account for, limit</strong>, prioritize, and isolate</strong> their resource usage — CPU, memory, block I/O, and so on, each handled by a per-resource "controller."</p> To avoid collision with the loaded word "container," it was renamed control groups</strong> — cgroups</strong> — in late 2007, and merged into the Linux kernel in 2.6.24 (January 2008).</strong></p> This is a kernel resource-management</em> primitive. It knows nothing about services, runlevels, or boot order. It just lets you draw a box around a set of processes and say "this box gets at most 2 GB of RAM and 10% of one CPU."</p> Why systemd cared: processes that won't stay caught#</span></a> </h3> Here's the bridge between the two family trees. Traditional Unix daemons double-fork</strong> to "daemonize" — detach from their parent and re-parent to PID</p> That's deliberate, and it's a disaster for a service manager: once a daemon double-forks, the thing that launched it has lost track of it</strong>. PID-based supervision is racy and unreliable — PIDs get recycled, children scatter and re-parent to init, and you can never be sure you've found (or killed) all</em> of a service's processes.</li> </ol> cgroups solved this cleanly, and it's exactly why "Rethinking PID 1" leaned on them. Put every service in its own cgroup</strong>, and the kernel maintains a reliable, race-free record of precisely which processes belong to that service</strong>, no matter how many times they fork. Poettering put it bluntly:</p> Unless it is privileged and has access to the cgroup file system it cannot escape its group.</p> </blockquote> That gives you accurate accounting and</em> the ability to reliably kill an entire service — every last worker — by acting on the group.</p> July 3, 2013 (v205): systemd grows slices and scopes#</span></a> </h3> For systemd's first few years it used</em> cgroups internally for this tracking, but there was a problem: lots of programs (systemd, logind, libvirt, your shell) were all writing to the cgroup tree at once, stepping on each other. So in version 205</strong> (released 2013-07-03), systemd introduced two new unit types and a new rule. From the NEWS file:</p> Two new unit types have been introduced: Scope units</strong> are very similar to service units, however, are created out of pre-existing processes […] By using scope units it is possible for system services and applications to group their own child processes […] Slice units</strong> may be used to partition system resources in an hierarchical fashion and then assign other units to them. By default there are now three slices: system.slice</code> (for all system services), user.slice</code> (for all user sessions), machine.slice</code> (for VMs and containers).</p> </blockquote> And the reason they appeared at all:</p> Slices and scopes have been introduced primarily in context of the work to move cgroup handling to a single-writer scheme</strong>, where only PID 1 creates/removes/manages cgroups.</p> </blockquote> That "single-writer" rule — only PID 1 touches the cgroup tree</strong> — is why modern systemd is the sole owner of cgroups on a Linux box. Everything else asks systemd.</p> So what is</em> a slice? From systemd.slice(5)</code>:</p> A slice unit is a concept for hierarchically managing resources of a group of processes</strong>. This management is performed by creating a node in the Linux Control Group (cgroup) tree</strong>. Units that manage processes (primarily scope and service units) may be assigned to a specific slice. For each slice, certain resource limits may be set that apply to all processes of all units contained in that slice.</p> </blockquote> A slice is</strong> a cgroup directory. Its name</em> encodes its path in the tree: -.slice</code> is the root, and foo-bar.slice</code> sits inside foo.slice</code> inside -.slice</code>. The default layout every systemd box boots with:</p> -.slice the root</span></span> ├── system.slice all system services</span></span> ├── user.slice all user sessions (each user gets user-UID.slice)</span></span> └── machine.slice VMs and containers (via systemd-machined)</span></span></code></pre> The third unit type from v205, the scope</strong>, is the odd cousin: like a service, but for processes systemd didn't</em> fork itself — login sessions, VMs, anything externally launched. Scopes and services are the leaves</strong> that hold actual processes; slices are the branches</strong> that hold them.</p> And that leaf/branch split isn't a metaphor — it's a kernel rule. systemd's CONTROL_GROUP_INTERFACE.md</code> spells it out:</p> Service, scope and slice units directly map to objects in the cgroup tree</strong>. […] Slices do not contain processes themselves</strong>, but the services and scopes contained in them do.</p> </blockquote> Under cgroup v2's "no processes in inner nodes" rule, an inner node can't hold processes directly — so slices (inner nodes) never contain processes, only their service/scope children (leaf nodes) do. Crucially, a unit is placed in exactly one</strong> slice: the Slice=</code> directive is a single value, defaulting to system.slice</code>. Membership in the cgroup tree is one-to-one</strong>.</p> The cgroup v2 detour#</span></a> </h3> One more thread, because it explains why old guides feel inconsistent. The original cgroups (v1) gave each controller its own independent hierarchy</em> — the cpu tree, the memory tree, the io tree were separate and could disagree about which group a process was in. It was a mess that couldn't be fixed without breaking the interface. Tejun Heo</strong> led the redesign — cgroup v2</strong>, the "unified hierarchy," where every controller shares one tree and a process belongs to exactly one cgroup. It landed as stable in Linux 4.5 (March 2016).</strong></p> systemd's migration tracks that maturing:</p> v233 (2017) hybrid v1+v2 default, with knobs to opt into unified</span></span> v243 (2019) build-time default flips to "unified" (pure cgroup v2)</span></span> v258 (2025) cgroup v1 support removed entirely — v2 always mounted</span></span></code></pre> So if you're on a current system, your slices are nodes in a single unified cgroup v2 tree. The "exactly one slice" cleanliness is partly because</em> of Tejun's redesign.</p> So: cgroups → slices.</strong> The lineage is resources, accounting, limits, containment.</em> Nothing about boot order.</p> The nuance: why they aren't siblings#</span></a> </h2> Lay the two family trees side by side and the "overlap" evaporates:</p> </th> Target</strong></th> Slice</strong></th></tr></thead> Ancestor</td> SysV init runlevels (early 1980s)</td> Linux cgroups (Google, 2006 → kernel 2.6.24, 2008)</td></tr> Arrived in systemd</td> day one (2010)</td> v205 (2013)</td></tr> What it is</em></td> a node in the dependency graph</strong></td> a node in the cgroup tree</strong></td></tr> Question it answers</td> what should run, and in what order?</em></td> whose resources, and what limits?</em></td></tr> Has processes?</td> no — pure synchronization point</td> not directly; its leaf children (services/scopes) do</td></tr> Kernel object?</td> no, it's bookkeeping</td> yes</strong>, a real cgroup directory</td></tr> A unit relates to it via</td> WantedBy=</code>, Wants=</code>, Requires=</code>, After=</code></td> Slice=</code></td></tr> Cardinality</td> a unit can be in many</strong> targets</td> a unit is in exactly one</strong> slice</td></tr> The knobs it carries</td> none (it does nothing itself)</td> CPUWeight=</code>, MemoryMax=</code>, IOWeight=</code>, TasksMax=</code>…</td></tr> </tbody></table> The deep point is orthogonality.</strong> Every service is simultaneously at two coordinates:</p> Somewhere in the dependency graph</strong> — pulled in by some targets, ordered before/after other units. This is the time / causality</em> axis. Targets live here.</li> Somewhere in the cgroup tree</strong> — inside exactly one slice, subject to its resource limits. This is the resource / containment</em> axis. Slices live here.</li> </ol> These axes don't compete; they're perpendicular. Asking "should I use a target or a slice?" is a bit like asking "should I describe this meeting by its time or its room?" Usually the honest answer is both, for different reasons.</em></p> So which one do I actually want?#</span></a> </h2> Back to my original itch — "let me see and operate on my own set of units as a group." Now I can answer it properly, because it depends on which axis</em> I care about.</p> If I want to bring them up/down together, order them, or depend on them as a set</strong> → that's the dependency-graph axis → a target.</strong> It's the runlevel-flavored answer: "here is a named bundle; activate it."</p> # ~/.config/systemd/user/personal.target</span></span> [</span>Unit</span>]</span></span> Description</span>=</span>My personal services</span></span></code></pre># in each of my units:</span></span> [</span>Install</span>]</span></span> WantedBy</span>=</span>personal.target</span></span></code></pre>systemctl</span> --user list-dependencies personal.target</span> # see the whole set</span></span> systemctl</span> --user start personal.target</span> # bring the set up</span></span></code></pre> If I want to cap or account their combined CPU/memory, or view them all as one resource group</strong> → that's the cgroup-tree axis → a slice.</strong> It's the cgroup-flavored answer, and status</code> on a slice gives you a live tree of everything inside it plus its resource usage:</p> # ~/.config/systemd/user/personal.slice</span></span> [</span>Slice</span>]</span></span> MemoryMax</span>=</span>2G</span></span> CPUWeight</span>=</span>50</span></span></code></pre># in each of my units:</span></span> [</span>Service</span>]</span></span> Slice</span>=</span>personal.slice</span></span></code></pre>systemctl</span> --user status personal.slice</span> # the cgroup tree + live usage</span></span> systemd-cgls</span> --user</span> # the whole user cgroup tree</span></span></code></pre> And the punchline that took me the whole history to appreciate: you can use both, and it isn't redundant.</strong> A personal.target</code> that pulls the group up and</em> a personal.slice</code> that boxes its resources are describing the same services along two independent axes. The target says when</em> they run; the slice says how much</em> they get.</p> (If all I really want is to list</em> mine and never operate on them as a unit, the cheapest trick of all is a shared name prefix — systemctl --user list-units 'personal-'</code>, since systemctl</code> accepts globs. No new unit types required. But the moment I want grouped behavior</em> — ordering or limits — I reach for the matching family tree.)</p> The takeaway#</span></a> </h2> systemd looks like one monolithic thing, but a lot of it is really two old Unix ideas wearing the same systemctl</code> uniform</strong>: the init state machine that goes back to AT&T System V, and the kernel resource controller that came out of Google's clusters. Targets are the grandchild of the runlevel. Slices are the grandchild of the cgroup. They share a command-line front-end and absolutely nothing else in their DNA.</p> Once you can glance at a unit type and ask "which family tree is this from — the dependency graph, or the cgroup tree?"</em>, the rest of systemd stops feeling arbitrary. It's just two lineages, finally living under one roof.</p> Sources & further reading#</span></a> </h3> Lennart Poettering, Rethinking PID 1</strong></a> (2010-04-30) — the founding essay.</li> systemd.target(5)</code>, systemd.slice(5)</code>, systemd.scope(5)</code>, systemd.special(7)</code> — the canonical definitions quoted above.</li> systemd NEWS</code> — v205 (slices/scopes + single-writer), v243 (unified default), v258 (v1 removed).</li> Control Group v2</a> — the kernel's own cgroup v2 documentation.</li> LWN, 14 years of systemd</a> — a retrospective.</li> </ul> Getting SASSy: Stealing Login Patterns from Codeberg and Sourcehut Unknown — Wed, 03 Jun 2026 00:00:00 +0000 I'm building a SaaS. I want it to be as FOSS, Linux-native, and self-hosted as I can make it — which means at some point I have to stop and solve the least glamorous, most important problem in the whole thing: letting users log in.</strong></p> Authentication is one of those areas where the wrong instinct is to invent something. The right instinct is to go read how people who already solved it — in the open, in production, for years — chose to do it. I picked two: Codeberg</strong>, which runs Forgejo</a> (a fork of Gitea), and Sourcehut</a></strong> (sr.ht</code>). Both are fully open source. Both are self-hostable. Both have been running real accounts for real people for a long time. And because they're open, I can read exactly how they do it instead of guessing.</p> I came away with two questions. This post is built around them.</p> "How do Codeberg and Sourcehut handle this problem?"#</span></a> </h2> The most useful thing I learned up front is that these two sit at opposite ends of one spectrum</strong>, and which one you copy depends entirely on whether you ever plan to split your app into multiple services.</p> Sourcehut treats identity as a service.</strong> meta.sr.ht</code> is a standalone account / billing / security hub. Every other service — git.sr.ht</code>, builds.sr.ht</code>, lists.sr.ht</code> — is an OAuth2 client of meta</strong>. Auth is a network boundary you cross.</p> Forgejo treats identity as a module.</strong> One Go process owns login, sessions, tokens, and 2FA directly, but it delegates credential checking through a common auth-source interface</strong> (local password, LDAP, OIDC, PAM…). Auth is a plugin point inside one binary.</p> If you're one app today — like I am — Forgejo's model is the closer fit. But the discipline</em> worth stealing is Sourcehut's: treat identity as the source of truth, behind an API, so you can extract it into its own service later without a rewrite.</p> Sourcehut, in detail#</span></a> </h3> meta.sr.ht-api</code> (Go, GraphQL) owns all auth state. The Python web frontend holds no credentials</em> — it just issues GraphQL queries. The UI is a client of the auth API, not a co-owner of the auth database. That separation is the whole point.</p> Login:</strong> password + TOTP</strong> 2FA. No WebAuthn. Public keys (SSH, PGP) are stored centrally in meta and consumed by the other services.</li> OAuth2</strong> is RFC 6749-conformant and deliberately hardened: Confidential clients only</strong> — public clients are prohibited</em>. A native or CLI app must run a server component, or use a personal access token.</li> redirect_uri</code> is not accepted at runtime</strong> — only the pre-registered URI is used. That kills a whole class of redirect attacks.</li> scope</code> is mandatory</strong> (stricter than the spec). Scopes look like meta.sr.ht/PROFILE:RO meta.sr.ht/SSH_KEYS:RW</code> — service/SCOPE:access</code>, where access is RO</code> or RW</code>, enforced per GraphQL resolver via an @access</code> directive.</li> Authorization codes are 16 hex chars, 5-minute TTL. Client secret is 64-byte base64.</li> </ul> </li> The token format is the clever bit.</strong> An access / personal token is a BARE</a>-encoded payload — username, client ID, authorized scopes, expiry — that is HMAC-signed</strong> and base64'd. So the token is stateless</em>: the server validates the signature without a database hit. The grant rides inside the token. Revocation is layered on top — a SHA hash of the token in Redis</strong> for instant lookup, plus a DB expiry timestamp as the durable fallback if the cache is lost.</li> Service-to-service</strong> uses an @internal</code> auth type — single-use tokens encrypted with Sourcehut's internal private keys — which unlocks resolvers not exposed to the public. That's how git.sr.ht</code> trusts a request that claims to come from meta</code>.</li> </ul> Forgejo / Codeberg, in detail#</span></a> </h3> Password hashing:</strong> PASSWORD_HASH_ALGO</code>, default PBKDF2-HMAC-SHA256 at ~310k–320k iterations</strong>; also argon2id, scrypt, bcrypt. Configurable length and complexity, plus an optional Have-I-Been-Pwned check</strong> (PASSWORD_CHECK_PWN</code>) that rejects known-breached passwords at signup. Cheap, high-signal win.</li> Sessions:</strong> cookie + middleware. A webAuth()</code> → AuthShared()</code> flow walks an authentication priority chain</strong> (try each method in order), then stashes the user in request context, gated on email confirmation / login-prohibition / forced-password-change. Session store is pluggable: memory / db / Redis</strong> / memcached.</li> API tokens</strong> are hashed in the DB with the same algorithm as passwords</strong> — never stored in plaintext — and scoped</strong> by permission, with an LRU cache (default 20) of recently-validated hashes so it doesn't re-run PBKDF2 on every request. A nice perf/security balance.</li> 2FA:</strong> TOTP and</em> WebAuthn/FIDO2 passkeys, including passwordless login. For basic-auth API calls with 2FA on, the code goes in an X-Gitea-OTP:</code> header.</li> The auth-source abstraction</strong> is the real lesson: one interface, many backends — local password, OAuth2/OIDC client, LDAP (service-account search or</em> simple DN-template bind), PAM, SMTP, SPNEGO/Kerberos, reverse-proxy header trust. Forgejo can also be</em> an OAuth2/OIDC provider for other apps.</li> </ul> "Where is it all stored? A SQL table? Unix?"#</span></a> </h2> This was my second question, and it's the one that actually changed how I'm going to build. Because the naïve mental model — "a Linux server, users, surely there are Unix accounts somewhere" — is wrong</strong>, and deliberately so.</p> The trick is to separate two things people conflate:</p> Where the identity record</em> lives</strong> → always a SQL row.</strong> Never /etc/passwd</code>.</li> What verifies the secret</em></strong> → usually a hash in that same SQL row, but optionally</em> delegated to Unix (/etc/shadow</code>) or an external directory (LDAP/OIDC).</li> </ol> That second point is the entire reason the auth-source abstraction exists.</p> Sourcehut: everything in PostgreSQL#</span></a> </h3> Owned by meta.sr.ht</code>:</p> Concern</th> Where it lives</th></tr></thead> Identification</strong> (who you are)</td> Postgres user</code> table: id, created, updated, username, email, user_type, url, location, bio</code>. Single source of truth; other services have their own DBs but defer to meta.</td></tr> Authentication</strong> (proving it)</td> Password hash + TOTP secret in meta's Postgres. SSH public keys in Postgres</strong>, cached in Redis.</td></tr> Authorization</strong> (what you can do)</td> A stateless HMAC-signed token</strong> carrying username/client/scopes/expiry. Revocation via Redis (SHA of token) + Postgres expiry.</td></tr> Unix accounts</strong></td> None per user.</strong> One shared system user per service</em> — you SSH as git@</code>, hg@</code>, builds@</code>.</td></tr> </tbody></table> Forgejo: everything in a configurable SQL DB#</span></a> </h3> SQLite / MySQL / PostgreSQL, your choice:</p> Concern</th> Where it lives</th></tr></thead> Identification</strong></td> user</code> table.</td></tr> Authentication</strong></td> Hashed password in the user</code> row; 2FA secrets + WebAuthn credentials in their own tables; SSH keys in public_key</code>; API tokens hashed</strong> in access_token</code>.</td></tr> Authorization</strong></td> Token scopes stored with the token; repo/org/team permissions in SQL. Sessions in memory / db / Redis.</td></tr> Unix accounts</strong></td> None per user.</strong> A single git</code> RUN_USER</code>.</td></tr> </tbody></table> So where does Unix actually come in?#</span></a> </h3> Two places, both optional, both Forgejo:</p> PAM</strong> verifies the password against /etc/shadow</code> (real Unix accounts) — but the identity row still lives in SQL.</strong> PAM only checks the secret. (It also has a footgun: an uploaded SSH key can bypass the PAM check entirely.)</li> LDAP / OIDC</strong> verify against an external directory — but Forgejo still mirrors a user</code> row into SQL</strong> so it has something to hang repos and permissions off of.</li> </ol> Even when the verifier</em> is Unix or LDAP, the system of record</em> is SQL. That's the rule.</p> The SSH trick worth stealing#</span></a> </h3> Neither forge runs useradd</code> per customer. So how does git push</code> over SSH know who you are with one shared system user?</p> sshd</code>'s AuthorizedKeysCommand</code>. Instead of reading ~/.ssh/authorized_keys</code>, OpenSSH runs a program, hands it the public key being offered, and that program looks the key up in the database</strong> and resolves it to a user.</p> Sourcehut</strong> runs gitsrht-dispatch</code>: it checks Redis, then Postgres, then meta</code>, caching the result back down the chain for the next push.</li> Forgejo</strong> either generates a managed authorized_keys</code> with a forced-command per key, or uses AuthorizedKeysCommand</code> (gitea keys …</code>) to do the same lookup.</li> </ul> So OpenSSH's key-auth mechanism gets reused</em>, but the "account" a key resolves to is a database row, not a Unix user</strong>. One system user, a key→user lookup, and you never touch /etc/passwd</code>.</p> What I'm taking into my own build#</span></a> </h2> Both forges converged on the same shape, so that's the shape I'm copying:</p> One SQL users</code> table is the source of truth</strong> for identity. Unix accounts don't enter into it.</li> Make the credential verifier pluggable</strong> behind that table: local hash now; LDAP/OIDC later when an enterprise customer asks for SSO — without touching the identity model.</li> Authorization travels separately</strong> — either DB-stored scoped tokens (Forgejo: simpler) or stateless signed tokens with a revocation list (Sourcehut: fewer DB hits). I'll probably start with the former.</li> Ship TOTP early, WebAuthn/passkeys later.</strong> That's the order both landed on.</li> Add the HIBP breached-password check.</strong> Free, and it stops the dumbest account takeovers.</li> If I ever do SSH/key access,</strong> copy the AuthorizedKeysCommand</code> dispatch trick. One shared system user, key→DB-user lookup. Never useradd</code> per customer.</li> </ul> The headline, if you skipped to the bottom: it's SQL all the way down for identity and the canonical credentials, and the avoidance of per-user Unix accounts is a feature, not an oversight.</strong> The Linux box hosts the service; it does not host the users.</p> Sources I read while writing this: the Sourcehut API 2.0 dev log</a>, OAuth 2.0 via meta.sr.ht</a>, What happens when you push to git.sr.ht</a>, Drew DeVault's Building interactive SSH applications</a>, the Forgejo authentication docs</a>, the Gitea config cheat sheet</a>, and the Gitea SSH & key management writeup</a>.</em></p> Mouse settings explained: DPI, report rate, and receiver modes Unknown — Sun, 17 May 2026 00:00:00 +0000 Modern gaming mice expose a few settings that are easy to misunderstand because the labels are short and the marketing around them is often imprecise. This post uses ZOWIE's current -DW wireless mice as the example: EC-DW, FK2-DW, ZA13-DW, S2-DW, and U2-DW. The exact shape differs by model, but the relevant controls and settings are the same.</p> The important distinction is simple: the bottom slider selects the active connection mode. It is not a recommendation for when to play, when to stop, or where to put the mouse after a session.</p> The receiver mode slider#</span></a> </h2> </p> The three-position slider on the bottom is a power and receiver selector:</p> • Enhanced wireless receiver mode — use this when the enhanced receiver is the device plugged into the computer. This is the larger directional receiver that also works as a pogo-pin charging dock. • Off — use this when you want the mouse powered down, such as for travel or storage. • USB wireless receiver mode — use this when the small USB receiver/dongle is the device plugged into the computer.</p> Set the slider to match the receiver currently connected to the computer. If the enhanced receiver is connected, use enhanced receiver mode. If the USB dongle is connected, use USB receiver mode. If neither receiver is connected and you do not want the mouse awake, turn it off.</p> The enhanced receiver can charge the mouse through the pogo pins, but that does not make the slider a "dock mode" in the everyday sense. Charging is a feature of the enhanced receiver; the slider's job is to choose which wireless receiver the mouse should talk to.</p> The two bottom buttons#</span></a> </h2> The two other controls are for DPI and report rate:</p> • DPI button — cycles through 400 , 800 , 1000 , 1200 , 1600 , and 3200 DPI. • Report-rate button — cycles through 125 , 500 , 1000 , 2000 , and 4000 Hz, depending on which receiver is in use.</p> ZOWIE's documentation notes that available report rates vary by receiver. In practical terms, the enhanced receiver is the one intended for the high report-rate modes. The small USB receiver is the simpler portable option and may not expose the same top-end settings.</p> What DPI actually changes#</span></a> </h2> DPI means "dots per inch," although "counts per inch" is the more accurate technical description for a mouse sensor. It is the number of movement counts the mouse reports for each inch of physical movement.</p> At 400 DPI, moving the mouse one inch reports about 400 counts. At 1600 DPI, the same movement reports about 1600 counts. If every other sensitivity setting stays the same, the cursor or crosshair moves four times as far.</p> DPI is not a quality rating. A higher number does not automatically mean better tracking, better aim, or a more accurate sensor. It is mostly a sensitivity input. The setting only makes sense together with the rest of the chain: operating-system pointer speed, in-game sensitivity, raw-input behavior, screen resolution, and mousepad space.</p> For games, the useful concept is effective sensitivity. Players often call this eDPI : mouse DPI multiplied by in-game sensitivity. This is useful when comparing settings inside the same game, but it does not transfer perfectly across different games because their sensitivity scales are different.</p> A sensible starting point is 800 or 1600 DPI, then adjust in-game sensitivity until large movements and small corrections both feel controlled. For desktop use on a high-resolution display, a higher DPI can be convenient. For aim-heavy games, an extremely high total sensitivity usually makes small corrections harder, even if the sensor itself is working correctly.</p> What report rate changes#</span></a> </h2> Report rate, often called polling rate, is how often the mouse sends position updates to the computer.</p> • 125 Hz = one report every 8 ms • 500 Hz = one report every 2 ms • 1000 Hz = one report every 1 ms • 2000 Hz = one report every 0.5 ms • 4000 Hz = one report every 0.25 ms</p> A higher report rate reduces the maximum time between a physical movement and the next mouse report. That can improve motion smoothness and reduce input latency, especially on high-refresh-rate displays where the system can present those updates more frequently.</p> The improvement is not linear in practice. Moving from 125 Hz to 1000 Hz is obvious. Moving from 1000 Hz to 4000 Hz is much smaller and depends on the rest of the system: monitor refresh rate, game engine, CPU load, USB behavior, and whether the game is already frame-time limited.</p> ZOWIE lists the S2-DW battery life as 80 hours at 1000 Hz, 30 hours at 2000 Hz, and 24 hours at 4000 Hz. Higher report rates also create more USB/input work for the system. On a fast machine this is usually fine; on a CPU-limited system or a poorly behaving game, dropping back to 1000 Hz can make sense.</p> Practical settings#</span></a> </h2> For most people, this is the clean baseline:</p> • Use enhanced wireless receiver mode when the enhanced receiver is plugged into the computer. • Use USB wireless receiver mode when the small USB dongle is plugged into the computer. • Use Off when storing or transporting the mouse. • Start at 800 or 1600 DPI, then tune sensitivity in the operating system or game. • Start at 1000 Hz report rate; try 2000 or 4000 Hz if you are on a high- refresh display and the system remains stable.</p> • Dock the mouse to charge it as frequently as needed per the report rate settings</p> References#</span></a> </h2> • ZOWIE S2-DW specifications: https://zowie.benq.com/en-us/mouse/s2-dw.html • ZOWIE DPI and polling-rate FAQ: https://zowie.benq.com/en- us/support/faq/product/application/zowiegear-faq-kn-00025.html • ZOWIE S2-DW quick start guide: https://zowie.benq.com/en-us/qsg/s2- dw/guide.html</p> Why Your OnePlus Says VOOC When It Isn't: USB-PD, PPS, and the SuperVOOC Architecture Unknown — Thu, 30 Apr 2026 00:00:00 +0000 Why Your OnePlus Reports VOOC When It Isn't#</span></a> </h1> A OnePlus phone connected to a 100W third-party GaN charger displays a "VOOC" fast-charge indicator in OxygenOS. The indicator is misleading. The phone is negotiating USB Power Delivery with PPS and drawing approximately a quarter of what its battery hardware will accept. OxygenOS labels any negotiated rate above an internal threshold with the family name, regardless of the protocol actually in use.</p> This post covers what is on the CC pin during that negotiation, why third-party chargers cap where they do, and what would have to change for the same phone to charge at its rated peak. The relevant pieces are dual-cell battery topology, charge pumps versus buck converters, and a cable-side ID resistor that gates OPPO's high-current path.</p> USB Power Delivery#</span></a> </h2> USB-PD is the open standard maintained by the USB-IF for negotiating power over USB-C. The model is straightforward: the source advertises a list of Power Data Objects</strong> (PDOs) — fixed voltage/current pairs such as 5V/3A</code>, 9V/3A</code>, 15V/3A</code>, 20V/5A</code> — and the sink selects one. Voltage steps are coarse and discrete. Conversion happens device-side: the phone receives 9V at the port and its internal buck IC steps that down to ~4.4V for the cell. Conversion losses dissipate as heat inside the phone.</p> The spec has gone through several revisions. Each adds capability without removing the older fixed-PDO model.</p> Spec</th> Year</th> Max power</th> Voltages</th> Key addition</th></tr></thead> USB-PD 1.0</strong></td> 2012</td> 100W</td> 5/12/20V profiles</td> First spec; ran over USB-A/B VBUS. Largely skipped.</td></tr> USB-PD 2.0</strong></td> 2014</td> 100W</td> 5V, 9V, 15V, 20V (fixed PDOs)</td> Rebuilt around USB-C. The "classic" PD everyone knows.</td></tr> USB-PD 3.0</strong></td> 2015–2017</td> 100W</td> Same fixed PDOs</td> Fast Role Swap, extended messages, battery status, firmware update over PD.</td></tr> USB-PD 3.0 + PPS</strong></td> 2017</td> 100W</td> 3.3–21V in 20mV</strong> steps, 50mA</strong> current steps</td> Programmable Power Supply — chargers track battery voltage.</td></tr> USB-PD 3.1 SPR</strong></td> 2021</td> 100W</td> Same as 3.0 (≤20V)</td> "Standard Power Range." Cleanups, better extended messages.</td></tr> USB-PD 3.1 EPR</strong></td> 2021</td> 240W</strong></td> Adds 28V, 36V, 48V fixed PDOs</td> "Extended Power Range." Requires EPR-rated e-marked cable (5A @ 50V).</td></tr> USB-PD 3.1 EPR + AVS</strong></td> 2021</td> 240W</td> 15–48V adjustable</strong> (100mV steps)</td> Adjustable Voltage Supply — PPS's bigger sibling for the EPR range.</td></tr> USB-PD 3.2</strong></td> 2023</td> 240W</td> Same envelope as 3.1</td> Tightens AVS, finer current granularity, multi-port budget signaling.</td></tr> </tbody></table> Reading a charger's spec label:</p> "PD 65W"</strong> with no PPS: fixed 20V/3.25A. Adequate for laptops, suboptimal for modern phones because it forces a high-ratio buck.</li> "PD 65W PPS"</strong>: at least one PPS PDO, typically 3.3–11V @ 5A</code> or 3.3–21V @ 3A</code>. This is what a 2024+ phone targets.</li> "PD 3.1 / EPR 140W+"</strong>: includes 28V or 36V fixed PDOs. Apple's 140W MacBook Pro charger and the higher-tier Anker / UGREEN units fall here. Delivery above 100W requires an EPR-rated cable.</li> "AVS"</strong> on the label: PD 3.1 EPR with adjustable voltage. Currently limited to enthusiast GaN units.</li> </ul> PPS specifically#</span></a> </h2> PPS is the section of PD 3.0 most relevant to phones. The sink requests voltage in 20mV</strong> increments and current in 50mA</strong> increments and adjusts continuously. Rather than "9V/3A, device copes," the phone requests "8.74V at 4.20A," then a few hundred ms later "8.78V at 4.18A," and the source tracks.</p> This matters for thermal behavior. The phone's charge IC has the least conversion work when the input voltage is close to twice the cell voltage. A buck or charge pump operates most efficiently at small, predictable conversion ratios. With fixed PD the input is always 9V or 15V regardless of state-of-charge; with PPS the input slides as the cell fills, holding the conversion stage near peak efficiency throughout the curve.</p> PPS adoption on Android is broad: Samsung Super Fast Charging 2.0 (45W), Pixel fast charging, and every modern OnePlus when paired with a non-OPPO charger all use PPS. iPhones do not request PPS; they negotiate fixed 9V or 15V PDOs and let the internal IC handle the rest.</p> The OPPO family: Dash, Warp, VOOC, SuperVOOC#</span></a> </h2> OPPO's proprietary fast-charge family began with VOOC in 2014. OnePlus rebranded the protocol twice — first as Dash, then as Warp — before consolidating to VOOC and SuperVOOC after the 2021 OnePlus/OPPO merger. The underlying technology in each generation is the same lineage; the renames are organizational.</p> Name</th> Era</th> Typical wire spec</th> Architectural trick</th></tr></thead> VOOC 1.0</strong></td> 2014, OPPO</td> 5V / 4A (20W)</td> Low voltage, high current. Charger does the heavy regulation.</td></tr> Dash Charge</strong></td> 2016–2018, OnePlus</td> 5V / 4A (20W)</td> OnePlus rebrand of VOOC. Same tech.</td></tr> Warp Charge 30</strong></td> 2018, OnePlus</td> 5V / 6A (30W)</td> More current at the same voltage.</td></tr> Warp Charge 65</strong></td> 2020, OnePlus</td> 10V / 6.5A (65W)</td> OnePlus rebrand of SuperVOOC 2.0.</td></tr> VOOC 4.0</strong></td> 2019, OPPO</td> 5V / 6A (30W)</td> Same tier as Warp 30.</td></tr> SuperVOOC 1.0</strong></td> 2018, OPPO</td> 10V / 5A (50W)</td> Dual-cell series battery</strong>. Charge pump in the phone.</td></tr> SuperVOOC 2.0</strong></td> 2020</td> 10V / 6.5A (65W)</td> —</td></tr> SuperVOOC 80W</strong></td> 2021</td> 10V / 8A</td> —</td></tr> SuperVOOC 100W</strong></td> 2022</td> 10V / 10A</td> —</td></tr> SuperVOOC 150W</strong></td> 2022</td> 20V / 7.5A</td> Higher voltage, deeper charge-pump ratio.</td></tr> SuperVOOC 240W</strong></td> 2022</td> 24V / 10A</td> Reference design. Production phones top out near 100W.</td></tr> </tbody></table> Everything from SuperVOOC 1.0 onward is architecturally distinct from PD and from the earlier Dash/Warp/VOOC line. The next section covers why.</p> What distinguishes SuperVOOC#</span></a> </h2> Four mechanisms, which compound.</p> 1. Dual-cell series battery#</span></a> </h3> A SuperVOOC phone uses two battery cells wired in series</strong>. A OnePlus 12, for example, ships two ~2700 mAh cells whose terminals are connected (+)–[cell A]–(–)/(+)–[cell B]–(–)</code>, giving a pack with ~8.7V nominal instead of the ~4.35V of a conventional single-cell phone. Same energy capacity. Double the pack voltage. Half the pack current at any given charging power.</p> Resistive losses in the cells, busbars, protection IC, and connecting traces scale as I². Halving the current quarters the I²R dissipation. A single-cell phone absorbing 100W must push ~22A through its cells; a dual-cell phone absorbs the same 100W at ~11A. The 4× heat reduction is a direct consequence of Ohm's law on a load whose resistance cannot be designed away.</p> 2. Charge pump rather than buck converter#</span></a> </h3> A typical phone charge IC is a synchronous buck converter — a pair of MOSFETs and an inductor switching at a few hundred kHz to step the input voltage down. Buck IC efficiency under heavy load is 88–93%. The remaining 7–12% becomes heat next to the battery.</p> A charge pump</strong> performs the same function with a different topology: capacitors instead of an inductor, with a fixed integer ratio (2:1, 4:1) rather than an arbitrary one. The constraint pays off — a 2:1 charge pump halves the input voltage and doubles the current at 97–98% efficiency. The cost is the fixed ratio: the input voltage must be approximately 2× or 4× what the cells require.</p> SuperVOOC combines the two. The charger delivers 10V at 8–10A; the phone's 2:1 charge pump halves that to 5V at 16–20A; this flows into the dual-cell pack at ~8.7V, with each cell seeing ~4.35V at half the pack current. The arithmetic only closes when input voltage, charge-pump ratio, and pack topology are co-designed.</p> 3. Charger-side regulation via MCU-to-MCU handshake#</span></a> </h3> Under USB-PD the device runs the charge curve. The phone reads cell voltage and temperature, decides what it wants, requests it from the charger, and converts the input internally. The charger is a voltage source that complies.</p> SuperVOOC moves the regulation loop into the charger. The charger contains its own MCU. At plug-in, charger MCU and phone MCU exchange a proprietary handshake over the CC pin (or D+/D- on earlier revisions). After the handshake, the phone streams real-time cell voltage and temperature telemetry to the charger and the charger adjusts its output accordingly — fine-grained, high-rate, and outside the USB-PD spec.</p> The consequence is that more conversion happens inside the charger and less inside the phone. The charger runs warm; the phone often does not. This inverts the thermal profile of a generic PD charger.</p> 4. Cable ID resistor#</span></a> </h3> USB-PD specifies an e-marker chip in cables rated above 3A. SuperVOOC piggybacks on the same mechanism with a different signal: a specific resistor value to ground on a particular pin tells the charger that the cable is rated for 8A / 10A / 12.5A continuous and that high-current mode is permitted. Without that resistor the charger will not deliver above the PD-PPS fallback rate, regardless of what the phone requests.</p> This is why a SuperVOOC charger paired with a generic USB-C cable charges a OnePlus at PD-PPS speeds: the charger does not recognize an unidentified cable. The reverse — an OPPO/OnePlus cable into a generic charger — is also a no-op, because the generic charger has no MCU on the other end.</p> What happens when a OnePlus plugs into an Anker#</span></a> </h2> Concrete numbers for the four chargers in question.</p> </th> </th></tr></thead> </td> </td></tr> INIU 100W 3-port GaN</strong> (B0DJ8C3MZD)</td> Anker Prime 67W 3-port</strong> (B0C4YTB56H)</td></tr> </td> </td></tr> Anker Prime 100W 3-port</strong> (B0C4DGBHY2)</td> Anker Prime 250W 6-port station</strong> (B0CYLL8Y89)</td></tr> </tbody></table> All four advertise PD 3.0 + PPS. The first three are single-unit GaN; the last is a desktop power station, which on its higher-power port also supports PD 3.1 EPR (28V) for laptops. None implements VOOC, SuperVOOC, or any other proprietary protocol — generic chargers cannot, lacking both the MCU and the proprietary firmware.</p> When a OnePlus plugs into any of them, the negotiation proceeds as follows:</p> Source capabilities.</strong> The charger advertises its PDO list, including PPS ranges such as 3.3–11V @ 5A</code> and 3.3–21V @ 3.25A</code>.</li> Phone selects PPS.</strong> The OnePlus selects the PPS PDO whose voltage range covers ~2 × battery voltage</code>.</li> Phone requests V/I.</strong> It requests, e.g., 8.94V @ 3.00A</code>.</li> Charger complies; the phone's charge pump halves the 9V to ~4.5V at ~6A into the cell.</strong></li> Loop.</strong> The phone re-issues an adjusted V/I request every few hundred milliseconds.</li> </ol> The constraint is at step 3. OnePlus phones cap their PPS current request at ~3A</strong>, regardless of what the charger offers. A 100W single-port unit with a 3.3–11V @ 5A</code> PPS PDO still receives a 3A request from the phone. OPPO/OnePlus reserves the high-current path for the proprietary protocol.</p> Measured draw at the USB-C connector with a power meter on a OnePlus 12-class phone:</p> Battery state</th> Phone's PPS request</th> Wattage at the port</th></tr></thead> 0–10% (warm-up)</td> ~5V × 2–3A</td> 10–15W</td></tr> 10–60% (bulk CC)</td> ~8.9–9.0V × 3.0A</strong></td> ~27W</strong></td></tr> 60–80% (transition)</td> ~9V × 2–2.5A</td> 18–22W</td></tr> 80–95% (CV taper)</td> ~8.7V × 1–1.5A</td> 8–13W</td></tr> 95–100% (trickle)</td> ~5V × 0.5A</td> 2–3W</td></tr> </tbody></table> Bulk-stage draw at the connector reads approximately 27W</strong>, sustained for 15–25 minutes before the taper.</p> The voltage choice is not arbitrary. The 2:1 charge pump in front of the single cell wants its input at approximately twice the cell voltage. A cell at 4.0–4.4V during bulk → request 8.0–8.9V at the port. The 3A figure is the phone-side cap.</p> Removing PPS — connecting the same phone to a fixed-PDO PD charger with no PPS PDOs — falls back to 9V/2A = 18W</code> on a fixed rail. This is slower and warmer because the conversion stage no longer has voltage tracking and the charge pump cannot remain at its efficiency sweet spot across the curve.</p> Among the four chargers shown:</p> The INIU 100W</strong>, Anker 67W</strong>, and Anker 100W</strong> deliver the same ~27W bulk to the phone. The additional wattage on the 100W units serves laptops and other devices.</li> The Anker Prime 250W station</strong> also delivers ~27W to the phone. Its advantage is sustaining that rate while simultaneously running a 100W laptop on another port without throttling either.</li> None triggers the SuperVOOC handshake. The "VOOC" indicator in OxygenOS labels any charge above an internal wattage threshold; it does not confirm the protocol.</li> </ul> PD-PPS versus SuperVOOC, head to head#</span></a> </h2> Same OnePlus 12-class phone, same 5400 mAh dual-cell pack, two different protocols on the wire.</p> Stage</th> PD-PPS (Anker / INIU)</th> SuperVOOC 80W brick</th> SuperVOOC 100W brick</th> SuperVOOC 150W brick</th></tr></thead> Negotiation</td> USB-PD messages on CC</td> Proprietary handshake on CC</td> same</td> same</td></tr> Bulk voltage</td> ~9V</td> ~10V</strong></td> ~10V</td> ~20V</strong></td></tr> Bulk current</td> 3A</strong> (phone-capped)</td> 8A</strong></td> 10A</strong></td> 7.5A</td></tr> Bulk wattage</td> ~27W</strong></td> ~80W</strong></td> ~100W</strong></td> ~150W</td></tr> Cable requirement</td> any USB-C</td> OPPO/OnePlus cable with ID resistor</td> same</td> same</td></tr> Battery topology assumed</td> dual-cell</td> dual-cell</td> dual-cell</td> dual-cell</td></tr> 0→100% time (5400 mAh)</td> 75–90 min</td> 30–35 min</td> ~26 min</strong></td> ~20 min</td></tr> Heat felt at</td> the phone</td> the brick</td> the brick</td> the brick</td></tr> </tbody></table> The 0→100% numbers do not scale linearly with peak wattage because the taper begins earlier at higher-power protocols. Cells absorb the peak rate only below ~70% state-of-charge; above that the curves converge as the battery itself becomes the bottleneck.</p> PD-PPS delivers approximately 27% of the phone's charging capacity</strong>. The remaining 73% sits behind a handshake that no third-party charger implements.</p> Summary#</span></a> </h2> SuperVOOC speeds require three components, all of them:</p> The OPPO / OnePlus charger with the SuperVOOC MCU at the matching wattage.</li> The OPPO / OnePlus cable with the ID resistor.</li> A SuperVOOC-capable phone — a flagship OnePlus from 2020 onward with the dual-cell pack and charge pump.</li> </ol> Missing any one falls the system back to USB-PD with PPS. The phone still charges, at approximately one-quarter of its peak rate, with the thermal load shifted from the charger into the phone, and a 0→100% time of roughly 90 minutes rather than 30.</p> The "VOOC" indicator in OxygenOS reports a wattage threshold, not a protocol. Generic 27W PD-PPS clears that threshold; so does 100W SuperVOOC. For ground-truth on the wire, a USB-C power meter — the ChargerLAB Power-Z KM003C</a> is the common enthusiast tool — reads the advertised PDO list and live V/A draw directly.</p> boot.initrd.secrets Unknown — Wed, 29 Apr 2026 00:00:00 +0000 boot.initrd.secrets#</span></a> </h1> Synopsis#</span></a> </h2> boot</span>.</span>initrd</span>.</span>secrets =</span> {</span></span> "/etc/ssh/ssh_host_ed25519_key"</span> =</span> /etc/secrets/initrd/ssh_host_ed25519_key</span>;</span></span> "/etc/cryptroot.key"</span> =</span> /etc/secrets/initrd/cryptroot.key</span>;</span></span> }</span>;</span></span></code></pre>Description#</span></a> </h2> boot.initrd.secrets</code> copies files into the initrd so they are available before the root filesystem is mounted. Typical contents include LUKS keyfiles, SSH host keys for initrd SSH, and VPN configurations.</p> The option type is attrsOf (nullOr path)</code>. Attribute names are destination paths inside the initrd. Values are source paths on the host, or null</code> to use the attribute name as both source and destination:</p> boot</span>.</span>initrd</span>.</span>secrets</span>.</span>"/etc/ssh/ssh_host_ed25519_key"</span> =</span> null</span>;</span></span> # copies /etc/ssh/ssh_host_ed25519_key to /etc/ssh/ssh_host_ed25519_key inside the initrd</span></span></code></pre> The option is defined in nixos/modules/system/boot/stage-1.nix</code>.</p> Implementation#</span></a> </h2> There are two code paths. Which one runs depends on the value of boot.loader.supportsInitrdSecrets</code>.</p> Append path (supportsInitrdSecrets = true</code>)#</span></a> </h3> GRUB, systemd-boot, and Limine all set supportsInitrdSecrets = true</code>.</p> In this path, the initrd is built as a normal Nix derivation without any secrets. At bootloader install time (nixos-rebuild switch</code>), the generated script append-initrd-secrets</code> constructs a second CPIO archive containing the secrets and appends it to the initrd file on the boot partition.</p> The Linux kernel treats concatenated CPIO archives as a single initramfs — it unpacks them sequentially. This is a documented property of the initramfs format.</p> Secrets handled this way never enter the Nix store. They reside on the boot partition only.</p> The script is built in stage-1.nix</code> and stored at config.system.build.initialRamdiskSecretAppender</code>. Bootloader installers call it: GRUB via install-grub.pl</code>, systemd-boot via systemd-boot-builder.py</code>. The path to the script is also recorded in bootspec.json</code> under the initrdSecrets</code> key for use by external boot managers.</p> Failure to create secrets for the current generation is fatal. Failure for older generations produces a warning and continues — this is expected after removing or renaming a secret source file.</p> Inline path (supportsInitrdSecrets = false</code>)#</span></a> </h3> When the bootloader does not support appending, secrets are copied into the initrd derivation at build time:</p> mkdir</span> -p</span> $(</span>dirname</span> "</span>$out</span>/secrets/</span>${dest}</span>"</span>)</span></span> cp</span> -Lr</span> ${source</span>'} "$out/secrets/${dest}"</span></span></code></pre> This places secrets in the Nix store. The store is world-readable. An assertion enforces that all values are unquoted Nix paths or store paths in this case, and the build emits:</p> Note that this will result in all secrets being stored world-readable in the Nix store!</p> </blockquote> The path</code> type on the option (rather than str</code>) exists to support this fallback.</p> append-initrd-secrets#</span></a> </h2> The script performs the following steps:</p> Exit immediately if boot.initrd.secrets</code> is empty.</li> Create a temporary directory.</li> For each secret, copy the source file to $tmp/.initrd-secrets/$dest</code>, preserving attributes (cp -a</code>).</li> Set all timestamps to the Unix epoch (touch -amt 197001010000</code>) for reproducibility.</li> Sort the file list (sort -z</code>) for deterministic ordering.</li> Generate a CPIO newc archive owned by root:root (cpio -H newc -R +0:+0 --reproducible</code>).</li> Compress with the same compressor used for the main initrd.</li> Append (>></code>) to the target initrd file.</li> </ol> Simplified:</p> tmp</span>=</span>$(</span>mktemp</span> -d</span> ${TMPDIR</span>:-/</span>tmp}</span>/initrd-secrets.XXXXXXXXXX</span>)</span></span> </span> mkdir</span> -p</span> $(</span>dirname</span> "</span>$tmp</span>/.initrd-secrets/</span>${dest}</span>"</span>)</span></span> cp</span> -a</span> ${source}</span> "</span>$tmp</span>/.initrd-secrets/</span>${dest}</span>"</span></span> </span> (</span>cd</span> "</span>$tmp</span>"</span> \</span></span> &&</span> find</span> . -mindepth</span> 1</span> \|</span> xargs</span> touch -amt</span> 197001010000</span> \</span></span> &&</span> find</span> . -mindepth</span> 1</span> -print0</span> \|</span> sort</span> -z</span> \</span></span> \|</span> cpio</span> --quiet -o -H newc -R +0:+0 --reproducible --null</span>)</span> \</span></span> \|</span> zstd</span> >></span> "</span>$1</span>"</span></span></code></pre> The compressor and its arguments are taken from config.boot.initrd.compressor</code> and config.boot.initrd.compressorArgs</code>.</p> systemd initrd integration#</span></a> </h2> When boot.initrd.systemd.enable = true</code>, secrets are not placed at their final paths directly. They are staged under /.initrd-secrets/</code> inside the initrd and copied into place by the initrd-nixos-copy-secrets</code> service.</p> This service is defined in nixos/modules/system/boot/systemd/initrd-secrets.nix</code>:</p> boot</span>.</span>initrd</span>.</span>systemd</span>.</span>services</span>.</span>initrd-nixos-copy-secrets =</span> {</span></span> description</span> =</span> "Copy secrets into place"</span>;</span></span> wantedBy</span> =</span> [</span> "sysinit.target"</span> ];</span></span> before</span> =</span> [</span> "cryptsetup-pre.target" "shutdown.target"</span> ];</span></span> conflicts</span> =</span> [</span> "shutdown.target"</span> ];</span></span> unitConfig</span>.</span>DefaultDependencies</span> =</span> false</span>;</span></span> </span> script</span> =</span> ''</span></span> for secret in $(cd /.initrd-secrets; find . -type f -o -type l); do</span></span> mkdir -p "$(dirname "/$secret")"</span></span> cp "/.initrd-secrets/$secret" "/$secret"</span></span> done</span></span> ''</span>;</span></span> </span> serviceConfig</span> =</span> {</span></span> Type</span> =</span> "oneshot"</span>;</span></span> RemainAfterExit</span> =</span> true</span>;</span></span> };</span></span> }</span>;</span></span></code></pre> The indirection exists because the initrd mounts a tmpfs on /run</code>. Secrets targeting paths under /run</code> would be shadowed by this mount. Staging them in /.initrd-secrets/</code> and copying after mount setup avoids this.</p> The service runs before cryptsetup-pre.target</code>, so secrets are available before LUKS devices attempt to unlock.</p> Consumers#</span></a> </h2> Several NixOS modules set boot.initrd.secrets</code> internally.</p> boot.initrd.network.ssh#</span></a> </h3> boot</span>.</span>initrd</span>.</span>network</span>.</span>ssh =</span> {</span></span> enable</span> =</span> true</span>;</span></span> hostKeys</span> =</span> [</span> /etc/secrets/initrd/ssh_host_ed25519_key</span> ];</span></span> }</span>;</span></span></code></pre> Generates:</p> boot</span>.</span>initrd</span>.</span>secrets</span>.</span>"/etc/ssh/ssh_host_ed25519_key"</span> =</span></span> /etc/secrets/initrd/ssh_host_ed25519_key</span>;</span></span></code></pre> The module warns against reusing regular host keys. With the append path, the initrd SSH key resides on the unencrypted boot partition.</p> boot.initrd.luks.devices..keyFile#</span></a> </h3> boot</span>.</span>initrd</span>.</span>secrets</span>.</span>"/etc/cryptroot.key"</span> =</span> /etc/secrets/initrd/cryptroot.key</span>;</span></span> </span> boot</span>.</span>initrd</span>.</span>luks</span>.</span>devices</span>.</span>cryptroot =</span> {</span></span> device</span> =</span> "/dev/disk/by-uuid/..."</span>;</span></span> keyFile</span> =</span> "/etc/cryptroot.key"</span>;</span></span> }</span>;</span></span></code></pre>boot.initrd.network.openvpn#</span></a> </h3> boot</span>.</span>initrd</span>.</span>network</span>.</span>openvpn =</span> {</span></span> enable</span> =</span> true</span>;</span></span> configuration</span> =</span> /etc/secrets/initrd/vpn.ovpn</span>;</span></span> }</span>;</span></span></code></pre> Generates:</p> boot</span>.</span>initrd</span>.</span>secrets</span>.</span>"/etc/initrd.ovpn"</span> =</span> /etc/secrets/initrd/vpn.ovpn</span>;</span></span></code></pre>Security#</span></a> </h2> Append path.</strong> Secrets are on the boot partition, owned by root, not world-readable. They are not encrypted. The security posture is equivalent to the rest of an unencrypted /boot</code> — kernel and initrd are equally exposed. Attacks against the boot partition (replacement, modification) are addressed by Secure Boot and measured boot (TPM), not by this module.</p> Inline path.</strong> Secrets are in /nix/store</code> and world-readable. Do not use this path for sensitive material.</p> Garbage collection.</strong> Old NixOS generations retain their initrd files with appended secrets. When a secret source file is removed, nixos-rebuild switch</code> warns but continues for old generations. The old initrd (with the old secret) remains on /boot</code> until the generation is deleted with nix-collect-garbage -d</code> or nix-env --delete-generations</code>.</p> Files#</span></a> </h2> File</th> Role</th></tr></thead> nixos/modules/system/boot/stage-1.nix</code></td> Option definition, append-initrd-secrets</code> script, inline fallback, assertion</td></tr> nixos/modules/system/boot/systemd/initrd-secrets.nix</code></td> systemd initrd copy service</td></tr> nixos/modules/system/boot/loader/grub/grub.nix</code></td> GRUB sets supportsInitrdSecrets = true</code></td></tr> nixos/modules/system/boot/loader/systemd-boot/systemd-boot.nix</code></td> systemd-boot sets supportsInitrdSecrets = true</code></td></tr> nixos/modules/system/boot/loader/grub/install-grub.pl</code></td> Calls append-initrd-secrets</code> per generation</td></tr> nixos/modules/system/boot/loader/systemd-boot/systemd-boot-builder.py</code></td> Calls append-initrd-secrets</code> per generation</td></tr> nixos/modules/system/boot/bootspec.nix</code></td> Records initrdSecrets</code> path in bootspec.json</td></tr> </tbody></table> My Recommended Software Stack for Leaving Gmail and Google Workspace Unknown — Thu, 23 Apr 2026 00:00:00 +0000 My Recommended Software Stack for Leaving Gmail and Google Workspace#</span></a> </h1> A lot of people want a simpler, more private, less Google-dependent setup without giving up the tools they rely on. Here’s the stack I use.</p> The philosophy: pick the best option per category, keep your computer at the center of your data. Favor local over cloud, speed over latency. Unix philosophy — do one thing well.</p> What this is NOT#</span></a> </h3> A guide for self-hosters. If that’s you, see 1</a></sup>.</p> Email and Calendar#</span></a> </h2> Use:</strong> Fastmail — fast, clean, focused on doing email well. Calendar works well enough.</p> Don’t use:</strong> Proton — was a customer for years. Expensive, slow, web-only, bloated.</p> Fastmail’s web app is faster and supports IETF standards, so you can drop the browser for a native desktop app.2</a></sup></p> !Important!#</span></a> </h3> Domain Registrar:</strong> Porkbun </p> Use your own domain. Switching providers becomes trivial — point MX and TXT records somewhere new and nobody notices. Your address is yours, not the vendor’s. Porkbun is cheap, simple, and supports WHOIS privacy.</p> Password Manager#</span></a> </h2> Use:</strong> 1Password — pretty good UX, cross-device, just don't use passkeys3</a></sup>. And then in the future if you want to migrate you can export all your TOTP secrets and data. It makes for a pretty good general purpose database too. Has native apps for all platforms.</p> File Storage#</span></a> </h2> Don’t use:</strong> Dropbox — billing is a labyrinth, they won’t delete your account, only cancel the next subscription. Prisoners, not customers.</p> Storage should be local first. For sharing on your network, NFS or SMB/CIFS does the job.4</a></sup></p> External Transfers:</strong> Filemail — simple, end-to-end encrypted, simple. You already have internet and storage; you don’t need the cloud to rent your files back to you.</p> Video Meetings#</span></a> </h2> Use:</strong> Signal , Zoom </p> Signal is peer-to-peer, end-to-end encrypted, and simple — but only reaches other Signal users. Use it where you can; fall back to Zoom for everyone else.</p> Writing and Notes#</span></a> </h2> Use:</strong> Obsidian + Markdown — portable, durable, not tied to any vendor. Use .docx</code> when compatibility wins.</p> Native editors beat web ones:</p> Linux</th> macOS</th> Windows</th></tr></thead> LibreOffice , OnlyOffice </td> Pages, Word </td> Word , LibreOffice </td></tr> </tbody></table> Spreadsheets#</span></a> </h3> Use:</strong> Grist — treats data like a database, not a grid. Typed columns, Python formulas, lightweight apps on top.</p> Chat#</span></a> </h2> Use:</strong> Signal , Slack </p> Signal for direct messages. Slack for teams. I’d rather recommend a peer-to-peer, channel-based alternative — but unless you like IRC, there isn’t one.5</a></sup></p> TL;DR#</span></a> </h2> Email / Calendar:</strong> Fastmail </li> Domain:</strong> Porkbun </li> Passwords:</strong> 1Password </li> File storage:</strong> NFS / SMB locally, Filemail for external</li> Video:</strong> Signal , Zoom </li> Writing:</strong> Obsidian , Markdown, .docx</code></li> Spreadsheets:</strong> Grist </li> Chat:</strong> Signal , Slack </li> </ul> Index#</span></a> </h2> Logo</th> Product</th> Category</th></tr></thead> </td> Fastmail</a></td> Email & Calendar</td></tr> </td> Proton</a></td> Email</td></tr> </td> Porkbun</a></td> Domain Registrar</td></tr> </td> 1Password</a></td> Password Manager</td></tr> </td> Dropbox</a></td> File Storage</td></tr> </td> Filemail</a></td> File Transfer</td></tr> </td> Signal</a></td> Chat & Video</td></tr> </td> Zoom</a></td> Video Meetings</td></tr> </td> Obsidian</a></td> Writing & Notes</td></tr> </td> LibreOffice</a></td> Office Suite</td></tr> </td> OnlyOffice</a></td> Office Suite</td></tr> </td> Microsoft Word</a></td> Word Processor</td></tr> </td> Grist</a></td> Spreadsheets</td></tr> </td> Slack</a></td> Chat</td></tr> </tbody></table> ^{1</sup> I'm working on some new projects to serve these gaps. See my git repos.</p> </div> ^{2</sup> The standards-protocol point is bigger than just native desktop software. It’s about having authority over your own data. That’s my biggest gripe with Proton and Gmail. You should be able to use your data the way you want — whether that’s through a desktop app or a CLI tool. Don’t settle for less, unless a better option simply doesn’t exist yet.</p> </div> ^{3</sup> They purposely ruined them by making them unexportable.</p> </div> ^{4</sup> microsoft Try To Name Something Challenge</p> </div> ^{5</sup> Don’t email me about Matrix. Do email me about IRC.</p> </div> About Me Unknown — Sun, 19 Apr 2026 00:00:00 +0000 Hi, I'm Scott</p> I'm a Senior Developer and Open Source project maintainer of electricEye.</p> I currently live in Texas with my partner and our cat🐈.</p> I use my site as a method of blogging about my learnings, as well as sharing information about projects I have previously, or am currently, working on in my spare time.</p> I specialize in Internet Protocol versions 4 and 6, software development, operations, system administration, and 🔥blazingly🔥 fast code</p> I'm a Linux user, a big advocate for the Open Source Software Movement, and the libre (free as in freedom) movement and I host my own services rather than handing it off to someone else's platform.</p> Lambdo's and Dont's Unknown — Sun, 19 Apr 2026 00:00:00 +0000 Lambdagony: You Don't Need a Lambda, You Need a Socket#</span></a> </h1> You know that Lambda function you wrote? The one that receives an HTTP request, does some small thing, and returns a response? The one with 200 lines of Terraform around it, an API Gateway in front of it, a CloudWatch log group you always forget how to query, a 3-second cold start that appears at the worst possible time, and a deployment pipeline held together with YAML and prayers?</p> What if I told you the exact same workload could be a 12-line systemd unit on a box you already have?</p> The Problem with Lambda#</span></a> </h2> Lambda is great for what Lambda is great for. The issue is that people reach for it when they don't need it. You don't have "thousands of concurrent requests with unpredictable spiky traffic." You have a webhook that fires twice an hour from your git forge and a cron job that runs nightly. You are not Netflix.</p> But you've been told serverless is the future, so now you have:</p> A Lambda function</li> An API Gateway (with its own fun config)</li> An IAM role with a policy document</li> A CloudWatch log group</li> A deployment artifact (zip or container image)</li> Some way to get that artifact there (CI, SAM, CDK, Terraform, clicking around in the console and hoping you remember what you did)</li> Cold starts</li> A 15 minute execution timeout</li> No local state, no filesystem (well, 512 MB of /tmp</code>), no persistent connections</li> A bill that's small but irritating to reason about</li> </ul> All for something that accepts a JSON payload, writes a row to a database, and returns {"ok": true}</code>.</p> What systemd Gives You for Free#</span></a> </h2> systemd has a feature called socket activation</strong>. It's been around since 2010. The idea is dead simple: systemd listens on a port. When a connection arrives, it starts your service and hands over the socket. When your service is idle, it can exit, and systemd goes back to listening.</p> Sound familiar? It's Lambda. It's literally Lambda. Except:</p> No cold start that you can't control (your binary is local, on an SSD)</li> No 15-minute timeout</li> Full filesystem access</li> Persistent state if you want it</li> Logs go to the journal, queryable with journalctl -u yourservice</code></li> Deployment is scp</code> and systemctl daemon-reload</code> (or NixOS rebuild, if you're civilized)</li> It's free. The box is already running.</li> </ul> How It Works#</span></a> </h2> You need two unit files. That's it.</p> The Socket Unit#</span></a> </h3> # /etc/systemd/system/myapi.socket</span></span> [</span>Unit</span>]</span></span> Description</span>=</span>My API Socket</span></span> </span> [</span>Socket</span>]</span></span> ListenStream</span>=</span>8080</span></span> </span> [</span>Install</span>]</span></span> WantedBy</span>=</span>sockets.target</span></span></code></pre> This tells systemd: listen on port 8080. When someone connects, start the corresponding .service</code>.</p> The Service Unit#</span></a> </h3> # /etc/systemd/system/myapi.service</span></span> [</span>Unit</span>]</span></span> Description</span>=</span>My API Service</span></span> Requires</span>=</span>myapi.socket</span></span> </span> [</span>Service</span>]</span></span> Type</span>=</span>notify</span></span> ExecStart</span>=</span>/usr/local/bin/myapi</span></span> NonBlocking</span>=</span>true</span></span> </span> # Hardening, because we're not animals</span></span> DynamicUser</span>=</span>yes</span></span> ProtectSystem</span>=</span>strict</span></span> ProtectHome</span>=</span>yes</span></span> PrivateTmp</span>=</span>yes</span></span> NoNewPrivileges</span>=</span>yes</span></span></code></pre> Your binary receives the already-open socket as file descriptor 3 (or via the LISTEN_FDS</code> protocol — sd_listen_fds(3) if you want to be proper about it). Most languages have a library for this. In Go:</p> listeners</span>,</span> _</span> :=</span> activation</span>.</span>Listeners</span>()</span></span> http</span>.</span>Serve</span>(</span>listeners</span>[</span>0</span>],</span> myHandler</span>)</span></span></code></pre> In Python:</p> import</span> socket</span>,</span> os</span></span> </span> fd</span> =</span> 3</span> # or use the sd-daemon library</span></span> sock</span> =</span> socket</span>.</span>fromfd</span>(</span>fd</span>,</span> socket</span>.</span>AF_INET</span>,</span> socket</span>.</span>SOCK_STREAM</span>)</span></span></code></pre> In Rust there's listenfd</code>. In C there's sd-daemon</code>. You get the idea.</p> Enable and Start#</span></a> </h3> systemctl</span> enable --now myapi.socket</span></span></code></pre> That's it. Port 8080 is now listening. Your service starts on first connection.</p> "But What About Scaling?"#</span></a> </h2> If you genuinely need to handle thousands of concurrent requests with autoscaling across regions, yes, use a managed service. Nobody is arguing you should run Twitter on socket-activated systemd units.</p> But be honest with yourself: most of the things people put in Lambdas are internal tools, webhooks, simple APIs, and cron jobs. These are workloads that a $5/month VPS handles without breaking a sweat. The "scale" argument is a thought-terminating cliche that prevents people from choosing the simple option.</p> "But What About High Availability?"#</span></a> </h2> You already have a server. Is it running? Then your service is available. If it's not running, your Lambda isn't going to help you either because the database it talks to is probably on the same infrastructure that's down.</p> If you genuinely need HA, run two boxes and put a load balancer in front. Still simpler than Lambda + API Gateway + custom domain + ACM certificate + Route53 record.</p> "But Lambda Scales to Zero!"#</span></a> </h2> So does a socket-activated service. That's the whole point. When nobody is connecting, your process isn't running. The socket is open, held by PID 1, using approximately zero resources.</p> Bonus: systemd Timers Instead of CloudWatch Events / EventBridge#</span></a> </h2> That cron-triggered Lambda? It's a systemd timer:</p> # /etc/systemd/system/nightly-job.timer</span></span> [</span>Unit</span>]</span></span> Description</span>=</span>Run nightly job</span></span> </span> [</span>Timer</span>]</span></span> OnCalendar</span>=</span>--* 03:00:00</span></span> Persistent</span>=</span>true</span></span> </span> [</span>Install</span>]</span></span> WantedBy</span>=</span>timers.target</span></span></code></pre># /etc/systemd/system/nightly-job.service</span></span> [</span>Unit</span>]</span></span> Description</span>=</span>Nightly job</span></span> </span> [</span>Service</span>]</span></span> Type</span>=</span>oneshot</span></span> ExecStart</span>=</span>/usr/local/bin/nightly-job</span></span> DynamicUser</span>=</span>yes</span></span></code></pre>systemctl</span> enable --now nightly-job.timer</span></span></code></pre> Want to see when it last ran? systemctl status nightly-job.timer</code>. Want logs? journalctl -u nightly-job</code>. Want to run it manually right now? systemctl start nightly-job</code>. No clicking through 4 AWS console pages.</p> When to Actually Use Lambda#</span></a> </h2> You have genuinely spiky, unpredictable traffic that spans orders of magnitude</li> You don't want to manage any infrastructure at all (but then you're managing AWS infrastructure, which is its own beast)</li> You're already deep in the AWS ecosystem and the integration points save you real time</li> You need to run in multiple regions with minimal effort</li> </ul> When to Use a Socket-Activated Service Instead#</span></a> </h2> You have a server</li> Your traffic is low to moderate</li> You want to actually understand your infrastructure</li> You're tired of 200 lines of IaC for a 50-line program</li> You value the ability to ssh</code> in and just look at things</li> You don't want to learn a new debugging/logging/deployment paradigm for what is fundamentally "a program that listens on a port"</li> </ul> The Punchline#</span></a> </h2> The cloud providers have done an incredible job of making people forget that computers are good at running programs. A modern Linux box with systemd can do socket activation, process supervision, resource limits (cgroups), sandboxing (namespaces, seccomp), logging, and scheduled tasks. It's been able to do all of this for over a decade.</p> Lambda is a product. Socket activation is a feature of your operating system. One of them has a marketing budget.</p> eyeFS: A Linux Kernel Filesystem for Multi-Camera H.265 Recording Unknown — Thu, 09 Apr 2026 00:00:00 +0000 Working title:</strong> eyeFS</code> Status:</strong> design document</p> 1. Abstract#</span></a> </h2> eyeFS is a Linux kernel filesystem that turns a dedicated block device into a circular log of H.265 chunks, supporting an arbitrary number of concurrent video feeds added and removed at runtime. The device is operated as a byte-addressable ring; chunks are packed back-to-back with no gaps and no container format; each chunk is an independently decodable H.265 Annex B bitstream. Multiple feeds share a single write head — a zipper merge across cameras — so that retention is uniform across feeds and the ring has one authoritative log-time axis. The userspace surface is a standard mount point: chunks appear as files under /mnt/eye/cameras/<id>/chunks/</code>, each camera has its own tail-able current</code> file at /mnt/eye/cameras/<id>/current</code> exposing the chunk-in-progress for that feed, and runtime control lives in sysfs. Crash recovery rescans the device for self-describing chunk markers.</p> This document covers the on-disk layout, the multi-feed architecture, the VFS and sysfs interfaces, the write and read paths, crash consistency, and the userspace/kernel split.</p> 2. Motivation (compressed)#</span></a> </h2> A surveillance workload is a single-writer, append-only, delete-oldest-first log of self-describing H.265 units. A conventional stack (filesystem + container + file rotation) solves problems the workload doesn't have, at a cost of 5–15% media waste, fragmentation, and bounded-space accounting headaches. The cleanest representation is a raw block device operated as a circular log of keyframe-aligned chunks.</p> 3. Why in-kernel#</span></a> </h2> The userspace design delivers the log but pushes every reader through an RPC. The kernel design delivers the log and</em> exposes it as a first-class filesystem, which unlocks properties a daemon cannot provide:</p> Page cache and zero-copy reads.</strong> Playback is sendfile(2)</code> / splice(2)</code> from a chunk fd directly to a socket. Hot ranges stay cached across readers. Userspace with O_DIRECT</code> actively avoids all of this.</li> mmap</code> on chunks.</strong> Decoders can map a chunk and decode in place; no read buffer, no copy.</li> Tail-able current</code> file.</strong> The chunk being written is a growing file. ffmpeg -i /mnt/eye/cameras/front/current</code> follows live. poll(2)</code> wakes on new data. inotify</code> fires on new chunks.</li> Standard tooling.</strong> ls</code>, find</code>, du</code>, rsync</code>, tar</code>, restic</code>, scp</code>, ffmpeg</code>, mpv</code>, NFS (exportfs</code>), Samba. No export subcommand; no replication daemon.</li> Kernel I/O integration.</strong> Writeback, merging, I/O scheduler interaction, blktrace</code>, bpftrace</code>, iostat</code>, perf</code> all work.</li> ZNS mapping.</strong> Zoned Namespace SSDs require sequential-per-zone writes and whole-zone resets. A circular log of append-only chunks is the canonical ZNS workload. In-kernel, blk-zoned</code> and zonefs</code> are right there.</li> Access control.</strong> Unix perms and mount namespaces. No custom auth layer.</li> </ul> What the kernel does not</em> do: RTSP. That stays userspace (see §9).</p> 4. Reference implementation: zonefs#</span></a> </h2> eyeFS is structurally zonefs-shaped — one file per logical unit (chunk), append-only, VFS-exposed, no hierarchical allocation — but adds H.265-specific semantics (sidecar SEI chunk markers, per-camera namespacing) and circular overwrite (zonefs's zones don't auto-evict). The zonefs source at fs/zonefs/</code> is ~2,000 lines and is the right model to read before writing eyeFS. Expect eyeFS to be 2–3× that, most of the delta being the camera registry, the chunk assembler, and the rescan/recovery path.</p> 5. On-disk layout#</span></a> </h2> The block device is laid out as:</p> offset 0 offset 4 KiB offset 36 KiB offset D</span></span> ┌──────────────────┬──────────────────┬────────────────────────────────────┐</span></span> │ Superblock │ Camera registry │ Ring region │</span></span> │ 4 KiB, immutable │ 32 KiB (128×256B)│ chunks packed back-to-back, wraps │</span></span> │ after format │ updated on │ to ring_start on overflow │</span></span> │ │ add/remove │ │</span></span> └──────────────────┴──────────────────┴────────────────────────────────────┘</span></span></code></pre>5.1 Superblock#</span></a> </h3> 4 KiB at offset 0, written once at mkfs.eyeFS</code> time. Fields are a superset of the userspace design:</p> Field</th> Size</th> Notes</th></tr></thead> magic</td> 8 B</td> "EYEFS\0\0\0"</code></td></tr> version</td> 4 B</td> on-disk format revision</td></tr> device_uuid</td> 16 B</td> identifies this volume</td></tr> logical_sector_sz</td> 4 B</td> 512 or 4096</td></tr> physical_sector_sz</td> 4 B</td> alignment hint</td></tr> registry_start</td> 8 B</td> typically 4096</td></tr> registry_size</td> 8 B</td> typically 32768</td></tr> ring_start</td> 8 B</td> typically 36864</td></tr> ring_end</td> 8 B</td> device capacity</td></tr> nominal_chunk_sz</td> 8 B</td> default target, overridable per-mount</td></tr> min_chunk_sz</td> 8 B</td> below this, wrap discards tail</td></tr> max_cameras</td> 4 B</td> registry slot count</td></tr> created_at</td> 8 B</td> unix ns</td></tr> last_write_head</td> 8 B</td> updated at each checkpoint</td></tr> last_global_seq</td> 8 B</td> checkpoint of sequence allocator</td></tr> crc32c</td> 4 B</td> </td></tr> reserved</td> pad to 4 KiB</td> </td></tr> </tbody></table> The superblock is not in the writable ring. last_write_head</code> and last_global_seq</code> are updated at checkpoints; see §12 for how this bounds recovery scan time.</p> 5.2 Camera registry#</span></a> </h3> A fixed-size table of 128 slots × 256 bytes = 32 KiB directly after the superblock. Each slot:</p> Field</th> Size</th> Notes</th></tr></thead> slot_flags</td> 4 B</td> `FREE</td></tr> camera_id</td> 2 B</td> stable, assigned once, never reused in-lifetime</td></tr> camera_uuid</td> 16 B</td> for cross-device identification</td></tr> display_name</td> 64 B</td> UTF-8, null-padded</td></tr> added_at</td> 8 B</td> wall-clock ns</td></tr> removed_at</td> 8 B</td> 0 = still active</td></tr> codec</td> 4 B</td> `H265_MAIN</td></tr> default_pts_rate</td> 4 B</td> Hz (e.g. 90000)</td></tr> vps_len</td> 2 B</td> stored length of VPS for reference</td></tr> sps_len</td> 2 B</td> </td></tr> pps_len</td> 2 B</td> </td></tr> vps/sps/pps</td> 112 B</td> inlined; actual chunks carry their own</td></tr> crc32c</td> 4 B</td> </td></tr> reserved</td> pad to 256 B</td> </td></tr> </tbody></table> The registry is updated in place by slot: add allocates a free slot, remove flips ACTIVE → REMOVED</code> and sets removed_at</code>. A removed camera's chunks remain readable until wrapped over; its slot is reclaimable for new cameras only after all its chunks have aged out of the ring.</p> Registry writes use the same ordering rule as chunk writes: write + fdatasync, then in-memory update, then WAL record on the NVMe-side index.</p> 5.3 Ring region#</span></a> </h3> Each chunk is a valid Annex B bitstream beginning with a sidecar SEI NAL unit (H.265 user_data_unregistered</code>, NAL unit type 39), followed by VPS, SPS, PPS, IDR, and the rest of the GOP. Decoders ignore the SEI; eyeFS uses it as a structured, scannable anchor.</p> Sidecar SEI payload for eyeFS (extended from the userspace design):</p> sidecar payload:</span></span> magic_uuid : 16 B (identifies the eyeFS chunk format)</span></span> global_seq : u64 (monotonic across all cameras on this device)</span></span> camera_id : u16 (registry slot ID)</span></span> chunk_length : u64 (total bytes including this SEI)</span></span> pts_start : i64 (stream's timescale, per camera)</span></span> pts_end : i64</span></span> wall_clock_ns : i64 (global log time)</span></span> prev_chunk_off : u64 (backpointer in global order)</span></span> flags : u32</span></span> crc32c : u32</span></span></code></pre> Per-camera sequence numbers are not</em> stored on disk; they are derived from (camera_id, global_seq)</code> at rebuild time by counting per-camera chunks in global-seq order. This keeps the on-device record fixed-size and avoids coordinating two counters during the flush critical section.</p> 6. Multi-feed architecture#</span></a> </h2> The core decision: one ring, one write head, chunks tagged by camera_id.</strong> Zipper-merge across feeds. Alternatives considered and rejected:</p> Partition table, one ring per camera.</strong> Not dynamic; adding a camera requires resizing existing partitions.</li> Multiple independent rings on one device, sized by bitrate share.</strong> Dynamic-capable, but rebalancing is complex and per-camera eviction makes cross-camera time queries awkward.</li> Per-camera virtual rings over shared physical space.</strong> Same semantics as the chosen design at significantly higher bookkeeping cost.</li> </ul> The chosen design is strictly simpler and gives uniform retention across cameras, which is what operators actually want (legal holds, compliance windows, "the last 30 days of everything").</p> 6.1 Zipper-merge mechanics#</span></a> </h3> Per camera, eyeFS maintains an in-kernel NAL accumulator: a buffer of access units since the last flush, plus a running size and the offset of the last IDR within the buffer. When the userspace ingestor feeds an access unit (§9) the kernel appends it.</p> A flush is triggered when, for some camera:</p> The accumulator size ≥ nominal_chunk_sz</code>, and</strong></li> The next incoming access unit is an IDR.</li> </ol> At that point the accumulator is cut immediately before the incoming IDR (the IDR becomes the first frame of the next chunk). The cut chunk is handed to the flush path.</p> The flush path takes the global write-head spinlock</strong>, stamps global_seq</code> and wall_clock_ns</code>, prepends the sidecar SEI, writes the chunk at the current write head, advances the head, releases the lock. Chunk writes are seconds apart per camera; at 32 cameras and 128 MiB chunks, the write rate is ~one chunk per few hundred milliseconds. Spinlock contention is not a factor.</p> 6.2 Dynamic add#</span></a> </h3> echo "rtsp://user:pass@cam4.lan/main front_entrance" > /sys/fs/eyeFS/<uuid>/cameras/add</code></p> The kernel:</p> Allocates a free registry slot, assigns camera_id</code>.</li> Writes the registry slot to disk, fdatasyncs.</li> Creates the VFS subtree /mnt/eye/cameras/<camera_id>/</code>.</li> Opens a per-camera write-side character device (or the current</code> file becomes writable; see §9).</li> </ol> The userspace ingestor daemon (a separate systemd service, watching for registry changes via inotify on cameras/</code>) picks up the new camera, opens the RTSP URL, and starts feeding access units. The first access unit that is an IDR starts the first chunk; earlier units arrive before the first IDR are discarded.</p> 6.3 Dynamic remove#</span></a> </h3> echo <camera_id> > /sys/fs/eyeFS/<uuid>/cameras/remove</code></p> Mark slot REMOVED</code>, set removed_at</code>, persist.</li> Close the per-camera write side; further access units are rejected.</li> Existing chunks remain visible under /mnt/eye/cameras/<camera_id>/</code> until overwritten.</li> When the last surviving chunk for this camera_id</code> is evicted, the slot is reclaimable.</li> </ol> 6.4 Offline / reconnecting cameras#</span></a> </h3> A camera whose RTSP connection drops simply stops feeding the kernel. No placeholder writes, no padding, no sentinel chunks. The ring sees fewer chunks from that camera during the outage; other cameras continue normally. When the camera reconnects, the userspace ingestor resumes feeding, and the next IDR starts a new chunk.</p> The retention window for the affected camera shrinks proportionally during the outage — if camera A was offline for 10% of a retention cycle, the last surviving chunk for A will be from 10% further back in time than the other cameras' last surviving chunks. This is correct behavior, not a bug: the ring preserves as much recent footage as possible.</p> 6.5 Log-time vs frame PTS#</span></a> </h3> Two timelines coexist and must not be conflated:</p> Log time</strong> (wall_clock_ns</code> in the sidecar SEI): a global, monotonic, NTP-ish timestamp assigned by the kernel at flush time. This is the axis for retention, eviction, and cross-camera time-range queries. All cameras on a device share this axis.</li> Frame PTS</strong> (pts_start</code>, pts_end</code> in the sidecar SEI): per-camera, in that camera's own timescale (typically 90 kHz). Preserved verbatim from the stream. This is the axis decoders see.</li> </ul> Time-range playback queries express T1..T2 in log time. The kernel resolves them against the global index and returns per-camera byte ranges. The decoder reconstructs frame timing from the in-stream PTS.</p> This separation is what the user called "synchronous timestamps." It is not that the cameras are frame-synchronized (they aren't, and can't be in general). It is that their log time</em> is the same single axis — which is all that eviction and queries need.</p> 6.6 Bandwidth sharing: no fairness#</span></a> </h3> eyeFS does not enforce per-camera quotas. A camera with double the bitrate consumes double the ring space and shortens everyone's retention together. This is deliberate: the ring is a shared resource with a single retention window, and any per-camera quota scheme would either waste ring space (reserving quota that isn't used) or require per-camera rings (rejected in §6 intro).</p> If quotas are ever needed, the correct mechanism is upstream: rate-limit the chatty camera at the ingestor, or give it its own device.</p> 7. VFS namespace#</span></a> </h2> /mnt/eye/</span></span> ├── .super (ro; superblock view, text)</span></span> ├── all/</span></span> │ └── by-seq/</span></span> │ └── 00000000000001847.h265 (global cross-camera, by global_seq)</span></span> └── cameras/</span></span> ├── 001/ (camera_id)</span></span> │ ├── info (ro; registry slot as text)</span></span> │ ├── chunks/</span></span> │ │ ├── 00000000000001847.h265 (named by global_seq)</span></span> │ │ └── 00000000000001851.h265</span></span> │ ├── by-time/</span></span> │ │ └── 2026-04-23T14:30:00.000Z (lookup-resolved symlink)</span></span> │ └── current (tail-able; grows as kernel writes)</span></span> ├── 002/</span></span> │ └── …</span></span> └── …</span></span></code></pre> Properties:</p> Chunk files are read-only, mmap</code>-able, and sendfile</code>/splice</code>-eligible.</li> by-time/</code> is a synthetic directory: readdir</code> enumerates known chunk wall-clock prefixes, lookup(ts)</code> returns the nearest chunk. Uses the kernel's dcache for negative caching.</li> current</code> is the only writable file and only the kernel-owned write side can append to it. Reads return the bytes written so far. poll(POLLIN)</code> wakes on new data. fstat.st_size</code> grows monotonically until the next flush, then resets (the new current</code> is the next in-progress chunk).</li> inotify(IN_CREATE)</code> on chunks/</code> fires when a chunk is flushed.</li> all/by-seq/</code> is a flat merged view, useful for archival scripts that don't care which camera.</li> </ul> Symlink targets across the synthetic trees are resolved through the same in-memory index; there is no persistent symlink data on disk.</p> 8. Sysfs interface#</span></a> </h2> 8.1 Per-device tree#</span></a> </h3> /sys/fs/eyeFS/<device_uuid>/</span></span> ├── device → ../../block/sdX (symlink)</span></span> ├── info/</span></span> │ ├── format_version</span></span> │ ├── capacity_bytes</span></span> │ ├── ring_start</span></span> │ ├── created_at</span></span> │ └── logical_sector_size</span></span> ├── state/</span></span> │ ├── write_head</span></span> │ ├── wrap_count</span></span> │ ├── oldest_global_seq</span></span> │ ├── newest_global_seq</span></span> │ ├── oldest_wall_clock</span></span> │ ├── newest_wall_clock</span></span> │ ├── chunks_live</span></span> │ └── bytes_live</span></span> ├── config/</span></span> │ ├── nominal_chunk_size (rw)</span></span> │ ├── min_chunk_size (rw)</span></span> │ ├── fsync_mode (rw; per_chunk \| batched:N)</span></span> │ ├── align_to_iu (rw)</span></span> │ └── paused (rw)</span></span> ├── stats/</span></span> │ ├── chunks_written</span></span> │ ├── chunks_evicted</span></span> │ ├── bytes_written</span></span> │ ├── wrap_tail_wasted</span></span> │ ├── fsync_latency_ns_p50</span></span> │ ├── fsync_latency_ns_p99</span></span> │ ├── scanner_runs</span></span> │ └── torn_chunks_recovered</span></span> ├── actions/ (write-only triggers)</span></span> │ ├── rebuild_index</span></span> │ ├── checkpoint_now</span></span> │ ├── force_cut (cut all cameras' current chunks immediately)</span></span> │ └── cameras/</span></span> │ ├── add (write: "<rtsp_url> <display_name>")</span></span> │ └── remove (write: <camera_id>)</span></span> └── cameras/</span></span> ├── 001/</span></span> │ ├── info/</span></span> │ │ ├── uuid</span></span> │ │ ├── display_name</span></span> │ │ ├── added_at</span></span> │ │ ├── codec</span></span> │ │ └── rtsp_url (stored for the ingestor; not used by FS)</span></span> │ ├── state/</span></span> │ │ ├── chunks_live</span></span> │ │ ├── bytes_live</span></span> │ │ ├── oldest_wall_clock</span></span> │ │ ├── newest_wall_clock</span></span> │ │ ├── last_keyframe_ns</span></span> │ │ └── online (0 = ingestor hasn't written recently)</span></span> │ └── stats/</span></span> │ ├── chunks_written</span></span> │ ├── bytes_written</span></span> │ ├── access_units_received</span></span> │ └── gaps_ns (total time with no data)</span></span> └── 002/…</span></span></code></pre>8.2 Debugfs (dev-only, not ABI)#</span></a> </h3> /sys/kernel/debug/eyeFS/<uuid>/</span></span> ├── chunks (one line per live chunk: global_seq cam_id off len pts wc)</span></span> ├── wal_tail (pending index WAL records)</span></span> ├── scan_trace (last rebuild scan, per-chunk timings)</span></span> └── slowlog (flushes > threshold)</span></span></code></pre> Debugfs is gated by CONFIG_EYEFS_DEBUG</code> and is not part of the stable interface.</p> 9. Userspace ingestor split#</span></a> </h2> RTSP is a TLS-capable, auth-bearing, vendor-quirky text protocol with reconnect logic that varies per camera firmware. It does not belong in the kernel. eyeFS splits the pipeline cleanly:</p> Userspace</strong> (electricEye-ingestor</code>, one systemd service per running daemon): maintains RTSP sessions, depacketizes RTP, assembles access units, hands them to the kernel.</li> Kernel</strong> (eyeFS): per-camera accumulator, chunk framing, circular write, VFS/sysfs surfaces, crash recovery.</li> </ul> The interface between them is the per-camera write side. Two viable shapes:</p> Shape A: write to cameras/<id>/current</code>.</strong> The ingestor opens the file O_WRONLY and write(2)</code>s framed access units. Framing is length-prefixed Annex B access units plus a small header carrying PTS and NAL flags. Simple, but conflates data and metadata on one fd.</li> Shape B: per-camera character device</strong> (/dev/eyeFS-<uuid>-<camera_id></code>) with a structured protocol: an ioctl registers codec params, then write(2)</code> calls carry (pts, flags, annex_b_bytes)</code> records. Cleaner separation; mirrors how other media subsystems handle this (v4l2).</li> </ul> Recommended: Shape B.</strong> The write-side is a private interface between the ingestor and the kernel; it does not need to be POSIX-file-shaped. Shape A would force the current</code> file to simultaneously serve "readable live stream" and "writable ingestion target," which couples unrelated concerns. With Shape B, current</code> is read-only to everyone including the ingestor.</p> The ingestor's total job: open RTSP, depacketize, call write</code> per access unit. No storage state, no retention logic, no index. It is a small Go program, drawn largely from the existing electricEye</code> codebase (src/ingest.go</code>), with the astiav/MKV muxer path removed.</p> 10. Write path#</span></a> </h2> Per-camera accumulation state (in kernel):</p> struct eyefs_camera_writer {</span></span> u16 camera_id;</span></span> struct list_head nal_buf; // accumulated access units</span></span> u64 buf_bytes;</span></span> u64 last_idr_off;</span></span> i64 pts_first, pts_last;</span></span> spinlock_t lock;</span></span> };</span></span></code></pre> Per-device shared state:</p> struct eyefs_device {</span></span> struct block_device bdev;</span></span> spinlock_t head_lock;</span></span> u64 write_head;</span></span> u64 global_seq;</span></span> struct eyefs_camera_writer cams[MAX_CAMERAS];</span></span> …</span></span> };</span></span></code></pre>10.1 Per access unit#</span></a> </h3> Ingestor write(2)</code>s (pts, flags, bytes)</code> on the per-camera chardev.</li> eyeFS takes cam->lock</code>, appends NAL bytes, updates buf_bytes</code>, updates pts_last</code>.</li> If the access unit is an IDR and buf_bytes ≥ nominal_chunk_sz</code>: trigger flush (§10.2) using the buffer up to but not including this IDR, then append this IDR as the start of the next buffer.</li> Release cam->lock</code>.</li> </ol> Keyframe detection is a two-byte check on the NAL header (nal_unit_type ∈ {19, 20} for IDR). Same as the userspace design.</p> 10.2 Flush#</span></a> </h3> Build sidecar SEI (global_seq</code> is stamped inside the critical section below).</li> Take device->head_lock</code>.</li> Assign global_seq = device->global_seq++</code>.</li> Compute write offset; handle wrap if chunk wouldn't fit before ring_end</code> (§10.3).</li> Compute the eviction set (all live chunks whose byte ranges intersect the new chunk's range, wrap-aware).</li> Issue submit_bio</code> for the chunk data at the write head.</li> Release device->head_lock</code>.</li> On I/O completion (callback): fdatasync</code>-equivalent barrier, then append index WAL records (EVICT ×N, ADD ×1) on the NVMe-side index file, fsync WAL, update in-memory index, create the new VFS dentry under chunks/</code>.</li> </ol> The head_lock</code> is held only long enough to stamp seq, compute offset+eviction set, and submit the bio. It is not held across I/O completion.</p> 10.3 Wrap#</span></a> </h3> If write_head + chunk_len > ring_end</code>:</p> If ring_end - write_head ≥ min_chunk_sz</code>: cut the flush candidate at its last-fitting IDR, write the shorter terminal chunk at write_head</code>, spill the remainder into the next chunk which starts at ring_start</code>.</li> If ring_end - write_head < min_chunk_sz</code>: abandon those bytes, wrap write_head</code> to ring_start</code>, write the chunk there.</li> </ul> The only wasted space on the device is at most one min_chunk_sz</code> per wrap cycle (default 16 MiB ≈ 1 ppm on a 10 TB device).</p> 10.4 Ordering#</span></a> </h3> bio complete → device barrier → WAL append → WAL fsync → in-memory index update</span></span></code></pre> Inverting any pair creates a window where the index advertises data that isn't durable, or the index stays stale for a durable chunk. The former is silent corruption; the latter is recoverable by rescan (§12).</p> 10.5 fsync mode#</span></a> </h3> per_chunk</code> (default): one barrier per flush. Cheap on NVMe, acceptable on HDD (~tens of ms amortized over minutes of footage).</li> batched:N</code>: barrier every N chunks. Widens the durability window but is safe: the rescan path finds and recovers any unindexed chunks on restart.</li> </ul> 11. Read path#</span></a> </h2> 11.1 Static chunks#</span></a> </h3> open</code> a chunk file → kernel looks up the chunk in the in-memory index → returns an inode whose data ops do bounded pread</code> against the block device within [offset, offset+length)</code>. Read-ahead, page cache, mmap</code>, sendfile</code>, splice</code> all work via standard VFS plumbing.</p> 11.2 current</code> (live)#</span></a> </h3> open</code> the per-camera current</code> → kernel returns an inode backed by the active accumulator (not yet on disk). read</code> returns bytes up to the current accumulator length. poll(POLLIN)</code> wakes on new access-unit append. When a flush happens, the in-progress chunk becomes a regular file in chunks/</code> and a new current</code> begins; consumers receive POLLHUP</code> (or a rename-style event) and reopen.</p> This is the killer feature: ffmpeg -i /mnt/eye/cameras/001/current -c copy out.mkv</code> is a live tap, no daemon in the middle.</p> 11.3 Time-range queries#</span></a> </h3> /mnt/eye/cameras/<id>/by-time/2026-04-23T14:30:00Z</code> → kernel looks up the nearest chunk with wall_clock_start ≤ T</code> → returns a symlink to that chunk's file. A time-range query is a readdir</code> of by-time/</code> bounded by T1..T2 and sequential cat</code> of the resolved chunks. Because every chunk begins with VPS/SPS/PPS/IDR, concatenation produces a valid Annex B stream.</p> 11.4 Cross-camera#</span></a> </h3> /mnt/eye/all/by-seq/</code> gives a global view indexed by global_seq</code>. Useful for archival ("copy everything from global_seq 100000 to 200000 to this tarball") and for tools that don't need per-camera separation.</p> 12. Crash consistency#</span></a> </h2> Three failure modes:</p> 12.1 Daemon dies mid-access-unit (ingestor side)#</span></a> </h3> Accumulator holds a partial AU. Kernel discards it on next IDR. No on-device state changes. Safe.</p> 12.2 Kernel panic with unflushed accumulators#</span></a> </h3> Accumulators are in RAM and lost. Last durable state is the last flushed chunk. Next boot: mount, replay index WAL, start accumulating again. Up to one nominal_chunk_sz</code> of footage per camera lost — by construction, since that's what the accumulator is.</p> 12.3 Power loss between bio completion and WAL fsync#</span></a> </h3> Device holds a chunk with a valid sidecar SEI that the index doesn't know about. Next mount:</p> Read superblock's last_write_head</code> and last_global_seq</code>.</li> Scan the ring forward from last_write_head</code>. At each position, read one logical block, look for the eyeFS sidecar SEI UUID. If found, validate CRC, read chunk_length</code>, advance by that amount, repeat. Each valid chunk found is ADDed to the WAL.</li> Stop when either an invalid SEI is hit (torn tail of a partial write — the prior chunk was the last durable one) or when global_seq</code> goes backward (we've walked past the tail into previous-wrap residue).</li> Update superblock's last_write_head</code> and last_global_seq</code>, resume.</li> </ol> Bounded by (current head − last checkpoint) which is controlled by the checkpoint cadence (default every 10,000 chunks or 5 min, whichever first). Worst case scan on unbounded skew is sequential read at device bandwidth — §12.4.</p> 12.4 Full rescan#</span></a> </h3> If the index is lost entirely (NVMe failure, operator wipe):</p> mount -t eyefs -o rebuild_index /dev/sdX /mnt/eye</span></span></code></pre> The kernel scans the entire ring, finds every valid sidecar SEI, and reconstructs the index from scratch. Per-camera sequence numbers are derived by counting per-camera chunks in global_seq</code> order.</p> Cost: sequential read at device bandwidth. 500 MB/s on a modern device → ~5.5 hours for 10 TB. Pure sequential I/O, CPU-negligible.</p> 12.5 Partial-wrap ambiguity#</span></a> </h3> After the ring has wrapped, the device contains current-cycle chunks followed by trailing residue from the previous cycle (the tail of an overwritten old chunk that wasn't fully clobbered by the most recent write). The scanner distinguishes them by global_seq</code> monotonicity: old residue has strictly smaller global_seq</code> than adjacent current-cycle chunks. Anything preceding a sequence-number discontinuity is discarded.</p> 12.6 Registry consistency#</span></a> </h3> The camera registry is updated out-of-band from chunk writes; a power loss mid-registry-update could leave a half-written slot. Each slot carries its own CRC; torn slots are treated as FREE on mount and logged. An active camera whose registry slot is invalidated this way will be re-added by the ingestor on next start (same RTSP URL, new camera_id</code>), at the cost of its prior chunks becoming orphans (still present on-device, reachable via all/by-seq/</code> but not under a named camera). This is tolerable; the operator recovers with a manual registry repair tool if it matters.</p> 13. Eviction semantics#</span></a> </h2> New chunk at [write_head, write_head+len)</code> evicts every live chunk whose byte range intersects that interval (wrap-aware). Variable chunk sizes mean 1..N evictions per write.</p> Consequence in the multi-feed world: eviction is global FIFO by write-time</strong>, not per-camera FIFO. The oldest live chunk is the oldest by wall_clock_ns</code>, not the oldest of any particular camera. All cameras share the retention window.</p> Transient dead region: if a new chunk is smaller than the oldest it overwrites, the residual bytes of the old chunk sit unreachable until the next</em> chunk comes through. Bounded by one chunk size at any moment.</p> 14. Mount options#</span></a> </h2> mount -t eyefs \</span></span> -o chunk_size=128M,min_chunk_size=16M,fsync_mode=per_chunk,\</span></span> align_to_iu=0,max_cameras=128,index_path=/var/lib/eyefs \</span></span> /dev/sdX /mnt/eye</span></span></code></pre>Option</th> Default</th> Notes</th></tr></thead> chunk_size</code></td> 128M</td> nominal flush target</td></tr> min_chunk_size</code></td> 16M</td> wrap discard threshold</td></tr> fsync_mode</code></td> per_chunk</code></td> per_chunk</code> or batched:N</code></td></tr> align_to_iu</code></td> 0</code></td> align chunk starts to indirection unit (ZNS/NVMe)</td></tr> max_cameras</code></td> 128</td> registry slot count (must match format-time)</td></tr> index_path</code></td> /var/lib/eyefs</code></td> NVMe-side index directory</td></tr> rebuild_index</code></td> 0</code></td> force full rescan at mount</td></tr> </tbody></table> Options marked format-time (e.g. max_cameras</code>) are set by mkfs.eyeFS</code> and cannot be changed at mount.</p> 15. Tradeoffs#</span></a> </h2> Zipper-merge vs per-camera rings.</strong> Chosen: zipper. Simpler, uniform retention, dynamic add/remove is trivial. Cost: no per-camera bandwidth isolation — a chatty camera shortens everyone's retention.</li> Chunk size vs camera count.</strong> At N cameras and nominal chunk size C, the total chunk rate is N × (bitrate/C). At 32 cameras × 10 Mbps / 128 MiB ≈ one chunk every 3.3 seconds. Index size scales linearly; still negligible (5 MiB per 10 TB at default C, 2.5 MiB at 256 MiB C).</li> Synchronized log-time simplification.</strong> Rules out per-camera retention policies ("keep camera A for 90 days, camera B for 7 days"). If ever needed, the right answer is two devices — one per retention class — not one device with two retention policies.</li> Kernel vs userspace.The kernel path is justified once you decide to own the storage path; the only reason to stay in userspace is to avoid kernel development, and this user has explicitly discounted that reason.</li> Writing RTSP in userspace.</strong> Not a tradeoff — it's the only sane place. Kernel-space TLS and auth and vendor quirks are a non-starter.</li> </ul> 16. Open questions#</span></a> </h2> ZNS as a first-class backend.</strong> Natural fit; a zoned-mode eyeFS uses one zone per chunk (or one zone per N chunks), and wrap becomes zone-reset. Worth a follow-up paper.</li> Per-camera priority.</strong> Not implemented; if needed, the mechanism is upstream at the ingestor (rate-limit) or downstream (separate devices).</li> Encryption at rest.</strong> dm-crypt underneath eyeFS is the no-brain answer. Per-chunk encryption with a scannable outer envelope is possible but deferred.</li> Hot-swap of block device.</strong> The superblock carries a UUID; the mount binds to a specific UUID. Unplug/replug with the same UUID should resume; mismatched UUID refuses. Behavior under degraded states (I/O errors mid-flush) needs a full write-up.</li> Multi-device striping / mirroring.</strong> Out of scope for eyeFS itself; push to md/dm below.</li> Registry slot reuse after full wrap.</strong> Mechanism exists (slot is reclaimable once all its chunks are overwritten); edge cases around camera_id recycling deserve care to avoid confusing archival tools that cached old associations.</li> Upstreaming.</strong> eyeFS is narrow enough that upstream inclusion is unlikely to be welcomed without a larger ZNS story. Out-of-tree is fine; DKMS via the NixOS monorepo handles distribution.</li> </ul> 17. Summary#</span></a> </h2> eyeFS is a kernel filesystem that treats a block device as a circular log of H.265 chunks and exposes it through standard VFS and sysfs surfaces. Multiple cameras share one ring via zipper-merge on a single write head, tagged by camera_id</code> in a decoder-safe sidecar SEI, dynamically added and removed at runtime, with uniform retention across feeds. The userspace surface is what users expect from a filesystem — chunks are files, current</code> tails live, sendfile</code> streams to sockets, inotify</code> fires on new chunks, NFS exports the whole thing. The kernel surface is what an operator expects from a tunable block-device subsystem — sysfs for state and config, debugfs for diagnostics, mount options for format-compatible defaults. RTSP stays in userspace, as it should.</p> Alpine Linux on arm install Part 3 Unknown — Thu, 26 Mar 2026 00:00:00 +0000 Step 3: Write U-Boot and Install Alpine Rootfs#</span></a> </h2> Overview#</span></a> </h2> Flash u-boot to the SD card, then mount the partitions and extract the Alpine Linux aarch64 rootfs.</p> Write u-boot to the SD card#</span></a> </h2> Sunxi (Allwinner) boards expect u-boot at the 8KB offset:</p> sudo dd if=./result/u-boot-sunxi-with-spl.bin of=/dev/sdd bs=1024 seek=8</span></span></code></pre>Mount the partitions#</span></a> </h2> sudo mkdir -p /mnt/sdcard/{boot,root}</span></span> sudo mount /dev/sdd1 /mnt/sdcard/boot</span></span> sudo mount /dev/sdd2 /mnt/sdcard/root</span></span></code></pre>Download and extract Alpine rootfs#</span></a> </h2> The Orange Pi Zero 2W is AArch64 (Allwinner H618), so use the aarch64 u-boot tarball:</p> wget https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/aarch64/alpine-uboot-3.21.3-aarch64.tar.gz</span></span></code></pre> Extract to the root partition:</p> sudo tar -xzf alpine-uboot-3.21.3-aarch64.tar.gz -C /mnt/sdcard/root</span></span></code></pre>Populate the boot partition#</span></a> </h2> Copy boot files (kernel, dtbs, extlinux config) to the FAT32 boot partition:</p> sudo cp -r /mnt/sdcard/root/boot/* /mnt/sdcard/boot/</span></span></code></pre> Verify that the DTB sun50i-h618-orangepi-zero2w.dtb</code> is present in the boot partition and referenced correctly in extlinux/extlinux.conf</code>.</p> Unmount#</span></a> </h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</span></span></code></pre> Alpine Linux on arm install Part 3 Unknown — Thu, 26 Mar 2026 00:00:00 +0000 Step 3: Write U-Boot and Install Alpine Rootfs</h2> Overview</h2> Flash u-boot to the SD card, then mount the partitions and extract the Alpine Linux aarch64 rootfs.</p> Write u-boot to the SD card</h2> Sunxi (Allwinner) boards expect u-boot at the 8KB offset:</p> sudo dd if=./result/u-boot-sunxi-with-spl.bin of=/dev/sdd bs=1024 seek=8</span></span></code></pre>Mount the partitions</h2> sudo mkdir -p /mnt/sdcard/{boot,root}</span></span> sudo mount /dev/sdd1 /mnt/sdcard/boot</span></span> sudo mount /dev/sdd2 /mnt/sdcard/root</span></span></code></pre>Download and extract Alpine rootfs</h2> The Orange Pi Zero 2W is AArch64 (Allwinner H618), so use the aarch64 u-boot tarball:</p> wget https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/aarch64/alpine-uboot-3.21.3-aarch64.tar.gz</span></span></code></pre> Extract to the root partition:</p> sudo tar -xzf alpine-uboot-3.21.3-aarch64.tar.gz -C /mnt/sdcard/root</span></span></code></pre>Populate the boot partition</h2> Copy boot files (kernel, dtbs, extlinux config) to the FAT32 boot partition:</p> sudo cp -r /mnt/sdcard/root/boot/* /mnt/sdcard/boot/</span></span></code></pre> Verify that the DTB sun50i-h618-orangepi-zero2w.dtb</code> is present in the boot partition and referenced correctly in extlinux/extlinux.conf</code>.</p> Unmount</h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</span></span></code></pre> Alpine Linux on arm install Part 2 Unknown — Wed, 18 Mar 2026 00:00:00 +0000 Step 2: Get U-Boot#</span></a> </h2> Overview#</span></a> </h2> Build u-boot for the Orange Pi Zero using Nix's cross-compilation infrastructure.</p> Build u-boot with Nix#</span></a> </h2> nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.ubootOrangePiZero2</span></span></code></pre> The resulting u-boot-sunxi-with-spl.bin</code> will be in ./result/</code>.</p> Notes from the NixOS U-Boot Wiki#</span></a> </h2> Reference: https://wiki.nixos.org/wiki/U-Boot</p> Cross-compilation package sets#</span></a> </h3> Nix provides cross-compilation package sets for ARM targets:</p> Target Architecture</th> Package Set</th></tr></thead> AArch64</td> aarch64-multiplatform</code></td></tr> ARMv7l</td> armv7l-hf-multiplatform</code></td></tr> ARMv6l</td> raspberryPi</code></td></tr> </tbody></table> General command format#</span></a> </h3> nix-build -A [pkgsCross.$cross_name.]$packageName</span></span></code></pre> When building from an x86_64 host for an ARM board, use the pkgsCross</code> prefix. On a native AArch64 system, the prefix can be omitted:</p> nix-build -A ubootOrangePiZero2</span></span></code></pre>Finding available u-boot packages#</span></a> </h3> To find board-specific u-boot packages in nixpkgs, search for uboot</code> in the package set:</p> nix-env -qaP -A nixpkgs.pkgsCross.aarch64-multiplatform \| grep -i uboot</span></span></code></pre> For board defconfigs and SD card write offsets, consult the official U-Boot documentation or board-specific pages, as the large variety of ecosystems makes a single guide impractical.</p> Alpine Linux on arm install Part 2 Unknown — Wed, 18 Mar 2026 00:00:00 +0000 Step 2: Get U-Boot</h2> Overview</h2> Build u-boot for the Orange Pi Zero using Nix's cross-compilation infrastructure.</p> Build u-boot with Nix</h2> nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.ubootOrangePiZero2</span></span></code></pre> The resulting u-boot-sunxi-with-spl.bin</code> will be in ./result/</code>.</p> Notes from the NixOS U-Boot Wiki</h2> Reference: https://wiki.nixos.org/wiki/U-Boot</p> Cross-compilation package sets</h3> Nix provides cross-compilation package sets for ARM targets:</p> Target Architecture</th> Package Set</th></tr></thead> AArch64</td> aarch64-multiplatform</code></td></tr> ARMv7l</td> armv7l-hf-multiplatform</code></td></tr> ARMv6l</td> raspberryPi</code></td></tr> </tbody></table> General command format</h3> nix-build -A [pkgsCross.$cross_name.]$packageName</span></span></code></pre> When building from an x86_64 host for an ARM board, use the pkgsCross</code> prefix. On a native AArch64 system, the prefix can be omitted:</p> nix-build -A ubootOrangePiZero2</span></span></code></pre>Finding available u-boot packages</h3> To find board-specific u-boot packages in nixpkgs, search for uboot</code> in the package set:</p> nix-env -qaP -A nixpkgs.pkgsCross.aarch64-multiplatform \| grep -i uboot</span></span></code></pre> For board defconfigs and SD card write offsets, consult the official U-Boot documentation or board-specific pages, as the large variety of ecosystems makes a single guide impractical.</p> Alpine Linux on arm install Part 1 Unknown — Sat, 07 Mar 2026 00:00:00 +0000 Step 1: Partition the SD Card#</span></a> </h1> Overview#</span></a> </h2> Partition an SD card for installing Alpine Linux on an Orange Pi Zero 2W (Allwinner H618, AArch64). The card needs two partitions: a FAT32 boot partition for u-boot and the kernel, and an ext4 root partition for the filesystem.</p> Identify the SD Card#</span></a> </h2> Find the SD card device with:</p> lsblk -d -o NAME,SIZE,MODEL,TRAN</span></span></code></pre> In this case, the device was /dev/sdd</code>.</p> Partition with sfdisk#</span></a> </h2> Use sfdisk</code> for scriptable partitioning:</p> echo '</span></span> label: dos</span></span> unit: sectors</span></span> </span> /dev/sdd1 : size=+512M, type=c, bootable</span></span> /dev/sdd2 : type=83</span></span> ' \| sudo sfdisk /dev/sdd</span></span></code></pre> This creates:</p> Partition 1</strong> — 512MB, FAT32 (LBA), bootable — boot partition (u-boot + kernel)</li> Partition 2</strong> — remaining space, Linux — root filesystem</li> </ul> Note:</strong> 256MB is too small for the boot partition — the Alpine kernel, modloop, and System.map won't fit. Use 512MB.</p> </blockquote> Format the Partitions#</span></a> </h2> sudo mkfs.vfat /dev/sdd1</span></span> sudo mkfs.ext4 /dev/sdd2</span></span></code></pre> Alpine Linux on arm install Part 1 Unknown — Sat, 07 Mar 2026 00:00:00 +0000 Step 1: Partition the SD Card</h1> Overview</h2> Partition an SD card for installing Alpine Linux on an Orange Pi Zero 2W (Allwinner H618, AArch64). The card needs two partitions: a FAT32 boot partition for u-boot and the kernel, and an ext4 root partition for the filesystem.</p> Identify the SD Card</h2> Find the SD card device with:</p> lsblk -d -o NAME,SIZE,MODEL,TRAN</span></span></code></pre> In this case, the device was /dev/sdd</code>.</p> Partition with sfdisk</h2> Use sfdisk</code> for scriptable partitioning:</p> echo '</span></span> label: dos</span></span> unit: sectors</span></span> </span> /dev/sdd1 : size=+512M, type=c, bootable</span></span> /dev/sdd2 : type=83</span></span> ' \| sudo sfdisk /dev/sdd</span></span></code></pre> This creates:</p> Partition 1</strong> — 512MB, FAT32 (LBA), bootable — boot partition (u-boot + kernel)</li> Partition 2</strong> — remaining space, Linux — root filesystem</li> </ul> Note:</strong> 256MB is too small for the boot partition — the Alpine kernel, modloop, and System.map won't fit. Use 512MB.</p> </blockquote> Format the Partitions</h2> sudo mkfs.vfat /dev/sdd1</span></span> sudo mkfs.ext4 /dev/sdd2</span></span></code></pre>}}}}}

There Are 2 Hard Things in Computer Science, and Sean Bean Said None of Them

Unknown — Sat, 15 Aug 2026 00:00:00 +0000

I had the quote stuck in my head all week and could not place it. What I had was roughly this:

There are only 2 problems in programming. 1. Naming things. 2. Cache invalidation. 3. Counting. </blockquote>
And I was fairly sure it had been read to me in a grave British voice, by Sean Bean, in Civilization V.
Every part of that is wrong, and the ways in which it is wrong turn out to be more interesting than the quote.
The actual line

There are only two hard things in Computer Science: cache invalidation and naming things. </blockquote>
Two things. Not three. The man who said it was Phil Karlton, and the reason you have never seen a citation is that there isn't one — he said it out loud, to people, and then he died.
Karlton is not a household name but his résumé is the back half of the twentieth century in computing: Xerox, Digital, Silicon Graphics, and finally Netscape, where his own home page</a> gives his title as Principal Curmudgeon. X11. OpenGL. Debuggers. Mail user agents. He was in the room for SSL 3.0 — you can still read him arguing about the transport layer on the IETF TLS list in 1996</a>. Every HTTPS connection you have ever made is downstream of meetings this man sat in.
On 20 June 1997 he and his wife Jan were killed in a car accident while on holiday in Italy. Their son David runs the site</a> that is now the closest thing the quote has to a primary source.
So the most-repeated sentence in software engineering has no paper. It is hearsay, all the way down.
The chain of custody

~1970, Carnegie Mellon. A commenter named Tom Bajzek says he heard Phil say it before an intramural game, back when they both belonged to the Friday Evening Volleyball Association. This is a guy on a blog, in 2017, recalling a remark from forty-seven years earlier.</li>
~1996–97. Tim Bray hears it from Karlton at Netscape and later writes it down on his blog — the earliest instance anyone has found online. Bray's own summary of his role is about as modest as it gets: Phil said it, I reported it, someone else added on the off-by-one.</li>
December 2005. Kellan Elliott-McCrea publishes a post whose title is just the quote, which is roughly the moment it stops being a thing Netscape people say and starts being a thing everyone says.</li>
July 2009. Martin Fowler gives it a bliki entry</a>, which is canonisation.</li>
December 2017. David Karlton writes Naming things is hard</a>, confirming his dad threw the line around more than once and that he's fairly confident it was original to him.</li> </ul>
That's it. That's the archive. The best corroboration for the volleyball story is that Karlton's own Netscape page, written by him, lists volleyball among his hobbies — beer, single malts, rock climbing, roller hockey, volleyball. The detail checks out even though the quote doesn't.
There is something perfect about this. The industry's favourite line about how hard it is to name things has spent thirty years unable to reliably name its own author.
The third oneThe version I had in my head — the one with the counting joke — is not Karlton's and was never meant to be mistaken for it. On New Year's Day 2010, Leon Bambrick — @secretGeek</a> — tweeted:

There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. </blockquote>
The joke is not the third item. The joke is the 2. The sentence promises two problems and delivers three, so the list demonstrates the bug it is describing while you are still reading it. It is a self-executing punchline, and it is better than the original — which is why it has almost entirely replaced it in casual use, and why my memory kept the shape of it while dropping every fact attached.
My reconstruction had even mangled that: I remembered the payload ("counting") rather than the mechanism ("off-by-one"), which is what you get when a brain compresses a joke down to its gist and throws away the executable part.
The format turned out to be extremely fertile. Mathias Verraes did the distributed-systems port, and it may be the best of the lot:

There are only two hard problems in distributed systems: 2. Exactly-once delivery. 1. Guaranteed order of messages. 2. Exactly-once delivery. </blockquote>
Out-of-order and delivered twice. The sentence is the failure mode.
Fowler's page also collects Phillip Scott Bowden's — there's two hard problems in computer science: we only have one joke and it's not funny — and Nat Pryce's observation that the sheer volume of variants is itself evidence that programming may not be as easy as everyone claims.
Now, Sean Bean#</a> </h2>
Here is where my memory was most confidently wrong.
Civilization has read historical quotes at you since the beginning, and the narrator is different in each game:
Game</th> Narrator</th></tr></thead>

Civilization IV</td> Leonard Nimoy</td></tr>
Civilization V</td> W. Morgan Sheppard</td></tr>
Civilization VI</td> Sean Bean</td></tr>
Civilization VII</td> Gwendoline Christie</td></tr> </tbody></table>
Sean Bean is Civ VI. Civ V — the one I was sure about — was W. Morgan Sheppard, the English character actor with the gravel in his voice, who did the tech quotes and the intro and died in 2019. Civ VII, in 2025, went to Gwendoline Christie, the first woman to narrate the series and the second Game of Thrones alumnus in a row, which is a hiring pattern I respect.
And the Karlton quote is in none of them. As far as I can tell it has never appeared in a Civilization game. What Sean Bean actually reads you when you finish Computers in Civ VI is:

To err is human, but to really foul things up you need a computer. — Paul R. Ehrlich </blockquote>

The good thing about computers is that they do what you tell them to do. The bad news is that they do what you tell them to do. — Ted Nelson </blockquote>
Same register. Same cadence. Same wry-aphorism-about-machines slot in the sentence. If you were going to falsely remember a programming joke as having been narrated to you over a tech-tree fanfare, these are exactly the neighbours that would make it feel true.
Why the misremembering is on-topic#</a> </h2>
Two decades of Civilization trained a generation of people who work with computers to receive pithy aphorisms in a serious British voice with strings under it. The Karlton line fits that slot so precisely — right length, right irony, right subject — that my brain filed it there and, when asked for a source, returned the one it had lying around.
Which is, and I'm aware of how neat this is, a cache invalidation bug. The aphorism was cached. Its provenance was evicted. Nothing signalled that the entry had gone stale, so the lookup succeeded and returned confident garbage: Sean Bean, Civ 5. No error. No miss. Just a plausible value served warm from a bad line.
Human memory is a cache with an excellent hit rate, no TTL, and no invalidation protocol whatsoever. It will hand you a wrong answer with exactly the same subjective confidence as a right one, which is the whole reason the original observation is any good: the hard problems aren't the ones that announce themselves by crashing.
So, corrected and with numbering intact:

There are only two hard things in Computer Science: cache invalidation and naming things. — Phil Karlton, out loud, sometime between 1970 and 1997, to somebody who was paying attention </blockquote>
Nobody read it to you. It doesn't need the voice.
Sources: Martin Fowler, Two Hard Things</a> · David Karlton, Naming things is hard</a> · karlton.org</a> · Phil Karlton's Netscape home page</a> · Kellan Elliott-McCrea (2005)</a> · PC Gamer on Sean Bean and Civ 6</a> · PC Gamer on Gwendoline Christie and Civ 7</a> · Civilization Wiki: Computers (Civ6)</a>

I Put Claude Code in a Cgroup Jail Because It Kept Eating My Workstation

Unknown — Fri, 31 Jul 2026 00:00:00 +0000

For two weeks, heracles — my main box, 128 GB of RAM, the machine I drive the whole fleet from — kept hanging solid and rebooting itself out from under me.
Not a crash you get to watch. A wedge. One second I'm working across a dozen terminals; the next the pointer won't move, the fans spin up like a jet on the apron, and every input I give the machine vanishes into a tarpit. Thirty seconds of that and the hardware watchdog gives up, yanks the reset line, and I'm looking at a login greeter with every tmux session, every browser tab, every train of thought I had going — gone.
It happened on the 13th. It happened again on the 26th. It happened on the 29th while I sat there and watched it die. Losing a day's context to a silent reboot is a special kind of rage, the kind that makes you want to physically fight a computer. So I stopped being angry at the symptom and went to find out what was actually killing my machine.
This is that story. It has a villain, and the villain is JavaScript. Obviously it's JavaScript.
The autopsy#</a> </h2>
Here's the thing about a machine that wedges hard: its own logs are useless, because the journal never gets flushed. heracles' on-disk journal just ends, mid-line, at the moment of death. So years ago I set up netconsole — the kernel screaming its dying words over UDP to a different box on the network before it loses the ability to do anything at all. ares catches heracles' last breath.
And heracles' last words, every single time, were the same:
oom-kill:constraint=CONSTRAINT_NONE,...,global_oom</code></pre> A global out-of-memory. The kernel ran out of RAM, invoked the OOM killer, and the OOM killer — in its infinite wisdom — started executing whatever. Not the thing eating the memory. Just... things. wireplumber. My emacs daemon. The accessibility bus. Tiny 30-megabyte desktop services, shot one after another, each death freeing nothing, while the actual gluttons sat there untouched and the box spiraled into a reclaim stall until the watchdog put it out of its misery. So: what filled 128 gigabytes? I ranked the processes from the OOM dump. The entire top of the list was one thing, over and over:7.36 GB .claude-unwrapped 7.35 GB .claude-unwrapped 7.35 GB .claude-unwrapped ... (twenty of them, five to seven gigs each)</code></pre> Twenty-odd Claude Code sessions, north of 100 GB combined. My AI assistant had eaten my workstation. "Bun Pool 5 invoked oom-killer"#</a> </h2> Now, a detail that made me tilt my head. The thread that actually tripped the final allocation was named Bun Pool 5</code>. Bun. As in the JavaScript runtime. I want to be precise about what I learned next, because it reframes everything. Claude Code is not a Node script. It's not a little CLI that shells out. Claude Code, as shipped, is a 257-megabyte single-file executable — the entire Bun runtime (v1.4.0) with JavaScriptCore baked in, plus the whole application bundle, bun build --compile</code>'d into one enormous ELF binary. Bun Pool 5</code> wasn't some side process. It was Claude Code itself. Those worker threads are the program. Which means every one of those 7-gigabyte lines was a single chat session's JavaScript heap. Seven gigabytes. For a terminal program. That mostly sits there waiting for me to type. The ratchet#</a> </h2> It gets worse, and this is the part that made me put my head on the desk. A few days later — after I'd started building the fix — I was mid-session, sitting at a prompt Claude had drawn for me, deciding how to answer. Hadn't typed anything in a while. And the session got killed. So I went and looked at what it had been holding at the moment of death. systemd logs it right there on the way out: claude-1654491.scope: Consumed 25min CPU over 3h 18min wall clock time, 21.5G memory peak.</code></pre> Twenty-one and a half gigabytes. Sitting idle. At a menu. For three hours. That one session was 83% of all the Claude memory on the box. A fresh session starts at about 400 megabytes. This thing had ratcheted to fifty times that and then just... held it, doing nothing, while I stared at a multiple-choice question. And "ratcheted" is exactly the right word, because this is the specific technical sin, and I want to name it precisely instead of just yelling "JavaScript bad." JavaScriptCore — like V8, like every one of these runtimes — allocates its heap in a big arena. When garbage collection runs, it frees your dead objects inside that arena. It reuses the space. What it does not do, essentially ever, is give the pages back to the operating system. So the arena is a one-way ratchet. The instant a session briefly needs 21 GB — you had it read a big file, or it captured a huge build log into its context, or you resumed a long conversation and it deserialized the whole transcript back into memory — the arena grows to fit. And then it stays that big. Forever. For the entire life of the process. The memory is "free" as far as the JS heap is concerned, and completely unavailable as far as the rest of the box is concerned. This is not a bug in Claude Code, exactly. It's the water the entire ecosystem swims in. It's what you get when an industry decides that the correct runtime for a systems tool — a thing that runs on my machine, in my resource budget, next to my real work — is a browser engine with a garbage collector that treats "give memory back" as somebody else's problem. Ship it as a 257 MB binary, let the heap grow without bound, and if the user's workstation falls over under ten concurrent copies, well, buy more RAM. I have 128 gigabytes. It wasn't enough, because the software was written by people for whom "it works on my machine" is the entire quality bar. I'm not going to fix their memory management. I can't; it's compiled into a quarter-gigabyte blob and there isn't even a heap-limit flag exposed. But I run NixOS, and I don't have to trust software. I can put it in a cage. Building the cage#</a> </h2> Here's the good news, and the reason I'll die on the NixOS hill: my entire fleet is one declarative repo. Every machine, every service, every dotfile, and — as of this saga — the exact resource jail every claude</code> process runs inside. I don't SSH into a box and tweak things. I change the source and rebuild. The first thing I tried was wrong, and I'm including it because pretending you got it right on the first go is how blogs lie to you. I added zram — compressed swap in RAM — figuring a swap cushion would let pressure build gracefully instead of slamming into a wall. Reader, it made the next crash worse. When your problem is a hot working set of live JavaScript heaps, compressing them into a different part of the same RAM buys you nothing and costs you CPU. At the next death, 56 GB of physical memory was consumed by zram's own compressed pages, on top of 60 GB of resident heaps, and the box thrashed itself to death faster than before. I ripped it back out. Wrong tool. Moving on. The fix that actually worked has three parts, and they're all just cgroups — the Linux kernel's resource-accounting jails — wired up declaratively. One: every Claude session runs in its own systemd scope with a hard memory cap. The wrapper launches claude</code> inside a transient --user</code> scope with MemoryMax=12G</code>. A single session that ratchets past 12 gigs now hits the wall of its own cgroup and gets OOM-killed inside its own jail — the kernel kills that one process tree and nothing else. The bloated session executes itself. Twelve gigs is still obscene for a chat client, but it's a number that lets ten of them coexist on this box, and a session that blows through it was going to hurt me anyway. Two: all of those scopes live under one parent claude.slice</code> with an aggregate cap — MemoryMax</code> at 66% of RAM. This is the one that actually addresses my real failure mode, which was never one runaway. It was twenty reasonable-looking sessions summing past 128 GB with no single one looking guilty. Now, all Claudes combined are bounded. When they collectively push the ceiling, a kill fires inside the slice — a Claude dies, never the compositor. Three: I un-inverted the OOM priorities. This was an embarrassing bug of my own making. I'd biased claude</code> toward being the preferred kill victim... but I'd set the bias too low. It turns out logind session services (wireplumber, emacs, the a11y bus) carry a higher OOM-priority by default than the timid value I'd picked, so the kernel would rather shoot a 30 MB audio daemon than a 7 GB Claude. That's exactly backwards, and it's why the desktop kept dying instead of the thing at fault. Fixed the numbers so a fat Claude is always first against the wall. Belt and suspenders on top: systemd-oomd watching memory-pressure (PSI) on the user slice and killing the worst offender from userspace before the kernel's blunt global killer ever wakes up, plus earlyoom as a dumb backstop. Did it work?#</a> </h2> The night I finished it, that idle 21.5 GB session I told you about got killed. And here is the entire point of two weeks of work, in the logs: kernel global-OOM events this boot: 0 systemd-oomd kill actions this boot: 1 -> claude-1654491.scope claude sessions still alive: 8 box reboots: 0</code></pre> One session died. The one holding 21.5 gigabytes for no reason. The other eight kept running. My desktop didn't even flicker. The machine that had hard-rebooted three times in two weeks took the exact same kind of memory pressure and handled it as a routine event — killed one recoverable chat session, which I brought right back with claude --resume</code>, and stayed up. Uptime held. Nothing got kicked to a greeter. I didn't lose a day. That's the difference between "the OS is a suggestion" and "the OS is in charge." The part where I make it a graph#</a> </h2> The last thing that bugged me: I only found the 21.5 GB session after it died, in a scope-exit line. I was blind to the ratchet while it was happening. So I wired Claude's cgroups into the Telegraf → InfluxDB → Grafana stack that already runs my fleet — every session's live memory, its high-water mark, its throttling events, sampled every 15 seconds, graphed next to everything else. Plus a little journal-scraper that turns every session teardown into an event so I can see how each one ended — clean exit, self-OOM'd on its own cap, or reaped by oomd — and backfills the peak history from those same scope-exit lines. Now I can watch the ratchet climb in real time. When some future Claude session starts marching toward 12 gigs while sitting at a prompt, I'll see the line go up and to the right, and I'll know exactly which JavaScript heap to be furious at. The lesson#</a> </h2> You are not going to fix the memory management of software you didn't write, especially not when it's shipped as a quarter-gigabyte compiled blob by an ecosystem that considers releasing memory back to the OS a quaint premodern concern. That fight is unwinnable and it's not yours. But you don't have to win it. The kernel has had the tools to contain a greedy process since forever — cgroups, memory limits, OOM scoring, PSI. The tragedy is how few people wire them up, because doing it by hand is fiddly and doesn't survive a reboot. That's the whole reason I build my machines the way I do: declaratively, in one repo, where "every claude</code> runs in a 12-gig jail under a slice capped at two-thirds of RAM, and here's the exact OOM priority of every service" is fifteen lines of Nix that are true on every boot forever. I can't make the JavaScript behave. But I can build a cage strong enough that it doesn't matter, and I can prove the cage holds, and I can graph the animal throwing itself against the bars. Now if you'll excuse me, I have a chat session at 11 gigs and climbing.
Booting Back Out of a Bad Timeshift Snapshot (Pop!_OS + BTRFS) Unknown — Fri, 26 Jun 2026 00:00:00 +0000 I upgraded a Pop!_OS box from 22.04 LTS to 24.04 LTS, hit a snag, and reached for Timeshift to "restore" an older snapshot. It booted — into the old system — and then every subsequent boot dropped me into the systemd emergency shell. systemd-modules-load.service</code> was failing with "Failed to look up module alias … Function not implemented" (ENOSYS), vfat</code> wouldn't load, and so /boot/efi</code> couldn't mount. One illness, several symptoms. The root cause: Timeshift's BTRFS restore doesn't roll back in place — it swaps subvolumes, so the old snapshot became my live root (@</code>). The booted system's /lib/modules</code> no longer matched the running kernel, which is why module loading died and the FAT EFI partition couldn't be mounted. Below is the full recovery, in fish-shell syntax, from the emergency shell back to a clean 24.04 boot. Shell note: the systemd emergency shell is usually bash</code>/sh</code>. To use these exact commands, start fish first with fish</code> (if the binary is present). The only fish-specific bits are variable handling (set NAME value</code> instead of NAME=value</code>, and command substitution with (...)</code> instead of $(...)</code>); everything else is identical to bash. </blockquote> How BTRFS "chooses" what to boot#</a> </h2> BTRFS doesn't choose — the bootloader and mount options do: The GRUB kernel line carries root=UUID=<fs-uuid> ... rootflags=subvol=@</code>. That subvol=@</code> selects which subvolume becomes /</code>.</li> With no subvol=</code>, BTRFS falls back to the filesystem's default subvolume (btrfs subvolume get-default</code>, normally subvolid 5 = the top level).</li> After root mounts, the /etc/fstab</code> inside that subvolume re-asserts subvol=@</code>.</li> </ol> On Ubuntu/Pop the live system is the subvolume named @</code> (home is @home</code>). The fix is to put your good 24.04 system back at @</code> so the existing subvol=@</code> boot entry lands on the right system. Timeshift snapshots are read-only, so you can't boot one directly — you create a fresh writable @</code> from the snapshot. Step 1 — Identify the BTRFS device#</a> </h2> btrfs filesystem show lsblk -f</code></pre> Note the device for your root filesystem (e.g. /dev/nvme0n1p3</code>). If it's LUKS-encrypted, unlock it first: cryptsetup luksOpen /dev/nvme0n1pX cryptroot # then use /dev/mapper/cryptroot</code></pre>Step 2 — Mount the top-level subvolume (subvolid=5)#</a> </h2> This exposes every subvolume, not just the currently-booted one. mkdir -p /mnt/top mount -o subvolid=5 /dev/nvme0n1p3 /mnt/top ls -la /mnt/top</code></pre>Step 3 — List all subvolumes#</a> </h2> btrfs subvolume list /mnt/top</code></pre> You'll see your live @</code> and @home</code>, plus Timeshift snapshots under timeshift-btrfs/snapshots/<timestamp>/@</code> and .../@home</code>. Step 4 — Pick the snapshot and VERIFY it's the 24.04 system#</a> </h2> Don't trust names — check the contents before changing anything. In fish, set the variable with set</code> and use (...)</code> for any command substitution: ls /mnt/top/timeshift-btrfs/snapshots/ set SNAP /mnt/top/timeshift-btrfs/snapshots/2026-06-19_XX-XX-XX # your real timestamp grep VERSION= $SNAP/@/etc/os-release # expect VERSION="24.04..." ls $SNAP/@/lib/modules # expect a 24.x kernel directory</code></pre> Or loop over every candidate subvolume at once (fish for ... end</code>): for s in /mnt/top/@* echo "=== $s ===" grep VERSION= $s/etc/os-release ls $s/lib/modules end</code></pre> Proceed only once you've confirmed which subvolume is the 24.04 system. Step 5 — Move the broken subvolumes aside (rename, never delete)#</a> </h2> cd /mnt/top mv @ @broken-2204 mv @home @home-broken-2204</code></pre> Keeping them (renamed) makes this whole operation reversible. Step 6 — Create new WRITABLE subvolumes from the snapshot#</a> </h2> btrfs subvolume snapshot $SNAP/@ /mnt/top/@ btrfs subvolume snapshot $SNAP/@home /mnt/top/@home</code></pre> No -r</code> flag → these are read-write, which is what a root filesystem requires. The original snapshots stay untouched as backups. Note on @home</code>: only restore it if the snapshot's home is the version you want. If you've created nothing in /home</code> since the breakage, restoring it is fine. If unsure, skip the @home</code> lines and keep your current @home</code> — the system will still boot. </blockquote> Step 7 — Sanity-check the mount options#</a> </h2> The new @</code> came from a system that already used subvol=@</code>, so its /etc/fstab</code> and the GRUB entry should already agree. Quick confirm: grep -E 'subvol' /mnt/top/@/etc/fstab</code></pre>Step 8 — Reboot#</a> </h2> cd / umount /mnt/top reboot</code></pre> GRUB still points at subvol=@</code>, but @</code> is now your real 24.04 root with a kernel that matches its /lib/modules</code>. Module loading works → vfat</code> loads → /boot/efi</code> mounts → boot completes. Step 9 — Confirm success and clean up#</a> </h2> After a clean graphical/login boot: grep VERSION /etc/os-release # 24.04 systemctl --failed # should be empty findmnt /boot/efi # mounted uname -r; ls /lib/modules # kernel matches a modules dir</code></pre> Once you're confident, delete the leftovers: sudo btrfs subvolume delete /@broken-2204 sudo btrfs subvolume delete /@home-broken-2204</code></pre> (Mount subvolid=5 again if you need the top-level path to reach them.) Troubleshooting#</a> </h2> Drops right back to emergency: a unit is still failing. Run systemctl --failed</code> and journalctl -b -p err</code>. A failed *.mount</code> usually means a stale UUID/PARTUUID in /etc/fstab</code> — compare against blkid</code>. Add nofail</code> to non-critical mounts (like /boot/efi</code>, /recovery</code>) so a missing mount can't abort the whole boot.</li> "Function not implemented" on modprobe: confirms a kernel/modules mismatch — i.e. you're still on the wrong subvolume. Re-check Step 4.</li> GRUB itself is broken after the swap: boot a live USB, mount the top-level, mount --bind</code> /dev /proc /sys, chroot into @</code>, then update-grub && grub-install</code>.</li> Never run btrfs check --repair</code> first — use btrfs check --readonly</code> to diagnose; --repair</code> can worsen damage.</li> </ul> Recovery complete — you're back on 24.04. Two Family Trees: systemd Targets and Slices Unknown — Thu, 25 Jun 2026 00:00:00 +0000 I went looking for a way to group a handful of my own systemd units — my sync timers, a mail fetcher, a couple of personal daemons — so I could see them and operate on them as a set instead of squinting at the full systemctl</code> output. Two answers kept coming up: targets and slices. They both "group units." They both show up in the same listings. And for an embarrassing minute I couldn't have told you the difference between them. So I went and read the history. It turns out the reason they feel like overlapping tools is an illusion of proximity: they live next to each other in systemctl</code> today, but they were born in different decades, at different companies, to solve completely unrelated problems. Targets are the descendants of init runlevels. Slices are the descendants of control groups. They only look like siblings because systemd dresses every unit type in the same uniform. Here's the one sentence to hold onto before we dig in: A target is a node in a dependency graph. A slice is a node in a cgroup tree. </blockquote> One answers "what should be running, and in what order?" The other answers "whose resources are these, and what are the limits?" Those are orthogonal questions, and once you see that, the whole model snaps into focus. Let me trace each family tree. Family tree #1: runlevels → targets#</a> </h2> The 1980s: a runlevel is a state#</a> </h3> The story starts with SysV init, the init system that came out of AT&T's UNIX System III in the early 1980s and carried into System V (hence "SysV"). Its central concept was the runlevel: a named configuration of the machine — a state you transition the system into, in which only a selected group of processes is allowed to run. The mapping of "what runs at which level" lived in /etc/inittab</code>, and init's entire job was to drag the machine into the requested state and keep the right processes alive. There were eight of them, 0–6 plus S</code>, and the ones at the edges were universal: 0 halt / poweroff 1 single-user (maintenance) S single-user (≈ 1) 6 reboot</code></pre> The interesting part is the middle, 2–5, because that's where the distros quietly disagreed — a fact that bites people to this day: Red Hat / Fedora: runlevel 3 was full multi-user with networking but a text console; runlevel 5 added the graphical X11 login. The famous "boot to a GUI vs boot to a console" toggle was literally 3</code> vs 5</code>.</li> Debian / Ubuntu: runlevels 2 through 5 were all identical — full multi-user mode, no text-vs-graphical distinction at all.</li> </ul> The mental model that matters: a runlevel is a state machine position. You are at a runlevel; you switch to another. It's about the global condition of the box, and which bundle of services that condition implies. The init wars (2005–2010)#</a> </h3> SysV init had an Achilles' heel: it ran service scripts sequentially, in a numbered order, each one blocking the next. On a machine with dozens of services, boot was a slow conga line. Two projects attacked this before systemd existed: launchd (Apple, 2005, shipped in Mac OS X 10.4 "Tiger"). One daemon to replace init, cron, inetd, and friends — and crucially, it started services on demand via socket activation rather than eagerly in a fixed order. Apple relicensed it under Apache 2.0 in 2006 to encourage adoption elsewhere.</li> Upstart (Canonical, 2006, by Scott James Remnant, first in Ubuntu 6.10). An event-based init: services reacted to emitted events ("a disk appeared", "a filesystem mounted") instead of marching through runlevel scripts.</li> </ul> Both were saying the same thing in different dialects: the rigid, sequential runlevel model is the wrong abstraction. April 30, 2010: "Rethinking PID 1"#</a> </h3> Then Lennart Poettering (at Red Hat), with design work from Kay Sievers (then at Novell), published an essay titled Rethinking PID 1</a> and announced systemd. The pitch: aggressive parallelization of boot, socket activation pushed to its limit (create all the listening sockets first, hand them to services, let everything start at once without hand-written ordering), D-Bus activation, cgroups to track processes (more on that in the second family tree), and — the relevant bit here — targets to replace runlevels. A target is systemd's answer to "what is a runlevel, really?" Stripped of the state-machine framing, a runlevel was just a name for a set of services that should be up together. So systemd made that explicit. From the systemd.target(5)</code> man page: Target units are used to group units and to set synchronization points for ordering dependencies with other unit files. […] Target units do not offer any additional functionality on top of the generic functionality provided by units. They merely group units. […] Target units provide a more flexible replacement for SysV runlevels in the classic SysV init system. </blockquote> Read that twice, because it's the whole thing: a target has no processes of its own, does nothing on its own, and exists purely as a named node in the dependency graph that other units can point at. The old runlevels are still there as compatibility aliases — and the mapping is hard-coded right in systemd's build files (units/meson.build</code>): runlevel0.target → poweroff.target runlevel1.target → rescue.target runlevel2/3/4.target → multi-user.target runlevel5.target → graphical.target runlevel6.target → reboot.target</code></pre> systemd's own FAQ doesn't mince words: "The concept of runlevels is obsolete. A set of target units are exposed that carry similar semantics, e.g. runlevel 5 -> graphical.target." Targets also do the job runlevels never could: they're synchronization points in a real dependency graph. Boot is a chain of them — sysinit.target</code> → basic.target</code> → multi-user.target</code> → graphical.target</code> — and units hook themselves in with WantedBy=</code>, Wants=</code>, Requires=</code>, After=</code>, Before=</code>. That last point matters for later: a single unit can be WantedBy</code> many targets at once. Membership in the dependency graph is many-to-many. You can see the graph directly: systemctl list-units --type=target # what's active systemctl list-dependencies graphical.target # the tree under a target</code></pre> So: runlevels → targets. The lineage is init, boot, state, ordering. Nothing about resources. Family tree #2: cgroups → slices#</a> </h2> Now rewind to a completely different room. 2006–2008: Google needs to box in processes#</a> </h3> Around 2006, engineers at Google — Paul Menage and Rohit Seth — were wrestling with enormous shared Linux clusters and built a kernel feature they called "process containers." The goal had nothing to do with booting: it was about hierarchically grouping processes so you could account for, limit, prioritize, and isolate their resource usage — CPU, memory, block I/O, and so on, each handled by a per-resource "controller." To avoid collision with the loaded word "container," it was renamed control groups — cgroups — in late 2007, and merged into the Linux kernel in 2.6.24 (January 2008). This is a kernel resource-management primitive. It knows nothing about services, runlevels, or boot order. It just lets you draw a box around a set of processes and say "this box gets at most 2 GB of RAM and 10% of one CPU." Why systemd cared: processes that won't stay caught#</a> </h3> Here's the bridge between the two family trees. Traditional Unix daemons double-fork to "daemonize" — detach from their parent and re-parent to PID That's deliberate, and it's a disaster for a service manager: once a daemon double-forks, the thing that launched it has lost track of it. PID-based supervision is racy and unreliable — PIDs get recycled, children scatter and re-parent to init, and you can never be sure you've found (or killed) all of a service's processes.</li> </ol> cgroups solved this cleanly, and it's exactly why "Rethinking PID 1" leaned on them. Put every service in its own cgroup, and the kernel maintains a reliable, race-free record of precisely which processes belong to that service, no matter how many times they fork. Poettering put it bluntly: Unless it is privileged and has access to the cgroup file system it cannot escape its group. </blockquote> That gives you accurate accounting and the ability to reliably kill an entire service — every last worker — by acting on the group. July 3, 2013 (v205): systemd grows slices and scopes#</a> </h3> For systemd's first few years it used cgroups internally for this tracking, but there was a problem: lots of programs (systemd, logind, libvirt, your shell) were all writing to the cgroup tree at once, stepping on each other. So in version 205 (released 2013-07-03), systemd introduced two new unit types and a new rule. From the NEWS file: Two new unit types have been introduced: Scope units are very similar to service units, however, are created out of pre-existing processes […] By using scope units it is possible for system services and applications to group their own child processes […] Slice units may be used to partition system resources in an hierarchical fashion and then assign other units to them. By default there are now three slices: system.slice</code> (for all system services), user.slice</code> (for all user sessions), machine.slice</code> (for VMs and containers). </blockquote> And the reason they appeared at all: Slices and scopes have been introduced primarily in context of the work to move cgroup handling to a single-writer scheme, where only PID 1 creates/removes/manages cgroups. </blockquote> That "single-writer" rule — only PID 1 touches the cgroup tree — is why modern systemd is the sole owner of cgroups on a Linux box. Everything else asks systemd. So what is a slice? From systemd.slice(5)</code>: A slice unit is a concept for hierarchically managing resources of a group of processes. This management is performed by creating a node in the Linux Control Group (cgroup) tree. Units that manage processes (primarily scope and service units) may be assigned to a specific slice. For each slice, certain resource limits may be set that apply to all processes of all units contained in that slice. </blockquote> A slice is a cgroup directory. Its name encodes its path in the tree: -.slice</code> is the root, and foo-bar.slice</code> sits inside foo.slice</code> inside -.slice</code>. The default layout every systemd box boots with: -.slice the root ├── system.slice all system services ├── user.slice all user sessions (each user gets user-UID.slice) └── machine.slice VMs and containers (via systemd-machined)</code></pre> The third unit type from v205, the scope, is the odd cousin: like a service, but for processes systemd didn't fork itself — login sessions, VMs, anything externally launched. Scopes and services are the leaves that hold actual processes; slices are the branches that hold them. And that leaf/branch split isn't a metaphor — it's a kernel rule. systemd's CONTROL_GROUP_INTERFACE.md</code> spells it out: Service, scope and slice units directly map to objects in the cgroup tree. […] Slices do not contain processes themselves, but the services and scopes contained in them do. </blockquote> Under cgroup v2's "no processes in inner nodes" rule, an inner node can't hold processes directly — so slices (inner nodes) never contain processes, only their service/scope children (leaf nodes) do. Crucially, a unit is placed in exactly one slice: the Slice=</code> directive is a single value, defaulting to system.slice</code>. Membership in the cgroup tree is one-to-one. The cgroup v2 detour#</a> </h3> One more thread, because it explains why old guides feel inconsistent. The original cgroups (v1) gave each controller its own independent hierarchy — the cpu tree, the memory tree, the io tree were separate and could disagree about which group a process was in. It was a mess that couldn't be fixed without breaking the interface. Tejun Heo led the redesign — cgroup v2, the "unified hierarchy," where every controller shares one tree and a process belongs to exactly one cgroup. It landed as stable in Linux 4.5 (March 2016). systemd's migration tracks that maturing: v233 (2017) hybrid v1+v2 default, with knobs to opt into unified v243 (2019) build-time default flips to "unified" (pure cgroup v2) v258 (2025) cgroup v1 support removed entirely — v2 always mounted</code></pre> So if you're on a current system, your slices are nodes in a single unified cgroup v2 tree. The "exactly one slice" cleanliness is partly because of Tejun's redesign. So: cgroups → slices. The lineage is resources, accounting, limits, containment. Nothing about boot order. The nuance: why they aren't siblings#</a> </h2> Lay the two family trees side by side and the "overlap" evaporates: </th> Target</th> Slice</th></tr></thead> Ancestor</td> SysV init runlevels (early 1980s)</td> Linux cgroups (Google, 2006 → kernel 2.6.24, 2008)</td></tr> Arrived in systemd</td> day one (2010)</td> v205 (2013)</td></tr> What it is</td> a node in the dependency graph</td> a node in the cgroup tree</td></tr> Question it answers</td> what should run, and in what order?</td> whose resources, and what limits?</td></tr> Has processes?</td> no — pure synchronization point</td> not directly; its leaf children (services/scopes) do</td></tr> Kernel object?</td> no, it's bookkeeping</td> yes, a real cgroup directory</td></tr> A unit relates to it via</td> WantedBy=</code>, Wants=</code>, Requires=</code>, After=</code></td> Slice=</code></td></tr> Cardinality</td> a unit can be in many targets</td> a unit is in exactly one slice</td></tr> The knobs it carries</td> none (it does nothing itself)</td> CPUWeight=</code>, MemoryMax=</code>, IOWeight=</code>, TasksMax=</code>…</td></tr> </tbody></table> The deep point is orthogonality. Every service is simultaneously at two coordinates: Somewhere in the dependency graph — pulled in by some targets, ordered before/after other units. This is the time / causality axis. Targets live here.</li> Somewhere in the cgroup tree — inside exactly one slice, subject to its resource limits. This is the resource / containment axis. Slices live here.</li> </ol> These axes don't compete; they're perpendicular. Asking "should I use a target or a slice?" is a bit like asking "should I describe this meeting by its time or its room?" Usually the honest answer is both, for different reasons. So which one do I actually want?#</a> </h2> Back to my original itch — "let me see and operate on my own set of units as a group." Now I can answer it properly, because it depends on which axis I care about. If I want to bring them up/down together, order them, or depend on them as a set → that's the dependency-graph axis → a target. It's the runlevel-flavored answer: "here is a named bundle; activate it." # ~/.config/systemd/user/personal.target [Unit] Description=My personal services</code></pre># in each of my units: [Install] WantedBy=personal.target</code></pre>systemctl --user list-dependencies personal.target # see the whole set systemctl --user start personal.target # bring the set up</code></pre> If I want to cap or account their combined CPU/memory, or view them all as one resource group → that's the cgroup-tree axis → a slice. It's the cgroup-flavored answer, and status</code> on a slice gives you a live tree of everything inside it plus its resource usage: # ~/.config/systemd/user/personal.slice [Slice] MemoryMax=2G CPUWeight=50</code></pre># in each of my units: [Service] Slice=personal.slice</code></pre>systemctl --user status personal.slice # the cgroup tree + live usage systemd-cgls --user # the whole user cgroup tree</code></pre> And the punchline that took me the whole history to appreciate: you can use both, and it isn't redundant. A personal.target</code> that pulls the group up and a personal.slice</code> that boxes its resources are describing the same services along two independent axes. The target says when they run; the slice says how much they get. (If all I really want is to list mine and never operate on them as a unit, the cheapest trick of all is a shared name prefix — systemctl --user list-units 'personal-*'</code>, since systemctl</code> accepts globs. No new unit types required. But the moment I want grouped behavior — ordering or limits — I reach for the matching family tree.) The takeaway#</a> </h2> systemd looks like one monolithic thing, but a lot of it is really two old Unix ideas wearing the same systemctl</code> uniform: the init state machine that goes back to AT&T System V, and the kernel resource controller that came out of Google's clusters. Targets are the grandchild of the runlevel. Slices are the grandchild of the cgroup. They share a command-line front-end and absolutely nothing else in their DNA. Once you can glance at a unit type and ask "which family tree is this from — the dependency graph, or the cgroup tree?", the rest of systemd stops feeling arbitrary. It's just two lineages, finally living under one roof. Sources & further reading#</a> </h3> Lennart Poettering, Rethinking PID 1</a> (2010-04-30) — the founding essay.</li> systemd.target(5)</code>, systemd.slice(5)</code>, systemd.scope(5)</code>, systemd.special(7)</code> — the canonical definitions quoted above.</li> systemd NEWS</code> — v205 (slices/scopes + single-writer), v243 (unified default), v258 (v1 removed).</li> Control Group v2</a> — the kernel's own cgroup v2 documentation.</li> LWN, 14 years of systemd</a> — a retrospective.</li> </ul> Getting SASSy: Stealing Login Patterns from Codeberg and Sourcehut Unknown — Wed, 03 Jun 2026 00:00:00 +0000 I'm building a SaaS. I want it to be as FOSS, Linux-native, and self-hosted as I can make it — which means at some point I have to stop and solve the least glamorous, most important problem in the whole thing: letting users log in. Authentication is one of those areas where the wrong instinct is to invent something. The right instinct is to go read how people who already solved it — in the open, in production, for years — chose to do it. I picked two: Codeberg, which runs Forgejo</a> (a fork of Gitea), and Sourcehut</a> (sr.ht</code>). Both are fully open source. Both are self-hostable. Both have been running real accounts for real people for a long time. And because they're open, I can read exactly how they do it instead of guessing. I came away with two questions. This post is built around them. "How do Codeberg and Sourcehut handle this problem?"#</a> </h2> The most useful thing I learned up front is that these two sit at opposite ends of one spectrum, and which one you copy depends entirely on whether you ever plan to split your app into multiple services. Sourcehut treats identity as a service. meta.sr.ht</code> is a standalone account / billing / security hub. Every other service — git.sr.ht</code>, builds.sr.ht</code>, lists.sr.ht</code> — is an OAuth2 client of meta. Auth is a network boundary you cross. Forgejo treats identity as a module. One Go process owns login, sessions, tokens, and 2FA directly, but it delegates credential checking through a common auth-source interface (local password, LDAP, OIDC, PAM…). Auth is a plugin point inside one binary. If you're one app today — like I am — Forgejo's model is the closer fit. But the discipline worth stealing is Sourcehut's: treat identity as the source of truth, behind an API, so you can extract it into its own service later without a rewrite. Sourcehut, in detail#</a> </h3> meta.sr.ht-api</code> (Go, GraphQL) owns all auth state. The Python web frontend holds no credentials — it just issues GraphQL queries. The UI is a client of the auth API, not a co-owner of the auth database. That separation is the whole point. Login: password + TOTP 2FA. No WebAuthn. Public keys (SSH, PGP) are stored centrally in meta and consumed by the other services.</li> OAuth2 is RFC 6749-conformant and deliberately hardened: Confidential clients only — public clients are prohibited. A native or CLI app must run a server component, or use a personal access token.</li> redirect_uri</code> is not accepted at runtime — only the pre-registered URI is used. That kills a whole class of redirect attacks.</li> scope</code> is mandatory (stricter than the spec). Scopes look like meta.sr.ht/PROFILE:RO meta.sr.ht/SSH_KEYS:RW</code> — service/SCOPE:access</code>, where access is RO</code> or RW</code>, enforced per GraphQL resolver via an @access</code> directive.</li> Authorization codes are 16 hex chars, 5-minute TTL. Client secret is 64-byte base64.</li> </ul> </li> The token format is the clever bit. An access / personal token is a BARE</a>-encoded payload — username, client ID, authorized scopes, expiry — that is HMAC-signed and base64'd. So the token is stateless: the server validates the signature without a database hit. The grant rides inside the token. Revocation is layered on top — a SHA hash of the token in Redis for instant lookup, plus a DB expiry timestamp as the durable fallback if the cache is lost.</li> Service-to-service uses an @internal</code> auth type — single-use tokens encrypted with Sourcehut's internal private keys — which unlocks resolvers not exposed to the public. That's how git.sr.ht</code> trusts a request that claims to come from meta</code>.</li> </ul> Forgejo / Codeberg, in detail#</a> </h3> Password hashing: PASSWORD_HASH_ALGO</code>, default PBKDF2-HMAC-SHA256 at ~310k–320k iterations; also argon2id, scrypt, bcrypt. Configurable length and complexity, plus an optional Have-I-Been-Pwned check (PASSWORD_CHECK_PWN</code>) that rejects known-breached passwords at signup. Cheap, high-signal win.</li> Sessions: cookie + middleware. A webAuth()</code> → AuthShared()</code> flow walks an authentication priority chain (try each method in order), then stashes the user in request context, gated on email confirmation / login-prohibition / forced-password-change. Session store is pluggable: memory / db / Redis / memcached.</li> API tokens are hashed in the DB with the same algorithm as passwords — never stored in plaintext — and scoped by permission, with an LRU cache (default 20) of recently-validated hashes so it doesn't re-run PBKDF2 on every request. A nice perf/security balance.</li> 2FA: TOTP and WebAuthn/FIDO2 passkeys, including passwordless login. For basic-auth API calls with 2FA on, the code goes in an X-Gitea-OTP:</code> header.</li> The auth-source abstraction is the real lesson: one interface, many backends — local password, OAuth2/OIDC client, LDAP (service-account search or simple DN-template bind), PAM, SMTP, SPNEGO/Kerberos, reverse-proxy header trust. Forgejo can also be an OAuth2/OIDC provider for other apps.</li> </ul> "Where is it all stored? A SQL table? Unix?"#</a> </h2> This was my second question, and it's the one that actually changed how I'm going to build. Because the naïve mental model — "a Linux server, users, surely there are Unix accounts somewhere" — is wrong, and deliberately so. The trick is to separate two things people conflate: Where the identity record lives → always a SQL row. Never /etc/passwd</code>.</li> What verifies the secret → usually a hash in that same SQL row, but optionally delegated to Unix (/etc/shadow</code>) or an external directory (LDAP/OIDC).</li> </ol> That second point is the entire reason the auth-source abstraction exists. Sourcehut: everything in PostgreSQL#</a> </h3> Owned by meta.sr.ht</code>: Concern</th> Where it lives</th></tr></thead> Identification (who you are)</td> Postgres user</code> table: id, created, updated, username, email, user_type, url, location, bio</code>. Single source of truth; other services have their own DBs but defer to meta.</td></tr> Authentication (proving it)</td> Password hash + TOTP secret in meta's Postgres. SSH public keys in Postgres, cached in Redis.</td></tr> Authorization (what you can do)</td> A stateless HMAC-signed token carrying username/client/scopes/expiry. Revocation via Redis (SHA of token) + Postgres expiry.</td></tr> Unix accounts</td> None per user. One shared system user per service — you SSH as git@</code>, hg@</code>, builds@</code>.</td></tr> </tbody></table> Forgejo: everything in a configurable SQL DB#</a> </h3> SQLite / MySQL / PostgreSQL, your choice: Concern</th> Where it lives</th></tr></thead> Identification</td> user</code> table.</td></tr> Authentication</td> Hashed password in the user</code> row; 2FA secrets + WebAuthn credentials in their own tables; SSH keys in public_key</code>; API tokens hashed in access_token</code>.</td></tr> Authorization</td> Token scopes stored with the token; repo/org/team permissions in SQL. Sessions in memory / db / Redis.</td></tr> Unix accounts</td> None per user. A single git</code> RUN_USER</code>.</td></tr> </tbody></table> So where does Unix actually come in?#</a> </h3> Two places, both optional, both Forgejo: PAM verifies the password against /etc/shadow</code> (real Unix accounts) — but the identity row still lives in SQL. PAM only checks the secret. (It also has a footgun: an uploaded SSH key can bypass the PAM check entirely.)</li> LDAP / OIDC verify against an external directory — but Forgejo still mirrors a user</code> row into SQL so it has something to hang repos and permissions off of.</li> </ol> Even when the verifier is Unix or LDAP, the system of record is SQL. That's the rule. The SSH trick worth stealing#</a> </h3> Neither forge runs useradd</code> per customer. So how does git push</code> over SSH know who you are with one shared system user? sshd</code>'s AuthorizedKeysCommand</code>. Instead of reading ~/.ssh/authorized_keys</code>, OpenSSH runs a program, hands it the public key being offered, and that program looks the key up in the database and resolves it to a user. Sourcehut runs gitsrht-dispatch</code>: it checks Redis, then Postgres, then meta</code>, caching the result back down the chain for the next push.</li> Forgejo either generates a managed authorized_keys</code> with a forced-command per key, or uses AuthorizedKeysCommand</code> (gitea keys …</code>) to do the same lookup.</li> </ul> So OpenSSH's key-auth mechanism gets reused, but the "account" a key resolves to is a database row, not a Unix user. One system user, a key→user lookup, and you never touch /etc/passwd</code>. What I'm taking into my own build#</a> </h2> Both forges converged on the same shape, so that's the shape I'm copying: One SQL users</code> table is the source of truth for identity. Unix accounts don't enter into it.</li> Make the credential verifier pluggable behind that table: local hash now; LDAP/OIDC later when an enterprise customer asks for SSO — without touching the identity model.</li> Authorization travels separately — either DB-stored scoped tokens (Forgejo: simpler) or stateless signed tokens with a revocation list (Sourcehut: fewer DB hits). I'll probably start with the former.</li> Ship TOTP early, WebAuthn/passkeys later. That's the order both landed on.</li> Add the HIBP breached-password check. Free, and it stops the dumbest account takeovers.</li> If I ever do SSH/key access, copy the AuthorizedKeysCommand</code> dispatch trick. One shared system user, key→DB-user lookup. Never useradd</code> per customer.</li> </ul> The headline, if you skipped to the bottom: it's SQL all the way down for identity and the canonical credentials, and the avoidance of per-user Unix accounts is a feature, not an oversight. The Linux box hosts the service; it does not host the users. Sources I read while writing this: the Sourcehut API 2.0 dev log</a>, OAuth 2.0 via meta.sr.ht</a>, What happens when you push to git.sr.ht</a>, Drew DeVault's Building interactive SSH applications</a>, the Forgejo authentication docs</a>, the Gitea config cheat sheet</a>, and the Gitea SSH & key management writeup</a>. Mouse settings explained: DPI, report rate, and receiver modes Unknown — Sun, 17 May 2026 00:00:00 +0000 Modern gaming mice expose a few settings that are easy to misunderstand because the labels are short and the marketing around them is often imprecise. This post uses ZOWIE's current -DW wireless mice as the example: EC-DW, FK2-DW, ZA13-DW, S2-DW, and U2-DW. The exact shape differs by model, but the relevant controls and settings are the same. The important distinction is simple: the bottom slider selects the active connection mode. It is not a recommendation for when to play, when to stop, or where to put the mouse after a session. The receiver mode slider#</a> </h2> The three-position slider on the bottom is a power and receiver selector: • Enhanced wireless receiver mode — use this when the enhanced receiver is the device plugged into the computer. This is the larger directional receiver that also works as a pogo-pin charging dock. • Off — use this when you want the mouse powered down, such as for travel or storage. • USB wireless receiver mode — use this when the small USB receiver/dongle is the device plugged into the computer. Set the slider to match the receiver currently connected to the computer. If the enhanced receiver is connected, use enhanced receiver mode. If the USB dongle is connected, use USB receiver mode. If neither receiver is connected and you do not want the mouse awake, turn it off. The enhanced receiver can charge the mouse through the pogo pins, but that does not make the slider a "dock mode" in the everyday sense. Charging is a feature of the enhanced receiver; the slider's job is to choose which wireless receiver the mouse should talk to. The two bottom buttons#</a> </h2> The two other controls are for DPI and report rate: • DPI button — cycles through 400 , 800 , 1000 , 1200 , 1600 , and 3200 DPI. • Report-rate button — cycles through 125 , 500 , 1000 , 2000 , and 4000 Hz, depending on which receiver is in use. ZOWIE's documentation notes that available report rates vary by receiver. In practical terms, the enhanced receiver is the one intended for the high report-rate modes. The small USB receiver is the simpler portable option and may not expose the same top-end settings. What DPI actually changes#</a> </h2> DPI means "dots per inch," although "counts per inch" is the more accurate technical description for a mouse sensor. It is the number of movement counts the mouse reports for each inch of physical movement. At 400 DPI, moving the mouse one inch reports about 400 counts. At 1600 DPI, the same movement reports about 1600 counts. If every other sensitivity setting stays the same, the cursor or crosshair moves four times as far. DPI is not a quality rating. A higher number does not automatically mean better tracking, better aim, or a more accurate sensor. It is mostly a sensitivity input. The setting only makes sense together with the rest of the chain: operating-system pointer speed, in-game sensitivity, raw-input behavior, screen resolution, and mousepad space. For games, the useful concept is effective sensitivity. Players often call this eDPI : mouse DPI multiplied by in-game sensitivity. This is useful when comparing settings inside the same game, but it does not transfer perfectly across different games because their sensitivity scales are different. A sensible starting point is 800 or 1600 DPI, then adjust in-game sensitivity until large movements and small corrections both feel controlled. For desktop use on a high-resolution display, a higher DPI can be convenient. For aim-heavy games, an extremely high total sensitivity usually makes small corrections harder, even if the sensor itself is working correctly. What report rate changes#</a> </h2> Report rate, often called polling rate, is how often the mouse sends position updates to the computer. • 125 Hz = one report every 8 ms • 500 Hz = one report every 2 ms • 1000 Hz = one report every 1 ms • 2000 Hz = one report every 0.5 ms • 4000 Hz = one report every 0.25 ms A higher report rate reduces the maximum time between a physical movement and the next mouse report. That can improve motion smoothness and reduce input latency, especially on high-refresh-rate displays where the system can present those updates more frequently. The improvement is not linear in practice. Moving from 125 Hz to 1000 Hz is obvious. Moving from 1000 Hz to 4000 Hz is much smaller and depends on the rest of the system: monitor refresh rate, game engine, CPU load, USB behavior, and whether the game is already frame-time limited. ZOWIE lists the S2-DW battery life as 80 hours at 1000 Hz, 30 hours at 2000 Hz, and 24 hours at 4000 Hz. Higher report rates also create more USB/input work for the system. On a fast machine this is usually fine; on a CPU-limited system or a poorly behaving game, dropping back to 1000 Hz can make sense. Practical settings#</a> </h2> For most people, this is the clean baseline: • Use enhanced wireless receiver mode when the enhanced receiver is plugged into the computer. • Use USB wireless receiver mode when the small USB dongle is plugged into the computer. • Use Off when storing or transporting the mouse. • Start at 800 or 1600 DPI, then tune sensitivity in the operating system or game. • Start at 1000 Hz report rate; try 2000 or 4000 Hz if you are on a high- refresh display and the system remains stable. • Dock the mouse to charge it as frequently as needed per the report rate settings References#</a> </h2> • ZOWIE S2-DW specifications: https://zowie.benq.com/en-us/mouse/s2-dw.html • ZOWIE DPI and polling-rate FAQ: https://zowie.benq.com/en- us/support/faq/product/application/zowiegear-faq-kn-00025.html • ZOWIE S2-DW quick start guide: https://zowie.benq.com/en-us/qsg/s2- dw/guide.html Why Your OnePlus Says VOOC When It Isn't: USB-PD, PPS, and the SuperVOOC Architecture Unknown — Thu, 30 Apr 2026 00:00:00 +0000 Why Your OnePlus Reports VOOC When It Isn't#</a> </h1> A OnePlus phone connected to a 100W third-party GaN charger displays a "VOOC" fast-charge indicator in OxygenOS. The indicator is misleading. The phone is negotiating USB Power Delivery with PPS and drawing approximately a quarter of what its battery hardware will accept. OxygenOS labels any negotiated rate above an internal threshold with the family name, regardless of the protocol actually in use. This post covers what is on the CC pin during that negotiation, why third-party chargers cap where they do, and what would have to change for the same phone to charge at its rated peak. The relevant pieces are dual-cell battery topology, charge pumps versus buck converters, and a cable-side ID resistor that gates OPPO's high-current path. USB Power Delivery#</a> </h2> USB-PD is the open standard maintained by the USB-IF for negotiating power over USB-C. The model is straightforward: the source advertises a list of Power Data Objects (PDOs) — fixed voltage/current pairs such as 5V/3A</code>, 9V/3A</code>, 15V/3A</code>, 20V/5A</code> — and the sink selects one. Voltage steps are coarse and discrete. Conversion happens device-side: the phone receives 9V at the port and its internal buck IC steps that down to ~4.4V for the cell. Conversion losses dissipate as heat inside the phone. The spec has gone through several revisions. Each adds capability without removing the older fixed-PDO model. Spec</th> Year</th> Max power</th> Voltages</th> Key addition</th></tr></thead> USB-PD 1.0</td> 2012</td> 100W</td> 5/12/20V profiles</td> First spec; ran over USB-A/B VBUS. Largely skipped.</td></tr> USB-PD 2.0</td> 2014</td> 100W</td> 5V, 9V, 15V, 20V (fixed PDOs)</td> Rebuilt around USB-C. The "classic" PD everyone knows.</td></tr> USB-PD 3.0</td> 2015–2017</td> 100W</td> Same fixed PDOs</td> Fast Role Swap, extended messages, battery status, firmware update over PD.</td></tr> USB-PD 3.0 + PPS</td> 2017</td> 100W</td> 3.3–21V in 20mV steps, 50mA current steps</td> Programmable Power Supply — chargers track battery voltage.</td></tr> USB-PD 3.1 SPR</td> 2021</td> 100W</td> Same as 3.0 (≤20V)</td> "Standard Power Range." Cleanups, better extended messages.</td></tr> USB-PD 3.1 EPR</td> 2021</td> 240W</td> Adds 28V, 36V, 48V fixed PDOs</td> "Extended Power Range." Requires EPR-rated e-marked cable (5A @ 50V).</td></tr> USB-PD 3.1 EPR + AVS</td> 2021</td> 240W</td> 15–48V adjustable (100mV steps)</td> Adjustable Voltage Supply — PPS's bigger sibling for the EPR range.</td></tr> USB-PD 3.2</td> 2023</td> 240W</td> Same envelope as 3.1</td> Tightens AVS, finer current granularity, multi-port budget signaling.</td></tr> </tbody></table> Reading a charger's spec label: "PD 65W" with no PPS: fixed 20V/3.25A. Adequate for laptops, suboptimal for modern phones because it forces a high-ratio buck.</li> "PD 65W PPS": at least one PPS PDO, typically 3.3–11V @ 5A</code> or 3.3–21V @ 3A</code>. This is what a 2024+ phone targets.</li> "PD 3.1 / EPR 140W+": includes 28V or 36V fixed PDOs. Apple's 140W MacBook Pro charger and the higher-tier Anker / UGREEN units fall here. Delivery above 100W requires an EPR-rated cable.</li> "AVS" on the label: PD 3.1 EPR with adjustable voltage. Currently limited to enthusiast GaN units.</li> </ul> PPS specifically#</a> </h2> PPS is the section of PD 3.0 most relevant to phones. The sink requests voltage in 20mV increments and current in 50mA increments and adjusts continuously. Rather than "9V/3A, device copes," the phone requests "8.74V at 4.20A," then a few hundred ms later "8.78V at 4.18A," and the source tracks. This matters for thermal behavior. The phone's charge IC has the least conversion work when the input voltage is close to twice the cell voltage. A buck or charge pump operates most efficiently at small, predictable conversion ratios. With fixed PD the input is always 9V or 15V regardless of state-of-charge; with PPS the input slides as the cell fills, holding the conversion stage near peak efficiency throughout the curve. PPS adoption on Android is broad: Samsung Super Fast Charging 2.0 (45W), Pixel fast charging, and every modern OnePlus when paired with a non-OPPO charger all use PPS. iPhones do not request PPS; they negotiate fixed 9V or 15V PDOs and let the internal IC handle the rest. The OPPO family: Dash, Warp, VOOC, SuperVOOC#</a> </h2> OPPO's proprietary fast-charge family began with VOOC in 2014. OnePlus rebranded the protocol twice — first as Dash, then as Warp — before consolidating to VOOC and SuperVOOC after the 2021 OnePlus/OPPO merger. The underlying technology in each generation is the same lineage; the renames are organizational. Name</th> Era</th> Typical wire spec</th> Architectural trick</th></tr></thead> VOOC 1.0</td> 2014, OPPO</td> 5V / 4A (20W)</td> Low voltage, high current. Charger does the heavy regulation.</td></tr> Dash Charge</td> 2016–2018, OnePlus</td> 5V / 4A (20W)</td> OnePlus rebrand of VOOC. Same tech.</td></tr> Warp Charge 30</td> 2018, OnePlus</td> 5V / 6A (30W)</td> More current at the same voltage.</td></tr> Warp Charge 65</td> 2020, OnePlus</td> 10V / 6.5A (65W)</td> OnePlus rebrand of SuperVOOC 2.0.</td></tr> VOOC 4.0</td> 2019, OPPO</td> 5V / 6A (30W)</td> Same tier as Warp 30.</td></tr> SuperVOOC 1.0</td> 2018, OPPO</td> 10V / 5A (50W)</td> Dual-cell series battery. Charge pump in the phone.</td></tr> SuperVOOC 2.0</td> 2020</td> 10V / 6.5A (65W)</td> —</td></tr> SuperVOOC 80W</td> 2021</td> 10V / 8A</td> —</td></tr> SuperVOOC 100W</td> 2022</td> 10V / 10A</td> —</td></tr> SuperVOOC 150W</td> 2022</td> 20V / 7.5A</td> Higher voltage, deeper charge-pump ratio.</td></tr> SuperVOOC 240W</td> 2022</td> 24V / 10A</td> Reference design. Production phones top out near 100W.</td></tr> </tbody></table> Everything from SuperVOOC 1.0 onward is architecturally distinct from PD and from the earlier Dash/Warp/VOOC line. The next section covers why. What distinguishes SuperVOOC#</a> </h2> Four mechanisms, which compound. 1. Dual-cell series battery#</a> </h3> A SuperVOOC phone uses two battery cells wired in series. A OnePlus 12, for example, ships two ~2700 mAh cells whose terminals are connected (+)–[cell A]–(–)/(+)–[cell B]–(–)</code>, giving a pack with ~8.7V nominal instead of the ~4.35V of a conventional single-cell phone. Same energy capacity. Double the pack voltage. Half the pack current at any given charging power. Resistive losses in the cells, busbars, protection IC, and connecting traces scale as I². Halving the current quarters the I²R dissipation. A single-cell phone absorbing 100W must push ~22A through its cells; a dual-cell phone absorbs the same 100W at ~11A. The 4× heat reduction is a direct consequence of Ohm's law on a load whose resistance cannot be designed away. 2. Charge pump rather than buck converter#</a> </h3> A typical phone charge IC is a synchronous buck converter — a pair of MOSFETs and an inductor switching at a few hundred kHz to step the input voltage down. Buck IC efficiency under heavy load is 88–93%. The remaining 7–12% becomes heat next to the battery. A charge pump performs the same function with a different topology: capacitors instead of an inductor, with a fixed integer ratio (2:1, 4:1) rather than an arbitrary one. The constraint pays off — a 2:1 charge pump halves the input voltage and doubles the current at 97–98% efficiency. The cost is the fixed ratio: the input voltage must be approximately 2× or 4× what the cells require. SuperVOOC combines the two. The charger delivers 10V at 8–10A; the phone's 2:1 charge pump halves that to 5V at 16–20A; this flows into the dual-cell pack at ~8.7V, with each cell seeing ~4.35V at half the pack current. The arithmetic only closes when input voltage, charge-pump ratio, and pack topology are co-designed. 3. Charger-side regulation via MCU-to-MCU handshake#</a> </h3> Under USB-PD the device runs the charge curve. The phone reads cell voltage and temperature, decides what it wants, requests it from the charger, and converts the input internally. The charger is a voltage source that complies. SuperVOOC moves the regulation loop into the charger. The charger contains its own MCU. At plug-in, charger MCU and phone MCU exchange a proprietary handshake over the CC pin (or D+/D- on earlier revisions). After the handshake, the phone streams real-time cell voltage and temperature telemetry to the charger and the charger adjusts its output accordingly — fine-grained, high-rate, and outside the USB-PD spec. The consequence is that more conversion happens inside the charger and less inside the phone. The charger runs warm; the phone often does not. This inverts the thermal profile of a generic PD charger. 4. Cable ID resistor#</a> </h3> USB-PD specifies an e-marker chip in cables rated above 3A. SuperVOOC piggybacks on the same mechanism with a different signal: a specific resistor value to ground on a particular pin tells the charger that the cable is rated for 8A / 10A / 12.5A continuous and that high-current mode is permitted. Without that resistor the charger will not deliver above the PD-PPS fallback rate, regardless of what the phone requests. This is why a SuperVOOC charger paired with a generic USB-C cable charges a OnePlus at PD-PPS speeds: the charger does not recognize an unidentified cable. The reverse — an OPPO/OnePlus cable into a generic charger — is also a no-op, because the generic charger has no MCU on the other end. What happens when a OnePlus plugs into an Anker#</a> </h2> Concrete numbers for the four chargers in question. </th> </th></tr></thead> </td> </td></tr> INIU 100W 3-port GaN (B0DJ8C3MZD)</td> Anker Prime 67W 3-port (B0C4YTB56H)</td></tr> </td> </td></tr> Anker Prime 100W 3-port (B0C4DGBHY2)</td> Anker Prime 250W 6-port station (B0CYLL8Y89)</td></tr> </tbody></table> All four advertise PD 3.0 + PPS. The first three are single-unit GaN; the last is a desktop power station, which on its higher-power port also supports PD 3.1 EPR (28V) for laptops. None implements VOOC, SuperVOOC, or any other proprietary protocol — generic chargers cannot, lacking both the MCU and the proprietary firmware. When a OnePlus plugs into any of them, the negotiation proceeds as follows: Source capabilities. The charger advertises its PDO list, including PPS ranges such as 3.3–11V @ 5A</code> and 3.3–21V @ 3.25A</code>.</li> Phone selects PPS. The OnePlus selects the PPS PDO whose voltage range covers ~2 × battery voltage</code>.</li> Phone requests V/I. It requests, e.g., 8.94V @ 3.00A</code>.</li> Charger complies; the phone's charge pump halves the 9V to ~4.5V at ~6A into the cell.</li> Loop. The phone re-issues an adjusted V/I request every few hundred milliseconds.</li> </ol> The constraint is at step 3. OnePlus phones cap their PPS current request at ~3A, regardless of what the charger offers. A 100W single-port unit with a 3.3–11V @ 5A</code> PPS PDO still receives a 3A request from the phone. OPPO/OnePlus reserves the high-current path for the proprietary protocol. Measured draw at the USB-C connector with a power meter on a OnePlus 12-class phone: Battery state</th> Phone's PPS request</th> Wattage at the port</th></tr></thead> 0–10% (warm-up)</td> ~5V × 2–3A</td> 10–15W</td></tr> 10–60% (bulk CC)</td> ~8.9–9.0V × 3.0A</td> ~27W</td></tr> 60–80% (transition)</td> ~9V × 2–2.5A</td> 18–22W</td></tr> 80–95% (CV taper)</td> ~8.7V × 1–1.5A</td> 8–13W</td></tr> 95–100% (trickle)</td> ~5V × 0.5A</td> 2–3W</td></tr> </tbody></table> Bulk-stage draw at the connector reads approximately 27W, sustained for 15–25 minutes before the taper. The voltage choice is not arbitrary. The 2:1 charge pump in front of the single cell wants its input at approximately twice the cell voltage. A cell at 4.0–4.4V during bulk → request 8.0–8.9V at the port. The 3A figure is the phone-side cap. Removing PPS — connecting the same phone to a fixed-PDO PD charger with no PPS PDOs — falls back to 9V/2A = 18W</code> on a fixed rail. This is slower and warmer because the conversion stage no longer has voltage tracking and the charge pump cannot remain at its efficiency sweet spot across the curve. Among the four chargers shown: The INIU 100W, Anker 67W, and Anker 100W deliver the same ~27W bulk to the phone. The additional wattage on the 100W units serves laptops and other devices.</li> The Anker Prime 250W station also delivers ~27W to the phone. Its advantage is sustaining that rate while simultaneously running a 100W laptop on another port without throttling either.</li> None triggers the SuperVOOC handshake. The "VOOC" indicator in OxygenOS labels any charge above an internal wattage threshold; it does not confirm the protocol.</li> </ul> PD-PPS versus SuperVOOC, head to head#</a> </h2> Same OnePlus 12-class phone, same 5400 mAh dual-cell pack, two different protocols on the wire. Stage</th> PD-PPS (Anker / INIU)</th> SuperVOOC 80W brick</th> SuperVOOC 100W brick</th> SuperVOOC 150W brick</th></tr></thead> Negotiation</td> USB-PD messages on CC</td> Proprietary handshake on CC</td> same</td> same</td></tr> Bulk voltage</td> ~9V</td> ~10V</td> ~10V</td> ~20V</td></tr> Bulk current</td> 3A (phone-capped)</td> 8A</td> 10A</td> 7.5A</td></tr> Bulk wattage</td> ~27W</td> ~80W</td> ~100W</td> ~150W</td></tr> Cable requirement</td> any USB-C</td> OPPO/OnePlus cable with ID resistor</td> same</td> same</td></tr> Battery topology assumed</td> dual-cell</td> dual-cell</td> dual-cell</td> dual-cell</td></tr> 0→100% time (5400 mAh)</td> 75–90 min</td> 30–35 min</td> ~26 min</td> ~20 min</td></tr> Heat felt at</td> the phone</td> the brick</td> the brick</td> the brick</td></tr> </tbody></table> The 0→100% numbers do not scale linearly with peak wattage because the taper begins earlier at higher-power protocols. Cells absorb the peak rate only below ~70% state-of-charge; above that the curves converge as the battery itself becomes the bottleneck. PD-PPS delivers approximately 27% of the phone's charging capacity. The remaining 73% sits behind a handshake that no third-party charger implements. Summary#</a> </h2> SuperVOOC speeds require three components, all of them: The OPPO / OnePlus charger with the SuperVOOC MCU at the matching wattage.</li> The OPPO / OnePlus cable with the ID resistor.</li> A SuperVOOC-capable phone — a flagship OnePlus from 2020 onward with the dual-cell pack and charge pump.</li> </ol> Missing any one falls the system back to USB-PD with PPS. The phone still charges, at approximately one-quarter of its peak rate, with the thermal load shifted from the charger into the phone, and a 0→100% time of roughly 90 minutes rather than 30. The "VOOC" indicator in OxygenOS reports a wattage threshold, not a protocol. Generic 27W PD-PPS clears that threshold; so does 100W SuperVOOC. For ground-truth on the wire, a USB-C power meter — the ChargerLAB Power-Z KM003C</a> is the common enthusiast tool — reads the advertised PDO list and live V/A draw directly. boot.initrd.secrets Unknown — Wed, 29 Apr 2026 00:00:00 +0000 boot.initrd.secrets#</a> </h1> Synopsis#</a> </h2> boot.initrd.secrets = { "/etc/ssh/ssh_host_ed25519_key" = /etc/secrets/initrd/ssh_host_ed25519_key; "/etc/cryptroot.key" = /etc/secrets/initrd/cryptroot.key; };</code></pre>Description#</a> </h2> boot.initrd.secrets</code> copies files into the initrd so they are available before the root filesystem is mounted. Typical contents include LUKS keyfiles, SSH host keys for initrd SSH, and VPN configurations. The option type is attrsOf (nullOr path)</code>. Attribute names are destination paths inside the initrd. Values are source paths on the host, or null</code> to use the attribute name as both source and destination: boot.initrd.secrets."/etc/ssh/ssh_host_ed25519_key" = null; # copies /etc/ssh/ssh_host_ed25519_key to /etc/ssh/ssh_host_ed25519_key inside the initrd</code></pre> The option is defined in nixos/modules/system/boot/stage-1.nix</code>. Implementation#</a> </h2> There are two code paths. Which one runs depends on the value of boot.loader.supportsInitrdSecrets</code>. Append path (supportsInitrdSecrets = true</code>)#</a> </h3> GRUB, systemd-boot, and Limine all set supportsInitrdSecrets = true</code>. In this path, the initrd is built as a normal Nix derivation without any secrets. At bootloader install time (nixos-rebuild switch</code>), the generated script append-initrd-secrets</code> constructs a second CPIO archive containing the secrets and appends it to the initrd file on the boot partition. The Linux kernel treats concatenated CPIO archives as a single initramfs — it unpacks them sequentially. This is a documented property of the initramfs format. Secrets handled this way never enter the Nix store. They reside on the boot partition only. The script is built in stage-1.nix</code> and stored at config.system.build.initialRamdiskSecretAppender</code>. Bootloader installers call it: GRUB via install-grub.pl</code>, systemd-boot via systemd-boot-builder.py</code>. The path to the script is also recorded in bootspec.json</code> under the initrdSecrets</code> key for use by external boot managers. Failure to create secrets for the current generation is fatal. Failure for older generations produces a warning and continues — this is expected after removing or renaming a secret source file. Inline path (supportsInitrdSecrets = false</code>)#</a> </h3> When the bootloader does not support appending, secrets are copied into the initrd derivation at build time: mkdir -p $(dirname "$out/secrets/${dest}") cp -Lr ${source'} "$out/secrets/${dest}"</code></pre> This places secrets in the Nix store. The store is world-readable. An assertion enforces that all values are unquoted Nix paths or store paths in this case, and the build emits: Note that this will result in all secrets being stored world-readable in the Nix store! </blockquote> The path</code> type on the option (rather than str</code>) exists to support this fallback. append-initrd-secrets#</a> </h2> The script performs the following steps: Exit immediately if boot.initrd.secrets</code> is empty.</li> Create a temporary directory.</li> For each secret, copy the source file to $tmp/.initrd-secrets/$dest</code>, preserving attributes (cp -a</code>).</li> Set all timestamps to the Unix epoch (touch -amt 197001010000</code>) for reproducibility.</li> Sort the file list (sort -z</code>) for deterministic ordering.</li> Generate a CPIO newc archive owned by root:root (cpio -H newc -R +0:+0 --reproducible</code>).</li> Compress with the same compressor used for the main initrd.</li> Append (>></code>) to the target initrd file.</li> </ol> Simplified: tmp=$(mktemp -d ${TMPDIR:-/tmp}/initrd-secrets.XXXXXXXXXX) mkdir -p $(dirname "$tmp/.initrd-secrets/${dest}") cp -a ${source} "$tmp/.initrd-secrets/${dest}" (cd "$tmp" \ && find . -mindepth 1 | xargs touch -amt 197001010000 \ && find . -mindepth 1 -print0 | sort -z \ | cpio --quiet -o -H newc -R +0:+0 --reproducible --null) \ | zstd >> "$1"</code></pre> The compressor and its arguments are taken from config.boot.initrd.compressor</code> and config.boot.initrd.compressorArgs</code>. systemd initrd integration#</a> </h2> When boot.initrd.systemd.enable = true</code>, secrets are not placed at their final paths directly. They are staged under /.initrd-secrets/</code> inside the initrd and copied into place by the initrd-nixos-copy-secrets</code> service. This service is defined in nixos/modules/system/boot/systemd/initrd-secrets.nix</code>: boot.initrd.systemd.services.initrd-nixos-copy-secrets = { description = "Copy secrets into place"; wantedBy = [ "sysinit.target" ]; before = [ "cryptsetup-pre.target" "shutdown.target" ]; conflicts = [ "shutdown.target" ]; unitConfig.DefaultDependencies = false; script = '' for secret in $(cd /.initrd-secrets; find . -type f -o -type l); do mkdir -p "$(dirname "/$secret")" cp "/.initrd-secrets/$secret" "/$secret" done ''; serviceConfig = { Type = "oneshot"; RemainAfterExit = true; }; };</code></pre> The indirection exists because the initrd mounts a tmpfs on /run</code>. Secrets targeting paths under /run</code> would be shadowed by this mount. Staging them in /.initrd-secrets/</code> and copying after mount setup avoids this. The service runs before cryptsetup-pre.target</code>, so secrets are available before LUKS devices attempt to unlock. Consumers#</a> </h2> Several NixOS modules set boot.initrd.secrets</code> internally. boot.initrd.network.ssh#</a> </h3> boot.initrd.network.ssh = { enable = true; hostKeys = [ /etc/secrets/initrd/ssh_host_ed25519_key ]; };</code></pre> Generates: boot.initrd.secrets."/etc/ssh/ssh_host_ed25519_key" = /etc/secrets/initrd/ssh_host_ed25519_key;</code></pre> The module warns against reusing regular host keys. With the append path, the initrd SSH key resides on the unencrypted boot partition. boot.initrd.luks.devices.*.keyFile#</a> </h3> boot.initrd.secrets."/etc/cryptroot.key" = /etc/secrets/initrd/cryptroot.key; boot.initrd.luks.devices.cryptroot = { device = "/dev/disk/by-uuid/..."; keyFile = "/etc/cryptroot.key"; };</code></pre>boot.initrd.network.openvpn#</a> </h3> boot.initrd.network.openvpn = { enable = true; configuration = /etc/secrets/initrd/vpn.ovpn; };</code></pre> Generates: boot.initrd.secrets."/etc/initrd.ovpn" = /etc/secrets/initrd/vpn.ovpn;</code></pre>Security#</a> </h2> Append path. Secrets are on the boot partition, owned by root, not world-readable. They are not encrypted. The security posture is equivalent to the rest of an unencrypted /boot</code> — kernel and initrd are equally exposed. Attacks against the boot partition (replacement, modification) are addressed by Secure Boot and measured boot (TPM), not by this module. Inline path. Secrets are in /nix/store</code> and world-readable. Do not use this path for sensitive material. Garbage collection. Old NixOS generations retain their initrd files with appended secrets. When a secret source file is removed, nixos-rebuild switch</code> warns but continues for old generations. The old initrd (with the old secret) remains on /boot</code> until the generation is deleted with nix-collect-garbage -d</code> or nix-env --delete-generations</code>. Files#</a> </h2> File</th> Role</th></tr></thead> nixos/modules/system/boot/stage-1.nix</code></td> Option definition, append-initrd-secrets</code> script, inline fallback, assertion</td></tr> nixos/modules/system/boot/systemd/initrd-secrets.nix</code></td> systemd initrd copy service</td></tr> nixos/modules/system/boot/loader/grub/grub.nix</code></td> GRUB sets supportsInitrdSecrets = true</code></td></tr> nixos/modules/system/boot/loader/systemd-boot/systemd-boot.nix</code></td> systemd-boot sets supportsInitrdSecrets = true</code></td></tr> nixos/modules/system/boot/loader/grub/install-grub.pl</code></td> Calls append-initrd-secrets</code> per generation</td></tr> nixos/modules/system/boot/loader/systemd-boot/systemd-boot-builder.py</code></td> Calls append-initrd-secrets</code> per generation</td></tr> nixos/modules/system/boot/bootspec.nix</code></td> Records initrdSecrets</code> path in bootspec.json</td></tr> </tbody></table> My Recommended Software Stack for Leaving Gmail and Google Workspace Unknown — Thu, 23 Apr 2026 00:00:00 +0000 My Recommended Software Stack for Leaving Gmail and Google Workspace#</a> </h1> A lot of people want a simpler, more private, less Google-dependent setup without giving up the tools they rely on. Here’s the stack I use. The philosophy: pick the best option per category, keep your computer at the center of your data. Favor local over cloud, speed over latency. Unix philosophy — do one thing well. What this is NOT#</a> </h3> A guide for self-hosters. If that’s you, see 1</a>. Email and Calendar#</a> </h2> Use: Fastmail — fast, clean, focused on doing email well. Calendar works well enough. Don’t use: Proton — was a customer for years. Expensive, slow, web-only, bloated. Fastmail’s web app is faster and supports IETF standards, so you can drop the browser for a native desktop app.2</a> !Important!#</a> </h3> Domain Registrar: Porkbun Use your own domain. Switching providers becomes trivial — point MX and TXT records somewhere new and nobody notices. Your address is yours, not the vendor’s. Porkbun is cheap, simple, and supports WHOIS privacy. Password Manager#</a> </h2> Use: 1Password — pretty good UX, cross-device, just don't use passkeys3</a>. And then in the future if you want to migrate you can export all your TOTP secrets and data. It makes for a pretty good general purpose database too. Has native apps for all platforms. File Storage#</a> </h2> Don’t use: Dropbox — billing is a labyrinth, they won’t delete your account, only cancel the next subscription. Prisoners, not customers. Storage should be local first. For sharing on your network, NFS or SMB/CIFS does the job.4</a> External Transfers: Filemail — simple, end-to-end encrypted, simple. You already have internet and storage; you don’t need the cloud to rent your files back to you. Video Meetings#</a> </h2> Use: Signal , Zoom Signal is peer-to-peer, end-to-end encrypted, and simple — but only reaches other Signal users. Use it where you can; fall back to Zoom for everyone else. Writing and Notes#</a> </h2> Use: Obsidian + Markdown — portable, durable, not tied to any vendor. Use .docx</code> when compatibility wins. Native editors beat web ones: Linux</th> macOS</th> Windows</th></tr></thead> LibreOffice , OnlyOffice </td> Pages, Word </td> Word , LibreOffice </td></tr> </tbody></table> Spreadsheets#</a> </h3> Use: Grist — treats data like a database, not a grid. Typed columns, Python formulas, lightweight apps on top. Chat#</a> </h2> Use: Signal , Slack Signal for direct messages. Slack for teams. I’d rather recommend a peer-to-peer, channel-based alternative — but unless you like IRC, there isn’t one.5</a> TL;DR#</a> </h2> Email / Calendar: Fastmail </li> Domain: Porkbun </li> Passwords: 1Password </li> File storage: NFS / SMB locally, Filemail for external</li> Video: Signal , Zoom </li> Writing: Obsidian , Markdown, .docx</code></li> Spreadsheets: Grist </li> Chat: Signal , Slack </li> </ul> Index#</a> </h2> Logo</th> Product</th> Category</th></tr></thead> </td> Fastmail</a></td> Email & Calendar</td></tr> </td> Proton</a></td> Email</td></tr> </td> Porkbun</a></td> Domain Registrar</td></tr> </td> 1Password</a></td> Password Manager</td></tr> </td> Dropbox</a></td> File Storage</td></tr> </td> Filemail</a></td> File Transfer</td></tr> </td> Signal</a></td> Chat & Video</td></tr> </td> Zoom</a></td> Video Meetings</td></tr> </td> Obsidian</a></td> Writing & Notes</td></tr> </td> LibreOffice</a></td> Office Suite</td></tr> </td> OnlyOffice</a></td> Office Suite</td></tr> </td> Microsoft Word</a></td> Word Processor</td></tr> </td> Grist</a></td> Spreadsheets</td></tr> </td> Slack</a></td> Chat</td></tr> </tbody></table> ^{1 I'm working on some new projects to serve these gaps. See my git repos. </div> ^{2 The standards-protocol point is bigger than just native desktop software. It’s about having authority over your own data. That’s my biggest gripe with Proton and Gmail. You should be able to use your data the way you want — whether that’s through a desktop app or a CLI tool. Don’t settle for less, unless a better option simply doesn’t exist yet. </div> ^{3 They purposely ruined them by making them unexportable. </div> ^{4 microsoft Try To Name Something Challenge </div> ^{5 Don’t email me about Matrix. Do email me about IRC. </div> About Me Unknown — Sun, 19 Apr 2026 00:00:00 +0000 Hi, I'm Scott I'm a Senior Developer and Open Source project maintainer of electricEye. I currently live in Texas with my partner and our cat🐈. I use my site as a method of blogging about my learnings, as well as sharing information about projects I have previously, or am currently, working on in my spare time. I specialize in Internet Protocol versions 4 and 6, software development, operations, system administration, and 🔥blazingly🔥 fast code I'm a Linux user, a big advocate for the Open Source Software Movement, and the libre (free as in freedom) movement and I host my own services rather than handing it off to someone else's platform. Lambdo's and Dont's Unknown — Sun, 19 Apr 2026 00:00:00 +0000 Lambdagony: You Don't Need a Lambda, You Need a Socket#</a> </h1> You know that Lambda function you wrote? The one that receives an HTTP request, does some small thing, and returns a response? The one with 200 lines of Terraform around it, an API Gateway in front of it, a CloudWatch log group you always forget how to query, a 3-second cold start that appears at the worst possible time, and a deployment pipeline held together with YAML and prayers? What if I told you the exact same workload could be a 12-line systemd unit on a box you already have? The Problem with Lambda#</a> </h2> Lambda is great for what Lambda is great for. The issue is that people reach for it when they don't need it. You don't have "thousands of concurrent requests with unpredictable spiky traffic." You have a webhook that fires twice an hour from your git forge and a cron job that runs nightly. You are not Netflix. But you've been told serverless is the future, so now you have: A Lambda function</li> An API Gateway (with its own fun config)</li> An IAM role with a policy document</li> A CloudWatch log group</li> A deployment artifact (zip or container image)</li> Some way to get that artifact there (CI, SAM, CDK, Terraform, clicking around in the console and hoping you remember what you did)</li> Cold starts</li> A 15 minute execution timeout</li> No local state, no filesystem (well, 512 MB of /tmp</code>), no persistent connections</li> A bill that's small but irritating to reason about</li> </ul> All for something that accepts a JSON payload, writes a row to a database, and returns {"ok": true}</code>. What systemd Gives You for Free#</a> </h2> systemd has a feature called socket activation. It's been around since 2010. The idea is dead simple: systemd listens on a port. When a connection arrives, it starts your service and hands over the socket. When your service is idle, it can exit, and systemd goes back to listening. Sound familiar? It's Lambda. It's literally Lambda. Except: No cold start that you can't control (your binary is local, on an SSD)</li> No 15-minute timeout</li> Full filesystem access</li> Persistent state if you want it</li> Logs go to the journal, queryable with journalctl -u yourservice</code></li> Deployment is scp</code> and systemctl daemon-reload</code> (or NixOS rebuild, if you're civilized)</li> It's free. The box is already running.</li> </ul> How It Works#</a> </h2> You need two unit files. That's it. The Socket Unit#</a> </h3> # /etc/systemd/system/myapi.socket [Unit] Description=My API Socket [Socket] ListenStream=8080 [Install] WantedBy=sockets.target</code></pre> This tells systemd: listen on port 8080. When someone connects, start the corresponding .service</code>. The Service Unit#</a> </h3> # /etc/systemd/system/myapi.service [Unit] Description=My API Service Requires=myapi.socket [Service] Type=notify ExecStart=/usr/local/bin/myapi NonBlocking=true # Hardening, because we're not animals DynamicUser=yes ProtectSystem=strict ProtectHome=yes PrivateTmp=yes NoNewPrivileges=yes</code></pre> Your binary receives the already-open socket as file descriptor 3 (or via the LISTEN_FDS</code> protocol — sd_listen_fds(3) if you want to be proper about it). Most languages have a library for this. In Go: listeners, _ := activation.Listeners() http.Serve(listeners[0], myHandler)</code></pre> In Python: import socket, os fd = 3 # or use the sd-daemon library sock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)</code></pre> In Rust there's listenfd</code>. In C there's sd-daemon</code>. You get the idea. Enable and Start#</a> </h3> systemctl enable --now myapi.socket</code></pre> That's it. Port 8080 is now listening. Your service starts on first connection. "But What About Scaling?"#</a> </h2> If you genuinely need to handle thousands of concurrent requests with autoscaling across regions, yes, use a managed service. Nobody is arguing you should run Twitter on socket-activated systemd units. But be honest with yourself: most of the things people put in Lambdas are internal tools, webhooks, simple APIs, and cron jobs. These are workloads that a $5/month VPS handles without breaking a sweat. The "scale" argument is a thought-terminating cliche that prevents people from choosing the simple option. "But What About High Availability?"#</a> </h2> You already have a server. Is it running? Then your service is available. If it's not running, your Lambda isn't going to help you either because the database it talks to is probably on the same infrastructure that's down. If you genuinely need HA, run two boxes and put a load balancer in front. Still simpler than Lambda + API Gateway + custom domain + ACM certificate + Route53 record. "But Lambda Scales to Zero!"#</a> </h2> So does a socket-activated service. That's the whole point. When nobody is connecting, your process isn't running. The socket is open, held by PID 1, using approximately zero resources. Bonus: systemd Timers Instead of CloudWatch Events / EventBridge#</a> </h2> That cron-triggered Lambda? It's a systemd timer: # /etc/systemd/system/nightly-job.timer [Unit] Description=Run nightly job [Timer] OnCalendar=*-*-* 03:00:00 Persistent=true [Install] WantedBy=timers.target</code></pre># /etc/systemd/system/nightly-job.service [Unit] Description=Nightly job [Service] Type=oneshot ExecStart=/usr/local/bin/nightly-job DynamicUser=yes</code></pre>systemctl enable --now nightly-job.timer</code></pre> Want to see when it last ran? systemctl status nightly-job.timer</code>. Want logs? journalctl -u nightly-job</code>. Want to run it manually right now? systemctl start nightly-job</code>. No clicking through 4 AWS console pages. When to Actually Use Lambda#</a> </h2> You have genuinely spiky, unpredictable traffic that spans orders of magnitude</li> You don't want to manage any infrastructure at all (but then you're managing AWS infrastructure, which is its own beast)</li> You're already deep in the AWS ecosystem and the integration points save you real time</li> You need to run in multiple regions with minimal effort</li> </ul> When to Use a Socket-Activated Service Instead#</a> </h2> You have a server</li> Your traffic is low to moderate</li> You want to actually understand your infrastructure</li> You're tired of 200 lines of IaC for a 50-line program</li> You value the ability to ssh</code> in and just look at things</li> You don't want to learn a new debugging/logging/deployment paradigm for what is fundamentally "a program that listens on a port"</li> </ul> The Punchline#</a> </h2> The cloud providers have done an incredible job of making people forget that computers are good at running programs. A modern Linux box with systemd can do socket activation, process supervision, resource limits (cgroups), sandboxing (namespaces, seccomp), logging, and scheduled tasks. It's been able to do all of this for over a decade. Lambda is a product. Socket activation is a feature of your operating system. One of them has a marketing budget. eyeFS: A Linux Kernel Filesystem for Multi-Camera H.265 Recording Unknown — Thu, 09 Apr 2026 00:00:00 +0000 Working title: eyeFS</code> Status: design document 1. Abstract#</a> </h2> eyeFS is a Linux kernel filesystem that turns a dedicated block device into a circular log of H.265 chunks, supporting an arbitrary number of concurrent video feeds added and removed at runtime. The device is operated as a byte-addressable ring; chunks are packed back-to-back with no gaps and no container format; each chunk is an independently decodable H.265 Annex B bitstream. Multiple feeds share a single write head — a zipper merge across cameras — so that retention is uniform across feeds and the ring has one authoritative log-time axis. The userspace surface is a standard mount point: chunks appear as files under /mnt/eye/cameras/<id>/chunks/</code>, each camera has its own tail-able current</code> file at /mnt/eye/cameras/<id>/current</code> exposing the chunk-in-progress for that feed, and runtime control lives in sysfs. Crash recovery rescans the device for self-describing chunk markers. This document covers the on-disk layout, the multi-feed architecture, the VFS and sysfs interfaces, the write and read paths, crash consistency, and the userspace/kernel split. 2. Motivation (compressed)#</a> </h2> A surveillance workload is a single-writer, append-only, delete-oldest-first log of self-describing H.265 units. A conventional stack (filesystem + container + file rotation) solves problems the workload doesn't have, at a cost of 5–15% media waste, fragmentation, and bounded-space accounting headaches. The cleanest representation is a raw block device operated as a circular log of keyframe-aligned chunks. 3. Why in-kernel#</a> </h2> The userspace design delivers the log but pushes every reader through an RPC. The kernel design delivers the log and exposes it as a first-class filesystem, which unlocks properties a daemon cannot provide: Page cache and zero-copy reads. Playback is sendfile(2)</code> / splice(2)</code> from a chunk fd directly to a socket. Hot ranges stay cached across readers. Userspace with O_DIRECT</code> actively avoids all of this.</li> mmap</code> on chunks. Decoders can map a chunk and decode in place; no read buffer, no copy.</li> Tail-able current</code> file. The chunk being written is a growing file. ffmpeg -i /mnt/eye/cameras/front/current</code> follows live. poll(2)</code> wakes on new data. inotify</code> fires on new chunks.</li> Standard tooling. ls</code>, find</code>, du</code>, rsync</code>, tar</code>, restic</code>, scp</code>, ffmpeg</code>, mpv</code>, NFS (exportfs</code>), Samba. No export subcommand; no replication daemon.</li> Kernel I/O integration. Writeback, merging, I/O scheduler interaction, blktrace</code>, bpftrace</code>, iostat</code>, perf</code> all work.</li> ZNS mapping. Zoned Namespace SSDs require sequential-per-zone writes and whole-zone resets. A circular log of append-only chunks is the canonical ZNS workload. In-kernel, blk-zoned</code> and zonefs</code> are right there.</li> Access control. Unix perms and mount namespaces. No custom auth layer.</li> </ul> What the kernel does not do: RTSP. That stays userspace (see §9). 4. Reference implementation: zonefs#</a> </h2> eyeFS is structurally zonefs-shaped — one file per logical unit (chunk), append-only, VFS-exposed, no hierarchical allocation — but adds H.265-specific semantics (sidecar SEI chunk markers, per-camera namespacing) and circular overwrite (zonefs's zones don't auto-evict). The zonefs source at fs/zonefs/</code> is ~2,000 lines and is the right model to read before writing eyeFS. Expect eyeFS to be 2–3× that, most of the delta being the camera registry, the chunk assembler, and the rescan/recovery path. 5. On-disk layout#</a> </h2> The block device is laid out as: offset 0 offset 4 KiB offset 36 KiB offset D ┌──────────────────┬──────────────────┬────────────────────────────────────┐ │ Superblock │ Camera registry │ Ring region │ │ 4 KiB, immutable │ 32 KiB (128×256B)│ chunks packed back-to-back, wraps │ │ after format │ updated on │ to ring_start on overflow │ │ │ add/remove │ │ └──────────────────┴──────────────────┴────────────────────────────────────┘</code></pre>5.1 Superblock#</a> </h3> 4 KiB at offset 0, written once at mkfs.eyeFS</code> time. Fields are a superset of the userspace design: Field</th> Size</th> Notes</th></tr></thead> magic</td> 8 B</td> "EYEFS\0\0\0"</code></td></tr> version</td> 4 B</td> on-disk format revision</td></tr> device_uuid</td> 16 B</td> identifies this volume</td></tr> logical_sector_sz</td> 4 B</td> 512 or 4096</td></tr> physical_sector_sz</td> 4 B</td> alignment hint</td></tr> registry_start</td> 8 B</td> typically 4096</td></tr> registry_size</td> 8 B</td> typically 32768</td></tr> ring_start</td> 8 B</td> typically 36864</td></tr> ring_end</td> 8 B</td> device capacity</td></tr> nominal_chunk_sz</td> 8 B</td> default target, overridable per-mount</td></tr> min_chunk_sz</td> 8 B</td> below this, wrap discards tail</td></tr> max_cameras</td> 4 B</td> registry slot count</td></tr> created_at</td> 8 B</td> unix ns</td></tr> last_write_head</td> 8 B</td> updated at each checkpoint</td></tr> last_global_seq</td> 8 B</td> checkpoint of sequence allocator</td></tr> crc32c</td> 4 B</td> </td></tr> reserved</td> pad to 4 KiB</td> </td></tr> </tbody></table> The superblock is not in the writable ring. last_write_head</code> and last_global_seq</code> are updated at checkpoints; see §12 for how this bounds recovery scan time. 5.2 Camera registry#</a> </h3> A fixed-size table of 128 slots × 256 bytes = 32 KiB directly after the superblock. Each slot: Field</th> Size</th> Notes</th></tr></thead> slot_flags</td> 4 B</td> `FREE</td></tr> camera_id</td> 2 B</td> stable, assigned once, never reused in-lifetime</td></tr> camera_uuid</td> 16 B</td> for cross-device identification</td></tr> display_name</td> 64 B</td> UTF-8, null-padded</td></tr> added_at</td> 8 B</td> wall-clock ns</td></tr> removed_at</td> 8 B</td> 0 = still active</td></tr> codec</td> 4 B</td> `H265_MAIN</td></tr> default_pts_rate</td> 4 B</td> Hz (e.g. 90000)</td></tr> vps_len</td> 2 B</td> stored length of VPS for reference</td></tr> sps_len</td> 2 B</td> </td></tr> pps_len</td> 2 B</td> </td></tr> vps/sps/pps</td> 112 B</td> inlined; actual chunks carry their own</td></tr> crc32c</td> 4 B</td> </td></tr> reserved</td> pad to 256 B</td> </td></tr> </tbody></table> The registry is updated in place by slot: add allocates a free slot, remove flips ACTIVE → REMOVED</code> and sets removed_at</code>. A removed camera's chunks remain readable until wrapped over; its slot is reclaimable for new cameras only after all its chunks have aged out of the ring. Registry writes use the same ordering rule as chunk writes: write + fdatasync, then in-memory update, then WAL record on the NVMe-side index. 5.3 Ring region#</a> </h3> Each chunk is a valid Annex B bitstream beginning with a sidecar SEI NAL unit (H.265 user_data_unregistered</code>, NAL unit type 39), followed by VPS, SPS, PPS, IDR, and the rest of the GOP. Decoders ignore the SEI; eyeFS uses it as a structured, scannable anchor. Sidecar SEI payload for eyeFS (extended from the userspace design): sidecar payload: magic_uuid : 16 B (identifies the eyeFS chunk format) global_seq : u64 (monotonic across all cameras on this device) camera_id : u16 (registry slot ID) chunk_length : u64 (total bytes including this SEI) pts_start : i64 (stream's timescale, per camera) pts_end : i64 wall_clock_ns : i64 (global log time) prev_chunk_off : u64 (backpointer in global order) flags : u32 crc32c : u32</code></pre> Per-camera sequence numbers are not stored on disk; they are derived from (camera_id, global_seq)</code> at rebuild time by counting per-camera chunks in global-seq order. This keeps the on-device record fixed-size and avoids coordinating two counters during the flush critical section. 6. Multi-feed architecture#</a> </h2> The core decision: one ring, one write head, chunks tagged by camera_id. Zipper-merge across feeds. Alternatives considered and rejected: Partition table, one ring per camera. Not dynamic; adding a camera requires resizing existing partitions.</li> Multiple independent rings on one device, sized by bitrate share. Dynamic-capable, but rebalancing is complex and per-camera eviction makes cross-camera time queries awkward.</li> Per-camera virtual rings over shared physical space. Same semantics as the chosen design at significantly higher bookkeeping cost.</li> </ul> The chosen design is strictly simpler and gives uniform retention across cameras, which is what operators actually want (legal holds, compliance windows, "the last 30 days of everything"). 6.1 Zipper-merge mechanics#</a> </h3> Per camera, eyeFS maintains an in-kernel NAL accumulator: a buffer of access units since the last flush, plus a running size and the offset of the last IDR within the buffer. When the userspace ingestor feeds an access unit (§9) the kernel appends it. A flush is triggered when, for some camera: The accumulator size ≥ nominal_chunk_sz</code>, and</li> The next incoming access unit is an IDR.</li> </ol> At that point the accumulator is cut immediately before the incoming IDR (the IDR becomes the first frame of the next chunk). The cut chunk is handed to the flush path. The flush path takes the global write-head spinlock, stamps global_seq</code> and wall_clock_ns</code>, prepends the sidecar SEI, writes the chunk at the current write head, advances the head, releases the lock. Chunk writes are seconds apart per camera; at 32 cameras and 128 MiB chunks, the write rate is ~one chunk per few hundred milliseconds. Spinlock contention is not a factor. 6.2 Dynamic add#</a> </h3> echo "rtsp://user:pass@cam4.lan/main front_entrance" > /sys/fs/eyeFS/<uuid>/cameras/add</code> The kernel: Allocates a free registry slot, assigns camera_id</code>.</li> Writes the registry slot to disk, fdatasyncs.</li> Creates the VFS subtree /mnt/eye/cameras/<camera_id>/</code>.</li> Opens a per-camera write-side character device (or the current</code> file becomes writable; see §9).</li> </ol> The userspace ingestor daemon (a separate systemd service, watching for registry changes via inotify on cameras/</code>) picks up the new camera, opens the RTSP URL, and starts feeding access units. The first access unit that is an IDR starts the first chunk; earlier units arrive before the first IDR are discarded. 6.3 Dynamic remove#</a> </h3> echo <camera_id> > /sys/fs/eyeFS/<uuid>/cameras/remove</code> Mark slot REMOVED</code>, set removed_at</code>, persist.</li> Close the per-camera write side; further access units are rejected.</li> Existing chunks remain visible under /mnt/eye/cameras/<camera_id>/</code> until overwritten.</li> When the last surviving chunk for this camera_id</code> is evicted, the slot is reclaimable.</li> </ol> 6.4 Offline / reconnecting cameras#</a> </h3> A camera whose RTSP connection drops simply stops feeding the kernel. No placeholder writes, no padding, no sentinel chunks. The ring sees fewer chunks from that camera during the outage; other cameras continue normally. When the camera reconnects, the userspace ingestor resumes feeding, and the next IDR starts a new chunk. The retention window for the affected camera shrinks proportionally during the outage — if camera A was offline for 10% of a retention cycle, the last surviving chunk for A will be from 10% further back in time than the other cameras' last surviving chunks. This is correct behavior, not a bug: the ring preserves as much recent footage as possible. 6.5 Log-time vs frame PTS#</a> </h3> Two timelines coexist and must not be conflated: Log time (wall_clock_ns</code> in the sidecar SEI): a global, monotonic, NTP-ish timestamp assigned by the kernel at flush time. This is the axis for retention, eviction, and cross-camera time-range queries. All cameras on a device share this axis.</li> Frame PTS (pts_start</code>, pts_end</code> in the sidecar SEI): per-camera, in that camera's own timescale (typically 90 kHz). Preserved verbatim from the stream. This is the axis decoders see.</li> </ul> Time-range playback queries express T1..T2 in log time. The kernel resolves them against the global index and returns per-camera byte ranges. The decoder reconstructs frame timing from the in-stream PTS. This separation is what the user called "synchronous timestamps." It is not that the cameras are frame-synchronized (they aren't, and can't be in general). It is that their log time is the same single axis — which is all that eviction and queries need. 6.6 Bandwidth sharing: no fairness#</a> </h3> eyeFS does not enforce per-camera quotas. A camera with double the bitrate consumes double the ring space and shortens everyone's retention together. This is deliberate: the ring is a shared resource with a single retention window, and any per-camera quota scheme would either waste ring space (reserving quota that isn't used) or require per-camera rings (rejected in §6 intro). If quotas are ever needed, the correct mechanism is upstream: rate-limit the chatty camera at the ingestor, or give it its own device. 7. VFS namespace#</a> </h2> /mnt/eye/ ├── .super (ro; superblock view, text) ├── all/ │ └── by-seq/ │ └── 00000000000001847.h265 (global cross-camera, by global_seq) └── cameras/ ├── 001/ (camera_id) │ ├── info (ro; registry slot as text) │ ├── chunks/ │ │ ├── 00000000000001847.h265 (named by global_seq) │ │ └── 00000000000001851.h265 │ ├── by-time/ │ │ └── 2026-04-23T14:30:00.000Z (lookup-resolved symlink) │ └── current (tail-able; grows as kernel writes) ├── 002/ │ └── … └── …</code></pre> Properties: Chunk files are read-only, mmap</code>-able, and sendfile</code>/splice</code>-eligible.</li> by-time/</code> is a synthetic directory: readdir</code> enumerates known chunk wall-clock prefixes, lookup(ts)</code> returns the nearest chunk. Uses the kernel's dcache for negative caching.</li> current</code> is the only writable file and only the kernel-owned write side can append to it. Reads return the bytes written so far. poll(POLLIN)</code> wakes on new data. fstat.st_size</code> grows monotonically until the next flush, then resets (the new current</code> is the next in-progress chunk).</li> inotify(IN_CREATE)</code> on chunks/</code> fires when a chunk is flushed.</li> all/by-seq/</code> is a flat merged view, useful for archival scripts that don't care which camera.</li> </ul> Symlink targets across the synthetic trees are resolved through the same in-memory index; there is no persistent symlink data on disk. 8. Sysfs interface#</a> </h2> 8.1 Per-device tree#</a> </h3> /sys/fs/eyeFS/<device_uuid>/ ├── device → ../../block/sdX (symlink) ├── info/ │ ├── format_version │ ├── capacity_bytes │ ├── ring_start │ ├── created_at │ └── logical_sector_size ├── state/ │ ├── write_head │ ├── wrap_count │ ├── oldest_global_seq │ ├── newest_global_seq │ ├── oldest_wall_clock │ ├── newest_wall_clock │ ├── chunks_live │ └── bytes_live ├── config/ │ ├── nominal_chunk_size (rw) │ ├── min_chunk_size (rw) │ ├── fsync_mode (rw; per_chunk | batched:N) │ ├── align_to_iu (rw) │ └── paused (rw) ├── stats/ │ ├── chunks_written │ ├── chunks_evicted │ ├── bytes_written │ ├── wrap_tail_wasted │ ├── fsync_latency_ns_p50 │ ├── fsync_latency_ns_p99 │ ├── scanner_runs │ └── torn_chunks_recovered ├── actions/ (write-only triggers) │ ├── rebuild_index │ ├── checkpoint_now │ ├── force_cut (cut all cameras' current chunks immediately) │ └── cameras/ │ ├── add (write: "<rtsp_url> <display_name>") │ └── remove (write: <camera_id>) └── cameras/ ├── 001/ │ ├── info/ │ │ ├── uuid │ │ ├── display_name │ │ ├── added_at │ │ ├── codec │ │ └── rtsp_url (stored for the ingestor; not used by FS) │ ├── state/ │ │ ├── chunks_live │ │ ├── bytes_live │ │ ├── oldest_wall_clock │ │ ├── newest_wall_clock │ │ ├── last_keyframe_ns │ │ └── online (0 = ingestor hasn't written recently) │ └── stats/ │ ├── chunks_written │ ├── bytes_written │ ├── access_units_received │ └── gaps_ns (total time with no data) └── 002/…</code></pre>8.2 Debugfs (dev-only, not ABI)#</a> </h3> /sys/kernel/debug/eyeFS/<uuid>/ ├── chunks (one line per live chunk: global_seq cam_id off len pts wc) ├── wal_tail (pending index WAL records) ├── scan_trace (last rebuild scan, per-chunk timings) └── slowlog (flushes > threshold)</code></pre> Debugfs is gated by CONFIG_EYEFS_DEBUG</code> and is not part of the stable interface. 9. Userspace ingestor split#</a> </h2> RTSP is a TLS-capable, auth-bearing, vendor-quirky text protocol with reconnect logic that varies per camera firmware. It does not belong in the kernel. eyeFS splits the pipeline cleanly: Userspace (electricEye-ingestor</code>, one systemd service per running daemon): maintains RTSP sessions, depacketizes RTP, assembles access units, hands them to the kernel.</li> Kernel (eyeFS): per-camera accumulator, chunk framing, circular write, VFS/sysfs surfaces, crash recovery.</li> </ul> The interface between them is the per-camera write side. Two viable shapes: Shape A: write to cameras/<id>/current</code>. The ingestor opens the file O_WRONLY and write(2)</code>s framed access units. Framing is length-prefixed Annex B access units plus a small header carrying PTS and NAL flags. Simple, but conflates data and metadata on one fd.</li> Shape B: per-camera character device (/dev/eyeFS-<uuid>-<camera_id></code>) with a structured protocol: an ioctl registers codec params, then write(2)</code> calls carry (pts, flags, annex_b_bytes)</code> records. Cleaner separation; mirrors how other media subsystems handle this (v4l2).</li> </ul> Recommended: Shape B. The write-side is a private interface between the ingestor and the kernel; it does not need to be POSIX-file-shaped. Shape A would force the current</code> file to simultaneously serve "readable live stream" and "writable ingestion target," which couples unrelated concerns. With Shape B, current</code> is read-only to everyone including the ingestor. The ingestor's total job: open RTSP, depacketize, call write</code> per access unit. No storage state, no retention logic, no index. It is a small Go program, drawn largely from the existing electricEye</code> codebase (src/ingest.go</code>), with the astiav/MKV muxer path removed. 10. Write path#</a> </h2> Per-camera accumulation state (in kernel): struct eyefs_camera_writer { u16 camera_id; struct list_head nal_buf; // accumulated access units u64 buf_bytes; u64 last_idr_off; i64 pts_first, pts_last; spinlock_t lock; };</code></pre> Per-device shared state: struct eyefs_device { struct block_device *bdev; spinlock_t head_lock; u64 write_head; u64 global_seq; struct eyefs_camera_writer *cams[MAX_CAMERAS]; … };</code></pre>10.1 Per access unit#</a> </h3> Ingestor write(2)</code>s (pts, flags, bytes)</code> on the per-camera chardev.</li> eyeFS takes cam->lock</code>, appends NAL bytes, updates buf_bytes</code>, updates pts_last</code>.</li> If the access unit is an IDR and buf_bytes ≥ nominal_chunk_sz</code>: trigger flush (§10.2) using the buffer up to but not including this IDR, then append this IDR as the start of the next buffer.</li> Release cam->lock</code>.</li> </ol> Keyframe detection is a two-byte check on the NAL header (nal_unit_type ∈ {19, 20} for IDR). Same as the userspace design. 10.2 Flush#</a> </h3> Build sidecar SEI (global_seq</code> is stamped inside the critical section below).</li> Take device->head_lock</code>.</li> Assign global_seq = device->global_seq++</code>.</li> Compute write offset; handle wrap if chunk wouldn't fit before ring_end</code> (§10.3).</li> Compute the eviction set (all live chunks whose byte ranges intersect the new chunk's range, wrap-aware).</li> Issue submit_bio</code> for the chunk data at the write head.</li> Release device->head_lock</code>.</li> On I/O completion (callback): fdatasync</code>-equivalent barrier, then append index WAL records (EVICT ×N, ADD ×1) on the NVMe-side index file, fsync WAL, update in-memory index, create the new VFS dentry under chunks/</code>.</li> </ol> The head_lock</code> is held only long enough to stamp seq, compute offset+eviction set, and submit the bio. It is not held across I/O completion. 10.3 Wrap#</a> </h3> If write_head + chunk_len > ring_end</code>: If ring_end - write_head ≥ min_chunk_sz</code>: cut the flush candidate at its last-fitting IDR, write the shorter terminal chunk at write_head</code>, spill the remainder into the next chunk which starts at ring_start</code>.</li> If ring_end - write_head < min_chunk_sz</code>: abandon those bytes, wrap write_head</code> to ring_start</code>, write the chunk there.</li> </ul> The only wasted space on the device is at most one min_chunk_sz</code> per wrap cycle (default 16 MiB ≈ 1 ppm on a 10 TB device). 10.4 Ordering#</a> </h3> bio complete → device barrier → WAL append → WAL fsync → in-memory index update</code></pre> Inverting any pair creates a window where the index advertises data that isn't durable, or the index stays stale for a durable chunk. The former is silent corruption; the latter is recoverable by rescan (§12). 10.5 fsync mode#</a> </h3> per_chunk</code> (default): one barrier per flush. Cheap on NVMe, acceptable on HDD (~tens of ms amortized over minutes of footage).</li> batched:N</code>: barrier every N chunks. Widens the durability window but is safe: the rescan path finds and recovers any unindexed chunks on restart.</li> </ul> 11. Read path#</a> </h2> 11.1 Static chunks#</a> </h3> open</code> a chunk file → kernel looks up the chunk in the in-memory index → returns an inode whose data ops do bounded pread</code> against the block device within [offset, offset+length)</code>. Read-ahead, page cache, mmap</code>, sendfile</code>, splice</code> all work via standard VFS plumbing. 11.2 current</code> (live)#</a> </h3> open</code> the per-camera current</code> → kernel returns an inode backed by the active accumulator (not yet on disk). read</code> returns bytes up to the current accumulator length. poll(POLLIN)</code> wakes on new access-unit append. When a flush happens, the in-progress chunk becomes a regular file in chunks/</code> and a new current</code> begins; consumers receive POLLHUP</code> (or a rename-style event) and reopen. This is the killer feature: ffmpeg -i /mnt/eye/cameras/001/current -c copy out.mkv</code> is a live tap, no daemon in the middle. 11.3 Time-range queries#</a> </h3> /mnt/eye/cameras/<id>/by-time/2026-04-23T14:30:00Z</code> → kernel looks up the nearest chunk with wall_clock_start ≤ T</code> → returns a symlink to that chunk's file. A time-range query is a readdir</code> of by-time/</code> bounded by T1..T2 and sequential cat</code> of the resolved chunks. Because every chunk begins with VPS/SPS/PPS/IDR, concatenation produces a valid Annex B stream. 11.4 Cross-camera#</a> </h3> /mnt/eye/all/by-seq/</code> gives a global view indexed by global_seq</code>. Useful for archival ("copy everything from global_seq 100000 to 200000 to this tarball") and for tools that don't need per-camera separation. 12. Crash consistency#</a> </h2> Three failure modes: 12.1 Daemon dies mid-access-unit (ingestor side)#</a> </h3> Accumulator holds a partial AU. Kernel discards it on next IDR. No on-device state changes. Safe. 12.2 Kernel panic with unflushed accumulators#</a> </h3> Accumulators are in RAM and lost. Last durable state is the last flushed chunk. Next boot: mount, replay index WAL, start accumulating again. Up to one nominal_chunk_sz</code> of footage per camera lost — by construction, since that's what the accumulator is. 12.3 Power loss between bio completion and WAL fsync#</a> </h3> Device holds a chunk with a valid sidecar SEI that the index doesn't know about. Next mount: Read superblock's last_write_head</code> and last_global_seq</code>.</li> Scan the ring forward from last_write_head</code>. At each position, read one logical block, look for the eyeFS sidecar SEI UUID. If found, validate CRC, read chunk_length</code>, advance by that amount, repeat. Each valid chunk found is ADDed to the WAL.</li> Stop when either an invalid SEI is hit (torn tail of a partial write — the prior chunk was the last durable one) or when global_seq</code> goes backward (we've walked past the tail into previous-wrap residue).</li> Update superblock's last_write_head</code> and last_global_seq</code>, resume.</li> </ol> Bounded by (current head − last checkpoint) which is controlled by the checkpoint cadence (default every 10,000 chunks or 5 min, whichever first). Worst case scan on unbounded skew is sequential read at device bandwidth — §12.4. 12.4 Full rescan#</a> </h3> If the index is lost entirely (NVMe failure, operator wipe): mount -t eyefs -o rebuild_index /dev/sdX /mnt/eye</code></pre> The kernel scans the entire ring, finds every valid sidecar SEI, and reconstructs the index from scratch. Per-camera sequence numbers are derived by counting per-camera chunks in global_seq</code> order. Cost: sequential read at device bandwidth. 500 MB/s on a modern device → ~5.5 hours for 10 TB. Pure sequential I/O, CPU-negligible. 12.5 Partial-wrap ambiguity#</a> </h3> After the ring has wrapped, the device contains current-cycle chunks followed by trailing residue from the previous cycle (the tail of an overwritten old chunk that wasn't fully clobbered by the most recent write). The scanner distinguishes them by global_seq</code> monotonicity: old residue has strictly smaller global_seq</code> than adjacent current-cycle chunks. Anything preceding a sequence-number discontinuity is discarded. 12.6 Registry consistency#</a> </h3> The camera registry is updated out-of-band from chunk writes; a power loss mid-registry-update could leave a half-written slot. Each slot carries its own CRC; torn slots are treated as FREE on mount and logged. An active camera whose registry slot is invalidated this way will be re-added by the ingestor on next start (same RTSP URL, new camera_id</code>), at the cost of its prior chunks becoming orphans (still present on-device, reachable via all/by-seq/</code> but not under a named camera). This is tolerable; the operator recovers with a manual registry repair tool if it matters. 13. Eviction semantics#</a> </h2> New chunk at [write_head, write_head+len)</code> evicts every live chunk whose byte range intersects that interval (wrap-aware). Variable chunk sizes mean 1..N evictions per write. Consequence in the multi-feed world: eviction is global FIFO by write-time, not per-camera FIFO. The oldest live chunk is the oldest by wall_clock_ns</code>, not the oldest of any particular camera. All cameras share the retention window. Transient dead region: if a new chunk is smaller than the oldest it overwrites, the residual bytes of the old chunk sit unreachable until the next chunk comes through. Bounded by one chunk size at any moment. 14. Mount options#</a> </h2> mount -t eyefs \ -o chunk_size=128M,min_chunk_size=16M,fsync_mode=per_chunk,\ align_to_iu=0,max_cameras=128,index_path=/var/lib/eyefs \ /dev/sdX /mnt/eye</code></pre>Option</th> Default</th> Notes</th></tr></thead> chunk_size</code></td> 128M</td> nominal flush target</td></tr> min_chunk_size</code></td> 16M</td> wrap discard threshold</td></tr> fsync_mode</code></td> per_chunk</code></td> per_chunk</code> or batched:N</code></td></tr> align_to_iu</code></td> 0</code></td> align chunk starts to indirection unit (ZNS/NVMe)</td></tr> max_cameras</code></td> 128</td> registry slot count (must match format-time)</td></tr> index_path</code></td> /var/lib/eyefs</code></td> NVMe-side index directory</td></tr> rebuild_index</code></td> 0</code></td> force full rescan at mount</td></tr> </tbody></table> Options marked format-time (e.g. max_cameras</code>) are set by mkfs.eyeFS</code> and cannot be changed at mount. 15. Tradeoffs#</a> </h2> Zipper-merge vs per-camera rings. Chosen: zipper. Simpler, uniform retention, dynamic add/remove is trivial. Cost: no per-camera bandwidth isolation — a chatty camera shortens everyone's retention.</li> Chunk size vs camera count. At N cameras and nominal chunk size C, the total chunk rate is N × (bitrate/C). At 32 cameras × 10 Mbps / 128 MiB ≈ one chunk every 3.3 seconds. Index size scales linearly; still negligible (5 MiB per 10 TB at default C, 2.5 MiB at 256 MiB C).</li> Synchronized log-time simplification. Rules out per-camera retention policies ("keep camera A for 90 days, camera B for 7 days"). If ever needed, the right answer is two devices — one per retention class — not one device with two retention policies.</li> **Kernel vs userspace.**The kernel path is justified once you decide to own the storage path; the only reason to stay in userspace is to avoid kernel development, and this user has explicitly discounted that reason.</li> Writing RTSP in userspace. Not a tradeoff — it's the only sane place. Kernel-space TLS and auth and vendor quirks are a non-starter.</li> </ul> 16. Open questions#</a> </h2> ZNS as a first-class backend. Natural fit; a zoned-mode eyeFS uses one zone per chunk (or one zone per N chunks), and wrap becomes zone-reset. Worth a follow-up paper.</li> Per-camera priority. Not implemented; if needed, the mechanism is upstream at the ingestor (rate-limit) or downstream (separate devices).</li> Encryption at rest. dm-crypt underneath eyeFS is the no-brain answer. Per-chunk encryption with a scannable outer envelope is possible but deferred.</li> Hot-swap of block device. The superblock carries a UUID; the mount binds to a specific UUID. Unplug/replug with the same UUID should resume; mismatched UUID refuses. Behavior under degraded states (I/O errors mid-flush) needs a full write-up.</li> Multi-device striping / mirroring. Out of scope for eyeFS itself; push to md/dm below.</li> Registry slot reuse after full wrap. Mechanism exists (slot is reclaimable once all its chunks are overwritten); edge cases around camera_id recycling deserve care to avoid confusing archival tools that cached old associations.</li> Upstreaming. eyeFS is narrow enough that upstream inclusion is unlikely to be welcomed without a larger ZNS story. Out-of-tree is fine; DKMS via the NixOS monorepo handles distribution.</li> </ul> 17. Summary#</a> </h2> eyeFS is a kernel filesystem that treats a block device as a circular log of H.265 chunks and exposes it through standard VFS and sysfs surfaces. Multiple cameras share one ring via zipper-merge on a single write head, tagged by camera_id</code> in a decoder-safe sidecar SEI, dynamically added and removed at runtime, with uniform retention across feeds. The userspace surface is what users expect from a filesystem — chunks are files, current</code> tails live, sendfile</code> streams to sockets, inotify</code> fires on new chunks, NFS exports the whole thing. The kernel surface is what an operator expects from a tunable block-device subsystem — sysfs for state and config, debugfs for diagnostics, mount options for format-compatible defaults. RTSP stays in userspace, as it should. Alpine Linux on arm install Part 3 Unknown — Thu, 26 Mar 2026 00:00:00 +0000 Step 3: Write U-Boot and Install Alpine Rootfs#</a> </h2> Overview#</a> </h2> Flash u-boot to the SD card, then mount the partitions and extract the Alpine Linux aarch64 rootfs. Write u-boot to the SD card#</a> </h2> Sunxi (Allwinner) boards expect u-boot at the 8KB offset: sudo dd if=./result/u-boot-sunxi-with-spl.bin of=/dev/sdd bs=1024 seek=8</code></pre>Mount the partitions#</a> </h2> sudo mkdir -p /mnt/sdcard/{boot,root} sudo mount /dev/sdd1 /mnt/sdcard/boot sudo mount /dev/sdd2 /mnt/sdcard/root</code></pre>Download and extract Alpine rootfs#</a> </h2> The Orange Pi Zero 2W is AArch64 (Allwinner H618), so use the aarch64 u-boot tarball: wget https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/aarch64/alpine-uboot-3.21.3-aarch64.tar.gz</code></pre> Extract to the root partition: sudo tar -xzf alpine-uboot-3.21.3-aarch64.tar.gz -C /mnt/sdcard/root</code></pre>Populate the boot partition#</a> </h2> Copy boot files (kernel, dtbs, extlinux config) to the FAT32 boot partition: sudo cp -r /mnt/sdcard/root/boot/* /mnt/sdcard/boot/</code></pre> Verify that the DTB sun50i-h618-orangepi-zero2w.dtb</code> is present in the boot partition and referenced correctly in extlinux/extlinux.conf</code>. Unmount#</a> </h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</code></pre> Alpine Linux on arm install Part 3 Unknown — Thu, 26 Mar 2026 00:00:00 +0000 Step 3: Write U-Boot and Install Alpine Rootfs</h2> Overview</h2> Flash u-boot to the SD card, then mount the partitions and extract the Alpine Linux aarch64 rootfs. Write u-boot to the SD card</h2> Sunxi (Allwinner) boards expect u-boot at the 8KB offset: sudo dd if=./result/u-boot-sunxi-with-spl.bin of=/dev/sdd bs=1024 seek=8</code></pre>Mount the partitions</h2> sudo mkdir -p /mnt/sdcard/{boot,root} sudo mount /dev/sdd1 /mnt/sdcard/boot sudo mount /dev/sdd2 /mnt/sdcard/root</code></pre>Download and extract Alpine rootfs</h2> The Orange Pi Zero 2W is AArch64 (Allwinner H618), so use the aarch64 u-boot tarball: wget https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/aarch64/alpine-uboot-3.21.3-aarch64.tar.gz</code></pre> Extract to the root partition: sudo tar -xzf alpine-uboot-3.21.3-aarch64.tar.gz -C /mnt/sdcard/root</code></pre>Populate the boot partition</h2> Copy boot files (kernel, dtbs, extlinux config) to the FAT32 boot partition: sudo cp -r /mnt/sdcard/root/boot/* /mnt/sdcard/boot/</code></pre> Verify that the DTB sun50i-h618-orangepi-zero2w.dtb</code> is present in the boot partition and referenced correctly in extlinux/extlinux.conf</code>. Unmount</h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</code></pre> Alpine Linux on arm install Part 2 Unknown — Wed, 18 Mar 2026 00:00:00 +0000 Step 2: Get U-Boot#</a> </h2> Overview#</a> </h2> Build u-boot for the Orange Pi Zero using Nix's cross-compilation infrastructure. Build u-boot with Nix#</a> </h2> nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.ubootOrangePiZero2</code></pre> The resulting u-boot-sunxi-with-spl.bin</code> will be in ./result/</code>. Notes from the NixOS U-Boot Wiki#</a> </h2> Reference: https://wiki.nixos.org/wiki/U-Boot Cross-compilation package sets#</a> </h3> Nix provides cross-compilation package sets for ARM targets: Target Architecture</th> Package Set</th></tr></thead> AArch64</td> aarch64-multiplatform</code></td></tr> ARMv7l</td> armv7l-hf-multiplatform</code></td></tr> ARMv6l</td> raspberryPi</code></td></tr> </tbody></table> General command format#</a> </h3> nix-build -A [pkgsCross.$cross_name.]$packageName</code></pre> When building from an x86_64 host for an ARM board, use the pkgsCross</code> prefix. On a native AArch64 system, the prefix can be omitted: nix-build -A ubootOrangePiZero2</code></pre>Finding available u-boot packages#</a> </h3> To find board-specific u-boot packages in nixpkgs, search for uboot</code> in the package set: nix-env -qaP -A nixpkgs.pkgsCross.aarch64-multiplatform | grep -i uboot</code></pre> For board defconfigs and SD card write offsets, consult the official U-Boot documentation or board-specific pages, as the large variety of ecosystems makes a single guide impractical. Alpine Linux on arm install Part 2 Unknown — Wed, 18 Mar 2026 00:00:00 +0000 Step 2: Get U-Boot</h2> Overview</h2> Build u-boot for the Orange Pi Zero using Nix's cross-compilation infrastructure. Build u-boot with Nix</h2> nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.ubootOrangePiZero2</code></pre> The resulting u-boot-sunxi-with-spl.bin</code> will be in ./result/</code>. Notes from the NixOS U-Boot Wiki</h2> Reference: https://wiki.nixos.org/wiki/U-Boot Cross-compilation package sets</h3> Nix provides cross-compilation package sets for ARM targets: Target Architecture</th> Package Set</th></tr></thead> AArch64</td> aarch64-multiplatform</code></td></tr> ARMv7l</td> armv7l-hf-multiplatform</code></td></tr> ARMv6l</td> raspberryPi</code></td></tr> </tbody></table> General command format</h3> nix-build -A [pkgsCross.$cross_name.]$packageName</code></pre> When building from an x86_64 host for an ARM board, use the pkgsCross</code> prefix. On a native AArch64 system, the prefix can be omitted: nix-build -A ubootOrangePiZero2</code></pre>Finding available u-boot packages</h3> To find board-specific u-boot packages in nixpkgs, search for uboot</code> in the package set: nix-env -qaP -A nixpkgs.pkgsCross.aarch64-multiplatform | grep -i uboot</code></pre> For board defconfigs and SD card write offsets, consult the official U-Boot documentation or board-specific pages, as the large variety of ecosystems makes a single guide impractical. Alpine Linux on arm install Part 1 Unknown — Sat, 07 Mar 2026 00:00:00 +0000 Step 1: Partition the SD Card#</a> </h1> Overview#</a> </h2> Partition an SD card for installing Alpine Linux on an Orange Pi Zero 2W (Allwinner H618, AArch64). The card needs two partitions: a FAT32 boot partition for u-boot and the kernel, and an ext4 root partition for the filesystem. Identify the SD Card#</a> </h2> Find the SD card device with: lsblk -d -o NAME,SIZE,MODEL,TRAN</code></pre> In this case, the device was /dev/sdd</code>. Partition with sfdisk#</a> </h2> Use sfdisk</code> for scriptable partitioning: echo ' label: dos unit: sectors /dev/sdd1 : size=+512M, type=c, bootable /dev/sdd2 : type=83 ' | sudo sfdisk /dev/sdd</code></pre> This creates: Partition 1 — 512MB, FAT32 (LBA), bootable — boot partition (u-boot + kernel)</li> Partition 2 — remaining space, Linux — root filesystem</li> </ul> Note: 256MB is too small for the boot partition — the Alpine kernel, modloop, and System.map won't fit. Use 512MB. </blockquote> Format the Partitions#</a> </h2> sudo mkfs.vfat /dev/sdd1 sudo mkfs.ext4 /dev/sdd2</code></pre> Alpine Linux on arm install Part 1 Unknown — Sat, 07 Mar 2026 00:00:00 +0000 Step 1: Partition the SD Card</h1> Overview</h2> Partition an SD card for installing Alpine Linux on an Orange Pi Zero 2W (Allwinner H618, AArch64). The card needs two partitions: a FAT32 boot partition for u-boot and the kernel, and an ext4 root partition for the filesystem. Identify the SD Card</h2> Find the SD card device with: lsblk -d -o NAME,SIZE,MODEL,TRAN</code></pre> In this case, the device was /dev/sdd</code>. Partition with sfdisk</h2> Use sfdisk</code> for scriptable partitioning: echo ' label: dos unit: sectors /dev/sdd1 : size=+512M, type=c, bootable /dev/sdd2 : type=83 ' | sudo sfdisk /dev/sdd</code></pre> This creates: Partition 1 — 512MB, FAT32 (LBA), bootable — boot partition (u-boot + kernel)</li> Partition 2 — remaining space, Linux — root filesystem</li> </ul> Note: 256MB is too small for the boot partition — the Alpine kernel, modloop, and System.map won't fit. Use 512MB. </blockquote> Format the Partitions</h2> sudo mkfs.vfat /dev/sdd1 sudo mkfs.ext4 /dev/sdd2</code></pre>}}}}}

I Put Claude Code in a Cgroup Jail Because It Kept Eating My Workstation

Booting Back Out of a Bad Timeshift Snapshot (Pop!_OS + BTRFS)

Two Family Trees: systemd Targets and Slices

Getting SASSy: Stealing Login Patterns from Codeberg and Sourcehut

Mouse settings explained: DPI, report rate, and receiver modes

References

Why Your OnePlus Says VOOC When It Isn't: USB-PD, PPS, and the SuperVOOC Architecture

boot.initrd.secrets

My Recommended Software Stack for Leaving Gmail and Google Workspace

What this is NOT
A guide for self-hosters. If that’s you, see 1</a></sup>.</p>

Email and Calendar
Fastmail’s web app is faster and supports IETF standards, so you can drop the browser for a native desktop app.2</a></sup></p>

About Me

Lambdo's and Dont's

eyeFS: A Linux Kernel Filesystem for Multi-Camera H.265 Recording

Alpine Linux on arm install Part 3

`Unmount#</span></a> </h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</span></span></code></pre>`

Alpine Linux on arm install Part 3

Step 3: Write U-Boot and Install Alpine Rootfs</h2>

Overview</h2>
Flash u-boot to the SD card, then mount the partitions and extract the Alpine Linux aarch64 rootfs.</p>

`Unmount</h2> sudo umount /mnt/sdcard/boot /mnt/sdcard/root</span></span></code></pre>`

Alpine Linux on arm install Part 2

Alpine Linux on arm install Part 2

Step 2: Get U-Boot</h2>

Overview</h2>
Build u-boot for the Orange Pi Zero using Nix's cross-compilation infrastructure.</p>

Alpine Linux on arm install Part 1

Format the Partitions

Alpine Linux on arm install Part 1

Step 1: Partition the SD Card</h1>

Overview</h2>
Partition an SD card for installing Alpine Linux on an Orange Pi Zero 2W (Allwinner H618, AArch64). The card needs two partitions: a FAT32 boot partition for u-boot and the kernel, and an ext4 root partition for the filesystem.</p>

Format the Partitions</h2>
`sudo mkfs.vfat /dev/sdd1</span></span> sudo mkfs.ext4 /dev/sdd2</span></span></code></pre>`

Sc0tt

Mostly Harmless: A Hitchhiker's Guide to Nintendo Emulation and the Linux Hackers Who Won't Let It Die

There Are 2 Hard Things in Computer Science, and Sean Bean Said None of Them