eyeFS: A Linux Kernel Filesystem for Multi-Camera H.265 Recording

Working title: eyeFS Status: design document


1. Abstract

eyeFS is a Linux kernel filesystem that turns a dedicated block device into a circular log of H.265 chunks, supporting an arbitrary number of concurrent video feeds added and removed at runtime. The device is operated as a byte-addressable ring; chunks are packed back-to-back with no gaps and no container format; each chunk is an independently decodable H.265 Annex B bitstream. Multiple feeds share a single write head — a zipper merge across cameras — so that retention is uniform across feeds and the ring has one authoritative log-time axis. The userspace surface is a standard mount point: chunks appear as files under /mnt/eye/cameras/<id>/chunks/, each camera has its own tail-able current file at /mnt/eye/cameras/<id>/current exposing the chunk-in-progress for that feed, and runtime control lives in sysfs. Crash recovery rescans the device for self-describing chunk markers.

This document covers the on-disk layout, the multi-feed architecture, the VFS and sysfs interfaces, the write and read paths, crash consistency, and the userspace/kernel split.


2. Motivation (compressed)

A surveillance workload is a single-writer, append-only, delete-oldest-first log of self-describing H.265 units. A conventional stack (filesystem + container + file rotation) solves problems the workload doesn't have, at a cost of 5–15% media waste, fragmentation, and bounded-space accounting headaches. The cleanest representation is a raw block device operated as a circular log of keyframe-aligned chunks.


3. Why in-kernel

The userspace design delivers the log but pushes every reader through an RPC. The kernel design delivers the log and exposes it as a first-class filesystem, which unlocks properties a daemon cannot provide:

What the kernel does not do: RTSP. That stays userspace (see §9).


4. Reference implementation: zonefs

eyeFS is structurally zonefs-shaped — one file per logical unit (chunk), append-only, VFS-exposed, no hierarchical allocation — but adds H.265-specific semantics (sidecar SEI chunk markers, per-camera namespacing) and circular overwrite (zonefs's zones don't auto-evict). The zonefs source at fs/zonefs/ is ~2,000 lines and is the right model to read before writing eyeFS. Expect eyeFS to be 2–3× that, most of the delta being the camera registry, the chunk assembler, and the rescan/recovery path.


5. On-disk layout

The block device is laid out as:

  offset 0             offset 4 KiB        offset 36 KiB              offset D
  ┌──────────────────┬──────────────────┬────────────────────────────────────┐
  │ Superblock       │ Camera registry  │ Ring region                        │
  │ 4 KiB, immutable │ 32 KiB (128×256B)│ chunks packed back-to-back, wraps  │
  │ after format     │ updated on       │ to ring_start on overflow          │
  │                  │ add/remove       │                                    │
  └──────────────────┴──────────────────┴────────────────────────────────────┘

5.1 Superblock

4 KiB at offset 0, written once at mkfs.eyeFS time. Fields are a superset of the userspace design:

FieldSizeNotes
magic8 B"EYEFS\0\0\0"
version4 Bon-disk format revision
device_uuid16 Bidentifies this volume
logical_sector_sz4 B512 or 4096
physical_sector_sz4 Balignment hint
registry_start8 Btypically 4096
registry_size8 Btypically 32768
ring_start8 Btypically 36864
ring_end8 Bdevice capacity
nominal_chunk_sz8 Bdefault target, overridable per-mount
min_chunk_sz8 Bbelow this, wrap discards tail
max_cameras4 Bregistry slot count
created_at8 Bunix ns
last_write_head8 Bupdated at each checkpoint
last_global_seq8 Bcheckpoint of sequence allocator
crc32c4 B
reservedpad to 4 KiB

The superblock is not in the writable ring. last_write_head and last_global_seq are updated at checkpoints; see §12 for how this bounds recovery scan time.

5.2 Camera registry

A fixed-size table of 128 slots × 256 bytes = 32 KiB directly after the superblock. Each slot:

FieldSizeNotes
slot_flags4 B`FREE
camera_id2 Bstable, assigned once, never reused in-lifetime
camera_uuid16 Bfor cross-device identification
display_name64 BUTF-8, null-padded
added_at8 Bwall-clock ns
removed_at8 B0 = still active
codec4 B`H265_MAIN
default_pts_rate4 BHz (e.g. 90000)
vps_len2 Bstored length of VPS for reference
sps_len2 B
pps_len2 B
vps/sps/pps112 Binlined; actual chunks carry their own
crc32c4 B
reservedpad to 256 B

The registry is updated in place by slot: add allocates a free slot, remove flips ACTIVE → REMOVED and sets removed_at. A removed camera's chunks remain readable until wrapped over; its slot is reclaimable for new cameras only after all its chunks have aged out of the ring.

Registry writes use the same ordering rule as chunk writes: write + fdatasync, then in-memory update, then WAL record on the NVMe-side index.

5.3 Ring region

Each chunk is a valid Annex B bitstream beginning with a sidecar SEI NAL unit (H.265 user_data_unregistered, NAL unit type 39), followed by VPS, SPS, PPS, IDR, and the rest of the GOP. Decoders ignore the SEI; eyeFS uses it as a structured, scannable anchor.

Sidecar SEI payload for eyeFS (extended from the userspace design):

  sidecar payload:
    magic_uuid      : 16 B  (identifies the eyeFS chunk format)
    global_seq      : u64   (monotonic across all cameras on this device)
    camera_id       : u16   (registry slot ID)
    chunk_length    : u64   (total bytes including this SEI)
    pts_start       : i64   (stream's timescale, per camera)
    pts_end         : i64
    wall_clock_ns   : i64   (global log time)
    prev_chunk_off  : u64   (backpointer in global order)
    flags           : u32
    crc32c          : u32

Per-camera sequence numbers are not stored on disk; they are derived from (camera_id, global_seq) at rebuild time by counting per-camera chunks in global-seq order. This keeps the on-device record fixed-size and avoids coordinating two counters during the flush critical section.


6. Multi-feed architecture

The core decision: one ring, one write head, chunks tagged by camera_id. Zipper-merge across feeds. Alternatives considered and rejected:

The chosen design is strictly simpler and gives uniform retention across cameras, which is what operators actually want (legal holds, compliance windows, "the last 30 days of everything").

6.1 Zipper-merge mechanics

Per camera, eyeFS maintains an in-kernel NAL accumulator: a buffer of access units since the last flush, plus a running size and the offset of the last IDR within the buffer. When the userspace ingestor feeds an access unit (§9) the kernel appends it.

A flush is triggered when, for some camera:

  1. The accumulator size ≥ nominal_chunk_sz, and
  2. The next incoming access unit is an IDR.

At that point the accumulator is cut immediately before the incoming IDR (the IDR becomes the first frame of the next chunk). The cut chunk is handed to the flush path.

The flush path takes the global write-head spinlock, stamps global_seq and wall_clock_ns, prepends the sidecar SEI, writes the chunk at the current write head, advances the head, releases the lock. Chunk writes are seconds apart per camera; at 32 cameras and 128 MiB chunks, the write rate is ~one chunk per few hundred milliseconds. Spinlock contention is not a factor.

6.2 Dynamic add

echo "rtsp://user:pass@cam4.lan/main front_entrance" > /sys/fs/eyeFS/<uuid>/cameras/add

The kernel:

  1. Allocates a free registry slot, assigns camera_id.
  2. Writes the registry slot to disk, fdatasyncs.
  3. Creates the VFS subtree /mnt/eye/cameras/<camera_id>/.
  4. Opens a per-camera write-side character device (or the current file becomes writable; see §9).

The userspace ingestor daemon (a separate systemd service, watching for registry changes via inotify on cameras/) picks up the new camera, opens the RTSP URL, and starts feeding access units. The first access unit that is an IDR starts the first chunk; earlier units arrive before the first IDR are discarded.

6.3 Dynamic remove

echo <camera_id> > /sys/fs/eyeFS/<uuid>/cameras/remove

  1. Mark slot REMOVED, set removed_at, persist.
  2. Close the per-camera write side; further access units are rejected.
  3. Existing chunks remain visible under /mnt/eye/cameras/<camera_id>/ until overwritten.
  4. When the last surviving chunk for this camera_id is evicted, the slot is reclaimable.

6.4 Offline / reconnecting cameras

A camera whose RTSP connection drops simply stops feeding the kernel. No placeholder writes, no padding, no sentinel chunks. The ring sees fewer chunks from that camera during the outage; other cameras continue normally. When the camera reconnects, the userspace ingestor resumes feeding, and the next IDR starts a new chunk.

The retention window for the affected camera shrinks proportionally during the outage — if camera A was offline for 10% of a retention cycle, the last surviving chunk for A will be from 10% further back in time than the other cameras' last surviving chunks. This is correct behavior, not a bug: the ring preserves as much recent footage as possible.

6.5 Log-time vs frame PTS

Two timelines coexist and must not be conflated:

Time-range playback queries express T1..T2 in log time. The kernel resolves them against the global index and returns per-camera byte ranges. The decoder reconstructs frame timing from the in-stream PTS.

This separation is what the user called "synchronous timestamps." It is not that the cameras are frame-synchronized (they aren't, and can't be in general). It is that their log time is the same single axis — which is all that eviction and queries need.

6.6 Bandwidth sharing: no fairness

eyeFS does not enforce per-camera quotas. A camera with double the bitrate consumes double the ring space and shortens everyone's retention together. This is deliberate: the ring is a shared resource with a single retention window, and any per-camera quota scheme would either waste ring space (reserving quota that isn't used) or require per-camera rings (rejected in §6 intro).

If quotas are ever needed, the correct mechanism is upstream: rate-limit the chatty camera at the ingestor, or give it its own device.


7. VFS namespace

/mnt/eye/
├── .super                                (ro; superblock view, text)
├── all/
│   └── by-seq/
│       └── 00000000000001847.h265        (global cross-camera, by global_seq)
└── cameras/
    ├── 001/                              (camera_id)
    │   ├── info                          (ro; registry slot as text)
    │   ├── chunks/
    │   │   ├── 00000000000001847.h265    (named by global_seq)
    │   │   └── 00000000000001851.h265
    │   ├── by-time/
    │   │   └── 2026-04-23T14:30:00.000Z  (lookup-resolved symlink)
    │   └── current                       (tail-able; grows as kernel writes)
    ├── 002/
    │   └── …
    └── …

Properties:

Symlink targets across the synthetic trees are resolved through the same in-memory index; there is no persistent symlink data on disk.


8. Sysfs interface

8.1 Per-device tree

/sys/fs/eyeFS/<device_uuid>/
├── device                           → ../../block/sdX       (symlink)
├── info/
│   ├── format_version
│   ├── capacity_bytes
│   ├── ring_start
│   ├── created_at
│   └── logical_sector_size
├── state/
│   ├── write_head
│   ├── wrap_count
│   ├── oldest_global_seq
│   ├── newest_global_seq
│   ├── oldest_wall_clock
│   ├── newest_wall_clock
│   ├── chunks_live
│   └── bytes_live
├── config/
│   ├── nominal_chunk_size           (rw)
│   ├── min_chunk_size               (rw)
│   ├── fsync_mode                   (rw; per_chunk | batched:N)
│   ├── align_to_iu                  (rw)
│   └── paused                       (rw)
├── stats/
│   ├── chunks_written
│   ├── chunks_evicted
│   ├── bytes_written
│   ├── wrap_tail_wasted
│   ├── fsync_latency_ns_p50
│   ├── fsync_latency_ns_p99
│   ├── scanner_runs
│   └── torn_chunks_recovered
├── actions/                         (write-only triggers)
│   ├── rebuild_index
│   ├── checkpoint_now
│   ├── force_cut                    (cut all cameras' current chunks immediately)
│   └── cameras/
│       ├── add                      (write: "<rtsp_url> <display_name>")
│       └── remove                   (write: <camera_id>)
└── cameras/
    ├── 001/
    │   ├── info/
    │   │   ├── uuid
    │   │   ├── display_name
    │   │   ├── added_at
    │   │   ├── codec
    │   │   └── rtsp_url             (stored for the ingestor; not used by FS)
    │   ├── state/
    │   │   ├── chunks_live
    │   │   ├── bytes_live
    │   │   ├── oldest_wall_clock
    │   │   ├── newest_wall_clock
    │   │   ├── last_keyframe_ns
    │   │   └── online               (0 = ingestor hasn't written recently)
    │   └── stats/
    │       ├── chunks_written
    │       ├── bytes_written
    │       ├── access_units_received
    │       └── gaps_ns              (total time with no data)
    └── 002/…

8.2 Debugfs (dev-only, not ABI)

/sys/kernel/debug/eyeFS/<uuid>/
├── chunks           (one line per live chunk: global_seq cam_id off len pts wc)
├── wal_tail         (pending index WAL records)
├── scan_trace       (last rebuild scan, per-chunk timings)
└── slowlog          (flushes > threshold)

Debugfs is gated by CONFIG_EYEFS_DEBUG and is not part of the stable interface.


9. Userspace ingestor split

RTSP is a TLS-capable, auth-bearing, vendor-quirky text protocol with reconnect logic that varies per camera firmware. It does not belong in the kernel. eyeFS splits the pipeline cleanly:

The interface between them is the per-camera write side. Two viable shapes:

Recommended: Shape B. The write-side is a private interface between the ingestor and the kernel; it does not need to be POSIX-file-shaped. Shape A would force the current file to simultaneously serve "readable live stream" and "writable ingestion target," which couples unrelated concerns. With Shape B, current is read-only to everyone including the ingestor.

The ingestor's total job: open RTSP, depacketize, call write per access unit. No storage state, no retention logic, no index. It is a small Go program, drawn largely from the existing electricEye codebase (src/ingest.go), with the astiav/MKV muxer path removed.


10. Write path

Per-camera accumulation state (in kernel):

struct eyefs_camera_writer {
    u16             camera_id;
    struct list_head nal_buf;       // accumulated access units
    u64             buf_bytes;
    u64             last_idr_off;
    i64             pts_first, pts_last;
    spinlock_t      lock;
};

Per-device shared state:

struct eyefs_device {
    struct block_device *bdev;
    spinlock_t      head_lock;
    u64             write_head;
    u64             global_seq;
    struct eyefs_camera_writer *cams[MAX_CAMERAS];

};

10.1 Per access unit

  1. Ingestor write(2)s (pts, flags, bytes) on the per-camera chardev.
  2. eyeFS takes cam->lock, appends NAL bytes, updates buf_bytes, updates pts_last.
  3. If the access unit is an IDR and buf_bytes ≥ nominal_chunk_sz: trigger flush (§10.2) using the buffer up to but not including this IDR, then append this IDR as the start of the next buffer.
  4. Release cam->lock.

Keyframe detection is a two-byte check on the NAL header (nal_unit_type ∈ {19, 20} for IDR). Same as the userspace design.

10.2 Flush

  1. Build sidecar SEI (global_seq is stamped inside the critical section below).
  2. Take device->head_lock.
  3. Assign global_seq = device->global_seq++.
  4. Compute write offset; handle wrap if chunk wouldn't fit before ring_end (§10.3).
  5. Compute the eviction set (all live chunks whose byte ranges intersect the new chunk's range, wrap-aware).
  6. Issue submit_bio for the chunk data at the write head.
  7. Release device->head_lock.
  8. On I/O completion (callback): fdatasync-equivalent barrier, then append index WAL records (EVICT ×N, ADD ×1) on the NVMe-side index file, fsync WAL, update in-memory index, create the new VFS dentry under chunks/.

The head_lock is held only long enough to stamp seq, compute offset+eviction set, and submit the bio. It is not held across I/O completion.

10.3 Wrap

If write_head + chunk_len > ring_end:

The only wasted space on the device is at most one min_chunk_sz per wrap cycle (default 16 MiB ≈ 1 ppm on a 10 TB device).

10.4 Ordering

  bio complete → device barrier → WAL append → WAL fsync → in-memory index update

Inverting any pair creates a window where the index advertises data that isn't durable, or the index stays stale for a durable chunk. The former is silent corruption; the latter is recoverable by rescan (§12).

10.5 fsync mode


11. Read path

11.1 Static chunks

open a chunk file → kernel looks up the chunk in the in-memory index → returns an inode whose data ops do bounded pread against the block device within [offset, offset+length). Read-ahead, page cache, mmap, sendfile, splice all work via standard VFS plumbing.

11.2 current (live)

open the per-camera current → kernel returns an inode backed by the active accumulator (not yet on disk). read returns bytes up to the current accumulator length. poll(POLLIN) wakes on new access-unit append. When a flush happens, the in-progress chunk becomes a regular file in chunks/ and a new current begins; consumers receive POLLHUP (or a rename-style event) and reopen.

This is the killer feature: ffmpeg -i /mnt/eye/cameras/001/current -c copy out.mkv is a live tap, no daemon in the middle.

11.3 Time-range queries

/mnt/eye/cameras/<id>/by-time/2026-04-23T14:30:00Z → kernel looks up the nearest chunk with wall_clock_start ≤ T → returns a symlink to that chunk's file. A time-range query is a readdir of by-time/ bounded by T1..T2 and sequential cat of the resolved chunks. Because every chunk begins with VPS/SPS/PPS/IDR, concatenation produces a valid Annex B stream.

11.4 Cross-camera

/mnt/eye/all/by-seq/ gives a global view indexed by global_seq. Useful for archival ("copy everything from global_seq 100000 to 200000 to this tarball") and for tools that don't need per-camera separation.


12. Crash consistency

Three failure modes:

12.1 Daemon dies mid-access-unit (ingestor side)

Accumulator holds a partial AU. Kernel discards it on next IDR. No on-device state changes. Safe.

12.2 Kernel panic with unflushed accumulators

Accumulators are in RAM and lost. Last durable state is the last flushed chunk. Next boot: mount, replay index WAL, start accumulating again. Up to one nominal_chunk_sz of footage per camera lost — by construction, since that's what the accumulator is.

12.3 Power loss between bio completion and WAL fsync

Device holds a chunk with a valid sidecar SEI that the index doesn't know about. Next mount:

  1. Read superblock's last_write_head and last_global_seq.
  2. Scan the ring forward from last_write_head. At each position, read one logical block, look for the eyeFS sidecar SEI UUID. If found, validate CRC, read chunk_length, advance by that amount, repeat. Each valid chunk found is ADDed to the WAL.
  3. Stop when either an invalid SEI is hit (torn tail of a partial write — the prior chunk was the last durable one) or when global_seq goes backward (we've walked past the tail into previous-wrap residue).
  4. Update superblock's last_write_head and last_global_seq, resume.

Bounded by (current head − last checkpoint) which is controlled by the checkpoint cadence (default every 10,000 chunks or 5 min, whichever first). Worst case scan on unbounded skew is sequential read at device bandwidth — §12.4.

12.4 Full rescan

If the index is lost entirely (NVMe failure, operator wipe):

  mount -t eyefs -o rebuild_index /dev/sdX /mnt/eye

The kernel scans the entire ring, finds every valid sidecar SEI, and reconstructs the index from scratch. Per-camera sequence numbers are derived by counting per-camera chunks in global_seq order.

Cost: sequential read at device bandwidth. 500 MB/s on a modern device → ~5.5 hours for 10 TB. Pure sequential I/O, CPU-negligible.

12.5 Partial-wrap ambiguity

After the ring has wrapped, the device contains current-cycle chunks followed by trailing residue from the previous cycle (the tail of an overwritten old chunk that wasn't fully clobbered by the most recent write). The scanner distinguishes them by global_seq monotonicity: old residue has strictly smaller global_seq than adjacent current-cycle chunks. Anything preceding a sequence-number discontinuity is discarded.

12.6 Registry consistency

The camera registry is updated out-of-band from chunk writes; a power loss mid-registry-update could leave a half-written slot. Each slot carries its own CRC; torn slots are treated as FREE on mount and logged. An active camera whose registry slot is invalidated this way will be re-added by the ingestor on next start (same RTSP URL, new camera_id), at the cost of its prior chunks becoming orphans (still present on-device, reachable via all/by-seq/ but not under a named camera). This is tolerable; the operator recovers with a manual registry repair tool if it matters.


13. Eviction semantics

New chunk at [write_head, write_head+len) evicts every live chunk whose byte range intersects that interval (wrap-aware). Variable chunk sizes mean 1..N evictions per write.

Consequence in the multi-feed world: eviction is global FIFO by write-time, not per-camera FIFO. The oldest live chunk is the oldest by wall_clock_ns, not the oldest of any particular camera. All cameras share the retention window.

Transient dead region: if a new chunk is smaller than the oldest it overwrites, the residual bytes of the old chunk sit unreachable until the next chunk comes through. Bounded by one chunk size at any moment.


14. Mount options

mount -t eyefs \
    -o chunk_size=128M,min_chunk_size=16M,fsync_mode=per_chunk,\
       align_to_iu=0,max_cameras=128,index_path=/var/lib/eyefs \
    /dev/sdX /mnt/eye
OptionDefaultNotes
chunk_size128Mnominal flush target
min_chunk_size16Mwrap discard threshold
fsync_modeper_chunkper_chunk or batched:N
align_to_iu0align chunk starts to indirection unit (ZNS/NVMe)
max_cameras128registry slot count (must match format-time)
index_path/var/lib/eyefsNVMe-side index directory
rebuild_index0force full rescan at mount

Options marked format-time (e.g. max_cameras) are set by mkfs.eyeFS and cannot be changed at mount.


15. Tradeoffs


16. Open questions


17. Summary

eyeFS is a kernel filesystem that treats a block device as a circular log of H.265 chunks and exposes it through standard VFS and sysfs surfaces. Multiple cameras share one ring via zipper-merge on a single write head, tagged by camera_id in a decoder-safe sidecar SEI, dynamically added and removed at runtime, with uniform retention across feeds. The userspace surface is what users expect from a filesystem — chunks are files, current tails live, sendfile streams to sockets, inotify fires on new chunks, NFS exports the whole thing. The kernel surface is what an operator expects from a tunable block-device subsystem — sysfs for state and config, debugfs for diagnostics, mount options for format-compatible defaults. RTSP stays in userspace, as it should.


Comments