Architectural decisions, design records (ADRs), system specifications, and structural evolution behind momo’s distributed object store.
Object Lock is the S3 feature regulated workloads ask for first: write-once-read-many retention that the storage server itself enforces, not a policy the client promises to honor. This phase gives momo bucket-level lock enablement at creation, per-version retention and legal holds, and a delete path that answers 403 instead of quietly removing bytes. Teaches: how COMPLIANCE and GOVERNANCE differ in who can override them, why lock state must live beside the version record rather than inside object metadata, and which parts of AWS’s surface momo deliberately does not pretend to implement.
Implemented S3 bucket notification configurations with real webhook delivery, prefix/suffix filtering, exponential-backoff retries, and all six S3 event types — so backup tools and pipelines can react to object changes instead of polling. Teaches: why a notification dispatcher that returns nil without sending is the trap to watch for, and how events stay honest across plain PUT, multipart, and delete paths.
Lifecycle rules are the first feature that makes momo delete data on a schedule rather than on request — which turns age, storage class, and delete markers into correctness problems. This post walks through the expiration and transition engine, why a background worker is the only safe place to run it, and how expiration composes with versioning’s delete markers.
Why S3 versioning and content-addressed storage seem to conflict — mutation identity vs content identity — and how momo keeps both: UUIDv4 version records pointing at deduplicated blobs, delete markers that shadow without deleting, and pagination that can only be computed after sorting. Teaches: the real semantics backup tools depend on, and where the version chain deliberately stops being distributed.
Why millions of small files per directory collapse a filesystem, and how a SeaweedFS-style volume layout fixes it: append-only superblocks, 8-byte-aligned needles with CRC32C frames, hole-punching deletion, and crash recovery by truncation. Teaches: the real cost of per-blob files, why alignment beats variable-length framing, and how to make crash recovery O(1).
The momofs FUSE mount traded its hand-rolled bazil.org/fuse adapter for the go-fuse/v2 high-level fs API, mapped all 22 node callbacks to go-fuse interfaces, and deprecated consistency=cached.
The FUSE transport (momo -imp fs): bazil.org/fuse binding, per-handle buffered writes, /dev/fuse-gated e2e, and a clean unmount lifecycle.
The R4 POSIX core: inodes as content-addressed manifests, files as blobs, atomic rename, hardlinks, permission enforcement — a filesystem on momo.
Ack only what survived: fsync/group-commit durability barrier, survivor quorum, and read-your-writes — the write-side of the P0 stack.
When a local replica is corrupted or missing, degraded read fetches from healthy survivors transparently, while a background rebuild loop restores cluster redundancy.