/categories/s3/Recent content in S3 on Momo Engineering Journal/images/og-image.png/images/og-image.pngHugoen-usWed, 28 Oct 2026 17:47:15 -0600/posts/060-r10-object-lock/Wed, 28 Oct 2026 12:00:00 +0000/posts/060-r10-object-lock/Object Lock is the S3 feature regulated workloads ask for first: write-once-read-many retention that the storage server itself enforces, not a policy the client promises to honor. This phase gives momo bucket-level lock enablement at creation, per-version retention and legal holds, and a delete path that answers 403 instead of quietly removing bytes. Teaches: how COMPLIANCE and GOVERNANCE differ in who can override them, why lock state must live beside the version record rather than inside object metadata, and which parts of AWS’s surface momo deliberately does not pretend to implement./posts/059-r10-notification/Wed, 21 Oct 2026 07:14:25 +0000/posts/059-r10-notification/Implemented S3 bucket notification configurations with real webhook delivery, prefix/suffix filtering, exponential-backoff retries, and all six S3 event types — so backup tools and pipelines can react to object changes instead of polling. Teaches: why a notification dispatcher that returns nil without sending is the trap to watch for, and how events stay honest across plain PUT, multipart, and delete paths./posts/058-s3-lifecycle-on-cas/Wed, 21 Oct 2026 03:35:13 +0000/posts/058-s3-lifecycle-on-cas/Lifecycle rules are the first feature that makes momo delete data on a schedule rather than on request — which turns age, storage class, and delete markers into correctness problems. This post walks through the expiration and transition engine, why a background worker is the only safe place to run it, and how expiration composes with versioning’s delete markers./posts/056-s3-versioning-on-cas/Sun, 11 Oct 2026 08:56:13 +0000/posts/056-s3-versioning-on-cas/Why S3 versioning and content-addressed storage seem to conflict — mutation identity vs content identity — and how momo keeps both: UUIDv4 version records pointing at deduplicated blobs, delete markers that shadow without deleting, and pagination that can only be computed after sorting. Teaches: the real semantics backup tools depend on, and where the version chain deliberately stops being distributed./posts/052-bolt-etag-matches-zero-alloc/Tue, 06 Oct 2026 11:16:37 +0000/posts/052-bolt-etag-matches-zero-alloc/How we removed the heap allocation from ETag list parsing on the S3 conditional-request path by replacing strings.Split with a strings.IndexByte scan. Teaches: what If-Match/If-None-Match actually carry, why Split allocates per request, and how a microbenchmark proves the win./posts/051-bolt-aws-chunked-zero-alloc/Thu, 01 Oct 2026 11:32:41 +0000/posts/051-bolt-aws-chunked-zero-alloc/How we removed the last heap allocation from the aws-chunked ingest path by replacing strings.Split and strings.TrimPrefix with a strings.IndexByte scan. Teaches: what a streaming S3 upload body looks like, why Split allocates per chunk, and how a microbenchmark proves the win./posts/048-bolt-listparts-appendformat/Tue, 08 Sep 2026 11:29:58 +0000/posts/048-bolt-listparts-appendformat/How we eliminated the final per-response heap allocation in S3 XML rendering by swapping time.Format for time.AppendFormat in the ListParts handler. Teaches: the stack-buffer time formatting pattern, why Format allocates, and how microbenchmarks prove the win./posts/047-bolt-s3-copyresult-time-alloc/Sun, 30 Aug 2026 11:20:58 +0000/posts/047-bolt-s3-copyresult-time-alloc/How we optimized S3 XML responses by replacing time.Format with time.AppendFormat to eliminate heap allocations. Teaches: stack-buffer time formatting pattern, hot path identification via pprof./posts/053-multitenancy-authorization-audit/Tue, 25 Aug 2026 03:25:29 +0000/posts/053-multitenancy-authorization-audit/How momo isolates tenants without a central secret server: HKDF-derived per-tenant keys, default-deny ACLs evaluated per request, and a SHA-256 hash-chained audit log that makes tampering detectable. Teaches: key derivation vs. key storage, why ACL evaluation order matters, and how hash chaining turns a log into evidence./posts/035-s3-501-discipline-remaining-ops/Mon, 24 Aug 2026 21:44:12 +0000/posts/035-s3-501-discipline-remaining-ops/Final 501 sweep: SelectObjectContent, UploadPartCopy, analytics, inventory, metrics, intelligent-tiering now return honest 501 NotImplemented./posts/034-s3-501-discipline-object-subresources/Mon, 24 Aug 2026 21:13:07 +0000/posts/034-s3-501-discipline-object-subresources/5 object-level query parameters now return honest 501 NotImplemented instead of silently falling into GetObject/PutObject/DeleteObject./posts/033-s3-501-discipline-bucket-config/Mon, 24 Aug 2026 20:00:50 +0000/posts/033-s3-501-discipline-bucket-config/16 bucket config subresources (versioning, ACL, policy, CORS, lifecycle, etc.) now return honest 501 NotImplemented instead of silently misrouting to ListObjects/GetObject./posts/012-s3-integrity-checksums/Mon, 24 Aug 2026 16:36:34 +0000/posts/012-s3-integrity-checksums/x-amz-checksum-CRC32/SHA256 echo on PUT; integrity surfaced to clients so dirty bytes never masquerade as clean. Teaches: client-verifiable integrity, Bolt zero-allocation on read path, Sentinel honesty guarantee./posts/036-s3-listxml-appendformat-optimization/Mon, 24 Aug 2026 11:24:01 +0000/posts/036-s3-listxml-appendformat-optimization/S3 ListObjectsV2 XML serialization dropped from ~1000 allocations to 16 per op via inlined time formatting and a pre-allocated escape buffer./posts/009-s3-multipart-and-breadth/Thu, 13 Aug 2026 06:02:26 +0000/posts/009-s3-multipart-and-breadth/Multipart upload support plus a rigorous 501 strategy for unsupported S3 subresources — honest breadths over silent misbehavior./posts/011-s3-https-tls-enforcement/Wed, 12 Aug 2026 07:23:29 +0000/posts/011-s3-https-tls-enforcement/The S3 gateway turned HTTPS-or-explicit-insecure into the default: no silent plaintext for the gateway or inbound tcp/quic./posts/010-s3-auth-presigned-sigv4/Tue, 11 Aug 2026 23:11:16 +0000/posts/010-s3-auth-presigned-sigv4/Query-string SigV4 for presigned URLs, mandatory X-Amz-Date freshness, and decoupling gateway access/secret keys from the native auth token./posts/008-s3-gateway-core/Tue, 11 Aug 2026 18:25:16 +0000/posts/008-s3-gateway-core/The S3 gateway: SigV4, S3-compliant XML errors, Head/List/Copy, pagination, ranges, and metadata — a compatible API surface for real clients./posts/039-signed-payload-sse-s3/Tue, 11 Aug 2026 05:14:24 +0000/posts/039-signed-payload-sse-s3/SigV4 signed payload verification for S3 PUT/POST and Server-Sent Events for async operation notifications./posts/040-aws-chunked-streaming/Tue, 11 Aug 2026 04:52:58 +0000/posts/040-aws-chunked-streaming/aws-chunked payload encoding streams large S3 uploads in signed chunks — no full-body buffering, per-chunk SigV4 integrity./posts/030-external-s3-client-replication-downgrade/Wed, 01 Jul 2026 23:36:39 +0000/posts/030-external-s3-client-replication-downgrade/External S3 clients like aws-cli don’t send momo headers — the server detects this and downgrades client-side replication modes to server-side alternatives, ensuring replication never silently drops.