/tags/s3/Recent content in S3 on Momo Engineering Journal/images/og-image.png/images/og-image.pngHugoen-usWed, 28 Oct 2026 17:47:15 -0600/posts/060-r10-object-lock/Wed, 28 Oct 2026 12:00:00 +0000/posts/060-r10-object-lock/Object Lock is the S3 feature regulated workloads ask for first: write-once-read-many retention that the storage server itself enforces, not a policy the client promises to honor. This phase gives momo bucket-level lock enablement at creation, per-version retention and legal holds, and a delete path that answers 403 instead of quietly removing bytes. Teaches: how COMPLIANCE and GOVERNANCE differ in who can override them, why lock state must live beside the version record rather than inside object metadata, and which parts of AWS&rsquo;s surface momo deliberately does not pretend to implement./posts/059-r10-notification/Wed, 21 Oct 2026 07:14:25 +0000/posts/059-r10-notification/Implemented S3 bucket notification configurations with real webhook delivery, prefix/suffix filtering, exponential-backoff retries, and all six S3 event types — so backup tools and pipelines can react to object changes instead of polling. Teaches: why a notification dispatcher that returns nil without sending is the trap to watch for, and how events stay honest across plain PUT, multipart, and delete paths./posts/058-s3-lifecycle-on-cas/Wed, 21 Oct 2026 03:35:13 +0000/posts/058-s3-lifecycle-on-cas/Lifecycle rules are the first feature that makes momo delete data on a schedule rather than on request — which turns age, storage class, and delete markers into correctness problems. This post walks through the expiration and transition engine, why a background worker is the only safe place to run it, and how expiration composes with versioning&rsquo;s delete markers./posts/057-bolt-sigv4-zero-alloc-headers/Thu, 15 Oct 2026 11:43:50 +0000/posts/057-bolt-sigv4-zero-alloc-headers/Every authenticated S3 request rebuilds a SigV4 canonical-headers block — and the two obvious ways to write it (strings.Split and Header.Get) each allocate on the hottest authorization path. This post walks through replacing both with an IndexByte scan and an EqualFold lookup, and the golden-equivalence test that proves the rewrite is byte-identical for every request AWS clients actually send./posts/056-s3-versioning-on-cas/Sun, 11 Oct 2026 08:56:13 +0000/posts/056-s3-versioning-on-cas/Why S3 versioning and content-addressed storage seem to conflict — mutation identity vs content identity — and how momo keeps both: UUIDv4 version records pointing at deduplicated blobs, delete markers that shadow without deleting, and pagination that can only be computed after sorting. Teaches: the real semantics backup tools depend on, and where the version chain deliberately stops being distributed./posts/052-bolt-etag-matches-zero-alloc/Tue, 06 Oct 2026 11:16:37 +0000/posts/052-bolt-etag-matches-zero-alloc/How we removed the heap allocation from ETag list parsing on the S3 conditional-request path by replacing strings.Split with a strings.IndexByte scan. Teaches: what If-Match/If-None-Match actually carry, why Split allocates per request, and how a microbenchmark proves the win./posts/051-bolt-aws-chunked-zero-alloc/Thu, 01 Oct 2026 11:32:41 +0000/posts/051-bolt-aws-chunked-zero-alloc/How we removed the last heap allocation from the aws-chunked ingest path by replacing strings.Split and strings.TrimPrefix with a strings.IndexByte scan. Teaches: what a streaming S3 upload body looks like, why Split allocates per chunk, and how a microbenchmark proves the win./posts/049-eliminate-redundant-time-format-allocations/Sat, 12 Sep 2026 00:00:00 +0000/posts/049-eliminate-redundant-time-format-allocations/By replacing a second <code>time.Format</code> call with a simple string slice (<code>amzDate[:8]</code>), we eliminated redundant heap allocations and CPU overhead when generating timestamps for AWS SigV4 requests./posts/045-bolt-lastmodified-header/Tue, 01 Sep 2026 11:19:08 +0000/posts/045-bolt-lastmodified-header/time.Format allocates a fresh string on every S3 GET/HEAD response. Swapping to time.AppendFormat drops that heap escape to zero — measured 32 B/op saved per request on the hottest read path./posts/035-s3-501-discipline-remaining-ops/Mon, 24 Aug 2026 21:44:12 +0000/posts/035-s3-501-discipline-remaining-ops/Final 501 sweep: SelectObjectContent, UploadPartCopy, analytics, inventory, metrics, intelligent-tiering now return honest 501 NotImplemented./posts/034-s3-501-discipline-object-subresources/Mon, 24 Aug 2026 21:13:07 +0000/posts/034-s3-501-discipline-object-subresources/5 object-level query parameters now return honest 501 NotImplemented instead of silently falling into GetObject/PutObject/DeleteObject./posts/033-s3-501-discipline-bucket-config/Mon, 24 Aug 2026 20:00:50 +0000/posts/033-s3-501-discipline-bucket-config/16 bucket config subresources (versioning, ACL, policy, CORS, lifecycle, etc.) now return honest 501 NotImplemented instead of silently misrouting to ListObjects/GetObject./posts/012-s3-integrity-checksums/Mon, 24 Aug 2026 16:36:34 +0000/posts/012-s3-integrity-checksums/x-amz-checksum-CRC32/SHA256 echo on PUT; integrity surfaced to clients so dirty bytes never masquerade as clean. Teaches: client-verifiable integrity, Bolt zero-allocation on read path, Sentinel honesty guarantee./posts/036-s3-listxml-appendformat-optimization/Mon, 24 Aug 2026 11:24:01 +0000/posts/036-s3-listxml-appendformat-optimization/S3 ListObjectsV2 XML serialization dropped from ~1000 allocations to 16 per op via inlined time formatting and a pre-allocated escape buffer./posts/009-s3-multipart-and-breadth/Thu, 13 Aug 2026 06:02:26 +0000/posts/009-s3-multipart-and-breadth/Multipart upload support plus a rigorous 501 strategy for unsupported S3 subresources — honest breadths over silent misbehavior./posts/011-s3-https-tls-enforcement/Wed, 12 Aug 2026 07:23:29 +0000/posts/011-s3-https-tls-enforcement/The S3 gateway turned HTTPS-or-explicit-insecure into the default: no silent plaintext for the gateway or inbound tcp/quic./posts/010-s3-auth-presigned-sigv4/Tue, 11 Aug 2026 23:11:16 +0000/posts/010-s3-auth-presigned-sigv4/Query-string SigV4 for presigned URLs, mandatory X-Amz-Date freshness, and decoupling gateway access/secret keys from the native auth token./posts/008-s3-gateway-core/Tue, 11 Aug 2026 18:25:16 +0000/posts/008-s3-gateway-core/The S3 gateway: SigV4, S3-compliant XML errors, Head/List/Copy, pagination, ranges, and metadata — a compatible API surface for real clients./posts/039-signed-payload-sse-s3/Tue, 11 Aug 2026 05:14:24 +0000/posts/039-signed-payload-sse-s3/SigV4 signed payload verification for S3 PUT/POST and Server-Sent Events for async operation notifications./posts/040-aws-chunked-streaming/Tue, 11 Aug 2026 04:52:58 +0000/posts/040-aws-chunked-streaming/aws-chunked payload encoding streams large S3 uploads in signed chunks — no full-body buffering, per-chunk SigV4 integrity./posts/006-pluggable-storage-backends/Sun, 26 Jul 2026 14:33:59 +0000/posts/006-pluggable-storage-backends/The <code>[storage] backend</code> seam swaps blob storage (local, nfs, s3, raw) behind one Store interface, keeping CAS metadata local./posts/030-external-s3-client-replication-downgrade/Wed, 01 Jul 2026 23:36:39 +0000/posts/030-external-s3-client-replication-downgrade/External S3 clients like aws-cli don&rsquo;t send momo headers — the server detects this and downgrades client-side replication modes to server-side alternatives, ensuring replication never silently drops.