Vec vs. boxed slices for write-once Rust cache payloads

Consider Box<[T]> for cache payloads whose length will not change after insertion; keep Vec<T> for construction or later resizing. A boxed slice drops capacity bookkeeping and the reservation of element slots for future growth. That makes it a useful storage choice—not a guarantee of faster lookups or a particular reduction in process memory. Cloudflare’s DNS cache provides a production example of this trade-off. (Cloudflare)

Treat this as a payload representation decision, not a requirement that the entire cache be static. Separate the work of building a payload from the work of retaining it.

Separate metadata savings from unused capacity

Rust documents an ordinary Vec<T> as a pointer, capacity, and length triplet. When it has a backing allocation, that allocation contains the initialized elements followed by space for capacity - len additional elements. A vector does not automatically shrink when elements are removed. Vectors of zero-sized types do not allocate element storage, so unused capacity does not always represent unused heap bytes. (Rust’s Vec documentation)

For a payload that will never grow again, there are two distinct optimization opportunities:

• The collection value itself: a boxed slice carries a data pointer and slice length, without the vector’s capacity field.
• The backing allocation: slots reserved for future elements are no longer useful.

For example, a vector holding five non-zero-sized elements with capacity for eight has three unused slots. Converting it to a boxed slice removes the growable-capacity bookkeeping and can reduce the backing allocation. Cloudflare identifies both as sources of savings in its cache. (Cloudflare)

Do not equate those savings with an exact drop in resident memory. Rust’s documentation notes that an allocator can provide more memory than requested, and Cloudflare’s measurements distinguish per-entry savings from whole-process memory. A boxed slice’s length is not a measurement of the physical memory an allocator retains. (Rust’s Vec documentation, Cloudflare)

Build with Vec, then convert at the storage boundary

There is no need to abandon a growable collection during construction. Use Vec<T> while accumulating data, then consume it with into_boxed_slice() when preparing the stored entry. Rust documents this conversion and its relationship to vector capacity. (Rust’s Vec documentation)

For a byte payload, an illustrative interface is:

struct CachedPayload {
    bytes: Box<[u8]>,
}

impl CachedPayload {
    fn new(bytes: Vec<u8>) -> Self {
        Self {
            bytes: bytes.into_boxed_slice(),
        }
    }

    fn bytes(&self) -> &[u8] {
        &self.bytes
    }
}

This separates the construction representation from the stored representation. The accessor gives readers a slice rather than access to a growable collection. Rust’s documentation likewise recommends slice arguments when a function only needs read access to a vector’s elements. (Rust’s Vec documentation)

The same lifecycle argument applies to owned text. Cloudflare reports replacing stored String fields with Box<str> where it no longer needed growth capacity. (Cloudflare)

Fixed length does not mean immutable elements

A boxed slice is not an immutability mechanism. Box uniquely owns its allocation, but mutable access can still allow changes to the contained values. (Rust’s Box documentation)

For example:

let mut values = vec![1, 2, 3].into_boxed_slice();
values[0] = 9;

The slice’s length has not changed, but an element has. Keep the storage decision separate from the API decision about who may mutate the payload.

For the byte-payload interface above, the private field and &[u8] accessor provide read access without exposing element mutation through that accessor. That is an API choice, not an inherent immutability guarantee supplied by Box<[u8]>. Keep any changing operational metadata separate from the write-once payload.

Account for conversion work during insertion

Rust gives a useful guarantee: when len == capacity, a Vec<T> can be converted to and from Box<[T]> without reallocating or moving its elements. If spare capacity exists, that condition is not satisfied; do not use this guarantee to assume conversion is allocation-free. (Rust’s Vec documentation)

Include conversion in the insertion benchmark. Measuring only lookups misses the point where the representation changes.

Calling shrink_to_fit() and retaining a vector is another option. The method attempts to discard excess capacity, but the value remains a pointer-capacity-length triplet. It therefore does not remove the capacity field, even if the backing allocation is successfully reduced. (Rust’s Vec documentation)

For a useful comparison, benchmark three storage paths using equivalent payloads:

1. Store the constructed Vec<T> unchanged.
2. Call shrink_to_fit() and store the vector.
3. Convert with into_boxed_slice() and store the boxed slice.

This comparison helps distinguish the effects of reducing spare capacity from changing the stored collection representation.

What Cloudflare’s numbers establish

Cloudflare reports that each DNS cache entry contained eight fields using Vec or String. Replacing them with Box<[T]> and Box<str> saved eight bytes per field, or 64 bytes per entry, before accounting for unused heap capacity. Across more than 250 billion entries, it attributed over 15 terabytes of savings to the combined metadata and spare-capacity reductions. These are deployment-specific figures, not architecture-independent Rust size guarantees. (Cloudflare)

The larger results belong to five optimizations together, not boxed slices alone. Cloudflare reports roughly 100 terabytes less fleet-wide working-set memory after the rollout settled. In its benchmarks, the complete set of changes reduced the per-entry footprint by 56%, increased insert throughput by 43%, and reduced lookup latency by 19%. The benchmark improvements and fleet memory measurement are different kinds of results. (Cloudflare)

The lesson is that small per-field costs can matter at high entry counts—not that changing one type reproduces those results.

Measure the whole entry and the whole cache

Before adopting the change, measure these separately:

• Inline entry size: compare the complete old and new entry types, not just the collection fields. Cloudflare’s account shows how alignment and padding can change the savings from a field-level modification.
• Backing allocations: record payload lengths, vector capacities, allocation counts, and allocated bytes.
• Insertion and lookup performance: include construction, conversion, insertion, and representative reads.
• Steady-state resident memory: compare runs with similar cache occupancy and workload distributions.

Cloudflare used a custom allocator wrapping Rust’s System allocator to track allocation counts and sizes, measured insert throughput and lookup latency, and checked resident memory during production rollout. It says its synthetic workload approximated production rather than reproducing it exactly; traffic mix, occupancy, allocator state, and non-cache memory also affected process memory. (Cloudflare)

Keep Vec<T> when growth or buffer reuse is part of the lifecycle. Rust documents that emptying a vector and refilling it to its previous length should require no allocator calls, so retained capacity can be useful rather than wasteful. (Rust’s Vec documentation)

For write-once payloads, Vec during construction and Box<[T]> during storage is a sensible candidate. Validate it against insertion cost and whole-cache memory rather than treating a smaller collection value as the final result.