How Rasa reads: descriptors, bulk reads, and the cost of moving memory
A one-sided RDMA read does one thing: move bytes from a remote address. Rasa's read path is mostly about making "that remote address" safe, cacheable, and boring enough to keep on every client.
The read is not the lookup
A normal cache GET asks a server to find a value. Rasa does that work before the
data path. The client resolves a key to a descriptor, then the read itself is just
fi_read: remote address, remote key, length, local landing buffer.
The data node does not parse a request or copy bytes into a socket. Its registered
memory is already exposed to the NIC.
The descriptor carries everything the client needs. It says which node has the value, which registered region owns it, the exact remote address, the rkey that authorizes access, the byte length, and the version/checksum the client should validate after the read. Once the client has that record, the server is out of the hot path.
Bulk reads change the shape
A single lookup per key is fine for a benchmark and bad for a real cache. The better shape is: build the read plan first, bulk-load the descriptors with one metadata call, insert the node addresses once, then keep the NIC busy with a queue of outstanding reads. For a KV cache manifest, Rasa can go further: one manifest names every block, and the client lands them into one contiguous local buffer.
The map is smaller than it feels
People worry that every client needs an index. It does, but the raw index is not the scary part. Rasa's descriptor is 48 bytes. If metadata is signed, add a 32-byte signature. For 16 KiB values, that is still tiny.
| values | data at 16 KiB | 48 B descriptors | desc + 32 B sig |
|---|---|---|---|
| 1M | 16 GiB | 48 MiB | 80 MiB |
| 10M | 160 GiB | 480 MiB | 800 MiB |
The descriptor-only ratio is 48 / 16384 = 0.29%. With a 32-byte
signature it is 80 / 16384 = 0.49%. A client-side hash map adds overhead,
but even a few times the wire size is still a reasonable resident structure for a
machine serving inference traffic. For hot subsets, it is trivial.
The harder problem is not "can the client remember the pointers?" It is "can the system keep those pointers true?" A descriptor is not just metadata. It is a capability to read a specific memory range.
The catch: memory cannot casually move
Once a client has {remote_addr, rkey, len, version}, the data node has
made a promise. It cannot free that slot, reuse it for another key, or re-register the
region while a client may still read through the old descriptor. The remote CPU will
not get a chance to say "wait, that pointer is stale." The NIC will just move bytes.
This is why the boring parts matter: immutable values, versions, leases, grace periods, and rkey rotation. The fast read is easy after that. Keeping the memory lifecycle sound is the real system.
Implementation notes: Rasa's SDK resolves descriptors from Valkey, verifies signatures
when configured, caches node fabric addresses, and then issues libfabric fi_read
operations. Bulk cache reads use manifests so many blocks can be fetched with one metadata
object and a pipelined RDMA read plan.