SRD: the cloud-native transport under EFA
EFA's speed on ordinary cloud hardware comes down to the transport underneath it. AWS did not run RDMA over TCP or InfiniBand. It built a new protocol, Scalable Reliable Datagram (SRD), into the Nitro card. Here is what SRD is, why TCP and InfiniBand did not fit the cloud, and what SRD does instead.
The problem: TCP and InfiniBand inside a cloud network
A cloud datacenter network is a fat tree with enormous path diversity: any two hosts have many equal-cost paths between them. Routers spread traffic across those paths with ECMP (equal-cost multi-path) routing, which hashes each flow's 5-tuple and pins that whole flow to one of the paths. The 5-tuple is the five header fields that identify a flow: the IP protocol (for example TCP or UDP), the source IP, the destination IP, the source port, and the destination port. Pinning a flow to one path keeps its packets in order, but it wastes the path diversity for any single flow, and when two heavy flows hash onto the same link they collide and tail latency spikes with nothing either flow can do about it. It is also why one AWS flow tops out at a fixed rate. TCP adds its own problem on top: it reacts slowly to loss, with a retransmission timeout on the order of tens of milliseconds, an eternity at datacenter round-trip times, and because it delivers an ordered byte stream, one lost segment blocks everything queued behind it.
InfiniBand attacks latency from the other side, but it assumes a lossless, in-order fabric propped up by link-level flow control, which is hard to maintain across a large, lossy, multi-tenant cloud. Its reliable-connected transport also needs a queue pair per peer, so connection state grows with the cluster. Neither protocol was designed for the network AWS runs.
What SRD does instead
SRD keeps the good parts of RDMA, namely reliable delivery, kernel bypass, and hardware offload, and makes a few cloud-native bets. The first is to spray packets across many paths. Rather than pin a flow to one ECMP path, SRD spreads a single flow's packets over a large number of paths at once (AWS's ENA Express documents up to 64), so it uses the network's full bisection bandwidth and routes around congested or failed links.
The second bet, the one that makes the first possible, is to give up in-order delivery. SRD delivers packets out of order and leaves reordering to the layer above. With no ordering constraint at the transport, multipath comes for free and a single lost packet never blocks the rest. SRD also drops the classic Reliable Datagram model's single-outstanding-message limit, so there is no cap on messages in flight, while still guaranteeing at-most-once delivery.
The third bet is to retransmit fast. Reliability and retransmission live in the Nitro card with accurate round-trip measurement, so SRD detects and resends a lost packet in microseconds instead of waiting out a TCP-scale timeout. The fourth is a congestion control built for many paths that keeps queues shallow and reacts in microseconds, so the protocol does not become the source of tail latency it is trying to avoid.
All of this runs on the AWS Nitro networking card, not the host CPU or kernel. That placement is what gives SRD precise timing and keeps the work off your cores. And because SRD is a datagram protocol rather than a per-peer ordered connection, a node needs far fewer queues than reliable-connected InfiniBand, which is what lets it scale to large clusters.
Going deeper: how SRD stays reliable
SRD's reliability is retransmission-based, not erasure coding. It does not send forward-error-correction parity packets to reconstruct losses; instead the requester tracks which packets were acknowledged and resends the ones that were lost, guaranteeing at-most-once delivery. Because that bookkeeping and its timers live in the Nitro card, which measures round-trip time precisely, SRD can use a sub-millisecond retransmission timeout rather than TCP's coarse, tens-of-milliseconds one. It keeps no ordered byte stream and does no segmentation, so a lost packet triggers only its own resend and never stalls the packets behind it. (Erasure coding does show up elsewhere at AWS, in storage services like EBS and S3 for durability, but that is a separate layer from the SRD transport.)
Path choice is active, not random. The sender spreads a connection's packets across the available paths and uses per-path round-trip measurements to spot a path that is filling up, then steers new packets away from it. A congestion-control algorithm caps how much data each connection keeps in flight based on that RTT signal, aiming to hold network queues shallow so latency stays predictable instead of letting buffers fill the way TCP tends to. The datagram model beneath all of this is also what cuts queue-pair count: a node needs roughly one queue pair per local process rather than one per remote peer, which is the difference between a protocol that scales to a large cluster and one that does not.
Where SRD shows up
SRD is not EFA-only. It is the transport under several AWS services:
- EFA, the kernel-bypass interface HPC and ML use through MPI and NCCL, where SRD carries the RDMA traffic.
- ENA Express, which puts SRD under ordinary TCP and UDP sockets. AWS reports up to roughly 85% lower p99.9 tail latency and higher single-flow throughput (up to 25 Gbps) on supported instances, with no application changes.
- EBS io2 Block Express, where SRD carries storage traffic.
What out-of-order delivery means for your code
"SRD reassembles for you" is true and slightly too comforting. On an RDM endpoint you get send-after-send ordering and out-of-order packets are reassembled, but the endpoint is not fully ordered across operation types.
In practice: do not assume an RMA completion implies ordering with respect to an unrelated
message unless you asked for it with FI_FENCE. This is the kind of assumption
that holds in every test you write and then does not hold under load, so it is worth
deciding explicitly rather than discovering.
The tradeoff
Out-of-order delivery is the price of multipath: something above SRD has to reorder messages or tolerate disorder. ENA Express hides this by restoring TCP and UDP semantics on top; EFA leaves it to libfabric, MPI, or the application. For a system built on one-sided RDMA reads this is a good fit, because each read is an independent transfer. Disorder across reads does not matter, and a given value either arrives intact or the read is retried.
Why it matters
SRD is the reason EFA performs on the same commodity Ethernet that runs everything else in the cloud. It trades the comfortable in-order, lossless assumptions of InfiniBand for a protocol that embraces the cloud's many lossy paths, and pushes the whole thing into hardware so it stays fast and CPU-free. One-sided reads over EFA, which is what Rasa is built on, ride directly on top of it.
Sources: "A Cloud-Optimized Transport Protocol for Elastic and Scalable HPC" (Shalev et al., AWS Annapurna Labs, IEEE Micro 2020; IEEE Xplore), the amzn-drivers SRD notes, and the AWS ENA Express documentation. On ECMP flow hashing and how paths are picked: AWS's 5-tuple ECMP documentation and Dip Singh's Flow Distribution Across ECMP Paths.