The discussion of TCP’s congestion response so far has assumed that the only way a TCP infers that congestion is happening is observation of packet drops. In par- ticular, routers (the things that are mostly likely to become congested) do not ordi- narily help inform the TCP at each host that congestion is imminent. Instead, they simply drop arriving packets when no more buffer space is available (called “drop tail”) and send packets that have already arrived in a first-in-first-out (FIFO) man- ner. When Internet routers are passive like this (that is, they simply discard packets
Figure 16-21 The log-log plot shows the latency due to queuing delay experienced by data in fully congested queues of various sizes. When large buffers remain full (“buffer bloat”), interactive applications can experience unacceptable latencies in the multiple-second range.
ptg999 Section 16.11 Active Queue Management and ECN 783
when overloaded and provide no feedback regarding their congestion state), there is little a TCP can do other than react after the fact. If, however, these routers had a way to more actively manage their queues (i.e., by using a more sophisticated scheduling and buffer management policy than FIFO/drop tail), perhaps the situ- ation could be improved. If they could also signal their congestion state to TCP endpoints, so much the better.
Routers that apply scheduling and buffer management policies other than FIFO/drop tail are usually said to be active, and the corresponding methods they use to manage their queues are called active queue management (AQM) mechanisms.
The authors of [RFC2309] provide a discussion of the potential benefits of AQM.
Although AQM can be useful independently, it becomes more useful when rout- ers and switches implementing AQM have a common method for conveying their status to the end systems. For TCP, this is described in [RFC3168] and extended with additional security in an experimental specification [RFC3540]. These RFCs describe Explicit Congestion Notification (ECN), which is a way for routers to mark packets (by ensuring both of the ECN bits in the IP header are set) to indicate the onset of congestion.
Random Early Detection (RED) gateways [FJ93] are one mechanism suggested as being capable of detecting the onset of congestion and controlling the mark- ing of packets. These gateways implement a queue management discipline that measures the average queue occupancy over time. If the occupancy exceeds the minimum (called minthresh) and is less than the maximum (called maxthresh), a packet is marked with an increasing probability. If the average queue occupancy exceeds maxthresh, packets are marked with a configurable maximum probability (called MaxP), which could be 1.0. RED can also be configured to drop packets instead of marking them.
Note
The RED algorithm is the basis for a number of variants (e.g., Cisco’s WRED, which uses different RED instances based on IP DSCP or precedence values) that are supported on many routers and switches.
When received by a TCP, a congestion mark indicates that the packet has passed through a congested router. Of course, it is the sender (rather than the receiver) that really needs this information in order to react by slowing down. Thus, the receiver echoes this indication back to the sender in a series of ACK packets.
The ECN mechanism operates partially at the IP layer and so is potentially applicable to transport protocols other than TCP, although most of the work on ECN has been with TCP, and it is what we discuss here. When an ECN-capable router experiencing persistent congestion receives an IP packet, it looks in the IP header for an ECN-Capable Transport (ECT) indication (currently defined as either of the two ECN bits in the IP header being set). If set, the transport protocol responsible for sending the packet understands ECN. At this point, the router sets
ptg999 a Congestion Experienced indication in the IP header (by setting both ECN bits to 1)
and forwards the datagram. Routers are discouraged from setting a CE indication when congestion does not appear to be persistent (e.g., upon a single recent packet drop due to queue overrun) because the transport protocol is supposed to react given even a single CE indication.
The TCP receiver observing an incoming data packet with a CE set is obliged to return this indication to the sender (there is an experimental extension to add ECN to SYN + ACK segments as well [RFC5562]). Because the receiver normally returns information to the sender by using (unreliable) ACK packets, there is a significant chance that the congestion indicator could be lost. For this reason, TCP implements a small reliability-enhancing protocol for carrying the indication back to the sender. Upon receiving an incoming packet with CE set, the TCP receiver sets the ECN-Echo bit field in each ACK packet it sends until receiving a CWR bit field set to 1 from the TCP sender in a subsequent data packet. The CWR bit field being set indicates that the congestion window (i.e., sending rate) has been reduced.
Note
Although RED and ECN have been known for nearly two decades, they have not seen widespread Internet deployment. A variety of reasons have been asserted as to why (e.g., difficulty in setting RED parameters, a perception of limited ben- efits). In 2005, a “reexamination” of ECN [K05] pointed out that using ECN on only data packets limits its benefits substantially. An experimental extension [RFC5562] defines the use of ECN in SYN + ACK packets with the possibility of greatly increasing the utility of ECN for certain workloads (e.g., Web traffic).
A sending TCP receiving an ECN-Echo indicator in an ACK reacts the same way it would when detecting a single packet drop by adjusting cwnd, and it also arranges to set the CWR bit field in a subsequent data packet. The prescribed con- gestion response of the fast retransmit/recovery algorithms is invoked (of course, without the packet retransmission), causing the TCP to slow down prior to suffer- ing packet drops. Note that the TCP should not overreach; in particular, it should not react more than once for the same window of data. Doing so would overly penalize an ECN TCP relative to others.
In Windows Vista and later, ECN needs to be enabled to be used:
C:\> netsh int tcp set global ecncapability=enabled
In Linux, ECN is enabled if the Boolean sysctl variable net.ipv4.tcp_ecn is nonzero. The default varies based on which Linux distribution is used, with off being most common. On Mac OS 10.5 and later, the variables net.inet.tcp.
ecn_initiate_out and net.inet.tcp.ecn_negotiate_in control whether ECN is enabled for outgoing traffic and for incoming traffic with ECN flags set, respectively. Of course, without cooperation from routers or switches, the utility
ptg999 Section 16.12 Attacks Involving TCP Congestion Control 785
of ECN is limited in any case. Only time will tell if the vision for AQM will ever be fully realized in the global Internet.
Note
RED and ECN have been used successfully in a radically different operating envi- ronment from that for which they were designed. Microsoft and Stanford have developed Data Center TCP (DCTCP) [A10], which uses RED implemented in layer 2 switches with simplified parameters to mark packets when instantaneous congestion is experienced. They also modify the TCP receiver behavior to set ECN-Echo in ACKs only when the last received packet contains a CE mark. They report a 90% reduction in buffer occupancy for comparable TCP throughput, allowing a tenfold increase in background traffic to be supported.