The classic and standard TCP algorithms made a tremendous contribution to the operation of TCP, essentially addressing the major problem of Internet congestion collapse.
Note
The problem of Internet congestion collapse was a serious concern during the years 1986–1988. In October 1986 the NSFNET backbone, an important compo- nent of the early Internet, had been observed to operate with an effective capac- ity some 1000 times less than it should have (called the “NSFNET meltdown”).
The primary reason for the problem was aggressive retransmissions during times of loss without any controls. This behavior drove the network into a persistently congested state where packet loss was massive (causing more retransmissions) and throughput was low. Adoption of the classic congestion control algorithms effectively eliminated this problem.
However, there remained several areas for improvement. Given TCP’s popu- larity, a growing amount of effort was put into ensuring that TCP could be made to work well under a wider range of conditions. We now mention several of these that are found in many TCP implementations today.
16.3.1 NewReno
One problem with fast recovery is that when multiple packets are dropped in a window of data, once one packet is recovered (i.e., successfully delivered and ACKed), a good ACK can be received at the sender that causes the temporary window inflation in fast recovery to be erased before all the packets that were lost have been retransmitted. ACKs that trigger this behavior are called partial ACKs.
A Reno TCP reacting to a partial ACK by reducing its inflated congestion window can go idle until a retransmission timer fires. To understand why this happens, recall that (non-SACK) TCP depends on the signal of three (or dupthresh) duplicate ACKs to trigger its fast retransmit procedure. If there are not enough packets in the network, it is not possible to trigger this procedure on packet loss, ultimately
ptg999 leading to the expiration of the retransmission timer and invocation of the slow
start procedure, which drastically impacts TCP throughput performance.
To address this problem with Reno, a modification called NewReno [RFC3782]
has been developed. This procedure modifies fast recovery by keeping track of the highest sequence number from the last transmitted window of data (the recovery point, which we first saw in Chapter 14). Only when an ACK with an ACK num- ber at least as large as the recovery point is received is the inflation of fast recov- ery removed. This allows a TCP to continue sending one segment for each ACK it receives while recovering and reduces the occurrence of retransmission time- outs, especially when multiple packets are dropped in a single window of data.
NewReno is a popular variant of modern TCPs—it does not suffer from the prob- lems of the original fast recovery and is significantly less complicated to imple- ment than SACKs. With SACKs, however, a TCP can perform better than NewReno when multiple packets are lost in a window of data, but doing this requires careful attention to the congestion control procedures, which we discuss next.
16.3.2 TCP Congestion Control with SACK
With the introduction of SACKs and selective repeat to TCP, a sender is able to make better decisions about what segments to send in order to fill holes at the receiver (see Chapter 14). In filling the receiver’s holes, the sender generally sends each of the missing segments, in order, until all of the retransmissions for the lost segments have been received successfully. This procedure differs from the basic fast retransmit/recovery procedure mentioned previously in a somewhat subtle way.
In the case of fast retransmit/recovery, when a packet is lost, the sending TCP transmits only the segment it believes is lost and is able to send new data if the window W allows. Because the window is inflated for each arriving ACK dur- ing fast recovery, with larger windows TCP typically is able to send some addi- tional data after performing its retransmission. With SACK TCP, the sender can be informed of multiple missing segments and would theoretically be able to send them all immediately because they would all be in the valid window. However, this might involve sending too much data into the network at once, thereby com- promising the congestion control. The following issue arises with SACK TCP:
using only cwnd as a bound on the sender’s sliding window to indicate how many (and which) packets to send during recovery periods is not sufficient. Instead, the selection of which packets to send needs to be decoupled from the choice of when to send them. Said another way, SACK TCP underscores the need to separate the congestion management from the selection and mechanism of packet retransmis- sion. Conventional (non-SACK) TCP mixes these together.
One way to implement this decoupling is to have a TCP keep track of how much data it has injected into the network separately from the maintenance of the window. In [RFC3517] this is called the pipe variable, an estimate of the flight size. Importantly, the pipe variable counts bytes (or packets, depending on the
ptg999 Section 16.3 Evolution of the Standard Algorithms 741
implementation) of transmissions and retransmissions, provided they are not known to be lost. Assuming a large value of awnd, a SACK TCP is permitted to send a segment anytime the following relationship holds true: cwnd - pipe ≥ SMSS.
In other words, cwnd is still used to place a limit on the amount of data that can be outstanding in the network, but the amount of data estimated to be in the net- work is accounted for separately from the window itself. How SACK TCP using this approach to congestion control compares with conventional TCP was first explored in detail with a series of simulations in [FF96].
16.3.3 Forward Acknowledgment (FACK) and Rate Halving
For TCP variants based on Reno (including NewReno), the typical behavior is that when cwnd is reduced after a fast retransmit, ACKs for at least one-half of the current window’s outstanding data must be received before the sending TCP is allowed to continue transmitting. This is an expected consequence of reducing the congestion window by half immediately when a loss is detected. It causes the sending TCP to wait for about half of an RTT and then send any new data during the second half of the same RTT, a more bursty behavior than is really required.
In an effort to avoid the initial pause after loss but not violate the convention of emerging from recovery with a congestion window set to half of its size on entry, forward acknowledgment (FACK) was described in [MM96]. It consists of two algorithms called “overdamping” and “rampdown.” Since the initial proposal, the authors updated their approach to form a unified and improved algorithm they call rate halving, based on earlier work by Hoe [H96]. To ensure that it works as effectively as possible, they further govern its behavior by adding bounding parameters, resulting in the complete algorithm being called Rate-Halving with Bounding Parameters (RHBP) [PSCRH].
The basic operation of RHBP allows the TCP sender to send one packet for every two duplicate ACKs it receives during one RTT. This causes the recovering TCP to have sent the appropriate amount of data by the end of the recovery period, but it spaces or paces this data evenly, rather than bunching all the transmissions into the second half of the RTT period. Avoiding the bunching or burstiness is advantageous because bursts tend to persist across multiple RTTs, stressing router buffers more than required.
To keep an accurate estimate of the flight size, RHBP uses information from SACKs to determine the FACK: the highest sequence number known to have reached the receiver, plus 1. Taking the difference between the highest sequence number about to be sent by the sender (SND.NXT in Figure 15-9) and the FACK gives an estimate of the flight size, not including retransmissions.
With RHBP, a distinction is made between the adjustment interval (the period when cwnd is modified) and the repair interval (when some segments are retrans- mitted). The adjustment interval is entered immediately upon a loss or conges- tion indicator. The final value for cwnd when the interval completes is half of the correctly delivered portion of the window of data in the network at the time
ptg999 of detection. The following expression allows the RHBP sender to transmit, if
satisfied:
(SND.NXT – fack + retran_data + len) < cwnd
This expression captures the flight size, including retransmissions, and ensures that if injecting another packet of length len, cwnd will not be exceeded.
Provided all the data prior to the FACK is indeed no longer in the network (i.e., is lost or stored at the receiver), this causes the SACK sender to be appropriately controlled by cwnd. However, it can be overly aggressive if packets have been reor- dered in the network because the holes indicated by SACK have not been lost.
In Linux, FACK and rate halving are implemented and enabled by default.
FACK is activated only when SACK is enabled and the Boolean configuration vari- able net.ipv4.tcp_fack is set to 1. When reordering is detected in the network, the more aggressive behavior of FACK is disabled.
Rate halving is one of several ways of pacing TCP’s sending procedure to avoid or limit burstiness. Although it offers a number of benefits, it also has a few problems. In [ASA00], the authors analyze TCP pacing in some detail using simu- lations, concluding that in many cases it offers inferior performance to TCP Reno.
Furthermore, rate-halving TCP has been known to exhibit poor performance when the connection may become limited by the receiver’s advertised window [MM05].
16.3.4 Limited Transmit
In [RFC3042], the authors propose limited transmit, a small modification to TCP designed to help it perform better when the usable window is small. Recall from the experience with Reno TCP that when operating with a small window, there may not be enough packets in the network to trigger the fast retransmit/recovery algorithms when loss occurs, as these algorithms typically require three duplicate ACKs to be observed prior to initiation.
With limited transmit, a TCP with unsent data is permitted to send a new packet for each pair of consecutive duplicate ACKs it receives. Doing this helps to keep at least a minimal number of packets in the network—enough so that fast retransmit can be triggered upon packet loss. This is advantageous to TCP because waiting for an RTO (which can be a relatively large amount of time—sev- eral hundred milliseconds) can degrade throughput performance considerably.
As of [RFC5681], limited transmit is now a recommended TCP behavior. Note that rate halving is one form of limited transmit.
16.3.5 Congestion Window Validation (CWV)
One of the issues with congestion management in TCP arises when the TCP sender stops sending for a period of time, either because it has no more data to send, or because it has been prevented from sending when it wants to for some
ptg999 Section 16.3 Evolution of the Standard Algorithms 743
other reason. If all goes well, a sender never pauses, and it continues sending data and receiving ACKs from its peer. This continuous feedback enables it to keep a reasonably current (within one RTT) estimate of what cwnd and ssthresh should be.
If the TCP sender has been sending for some time, its cwnd may have grown to a substantial size. If it then fails to send for some time but resumes later, the large cwnd may allow the sender to inject an undesirably large number of packets (i.e., a high-rate burst) into the network without delay. Furthermore, if the pause is sufficiently long, its last cwnd value may no longer be appropriate for the path and congestion state.
In [RFC2861], the authors propose an experimental Congestion Window Valida- tion (CWV) mechanism. Essentially, the sender’s current value of cwnd decays over a period of nonuse, and ssthresh maintains the “memory” of it prior to the initia- tion of the decay. To understand the scheme, a distinction is made between an idle sender and an application-limited sender. The idle sender has stopped producing data it wants to send into the network; ACKs for all the data it has sent so far have been received. Thus, the connection is truly quiescent—no data is flowing, so no ACKs are either, except for occasional window updates (see Chapter 15). The application-limited sender does have more data to send but has been unable to for some reason. This could be because the sending computer is busy doing other tasks, or because some mechanism or protocol layer below TCP is preventing data from being sent. This case results in underutilization of the allowed congestion window, but the connection is not completely quiescent. In particular, ACKs may still be returning for previously sent data.
The CWV algorithm work as follows: Whenever a new packet is to be sent, the time since the last send event is measured to determine if it exceeds one RTO. If so,
• ssthresh is modified but not reduced—it is set to max(ssthresh, (3/4)*cwnd).
• cwnd is reduced by half for each RTT of idle time but is always at least 1 SMSS.
For application-limited periods that are not idle, the following similar behav- ior is used:
• The amount of window actually used is stored in W_used.
• ssthresh is modified but not reduced—it is set to max(ssthresh, (3/4)*cwnd).
• cwnd is set to the average of cwnd and W_used.
Both of these changes decay the value of cwnd while “remembering” it in ssthresh. The first case can dramatically affect cwnd in one operation, if the applica- tion has been idle for a long time. Handling the congestion window in this way can lead to better performance under some circumstances. As the authors report, reducing the burst of packets that can arise after an idle period eases the pressure
ptg999 on potentially limited buffer space in routers, ultimately leading to fewer dropped
packets. Note that because cwnd is decayed and ssthresh is not, the typical conse- quence of applying this algorithm is to place the sender into slow start after a long enough pause. CWV is enabled by default in Linux TCP implementations.