Redundant network buffering

The problem of excessive buffering occurs when there is a device with a buffer too large on the way from the source to the packet receiver. As a rule, buffers are present in almost all nodes of the network: switches, routers, stacks of operating systems, etc. They are designed to temporarily store “redundant” packets so that they are not lost when the sender transfers data to the node faster than it can get the recipient. This occurs when the bandwidth of the sender is higher than that of the recipient. Buffering delays the transmission of a packet by several milliseconds. If the buffer fills up, the next packet is discarded. Congestion control protocols detect this on the sender side and reduce the transmission speed. Data continues to be transmitted using the highest possible bandwidth.

But if the buffer at some network node is too large, then it continues to accumulate packets for a long time. Because of this long pause, congestion control algorithms fail and do not function as they should. Such a thing as a large and variable packet delay begins to occur, the "narrow neck" of the network "suffocates" from an excess of packets from one TCP stream, and other packets are discarded. Congestion occurs. After some time, the buffers are freed, then the transmission speed increases until it fills the buffers again, until the next jam.

From the point of view of ordinary users, the main symptoms of excessive network buffering are when the network is under load (a lot of data is transmitted), ordinary web pages load for a very long time (several seconds, or even minutes); any services that require constant bandwidth (no matter high or low), such as VoIP, network games, chat, interactive applications such as remote access, become impossible to use. Prioritization methods (QoS), in which a separate packet queue is created for a certain type of traffic, help little to solve the problem.

The problem of excessive buffering is caused mainly by manufacturers of routers, switches and developers of operating systems, which in recent years have begun to install too large buffers (several megabytes) on devices, which, in turn, is caused by a sharp decrease in the cost of memory.

The problem can be solved by simply reducing the size of the buffers on the network equipment. However, it is not configured on most routers and switches, especially if they are located outside the reach of ordinary users.

Sources of the problem

Any network service or any activity on the network that transfers large amounts of data or a large number of packets through the network can cause excessive network buffering. Among them:

• Uploading videos to youtube, files to Crash dump and other services
• Downloading movies from the Internet, watching movies online
• Copying files over the network, creating backups
• Emails with large attachments
• Torrent Clients
• Video conferencing
• Web surfing on sites with media content: youtube, google maps, etc.

Where is the

The phenomenon can occur wherever buffering occurs. First of all, a traffic jam is created on that node, after which there is the narrowest bandwidth. For example:

• Operating system network stack
• Network Card, Network Card Driver
• Wireless access point, home switch, home router, home modem
• DSLAM , CMTS , etc.
• All routers, bridges, gateways on the way from the source to the receiver
• Satellite Communication Devices

Consequences

Many protocols and network services suffer from congestion and long response times. For example:

• DNS 100-ms delays kill DNS queries from browsers
• ARP
• DHCP The machine cannot get into the network
• RA, ND
• VoIP, 1 packet every 10 ms should pass for normal operation. The spread should be no more than 30 ms.
• Network games.
• Streaming video.
• Any other network applications.

However, large network buffers are needed for normal processing of packets with large MTUs , for example, Jumbo frames .

Background

The problem of network congestion is an old network problem that existed from the first years of their existence. Network congestion causes bandwidth degradation, increased packet transit times, and packet loss. As a result of network congestion, some network services simply stop working.

The first manifestation of network congestion on a large scale occurred in 1986 on the NSFNet network ^[3] . In response to this event, in 1988 Van Jacobson's Congestion Control protocol was developed to control network congestion.

The Internet continued to grow. In 1993, S. Floyd and W. Jacobson developed the RED algorithm ( Random Early Detection) to control the overflow of queues of routers ^[4] .

RFC 2068 came out in 1997, which formulated the “principle of two connections”: one client should use no more than two connections simultaneously with the same server ^[5] . According to the same RFC, this recommendation is given "to reduce the HTTP response time and avoid overloading the Internet or other networks."

A year later, RFC 2309 came out : “Guidelines for managing queues and avoiding Internet congestion,” which suggested measures to improve and maintain Internet performance.

In 2001, RFC 3168 came out : “Adding Explicit Congestion Notification (ECN) to IP”, which suggested adding an ECN field in IP and TCP packets, reserving 2 bits for this.

In 2004-2007, one of the largest US Internet service providers, Comcast, experienced difficulties due to network congestion. This is evidenced by repeated Internet disconnections for “heavy" users ^[6] . And in 2007, Comcast, according to Jim Gettys, “gasps” from bittorrents ^[7] . The company is accused of blocking torrent traffic ^{[8] [9]} and is even harassed by lawsuits ^[10] .

Problem Detection

According to Jim Gettys, the first to discover the problem of excessive network buffering was Dave Clark ^[7] . In 2004, he observed this phenomenon on his DSLAM and used it to discourage his son from the habit of playing WOW for a long time ^[11] .

In 2007, Jim Gettys himself began to receive complaints from his family about the poor Internet and was experiencing damage to equipment from lightning surges ^[7] .

In 2009, Dave Reid announced too long a turn-around time (RTT) of packets (up to 30 seconds) with low packet losses on 3G networks, by making a message on the full-cycle mailing list. Jim Gettys himself recorded in 3G RTT networks up to 6 seconds.

Gettys continues to receive complaints from the family about the slow internet. In April 2010, he conducted the test “bandwidth / latency” ^[12] . The test result showed that the waiting time (delay) of packets increases with prolonged data transfer. On July 15, 2010 Gettys held a joint lunch with representatives of Comcast ^[13] , where the cause of the problem was proposed: the buffers are too large. The reason, in turn, was proposed to the company two years earlier by Dave Clark, but the company could not find evidence of this. Other reasons for buffer swelling: RED is often not included in networks because it requires complex configuration; ECNs are blocked on some networks because they cause home router crashes.

Gettys continued his research, taking measurements at home and on business trips. The measurement of September 20 showed a delay of 8 s on the path, which usually takes 10 ms ^[14] . Then Gettys reproduced this on other routers of different manufacturers.

In November, Gettys reproduced the problem on different operating systems. It turned out that under Linux and Mac this problem is easier to reproduce than under Windows. Gettys concluded: “it smacks of something in the network stacks of operating systems” ^[15] .

December 3, 2010 Jim Gettys publishes an article on The Criminal Mastermind: bufferbloat! On his blog, where he gives the name to this phenomenon - bufferbloat . In this and subsequent articles, Gettys talks about the essence of the phenomenon, places of occurrence, causes and consequences ^[16] .

Reaction

Robert Kringley, a journalist writing for InfoWorld, in his 2011 Predictions: One Word is bufferbloat. Or are these two words? ”Predicts that excessive network buffering will be the biggest challenge of 2011 ^[17] . Referring to Gettys, he gives a description of the problem. After 3 days, ars technica published an article by Ilyich van Beinum, in which he pointed out that some of the details described by Kringley were incorrect, but at the same time confirmed the existence of a problem of excessive network buffering ^[18] .

Soon, the www.bufferbloat.net project was created, the purpose of which was to coordinate the efforts of concerned individuals to solve the problem of excessive network buffering. The main objectives of the project:

• Bring together experts to tackle queue management in network stacks
• Disseminate problem information to end users
• Create tools to demonstrate and diagnose the problem
• Experiment with improved congestion management
• Create patches for popular OS, device drivers, and TCP / IP stacks.

• Jim Gettys Bufferbloat. Dark Buffers in the Internet (Febrary 21) . - Bell Labs, 2011 .-- 77 p.
• Jim Gettys Bufferbloat. Dark Buffers in the Internet (April 3) . - Bell Labs, 2011 .-- 36 p.
• Jim Gettys Bufferbloat. Dark Buffers in the Internet (March 24) . - Bell Labs, 2011 .-- 36 p.
• Jim Gettys Bufferbloat. Dark Buffers in the Internet (June 14) . - Bell Labs, 2011 .-- 44 p.
• Jim Gettys Bufferbloat: Dark Buffers in the Internet (English) // Internet Computing, IEEE: Journal. - IEEE Computer Society, 2011. - Vol. 15 , iss. May / June , no. 3 . - P. 96-95 . - DOI : 10.1109 / MIC.2011.56 .

• www.bufferbloat.net (English) (Site of a joint project to solve the problem of excessive network buffering)
• jg's ramblings ( J. Gettys blog , where he first described the problem of excessive network buffering)