Network Bandwidth vs. Network Latency
Purpose
It has always been curious to me that we accept a statement of bandwidth only from our ISPs, when latency is just as important. I may be able to send you an elephants weight (~8 tons) in gold over the next year, but wouldn’t you rather have a mouse’s weight every second until then (~700 tons)? Network latency is big deal. Too much network latency and we can bog down the performance on our servers and clients, and exhaust all of our bandwidth with unneeded retransmissions. The goal of this article is to talk about the difference between Network Bandwidth and Network Latency, and how both come into play in modern networks.
Network Bandwidth
It seems a good place to start to discuss what network bandwidth really is. Bandwidth is simple the amount of data that we can send/receive at any given second. Most typically, whether a home user or large enterprise when you go and look for connections to the internet, bandwidth is our first concern.
It is worth note that there is a big difference between the bandwidth that you are sold as a home user and that you get as an enterprise. That difference is your guaranteed rate, or CIR (Committed Information Rate). In the enterprise world (US), you typically buy something like a T1 or DS1, which is 1.544 Mbit/s up and down (meaning to and from the internet).
One tricky thing when talking about network connections is that the rates are in Mega Bits, not Mega Bytes (a difference of 8 times; 8 bits = 1 byte). I know you may be looking at that rate of a T1 (typically an enterprise solution around $250/month and up) and saying “My home ISP connection is much faster than 1.5 Mbit/s”, my answer to that is (if you are using traditional cable or DSL): “no, it’s not”. The CIR that comes with a T1 or similar connection promises that they will be able to send and receive 1.5 Mbit/s every second of every day. I know your ISP promised you 5 or 6 Mbit/s, but how often do you actually get that? Still, the gap is closing, and that is why ISPs now filter what services you can use on home internet connections.
Network Latency
As stated above, while network bandwidth determines how much you can send, network latency determines how long it takes to get there. This is measured in ms (milliseconds or 1/1,000,000 of a second). For the most part latency is determined by how close you are to something. The unit of measurement is so small that it accounts for the amount of “copper” or network medium between you and whatever you are trying to contact. Unfortunately this is not all that plays into network latency, as every network device that you touch along the way adds to the latency (in varying amounts) as it has to process that packet and send it on its way. Over all though you want all your connections to or across the internet to ideally have less than 120 ms RTT or Round-Trip-Time. Today though the standard is shrinking, applications are more demanding, and infrastructure is improving. I now like to see RTTs or no more than 90 ms, and optimally in the 30-60 ms range. This ensures that packets are delivered on time, and little to no delay is seen.
On the intranet, I like to see rates of no more than 60 ms. Ideally we are aiming for 10-30 ms RTTs. This of course increases if we go across a WAN connection, but in any single site, we should not get larger than this, especially when you consider that you can control you intranet latency mush easier than you can you internet latency (which WAN segments typically cross over).
Effects of High Network Latency
One of the greatest marvels of the modern network is the TCP/IP protocol suite, and specifically TCP. TCP is connection oriented which means that it will not send data until it knows you are listening, and it expects you to acknowledge every piece of data that it sends, or it will send it again. In almost all cases this works well to our benefits. Where it does not work well, is when we do not give it the right tools to work with (though it can be tuned for almost any environment, short of No network connection). In high-latency networks what you typically see is TCP trying to resend data. It does this, because it knows that you can only acknowledge data that you receive, so if it does not get an acknowledgement of the sent data it assumes it was lost in transit, and resends it.
On high-latency, low-bandwidth networks, this is devastating, because now you duplicate all your data, and eat up whatever bandwidth you may have. This retransmission is going to happen until either the connection is terminated prematurely, or you acknowledge that I sent you the data. So this can quickly grow into sending the data 2, 3, 4, even 5 times or more. Obviously this all causes further latency, as bandwidth is exhausted and packets are queued on the network. In addition there is the work of the client and server in re-packaging the same data over and over again, and its buffers filling for both the TCP driver and the application sending the data. Ever seen your outlook client go white when downloading mail? This is probably why.
Even on the networks with the widest bandwidth, you still end up having to resend the data, over and over again until the packet is acknowledge. Do you really want to eat up all your expensive bandwidth with unneeded retransmissions?
In short, always look at the network latency as well as the network bandwidth, otherwise you may be getting very little in the end. In a future post I will try to tackle how to tune TCP on Clients and Servers to work better in the situations where you cannot help but have high-latency connections.