Why VPN tunneling over TCP sucks!

TCP is one of today’s most popular protocol choices for VPN connections. In fact, some of the most widely used VPN protocols – including SonicWall SSL, Cisco AnyConnect SSL, Fortinet SSL, and even many OpenVPN connections – are based on TCP. With this fact in mind, it must be a pretty solid choice for VPN, right?

Wrong!

Keep reading to find out everything that’s wrong with TCP, what ‘TCP meltdown’ is, and why you should absolutely choose a different protocol for your VPN.

Why do we need additional protocol layers for VPN?

In order to work, a VPN protocol needs to tunnel network packets over a network. ‘Network’ today usually means an IP network. However, sending data directly with IP packets is not a good idea because IP is not designed for this task; rather the main task of the IP layer is purely to address packets.

Think of IP as an envelope with the sender's address and a destination address, simply so the postal service can ensure its contents (i.e. packets) are forwarded to the correct address and so the recipient knows who sent the letter – even without opening the envelope.

Think of IP as a simple envelope

However, this is where the identification process ends. IP does not care if the letter reaches its destination or what happens to it once it gets there, as the envelope contains no further identifying information (i.e. for a household, which family member is it for or for an office, which department?)

For this reason, there is almost always at least one extra protocol layer on top of IP before there is any actual content data, in order to tell the computer what to do with it once received.

Software typically has three options to choose from:

1. TCP (Transmission Control Protocol)
2. UDP (User Datagram Protocol)
3. Another more obscure or custom protocol

TCP and UDP sit on the Transport layer of the internet protocol suite

Which protocol options are available?

ESP & AH

IPsec chose the third option in the above list and specified two VPN protocols of its own that both sit directly on top of IP. They are known as ESP (Encapsulating Security Payload) and AH (Authentication Header). ESP is the protocol everyone uses today, as AH offers no encryption – making it pretty useless for a VPN connection.

P.S. Want to Learn more about the origins of IPsec? Check out our blog post about IKEv2!

GRE

PPTP does not have its own protocol, but instead repurposed and customized an existing protocol called GRE (Generic Routing Encapsulation). PPTP uses a GRE variant that differs slightly from the variant originally used by Cisco, from the later standardized variant and also from the later standardized Extended GRE.

Learn more about PPTP VPN.

UDP

UDP seems to be a logical choice for VPN protocols. The protocol is simple, fast and has a very low protocol overhead. There is no guarantee that sent data will ever arrive, but it ensures that corrupted data is detected by the receiver and can be discarded. Just like ESP and GRE, it is connectionless, meaning it does not require a handshake to explicitly establish a connection and leaves any necessary negotiation to another protocol layer or connection.

UDP is used by NAT-Traversal, an IPsec extension that wraps ESP packets in UDP to traverse gateways that use NAT and can often handle UDP and TCP, but may never have heard of ESP. GRE can also be packaged in UDP, but PPTP never uses that.

UPD is the default protocol for OpenVPN. (Note that OpenVPN can also use TCP, but UDP is strongly recommended, and you'll soon learn why!)

Finally, Cisco AnyConnect and Fortinet SSL can optionally use a secondary UDP tunnel, but this feature is not always available. They also use DTLS for the UDP channel and let's just call that "suboptimal".

TCP

TCP is connection-oriented, i.e. it can independently negotiate a connection with a remote peer, and it is reliable, i.e. TCP ensures that all data sent over a TCP connection reaches its destination. In fact, if it does not, the TCP connection as a whole fails at some point when TCP gives up.

If a data packet is lost or corrupted along the way, TCP retransmits it – multiple times if necessary – until it finally arrives at its destination. TCP can also split larger packets into smaller ones on retransmission if it suspects that the packet may have been lost because it was too large, or even as a way to reduce the amount of data retransmitted if packets keep arriving damaged.

That doesn't sound so bad, right? And with TLS (Transport Layer Security), officially known as SSL (Secure Sockets Layer), there is even a protocol that is built on top of TCP, offering everything a VPN needs:

• User authentication
• Data integrity
• Encryption

With this, all that's required for a VPN connection is to somehow wrap network packets locally, send them over a TLS connection, unwrap them at the destination, and release them onto the network. It doesn't get much simpler than that, which is why a whole bunch of VPN protocols exist that are just thin protocols built on top of TLS. This includes SonicWALL SSL, Cisco AnyConnect, and Fortinet SSL. Even OpenVPN offers it as an option. In fact, there are about a dozen more proprietary VPN protocols, all known as SSL VPN protocols.

However, in reality, TCP is a very poor choice as the basis for a VPN protocol. The problem is known as "TCP Meltdown".

What is a TCP Meltdown?

A VPN tunnels network packets, meaning it also tunnels packets from TCP connections. Consider the following case: A TCP connection over VPN sends a packet. This packet is encrypted, repackaged into TCP and sent over the internet. For some reason, this packet gets lost along the way and never arrives at its destination. What will the outer TCP VPN connection do? It will retransmit the packet, of course, and make sure that the packet eventually arrives, so the loss has already been compensated for by the outer connection.

But wait… What will the inner TCP connection do? The one that originally generated the packet and knows nothing about the VPN tunnel? It will also realize that the packet never arrived, so it will retransmit it as well. The problem? The VPN protocol has no way of knowing that it is a retransmission. Instead, it thinks it is a completely new packet and proceeds to tunnel it like any other packet. Can you already see where this is going?

Each lost packet is retransmitted at least twice, and by the time it has been lost a second time, it has already been retransmitted three times, because the outer TCP connection retransmits both lost packets: the original one and the first retransmitted one of the inner connection. However, at the same time, the inner TCP connection also retransmits the packet, meaning the same packet has now been retransmitted three times in a row, i.e. six times in total:

The first attempt, followed by two retransmission attempts, followed by three retransmission attempts and so on and so forth.

If packets continue getting lost, it quickly gets to the point where there are so many retransmission attempts on the line that it gets blocked, leading to other packets also becoming lost. These additional packet losses then proceed to generate even more retransmission attempts, which also pile up, with the whole thing resulting in a never-ending doom loop.

After a while, the line is filled up with old packets being retransmitted, leaving no room for new packets to get through. Eventually, the outer TCP connection gives up after realizing some sent packets never reached the other side. TCP terminates the connection and the VPN tunnel is lost, along with all the data that was still in circulation. This is known as TCP meltdown.

Why is UDP better than TCP?

This meltdown scenario does not happen with UDP, because UDP does not retransmit anything by itself. In the case of UDP, only the inner TCP connection retransmits lost packets, so nothing stacks up and the inbound problem is avoided. For this reason, UDP is a very good choice as a basis for a VPN protocol, unlike TCP!

Despite being considered the “more reliable protocol" of the two, a TCP VPN ironically only works reliably as long as the line is near perfect. Even if it doesn't break down completely, there will eventually be duplicate retransmissions that increase latency and noticeably decrease VPN throughput. Moreover, TCP is the slower protocol by design, since it has more processing overhead and also more protocol data overhead in each packet, meaning its performance is already worse to begin with.

Performance can be easily measured by configuring two OpenVPN server processes on the same host with identical configurations, one using UDP and one using TCP. The UDP connection will always win easily against the TCP connection in any benchmark, and if there is a more severe problem, both will be affected, but the UDP connection will simply slow down, while the TCP connection will get slower and slower until it comes to a complete stop. Shortly after that, the connection drops.

So what is TCP SSL good for anyway? Unless you have to use TCP because you're behind a firewall that lets TCP through but blocks UDP, it's really not good for anything. This is the only valid reason why you should use a TCP-based VPN instead of a UDP-based VPN.

In short, if you can use both, if both work, and someone asks you which type of VPN tunneling you prefer, "TCP or", the answer is always "or". TCP sucks (for VPN!)

Why VPN Tracker?

VPN Tracker is the best secure remote access solution for Mac, iPhone and iPad and is compatible with the most popular VPN gateways, including UniFi, NETGEAR, TP Link, Draytek, and many more.