Transparent Peer-to-Peer Network Connections

Back to My Mac

So, I was poking around the Internet today looking at tunneling technologies and stumbled across this little gem:

RFC 6281 - Understanding Apple's Back to My Mac (BTMM) Service

It's quite a tasty treat:

BTMM provides secure transport connections among a set of devices  that may be located over a dynamic and heterogeneous network environment.  Independent from whether a user is traveling and accessing the Internet via airport WiFi or staying at home behind a NAT, BTMM allows the user to connect to any Mac hosts with a click, after which the user can share files with remote computers or control the remote host through screen sharing. When a user changes locations and thus also changes the IP address of his computer (e.g., roaming around with a laptop and receiving dynamically allocated IP address), BTMM provides a means for the roaming host to update its reachability information to keep it reachable by the user's other Mac devices.  BTMM maintains end-to-end transport connections in the face of host IP address changes through the use of unique host identifiers.  It also provides a means to reach devices behind a NAT.

It proceeds to go into considerable detail as to how they pull this off. NAT, firewalls, IPSec, DDNS oh my!

Unfortunately, its achilles heel is the need to get incoming ports opened on your NAT/firewall device so that the BTMM peer-to-peer tunnel can connect. This is "addressed" by requiring your NAT/firewall to support NAT-PMP or UPnP. Not every provider (*cough* U-verse *cough*) supports these protocols in their equipment. So, you either have to do manual port opening and mapping or you are just out of luck.

Can a Server Make it Work?

This being a state-of-the-art implementation of transparent access to mobile devices stymied by such a fundamental problem, is there any hope for turn-key access for mere mortals? For some background, see this post at stackoverflow:

Is there a way to bridge two outgoing TCP connections in order to bypass firewalls and NAT?

NAT devices normally allow all traffic from the inside to the outside without restriction. When you start an outgoing TCP connection, the NAT device additionally sets up a mapping for the return traffic. The basic limitation, however, is on the other end of the connection - the destination device must either have an existing NAT mapping for the incoming traffic or it must be on the public Internet and not behind a NAT device.

As suggested by the stackoverflow poster, you therefore could use a server on the Internet to shuffle packets between TCP connections from two devices that want to communicate with each other. However, you end up with a server trying to act like a router and you have traffic running through the server rather than between the two peers. That's not an ideal situation but it does work - and works so well that this is the technique used by remote desktop tool TeamViewer for their transparent access:

When creating a session, TeamViewer determines the optimal type of connection. After the handshake through our master servers, in 70% of the cases a direct connection via UDP or TCP is established (even behind standard gateways, NATs and firewalls). The rest of the connections are routed through our highly redundant router network via TCP or http-tunnelling. You do not have to open any ports in order to work with TeamViewer! 

Working with NAT

Is there a way to avoid these drawbacks by using NAT itself? The idea is to leverage the standard NAT behavior so that two devices can simultaneously connect to each other and exchange traffic with no restriction. As far back as 2000 a Slashdot user sketched out a possible solution:

TCP splicing through simultaneous SYNs

TCP does handle simultaneous connections (see section 3.4 of the original TCP spec - RFC793). However, as the Slashdot poster indicates, the real problem is working with the NAT devices on each end to get the desired effect. The poster suggests using a third party server to assist with the connection but it still requires adjustments to NAT behavior in devices to have a chance of working.

More Information

The two techniques discussed above are the main basic approaches. For more detailed discussion and information about more variants see this excellent RFC:

RFC 5128 - State of Peer-to-Peer (P2P) Communication across Network Address Translators (NATs)

Circa 2008, this has some great information and even more references in it.

IPv6 Saves the Day?

Ultimately, this whole situation exists due to the fact that we use NAT devices at all. We were cursed with them as the available IPv4 addresses began running out. NAT allowed the Internet to continue to grow while kicking the address space issue down the road almost a decade. However, time finally ran out in January, 2011:

Address allocation kicks off IPv4 endgame

And the endgame won't last long:

IPv4 Address Report

By mid-2014, all public IPv4 addresses will be gone.

Of course, this situation was planned for starting back in 1998 with the design of IPv6. It directly dealt with the address space problem by quadrupling it in size. One participant said this amount of address space is "probably sufficient to uniquely address every molecule in the solar system".

With that much address space, there's no conceivable need for NAT boxes (ignoring any organizational want for them for policy purposes). And if you eliminate NAT boxes you eliminate the peer-to-peer communication problem.

Transparency at Last

With the forced adoption of IPv6 there is finally a direct and obvious path towards truly transparent peer-to-peer access. While NAT was probably directly responsible for the unfettered growth of the Internet in the face of address space shortage, it's timely death will finally enable truly turn-key peer-to-peer access.