References

TCP Fast Open (TFO) mechanism is described in https://tools.ietf.org/html/rfc7413

Other considerations can be found in this white paper.

Patches 

TFO is currently available available on Linux (full implementation in 4.1), OS X 10.12 and server side will be available for FreeBSd 10/11. There are differences between the implementations described below.

Patches for ldns, NSD and Unbound can be found in our Stash repo, which currently support TFO as implemented on Linux.

https://portal.sinodun.com/stash/projects/TDNS/repos/tcp_fast_open_patches/browse

Note that getdns 1.0 supports TFO on Linux and OS X.

To use these patches:

  1. Download the source code from the relevant repository

  2. Apply the relevant patch

  3. IMPORTANT! Run autoreconf 

  4. Add --enable-tcp-fastopen to the configure options

  5. make & make install as normal

Quick guide

TCP fastopen [I-D.ietf-tcpm-fastopen] (TFO) allows data to be carried in the SYN packet (and optionally in the SYN-ACK). It therefore saves up to one RTT compared to standard TCP. TFO clients request a server cookie in the initial SYN packet at the
start of a new connection. The server returns a cookie in its SYN-ACK. The client caches the cookie and reuses it when opening subsequent connections to the same server. The cookie is stored by the client's TCP stack (kernel) and persists if either the client or server processes are restarted. TFO also falls back to a regular TCP handshake gracefully. 

Message flow 

Requesting Fast Open Cookie in connection 1: 
TCP A (Client)                                   TCP B(Server) 
______________                                   _____________  
CLOSED                                                  LISTEN 
#1 SYN-SENT     ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD 
#2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD 
(caches cookie C) 
 
 
Performing TCP Fast Open in connection 2: 
 
TCP A (Client)                                   TCP B(Server) 
______________                                   _____________  
CLOSED                                                  LISTEN 
#1 SYN-SENT    ----- <SYN=x,CookieOpt=C,DATA_A> ----> SYN-RCVD 
#2 ESTABLISHED <---- <SYN=y,ACK=x+len(DATA_A)+1> ---- SYN-RCVD 
#3 ESTABLISHED <---- <ACK=x+len(DATA_A)+1,DATA_B>---- SYN-RCVD 
#4 ESTABLISHED ----- <ACK=y+1>-------------------->   ESTABLISHED 
#5 ESTABLISHED --- <ACK=y+len(DATA_B)+1>---------->   ESTABLISHED

 

Implementation details

Adding support for this to existing name server implementations is relatively easy, but does require source code modifications. It is also controlled via various kernel parameters documented below.

Client side code changes

Linux API

OS X API

connectx(fd, &endpoints, SAE_ASSOCID_ANY,  
	     CONNECT_DATA_IDEMPOTENT | CONNECT_RESUME_ON_READ_WRITE,
	      NULL, 0, NULL, NULL)

The Linux implementation follows the pattern suggested in the appendix of the RFC for maximum backwards compatibility, however the OS X API offers some advantages because TFO can be used in conjunction with libraries that require a 'connected' file descriptor to be available (e.g. OpenSSL). The hope is the Linux API will be extended to offer both modes (no FreeBSD client implemented yet).

Server side code changes

Kernel Parameters

Linux

OS X

FreeBSD

On-the-wire

The implementations have slightly different behaviour on the wire. Observations from testing include:

Clients

Servers