Wormhole - Functional Description

General

The wormhole is the gateway through which the packets exit and enter the Freedom network cloud. It is launched by the AIP and executes as a single independant process (not a child) on every AIP. The wormhole is responsible for launching the ident daemon, as an independant process also.


The wormhole uses a Unix Domain Socket (UDS) exchange traffic packets with the AIP. It also uses the UDS to send/receive messages to/from the AIP. The wormhole uses a second UDS to send messages to the ident daemon. Figure 1 illustrates the high-level architecture and information flow.




Figure 1: Wormhole data flow


We use a configuration file to select between the various modes of operation of the wormhole. During execution, it is also possible to send a limited set of commands to the wormhole using signals.


Main Algorithm

At startup, the wormhole reads the configuration parameters from the config file, creates a Unix Domain Socket (UDS) for communication with the AIP, creates the lists and hash tables to handle ACI-port mapping, launches the ident daemon and connects with its UDS.


Once the initialization is done, the execution is done in a main loop. For each loop, any signal that was received is first handled. Then, a check is done to see if a statistics report should be sent. In the case of the affirmative, the statistics report is assembled and sent to the AIP through the UDS. Then, execution is blocked until data is received on the AIP-wormhole UDS and/or from the networking stack (captured by the shim). Messages received on the AIP-wormhole UDS are processed first. See the "AIP-wormhole UDS commands/messages" section for details. Then, on Linux OS only, ARP request packets received by the shim are replied to. Finally, IP packets received by the shim are processed according to the type of transport layer they carry: TCP or UDP. Next, the loop is re-executed again... and again... and again... until termination order is received.


When the wormhole is ordered to terminate (either by UDS message from AIP or by a signal), it sends a termination message to through the AIP-wormhole UDS, frees its resources, terminates the ident daemon and exits.


ACI-Port Mapping (lists)

The wormhole emulates a client entity accessing a server entity on the internet. In order to achieve this, it replaces the packets IP source address with its own IP address and the TCP or UDP source port with one of its own pre-assigned ports before sending them on the internet. On the other hand, when packets come back from the internet, the inverse operation is done (on the destination fields this time). The correspondance is done between an ACI-client port pair and a wormhole pre-assigned port. This scheme avoids possible conflicts that could arise if two different routes were to use the same source port. This process is illustrated by Figure 2.




Figure 2: ACI-port to wormhole port mapping


In order to do those parameter substitutions, the wormhole must keep a mapping of the corresponding ACI-port to port pairs and keep track of the pre-assigned wormhole ports in use. To handle that matter, the wormhole manages a set of lists. The followings describe each of these lists.


A circular list is used to keep track of the free pre-assigned wormhole ports. The list is contained by an array, each element representing a free port number. Note that the port numbers are offsets from a base port number, so the free port numbers stored in the list can be from 0 to (number of pre-assigned ports - 1). The base port is identified by the baseport field in the wormhole context structure. The list is pointed to by the fracimap.freelist field, fracimap.freehead and fracimap.freetail are used to manipulate the list. Figure 3 illustrates this list.

Free wormhole ports




Figure 3: Circular list of free wormhole ports


ACI-port to wormhole port

A hash table is used to map packets that exit the Freedom cloud (ACI+port to wormhole port mapping). The hash table keys are formed with the ACI, the original port number and a source address. The source address is always set to 0. The keys refer to a memory location containing the wormhole port number to which the ACI-port pair is mapped. The port used for the key is an absolute value while the wormhole ports are relative to the baseport field in the wormhole context structure. The hash table is kept by the acimap.acimap field while the wormhole ports are stored in an array kept by acimap.portnum. The hash table and array are represented in Figure 4.




Figure 4: ACI-port to wormhole port map


Wormhole port to ACI-port

An array of structure is used to map packets that enter the Freedom cloud (wormhole port to ACI-port mapping). There is one element in the array for each possible wormhole port: element 0 represents wormhole port with offset 0 from baseport. Each element is formed of an ACI, a port number and an IP address. The ACI is used to build the Freedom packet header and indicates the route the packet should be forwarded to. The port number substitutes the destination port of the transport layer headder (comes in as wormhole port). The IP address replaces the destination IP address of the IP header (comes in as wormhole IP address).




Figure 5: wormhole port to ACI-port map


ACI ports inventory list

Each ACI corresponds to a client created route exiting at the wormhole. For a given ACI, there can potentially be an unlimited amount of UDP and/or TCP ports used simultaneously. However, because the number of available pre-assigned wormhole ports is finite, a configurable limit is set on the maximum amount of simultaneous ports per ACI. This is also necessary to avoid having a client keep all the wormhole ports for himself. Therefore, some kind of infrastructure is required to keep an inventory of the ports in use by each ACI. In addition, some extra information is kept on each port entry to allow for intelligent creation and deletion of ports.


The head of the inventory list is a structure containing the limit on simultaneous ACIs, the limit of simultaneous ports per ACI and a reference to a hash table. This structure can be referenced using acimap.portmap from the wormhole context structure. The hash table uses the ACI number as a key, and each key refers to a structure. This structure contains a pointer to a list of ports, the current number of ports in the list and the limit on the number of ports. Finally, each element of the port list is a structure containing the port number, the protocol (UDP or TCP), a timestamp of the last time a packet travelled on this port and, for use with TCP ports, a last TCP state field and the last sequence number. This infrastructure is illustrated by Figure 6.





Figure 6: ACI port inventory infrastructure


Map item creation rules

The limited amount of wormhole pre-assigned ports and the limit on the amount of ports per ACI create the need for a set of rules for ACI-port to port map creation. The creation rules are best illustrated by a flow-chart diagram in Figure 7. Note that a port will be considered timed out if no packets have travelled through it for 5 minutes (configurable).




Figure 7: Port creation flow-chart



AIP-wormhole UDS commands/messages

The AIP and the wormhole communicate via a UDS with commands and messages. The following table presents the commands that the AIP can send to the wormhole.


Command

Description

WORMCMD_CMD_OPENPORT

Enable a range of ports of a protocol type (UDP or TCP) by changing their access level. (Currently never used)

WORMCMD_CMD_BLOCKPORT

Block a range of ports of a protocol type (UDP or TCP) by changing their access level. (Currently never used)

WORMCMD_CMD_CONNECT

Notifies wormhole of which UDS socket to use. If the FLAG_FRESHCONNECT flag is set, it also destroys the ACI-port to port map and builds a fresh, empty one. (Currently never used)

WORMCMD_CMD_NEWMAP

Gives the name of the nym that corresponds to a specific ACI. Later, the wormhole can transmit it to the Ident daemon.

WORMCMD_CMD_DELMAP

Orders the wormhole to remove the name of the name corresponding to a specific ACI from its table and from the Ident daemon.

WORMCMD_CMD_KILL

Orders the wormhole to clean up, terminate and send an acknowledge message back.

WORMCMD_CMD_SEND

The AIP relays a packet to the wormhole so it can send it to the internet. For more details, see the "Freedom-Cloud-to-Internet packet handling" section.


Table 1: AIP to wormhole commands



Then, the following table describes the commands that wormhole can send to the AIP.


Command

Description

WORMCMD_CMD_SEND

The wormhole relays a packet to the AIP so it can route it to a client through the Freedom cloud. For more details, see the "Internet-to-Freedom-Cloud packet handling" section.

WORMCMD_CMD_MSG

Contains a message concerning the wormhole status. Consult the following table for message types and description.

WORMCMD_CMD_STAT

The wormhole sends it statistics report to the AIP, so they can be relayed to the NISS.


Table 2: Wormhole to AIP commands

The following table indicates the different messages the wormhole can send to the AIP using the WORMCMD_CMD_MSG commands.


Message

Description

WORMCMD_MSG_EACIFULL

No more available ACI connections. (Currently never used)

WORMCMD_MSG_EACIPRESENT

ACI is already active. (Currently never used)

WORMCMD_MSG_EADDACIFAILED

Operation to add an ACI failed.

WORMCMD_MSG_EDELACIFAILED

Operation to delete an ACI failed.

WORMCMD_MSG_EACINOTFOUND

Requested ACI could not be found. (Currently never used)

WORMCMD_MSG_TERMINATING

Wormhole is terminating.


Table 3: Wormhole to AIP messages


The format of a command is illustrated by the following figure. The length field indicates the number of bytes in the command. The length includes everything from the start of the length field to the end of the command specifics. The reserved bytes are unused for the moment. The command specifics contain the data that accompany the command. They vary for different commands and can be absent.





Figure 8: AIP-wormhole UDS command


Internet-to-Freedom-Cloud packet handling

Two types of packet can be received from the internet: ARP requests and IP packets.


On Linux, the ARP requests are handled by the wormhole while on Solaris, they are handled underneath the stack. Therefore, on Linux, the ARP requests are retreived, encapsulated in an Ethernet frame. First, the target IP address in the ARP request is verified, if it corresponds to the wormhole IP address, an ARP reply is built and sent back. Otherwise, the ARP request is dropped without further processing.


The IP packets are retreived from the shim. The packet can be encapsulated in an Ethernet frame or not, depending on whether it came from the network or the local host. Only the packets carrying TCP or UDP protocol are processed any further, others are discarded. Both TCP and UDP types are processed in the same way except for one difference. For both types, the destination port is used to retreive the corresponding ACI-port mapping. If no mapping is found, the packet is dropped. Otherwise, the destination port and destination address of the packet are replaced with the ones found in the mapping. In the case of a TCP segment, any option present in the header is stripped. If the TCP SYN flag is up, the MSS option is added/replaced by the MSS of the Freedom cloud. Then, the packet size is compared to the Freedom cloud MSS. If it is bigger or if the shim requested it, the packet is fragmented. Once all this processing is done, the resulting packet or fragments are encapsulated into WORMCMD_CMD_SEND command packets and sent to the AIP through the UDS.


Freedom-Cloud-to-Internet packet handling

The IP packets coming from the Freedom cloud and destined to the internet are communicated to the wormhole by the AIP through the UDS, encapsulated in a WORMCMD_CMD_SEND command. The wormhole only processes TCP and UDP segments, the rest are discarded. Most processing for both protocols is the same. First, the destination port access level is verified. If the access level is blocked, the packet is discarded. Otherwise, the ACI-port to port mapping is obtained or created, if it didn't already exist. The source port is then replaced with the mapped wormhole port and the source IP address with the wormhole IP address. The port timestamp is updated in the portmap list, as well as the TCP state and sequence number in the case of TCP ports. Also, TCP options are stripped off and MSS option is added/replaced with the MSS of the Freedom cloud. The resulting IP packet is then sent on the internet.


Statistics

The wormhole communicates its statistics to the NISS periodically. When a statistics period comes to an end, the wormhole gathers its statistics, packages them and send them inside a WORMCMD_CMD_STAT command to the AIP, via the UDS. The AIP is then responsible for relaying the statistics report to the NISS.


Ident Daemon

The Ident daemon is used by some internet servers (more specifically in our case: IRC) to obtain the name of the user of a specific port on a machine. Here, it returns the name of the nym using the port. (At least that's what it was originally intended to do, currently it always returns "Anonymous User").


Signals

Certain wormhole functionalities can be initiated using signals. The following table describes them.

Signal

Description

INT, TERM

Terminate the wormhole

HUP

Re-open wormhole log file

CHLD

Reap children


Table 4: Wormhole signals


Configuration File

Various parameters of the wormhole can be configured using a configuration file loaded during startup. The following table describes those parameters.


Parameter

Values

Description

debugFile

pathname

Log file pathname

debug

Yes/No

Enable debug messages

networkDevice

device name

Network device pathname

realAddress

IP address (x.x.x.x)

IP address of the wormhole host machine

wormholeAddress

IP address (x.x.x.x)

IP address of the wormhole

unixDevice

pathname

Unix Domain Socket pathname

maxPacketSize

integer

Maximum packet size that can enter the Freedom cloud

fragCacheSize

integer

Number of IP fragments that can be kept concurrently by the fragmentation cache

tcpTimeout

integer

Timeout (in seconds) of TCP port in the wormhole

udpTimeout

integer

Timeout (in seconds) of UDP port in the wormhole

basePort

integer

Base port number (lower limit) to use to send packets on the internet

endPort

integer

Last port number (upper limit) to use to send packets on the internet

maxNbPort

integer

Maximum number of wormhole ports that can be assigned to one route concurrently

redirectBasePort

integer

Base port number for the wormhole to redirect traffic to the local host

redirectEndPort

integer

Last port number for the wormhole to redirect traffic to the local host

redirectDuration

integer

Time that the local host redirection map will live

localDevice

pathname

Local loopback device pathname

pidFile

integer

PID file name

logFile

pathname

Wormhole log file pathname

mirrorMode

Yes/No

Bounce unsollicited traffic back to the sender

cleanOnConnect

Yes/No

Clean up ACI maps when a new AIP connects to wormhole

suppressArp

Yes/No

Suppress wormhole ARP behavior

stripIpHeader

Yes/No

Strip any IP option from header

stripTcpHeader

Yes/No

Strip any TCP option from header

hacks

Yes/No

Enable hacks (Kent, Bananarea, MSS)

frequency

integer

Time (in seconds) between statistics updates

MSSIncoming

integer

Max segment size of packets going in the Freedom cloud, from the Internet

MSSOutgoing

integer

Max segment size of packets going to the Internet, from the Freedom cloud

throughputIncoming

integer

Number of bytes per second that a wormhole should accept from the Internet

fragmentationSize

integer

Number of bytes for UDP segment fragments

serviceRedirect

Yes/No

Port redirection map so that wormhole can send traffic to known port to a server on local machine

useIdent

Yes/No

Use the ident daemon

serverPath (identd only)

pathname

Ident daemon path name

port (identd only)

integer

Ident daemon port number

command (identd only)

pathname

Wormhole-Ident daemon UDS pathname

logFile (identd only)

pathname

Ident daemon log file pathname

pidFile (identd only)

pathname

Ident daemon PID pathname


Table 5: Wormhole configuration parameters