The wormhole is the gateway through which the packets exit and enter the Freedom network cloud. It is launched by the AIP and executes as a single independant process (not a child) on every AIP. The wormhole is responsible for launching the ident daemon, as an independant process also.
The wormhole uses a Unix Domain Socket (UDS) exchange traffic packets with the AIP. It also uses the UDS to send/receive messages to/from the AIP. The wormhole uses a second UDS to send messages to the ident daemon. Figure 1 illustrates the high-level architecture and information flow.
Figure 1: Wormhole data flow
We use a configuration file to select between the various modes of operation of the wormhole. During execution, it is also possible to send a limited set of commands to the wormhole using signals.
At startup, the wormhole reads the configuration parameters from the config file, creates a Unix Domain Socket (UDS) for communication with the AIP, creates the lists and hash tables to handle ACI-port mapping, launches the ident daemon and connects with its UDS.
Once the initialization is done, the execution is done in a main loop. For each loop, any signal that was received is first handled. Then, a check is done to see if a statistics report should be sent. In the case of the affirmative, the statistics report is assembled and sent to the AIP through the UDS. Then, execution is blocked until data is received on the AIP-wormhole UDS and/or from the networking stack (captured by the shim). Messages received on the AIP-wormhole UDS are processed first. See the "AIP-wormhole UDS commands/messages" section for details. Then, on Linux OS only, ARP request packets received by the shim are replied to. Finally, IP packets received by the shim are processed according to the type of transport layer they carry: TCP or UDP. Next, the loop is re-executed again... and again... and again... until termination order is received.
When the wormhole is ordered to terminate (either by UDS message from AIP or by a signal), it sends a termination message to through the AIP-wormhole UDS, frees its resources, terminates the ident daemon and exits.
The wormhole emulates a client entity accessing a server entity on the internet. In order to achieve this, it replaces the packets IP source address with its own IP address and the TCP or UDP source port with one of its own pre-assigned ports before sending them on the internet. On the other hand, when packets come back from the internet, the inverse operation is done (on the destination fields this time). The correspondance is done between an ACI-client port pair and a wormhole pre-assigned port. This scheme avoids possible conflicts that could arise if two different routes were to use the same source port. This process is illustrated by Figure 2.
Figure 2: ACI-port to wormhole port mapping
In order to do those parameter substitutions, the wormhole must keep a mapping of the corresponding ACI-port to port pairs and keep track of the pre-assigned wormhole ports in use. To handle that matter, the wormhole manages a set of lists. The followings describe each of these lists.
A circular list is used to keep track of the free pre-assigned wormhole ports. The list is contained by an array, each element representing a free port number. Note that the port numbers are offsets from a base port number, so the free port numbers stored in the list can be from 0 to (number of pre-assigned ports - 1). The base port is identified by the baseport field in the wormhole context structure. The list is pointed to by the fracimap.freelist field, fracimap.freehead and fracimap.freetail are used to manipulate the list. Figure 3 illustrates this list.
Figure 3: Circular list of free wormhole ports
A hash table is used to map packets that exit the Freedom cloud (ACI+port to wormhole port mapping). The hash table keys are formed with the ACI, the original port number and a source address. The source address is always set to 0. The keys refer to a memory location containing the wormhole port number to which the ACI-port pair is mapped. The port used for the key is an absolute value while the wormhole ports are relative to the baseport field in the wormhole context structure. The hash table is kept by the acimap.acimap field while the wormhole ports are stored in an array kept by acimap.portnum. The hash table and array are represented in Figure 4.
Figure 4: ACI-port to wormhole port map
An array of structure is used to map packets that enter the Freedom cloud (wormhole port to ACI-port mapping). There is one element in the array for each possible wormhole port: element 0 represents wormhole port with offset 0 from baseport. Each element is formed of an ACI, a port number and an IP address. The ACI is used to build the Freedom packet header and indicates the route the packet should be forwarded to. The port number substitutes the destination port of the transport layer headder (comes in as wormhole port). The IP address replaces the destination IP address of the IP header (comes in as wormhole IP address).
Figure 5: wormhole port to ACI-port map
Each ACI corresponds to a client created route exiting at the wormhole. For a given ACI, there can potentially be an unlimited amount of UDP and/or TCP ports used simultaneously. However, because the number of available pre-assigned wormhole ports is finite, a configurable limit is set on the maximum amount of simultaneous ports per ACI. This is also necessary to avoid having a client keep all the wormhole ports for himself. Therefore, some kind of infrastructure is required to keep an inventory of the ports in use by each ACI. In addition, some extra information is kept on each port entry to allow for intelligent creation and deletion of ports.
The head of the inventory list is a structure containing the limit on simultaneous ACIs, the limit of simultaneous ports per ACI and a reference to a hash table. This structure can be referenced using acimap.portmap from the wormhole context structure. The hash table uses the ACI number as a key, and each key refers to a structure. This structure contains a pointer to a list of ports, the current number of ports in the list and the limit on the number of ports. Finally, each element of the port list is a structure containing the port number, the protocol (UDP or TCP), a timestamp of the last time a packet travelled on this port and, for use with TCP ports, a last TCP state field and the last sequence number. This infrastructure is illustrated by Figure 6.
Figure 6: ACI port inventory infrastructure
The limited amount of wormhole pre-assigned ports and the limit on the amount of ports per ACI create the need for a set of rules for ACI-port to port map creation. The creation rules are best illustrated by a flow-chart diagram in Figure 7. Note that a port will be considered timed out if no packets have travelled through it for 5 minutes (configurable).
Figure 7: Port creation flow-chart
The AIP and the wormhole communicate via a UDS with commands and messages. The following table presents the commands that the AIP can send to the wormhole.
Command |
Description |
---|---|
WORMCMD_CMD_OPENPORT |
Enable a range of ports of a protocol type (UDP or TCP) by changing their access level. (Currently never used) |
WORMCMD_CMD_BLOCKPORT |
Block a range of ports of a protocol type (UDP or TCP) by changing their access level. (Currently never used) |
WORMCMD_CMD_CONNECT |
Notifies wormhole of which UDS socket to use. If the FLAG_FRESHCONNECT flag is set, it also destroys the ACI-port to port map and builds a fresh, empty one. (Currently never used) |
WORMCMD_CMD_NEWMAP |
Gives the name of the nym that corresponds to a specific ACI. Later, the wormhole can transmit it to the Ident daemon. |
WORMCMD_CMD_DELMAP |
Orders the wormhole to remove the name of the name corresponding to a specific ACI from its table and from the Ident daemon. |
WORMCMD_CMD_KILL |
Orders the wormhole to clean up, terminate and send an acknowledge message back. |
WORMCMD_CMD_SEND |
The AIP relays a packet to the wormhole so it can send it to the internet. For more details, see the "Freedom-Cloud-to-Internet packet handling" section. |
Table 1: AIP to wormhole commands
Then, the following table describes the commands that wormhole can send to the AIP.
Command |
Description |
---|---|
WORMCMD_CMD_SEND |
The wormhole relays a packet to the AIP so it can route it to a client through the Freedom cloud. For more details, see the "Internet-to-Freedom-Cloud packet handling" section. |
WORMCMD_CMD_MSG |
Contains a message concerning the wormhole status. Consult the following table for message types and description. |
WORMCMD_CMD_STAT |
The wormhole sends it statistics report to the AIP, so they can be relayed to the NISS. |
Table 2: Wormhole to AIP commands
The following table indicates the different messages the wormhole can send to the AIP using the WORMCMD_CMD_MSG commands.
Message |
Description |
---|---|
WORMCMD_MSG_EACIFULL |
No more available ACI connections. (Currently never used) |
WORMCMD_MSG_EACIPRESENT |
ACI is already active. (Currently never used) |
WORMCMD_MSG_EADDACIFAILED |
Operation to add an ACI failed. |
WORMCMD_MSG_EDELACIFAILED |
Operation to delete an ACI failed. |
WORMCMD_MSG_EACINOTFOUND |
Requested ACI could not be found. (Currently never used) |
WORMCMD_MSG_TERMINATING |
Wormhole is terminating. |
Table 3: Wormhole to AIP messages
The format of a command is illustrated by the following figure. The length field indicates the number of bytes in the command. The length includes everything from the start of the length field to the end of the command specifics. The reserved bytes are unused for the moment. The command specifics contain the data that accompany the command. They vary for different commands and can be absent.
Figure 8: AIP-wormhole UDS command
Two types of packet can be received from the internet: ARP requests and IP packets.
On Linux, the ARP requests are handled by the wormhole while on Solaris, they are handled underneath the stack. Therefore, on Linux, the ARP requests are retreived, encapsulated in an Ethernet frame. First, the target IP address in the ARP request is verified, if it corresponds to the wormhole IP address, an ARP reply is built and sent back. Otherwise, the ARP request is dropped without further processing.
The IP packets are retreived from the shim. The packet can be encapsulated in an Ethernet frame or not, depending on whether it came from the network or the local host. Only the packets carrying TCP or UDP protocol are processed any further, others are discarded. Both TCP and UDP types are processed in the same way except for one difference. For both types, the destination port is used to retreive the corresponding ACI-port mapping. If no mapping is found, the packet is dropped. Otherwise, the destination port and destination address of the packet are replaced with the ones found in the mapping. In the case of a TCP segment, any option present in the header is stripped. If the TCP SYN flag is up, the MSS option is added/replaced by the MSS of the Freedom cloud. Then, the packet size is compared to the Freedom cloud MSS. If it is bigger or if the shim requested it, the packet is fragmented. Once all this processing is done, the resulting packet or fragments are encapsulated into WORMCMD_CMD_SEND command packets and sent to the AIP through the UDS.
The IP packets coming from the Freedom cloud and destined to the internet are communicated to the wormhole by the AIP through the UDS, encapsulated in a WORMCMD_CMD_SEND command. The wormhole only processes TCP and UDP segments, the rest are discarded. Most processing for both protocols is the same. First, the destination port access level is verified. If the access level is blocked, the packet is discarded. Otherwise, the ACI-port to port mapping is obtained or created, if it didn't already exist. The source port is then replaced with the mapped wormhole port and the source IP address with the wormhole IP address. The port timestamp is updated in the portmap list, as well as the TCP state and sequence number in the case of TCP ports. Also, TCP options are stripped off and MSS option is added/replaced with the MSS of the Freedom cloud. The resulting IP packet is then sent on the internet.
The wormhole communicates its statistics to the NISS periodically. When a statistics period comes to an end, the wormhole gathers its statistics, packages them and send them inside a WORMCMD_CMD_STAT command to the AIP, via the UDS. The AIP is then responsible for relaying the statistics report to the NISS.
The Ident daemon is used by some internet servers (more specifically in our case: IRC) to obtain the name of the user of a specific port on a machine. Here, it returns the name of the nym using the port. (At least that's what it was originally intended to do, currently it always returns "Anonymous User").
Certain wormhole functionalities can be initiated using signals. The following table describes them.
Signal |
Description |
---|---|
INT, TERM |
Terminate the wormhole |
HUP |
Re-open wormhole log file |
CHLD |
Reap children |
Table 4: Wormhole signals
Various parameters of the wormhole can be configured using a configuration file loaded during startup. The following table describes those parameters.
Parameter |
Values |
Description |
---|---|---|
debugFile |
pathname |
Log file pathname |
debug |
Yes/No |
Enable debug messages |
networkDevice |
device name |
Network device pathname |
realAddress |
IP address (x.x.x.x) |
IP address of the wormhole host machine |
wormholeAddress |
IP address (x.x.x.x) |
IP address of the wormhole |
unixDevice |
pathname |
Unix Domain Socket pathname |
maxPacketSize |
integer |
Maximum packet size that can enter the Freedom cloud |
fragCacheSize |
integer |
Number of IP fragments that can be kept concurrently by the fragmentation cache |
tcpTimeout |
integer |
Timeout (in seconds) of TCP port in the wormhole |
udpTimeout |
integer |
Timeout (in seconds) of UDP port in the wormhole |
basePort |
integer |
Base port number (lower limit) to use to send packets on the internet |
endPort |
integer |
Last port number (upper limit) to use to send packets on the internet |
maxNbPort |
integer |
Maximum number of wormhole ports that can be assigned to one route concurrently |
redirectBasePort |
integer |
Base port number for the wormhole to redirect traffic to the local host |
redirectEndPort |
integer |
Last port number for the wormhole to redirect traffic to the local host |
redirectDuration |
integer |
Time that the local host redirection map will live |
localDevice |
pathname |
Local loopback device pathname |
pidFile |
integer |
PID file name |
logFile |
pathname |
Wormhole log file pathname |
mirrorMode |
Yes/No |
Bounce unsollicited traffic back to the sender |
cleanOnConnect |
Yes/No |
Clean up ACI maps when a new AIP connects to wormhole |
suppressArp |
Yes/No |
Suppress wormhole ARP behavior |
stripIpHeader |
Yes/No |
Strip any IP option from header |
stripTcpHeader |
Yes/No |
Strip any TCP option from header |
hacks |
Yes/No |
Enable hacks (Kent, Bananarea, MSS) |
frequency |
integer |
Time (in seconds) between statistics updates |
MSSIncoming |
integer |
Max segment size of packets going in the Freedom cloud, from the Internet |
MSSOutgoing |
integer |
Max segment size of packets going to the Internet, from the Freedom cloud |
throughputIncoming |
integer |
Number of bytes per second that a wormhole should accept from the Internet |
fragmentationSize |
integer |
Number of bytes for UDP segment fragments |
serviceRedirect |
Yes/No |
Port redirection map so that wormhole can send traffic to known port to a server on local machine |
useIdent |
Yes/No |
Use the ident daemon |
serverPath (identd only) |
pathname |
Ident daemon path name |
port (identd only) |
integer |
Ident daemon port number |
command (identd only) |
pathname |
Wormhole-Ident daemon UDS pathname |
logFile (identd only) |
pathname |
Ident daemon log file pathname |
pidFile (identd only) |
pathname |
Ident daemon PID pathname |
Table 5: Wormhole configuration parameters