AIP Test Module Documentation
Brief Summary
This document outlines the AIP Test Module that was designed and
implemented for use in testing the 2.0 release of the AIP. It is functionally
similiar to other test tools currently available, the laip and the nymtool,
for example. However, it is radically different in its design, and hopefully
will allow greater reuse and extensibility of test modules. It attempts to
achieve this goal by breaking the code up into two different groups: Test
Modules, which define the interface and syntax; and the Freedom Test API, which
provides the functionality necessary to perform the tests specified by the
modules. By maintaining a clean division of these two groups, it is hoped
that larger and more complex tools will be developed quicker, be more robust
and easier to use.
Introduction
The AIP Test Module is a framework for assembling basic procedures into
advanced and/or specialized tests. It provides black-box tests, and is
therefore most useful for integration and system level testing. Most
importantly, it allows procedures to be combined, reused previous and extended
with no impact on existing tests. It provides a telnet port based interface
so it can be used remotely. Finally, it was written in C++ to take advantage
of the extra features provided by this language.
Motivation
There were several reasons for designing YATT (yet another test tool).
The main ones were:
#1 Need to perform and automate very specific tests.
#2 Need to extend, reuse and combine current test tool functionality.
#3 Need to have consistent, usuable interface.
#4 Need to move towards C++ and thread-safe tools.
These specific points are addressed below.
Need to perform and automate very specific tests
Currently, if there is a specific test that is not easily supported by one of
the current test tools, it is very difficult to alter them to support the
desired functionality. It is even more difficult to do this without breaking
current tests and scripts dependant on those tools. One reason why it is
difficult to alter current test tools to support specific tests is that they
were implemented with a high degree of interdependancy between functions. This
is resolved in the AIP Test Module by providing functionality thru logically
structured API. It will also be important not to break future tests that come
to depend on this API, while still maintaining the existing user interface.
By either tightly binding or completely separating objects, and even functions
within the objects, it is easy to find what needs to be changed to add or
modify certain functionality. Then the user can write a simple Test Module
that is inserted into the user interface which implements the desired tests.
Thus automating tests is much simpler, the tests can be reused without
affecting existing tests.
Need to extend, reuse and combine current test tool functionality
The existing set of test tools provides a large base of functionality that can
allow most desired tests. However, when APIs change, either all test tools
must be updated, or are lost. The current approach is for each subteam to
combine their own test modules into a handful that can be maintained.
However, as each individual test tools becomes more complex they start to
duplicate functionality even across teams; while also becoming more
specialized and less compatible. Another issue is that it is difficult to
combine tools that weren't designed with this intention, even if they are
extended properly. This hampers the development of complex test tools that
require the functionality of several independant tools. By maintaining one
basic Freedom Test API, along with one interface, that allows flexible syntax,
it is easy to migrate current tools and functionality to this module. The key
is that the Test Modules can be added with their own syntax. So if a user is
familiar with the nymtool syntax, it is trivial to write the Test Module to
interpret that syntax. It makes complex tools with a myriad of options
accessible to those who need them, while providing only the basic required
options to everyone else. This can be achieved by implementing 2 separate
Test Modules that provide different interfaces, but share the Freedom Test
API. Thus there is no need for code duplication, once the functionality is
embedded into a class within the API, it becomes trivial for any further test
modules to use it and build on it.
Need to have consistent, usuable interface
One of the greatest causes of code duplication and wasted effort is that those
who want to use certain tools cannot find them, cannot use them or become
frustrated with the lack of reliability of the current tools. Programmers
often build far too much functionality into a tool, just to be sure. New users
often have trouble determining how to use all the features of a complex tool.
The AIP Test Module provides one interface and uses virtual functions and
inheritance to enforce certain minimal standards. For example, every Test
Module must have a help message. This hopefully will encourage Test Module
writers to provide better online documentation. Also, with one interface to
all modules, it is easy to quickly sift thru all the available modules to find
one that does what is desired. Normally, this would be prohibited by resource
limits, but the AIP Test Module has the ability to activate and deactivate
each test module. This allows a test module to run, use 100 threads, and then
once finished, it can be deactivated and it is expected to shutdown all
threads. Sort of a 'power save' mode. This 'always available' aspect makes
the AIP Test Module much easier to use than other tools, as it is possible to
verify the network status, create a nym, and create an authenticated route,
all from the same interface. Furthermore, if it is found that another step is
needed (Verify the Nym's private keys, for example), this can be added quite
seemlessly. Therefore, it is possible to create modules with only the needed
options, because it will be trivial to add more options later.
Need to move towards C++ and thread-safe tools
Current test tools typically use C or Perl or Expect. As the regular
development is moving towards C++, it is important to have test tools that are
compatible with this environment. A C test tool would not be able to easily
adopt a C++ API, but a C++ test tool can simply wrap the C API. Also, as more
and more performance tests need the concurrency offered easily by threads, it
will be important to have a thread-safe API that can be executed concurrently.
It is important to note that the AIP Test Module is not currently thread-safe,
but it was a motivation and it was considered in several stages of
development. Therefore, it should not be overly difficult to add thread
support.
Design
As outlined in the previous section, there were many real requirements
that shaped the current design. It should be emphasized that this is almost
certainly not the best design, but it seems to address all the requirements
given. If new requirements are specified, the design should be reconsidered.
But it is hoped that most reasonable requirements could be supported simply
thru refactoring of the current implementation.
Once the motivations for YATT were determined, it was important to lay
out some design guidelines:
#1 KISS Principle.
#2 Only the functionality that is needed now should be included now.
Add the functionality that is needed later...later.
#3 Take from available code whenever possible, but rewrite where using
available code would sacrifice one of our requirements, or lead to a poor
design.
#4 All objects should either be tightly bound or completely
independant. No loose coupling.
#5 The test module must be reliable and usable, performance is secondary.
#6 The only constant is constant change (cliche, I know), and thus all
code must be sufficiently removed from the production environment to allow
improvements to be integrated quickly and with a minimum of red tape.
The AIP Test Module has 3 main parts:
#1 User Interface.
#2 Test Module Handler.
#3 Freedom Test API
They are fairly independant, with the User Interface being oblivious to
the existance of the Freedom Test API and vice versa. They are bound by the
Test Module Handler, which is the most unique aspect of this design.
User Interface
The User Interface is a message-based. The user connects to a pre-specified
port, usually by telnetting to it, and begins a session. Specifically, the
Test Module waits in an infinite loop and when a connection is accepted, a
User Session object is instantiated which handles that connection. As far as
the listening loop is concerned, that session is taken care of. The User
Session object then issues a welcome message and waits for a command. Every
command is packaged into a User Request object, which is passed to the Test
Module Handler. Once the Test Module Handler returns, the User Request object
has a response which is output via the socket. The only exception to this is
upon end of session. In this situation, the Test Module Handler recognizes
the command ('exit' or 'quit') and throws a specific exception. Upon catching
this exception, the User Session writes a goodbye message and exits. The only
other way to end a session is with an unexpected exception. Whether this will
remain the case is undecided.
Test Module Handler
The Test Module Handler accepts messages passed to it and indexes into a map
of all available Test Modules by using the first word of the command, which is
expected to be the name. There are a set of global commands that cannot be
used as module names, described further below. The test modules themselves all
inherit from the base class, Test Module, which is the interface that the Test
Module Handler exercises. The Test Modules are automatically indexed into the
map in their constructor, which is executed by the Test Module Handler's
constructor.
Freedom Test API
The classes in the Freedom Test API have a few requirements, they must all
recognize the ConfigManager, which handles dynamic configuration and global
variables. This allows the user to toggle debugging, change the network that
is being tested against, etc. Also, each object must have a PrintSelf() and a
SelfTest() function. The SelfTest() is intended to allow runtime diagnostics
of the test module, so that it can fulfill the reliability requirement.
Implementation
As previously mentioned, all code is done in C++. It uses most modern
aspects of C++, including inheritance, polymorphism, constructors and
destructors, exception handling and stl.
Test Modules
All the TestModules have Module in their name, and all their interfaces are
defined in one file, TestModuleSet.h. The Test Module interface has 3
functions of note: Activate(), Deactivate() and HandleRequest(). Activate()
and Deactivate() should obtain and release resources as the module is starting
up or shutting down. HandleRequest() invokes the Activate() function before
continuing, and as that is the only way that a user can access a given Test
Module, it transparently switches the module 'on' before the first usage. When
a request is processed, the proper functions are called from the Freedom Test
API, and the response is returned to the caller. If the command is
unrecognized, a 'bad command' response is returned. If an exception is
thrown, it is caught and returned to the user, so long as it is a known
exception (i.e. defined in the Freedom Test API). This allows the user to
continue, shutdown the module or exit completely. The reason why *all*
exceptions are not handled is to promote the removal of any possibility of
unexpected exceptions. Thus, by breaking the module, it forces the problem to
be tracked down and addressed.
Freedom Test API
The Freedom Test API is the complete set of functionality available to
the test modules.
Config Manager
This class holds all the global variables, including current network, the
home directory, the default timeout, etc. It is directly referenced by all the
other classes in the API, and since it is a Singleton, there is only one
instance of all the specified global variables. What this buys us is a
uniform interface for editing global variables, which makes dynamic
configuration quite easy. It also handles the loading of a configuration file
and saving the current configuration to a file.
CryptoWrapper
The CryptoWrapper wraps the crypto functionality. This is done mainly
because the current API is too low level and it was found that certain
functions (such as generating keys) were being repeatedly cut and pasted. It
also attempts to cover up some of the more glaring flaws of the current API,
trading efficiency for usuability. An example is that the ivec for symmetric
encryption is destroyed during encryption. However, the same ivec is needed
for decryption so it was the source of some subtle bugs to reuse the same ivec
for encryption and decryption. In order to avoid future occurences of this,
the ivec is now copied in the wrapper before being destroyed so the initial
copy is preserved..
KeyHandler
The KeyHandler maintains the key cache for public keys that have been
fetched from the kqd; maintains the key chain for private keys that have been
loaded; and fetches and loads keys when requested for the first time.
NetData Object
The NetData Object handles current network data. It updates on demand,
it has hooks to return relevant configuration information to core servers. It
also handles route verification. Although classically, this has been done
elsewhere, route verification is really a network data dependant action, so it
makes sense to do it inside this object.
NymManager Object
The NymManager Object has as it's purpose, creation, loading and saving of
Nyms. It has an active nym, which is the nym that has most recently been
created or loaded. The main function accepts a nym type and a nym name and creates
a nym corresponding to these.
TokenHandler
The Token Handler handles the token cache, exchanging serial numbers for
tokens which can then be used by the NymData Object to create Nyms. Currently,
the tokens are lost when the TokenHandler shuts down, but adding the
functionality to save the tokens to file is possible. It is also possible to
have the token handler fetch the serial number itself and create a token of
the desired type automatically.
RouteManager
The Route Manager coordinates all routes that currently exist. It
presents an interface that allows the user to add and remove routes, as well
as presenting a blocking loop to handle the route IO. It tracks the port
redirections that are active, the set of ACIs used/available, the number of
concurrent routes and offers the only interface to the shim.
RouteHandler
The RouteHandler is misnamed, it is actually a packet builder. It builds
route create and route destroy packets, as well as performing layered
encryption and decryption of data packets. It maintains all the state
required to use a route.
DHNegotiator
The DHNegotiator performs the DH exchange, either as a client, a
receiving AIP or a sending AIP. It then builds and returns the symkey. It is
designed to exist for the duration of a DH exchange, then be destroyed.
FreedomPkt
The FreedomPkt object is to the Freedom network stack as the sk_buff is
the Linux network stack. It hides the byte level details of building,
encrypting/decrypting and reading Freedom Packets. It is designed to be
reused, cached and it is intended that at some point it will be possible to
grow/shink FreedomPkt buffers just like sk_buffs.
Usage
Using the AIP Test Module should be fairly straightforward. The user
simply telnets to the command port, which defaults to 8888 unless specified on
startup (by placing it on the command line after the executable name). Once
the welcome message has been returned, the test module waits for a command.
There are two types of commands, base commands and module commands. Module
commands are specified by prefacing the command with the module name, and as
such are module specific. The base commands are embedded in the test module
itself and include the following:
help The basic help message.
list Lists all currently available test modules.
quit Quit the current session, but don't kill the Test Module.
exit Exit the current session and shutdown the Test Module.
As well each module is required to support certain commands, these
include:
help Lists and describes each command that the module recognizes.
shutdown Deactivates the module, freeing it's resources.
print Prints the internal state of the module's objects.
selftest Tests all objects currently instantiated by the module.
Applications
Network tracking: By creating a route to every AIP once every 5 minutes,
it is possible to have an accurate picture of the network state in real time.
Future Enhancements
Multi-threading, multiple concurrent users, more performance modules,
such as netperf and wget. Integrate libwww and libpcap to allow even more
powerful automated tests.
Conclusion
The AIP Test Module is good.
Last
modified: Mon Oct 30 11:57:43 EST 2000