Zero-Knowledge Database Server

Freedom Services 2.0 Archive Validated

Table Of Contents

  1. Introduction
  2. Software Architecture
  3. Database Server Configuration
  4. Performance Measurements

1. Introduction

Why a database server ?

The main problem faced with Core Services that uses database is that all Core Services sharing a database are required to all run on the same machine as the database. This has the effect of lowering the maximum traffic load they can handle as they all share the machine resources (like CPU). Providing access to a remote database allows each Core Services to share a common database while running on their own machine. The database server also allows for greater scalability as it is now possible to have multiple instances of the same Core Services running on different machines.

2. zkDatabase Server Software Architecture

The database server as been designed in a generic fashion so that other people needing to write new servers that interpret some commands and send back responses could benefit from our work. For that, the protocol and server parts have been decoupled from the database code and are offered as two new libraries. The ZK Command Protocol (zkcp) library provides an interface to create and send binary commands and to receive a response. The ZK Command Server (zkcs) library provides a server framework that can listen on a port waiting for commands (following the zkcp format). The library user should provide the ZK Command Server with functions for all defined commands.
With these new libraries, building a database server consisted of defining a DB protocol which covers the DB queries and their parameters. Also, function callbacks are provided to the ZK Command Server for the different commands available (as defined in the DB protocol). These callback will convert the received zkcp commands into local database queries and return the results.
Quick and Dirty Schema of Control flow and Data in zkDB Server

2.1) zk Command Protocol library

It's main purpose is packet handling. It provides utilities to encode/extract the command and its parameters to/from a packet.  The Command Protocol library allows the user to define the types that need to be supported. It also performs validation and authentication of the packets.

2.2) zk Command Server library

Provides a framework for designing servers based on the ZK Command Protocol . The server  listens to the specified port waiting for commands. On receipt of a command, the associated command handler is invoked. The server handles the port/socket, signal handling, PID file and logs error encountered. It also provides, as many other core servers do, user control through signals.

2.3) zkDatabase library

The database library is required to perform the local database query. Note that in 1.1 release, the database is enhanced to allow remote queries so that it is now possible to configure Freedom servers to use remote database. The DB interface remains the same however configuration files are changed to indicate if the database is remote or local. zkDatabases can now be seen as logical databases split into physical databases, called partitions, which can be each configured as remote or local.

2.4) zkDatabase Server

The main work in the Database Server layer is to perform the Database handling. For that, it defines data types that need to be supported by the Command Protocol and also defines the commands and handlers that the Command Server will support. The real work of the Database Server is performed in the command handlers.

3. zkDatabase Server Configuration

Since the zk Database Server was developed using the zkDatabase and the zkCommand Server libraries, the configuration file will contain both database and server options. These options are contained in a single configuration file which can be the default file:
/freedom/etc/zkdbsrv.conf
or the configuration file can be specified with the --config command line option. For more details on database and server configuration please consult:

4. Performance Measurements

To measure the performance of the database server, a new tool called dbperf was written to send and measure db requests to the zkDB server. This tool is located under freedom/ndb/test and the --help can be used to get information about the available tool options. Please refer to zkDB Server performance results for more details on the actual tests performed and their results.