$Id: zkDB.html,v 1.4 2000/07/04 18:23:03 igor Exp $
Revision | Comments |
---|---|
1.1 | Author(s): Igor Stolbikov (igor@zks.net);
Initial draft. |
2. Partitioning.
Partitioning means presenting one big logical
database as set of the physical databases (called partitions). Partitions
can be physically located on the different compute servers, different hard
drives or simply in the different directories. All operations like search,
update, scan are executed independently on each partitions. Partitions
allow with maximum efficiency use multi-computer , multi-processor and
multi hard drive hardware configurations.
Partitions allow to take out database file size limitation of
2 GBytes on the Linux servers. Current maximum number partitions
is 256. It is mean it can be up to 512 GBytes databases on the one Linux
server. (Current limitation is 2 GBytes). Using distributed database network
logical database size will be not limited!
Partitions can be remote or local. In case of remote partitions
zkRemotePartition_* functions will be used. In case of the local zkLocalPartition_*
functions.
zkDB layer execute role dispatcher directing database calls to
the necessary partitions. Depending from partition type
call will be directed to remote or local partition.
Pic. 2
Partitions = part_1;part_2;part_3;
[part_1]
MaxLimit = k
LimitStartPos = 3
[part_2]
MinLimit = l
MaxLimit = p
LimitStartPos = 3
[part_3]
MinLimit = q
LimitStartPos = 3
If input key value will start from any character from 'a'
to 'k' or 'A' to 'K' it will belong to the first partition. Else if it
start from 'l' to 'p' or 'L' to 'P' it will go to the
second. And in case 'q' or 'Q' and bigger it will go to the third partition.
The limit string can contain move than one character, but it is required
that all limit strings have same length and form
uninterrupted range values covered all possible keys variation.
In case of hashed values or binary data another method of the setting
limits can be used. Currently just 1 byte value is used to find partition.
All range of values is splited on the range from 0 to 255 (maximum number
of partitions). And the limit values has to be specified in this
range :
[part_1]
IsHashed = Yes
MinLimit = 0
MaxLimit = 96
LimitStartPos = 3
[part_2]
IsHashed = Yes
MinLimit = 97
MaxLimit = 180
LimitStartPos = 3
[part_3]
IsHashed = Yes
MinLimit = 181
MaxLimit = 255
LimitStartPos = 3
Default value of the MinLimit = 0 and MaxLimit = 255. It is possible
that in future versions maximum number of partitions will be changed.
It
is important that LimitStartPos reference the changeable part of
the key and MinLimit and MaxLimit values take in consideration possible
range values.
For example if all the keys has following pattern:
"Key 1" , "Key 2" ... "Key N". If partition
have LimitStartPos set to anything else than 4 , it will put all records
to the same partition. The same can happened if MinLimit and
MaxLimit values did not respect range of values.
Changing MinLimit and MaxLimit allow to
5. Library structure
ZkDB library contains 3 layer.
Dispatching layer , which has all interface functions:
Open/Close functions
int zkDB_Open (ZkError err, const char * dbfile, ZkDB * db);
int zkDB_Close (ZkError err, ZkDB * db);
Operation functions
zkDB_Fetch
(ZkError err, ZkDB db, ZkDBT key, ZkDBT data, int * found);
and so on.
This layer uses ZkDB object which is a complex object
include a array different kind of partitions (local or remote).
Each function call is dispatched by the layer to
corresponding partitions or set of partitions function calls, remote
or local.
To see the list of all ZkDB API functions see
Remote connection layer. Operate on the remote partitions.
Responsible for the connection with the database server,
sending commands and reciving response.
Contains similar set of functions
Open/Close functions
int zkRemotePartition_Connect (ZkError err, ZkRemotePartition partition);
int zkRemotePartition_Disconnect(ZkError err, ZkRemotePartition partition);
Operation functions
zkRemotePartition_Fetch
(ZkError err, ZkRemotePartition partition, ZkDBT key, ZkDBT data, int *
found);
and so on.
Local database access layer. Operate on local physical
databases (partitions).
Open/Close functions
int zkLocalPartition_Open (ZkError err, ZkLocalPartition partition);
int zkLocalPartition_Close(ZkError err, ZkLocalPartition partition);
Operation functions
zkLocalPartition_Fetch
(ZkError err, ZkLocalPartition partition, ZkDBT key, ZkDBT data, int *
found);
and so on.
6. Error Handling
zkDB process each error
differently depending on the condition in which error happened. If error
happened during open or close process, this error is consider to be critical
and it's error type is urgent. Errors which are inform about bad database
state and require database recovery, consider to be urgent as well. This
approach allow Database server distinguish critical, fatal error from the
regular error and behave differently depending from the error type.
The current standard behavior is to restart database child process
each time it has not fatal error. Fatal error cause child process to shut
down without restarting!