Full-node (low-level)

caution

This section describes instructions and manuals for interacting with TOS at a low level.

The aim of this document is to provide step-by-step instructions for setting up a full node for TOS Blockchain. We assume some familiarity with TOS Blockchain Lite Client, at least to the extent explained in the Getting Started section.

Note that you will need a machine with a public IP address and a high-bandwidth network connection to run a TOS Blockchain Full Node. Typically you'll need a sufficiently powerful server in a data center with good network connectivity, using at least a 1 Gbit/s connection to reliably accommodate peak loads (the average load is expected to be approximately 100 Mbit/s).

It is a bad idea to run a Full Node on your home computer; instead, you could run a Full Node on a remote server and use TOS Blockchain Lite Client to connect to it from home.

Running a Full Node (video)

0. Downloading and compiling

The complete the TOS Blockchain Library and Validator software is downloaded and compiled similarly to the Lite Client. This process is outlined in the Getting Started page. The most important difference is that you have to download the complete sources from the public GitHub repository https://github.com/tos-network/tos (e.g. by running git clone https://github.com/tos-network/tos and git submodule update --init afterwards) instead of downloading the smaller Lite Client source archive. You should also build all goals defined in CMakeLists.txt (e.g. by running cmake <path-to-source-directory> and make in your build directory), not only those specifically related to the Lite Client (which is also included in the larger distribution; you don't have to download and build it separately). We strongly recommend building a "release" or a "release with debug information" version of TOS Blockchain Library and especially of the Validator/Full Node bypassing -DCMAKE_BUILD_TYPE=Release or -DCMAKE_BUILD_TYPE=RelWithDebInfo as an extra argument to cmake during its first run (if you forgot to do this, you can later delete the CMakeCache.txt file from your build directory and re-run cmake with the appropriate options).

1. Full Node binaries

After the sources have been compiled successfully, you should obtain executable files validator-engine/validator-engine and validator-engine-console/validator-engine-console in your build directory. These are the most important files you need to run and control a TOS Blockchain Full Node (or even a Validator). You might wish to install them into your /usr/bin or similar directory. You are also likely to need the generate-random-id utility during setup.

2. Working directory of the Full Node

The Full Node (also known as the "validator-engine") stores its data in the subdirectories of its working directory (e.g. /var/tos-work/db). It requires write access to this directory. If you want to use another directory as the working directory of the Full Node, you can use the command line option --db <path-to-work-dir>:

$ validator-engine --db ${DB_ROOT}

where ${DB_ROOT} is /var/tos-work/db or any other directory where the validator-engine has write permissions.

3. Working directory layout

An approximate layout of the working directory of the TOS Blockchain Full Node software is as follows:

• ${DB_ROOT}/config.json -- Automatically generated configuration file. It is automatically regenerated by the validator-engine on some occasions. When the validator-engine is not running, you can edit this file in a text editor because it is a JSON file.
• ${DB_ROOT}/static -- A directory with files that cannot be downloaded from the network, such as the "zerostate" (corresponding to the Genesis block of other blockchain architectures) of the masterchain and active workchains. Normally you don't have to initialize this directory unless you want to run your own instance of TOS Blockchain (for example, for testing or development purposes). Full nodes of existing instances of the TOS Blockchain (such as the "testnet" and the "mainnet") will be able to download all required files from already running full nodes.
• ${DB_ROOT}/keyring -- Stores public and private keys known to the validator-engine. For example, if your full node runs as a validator for some TOS Blockchain shardchains, the validator block signing key is kept here. You may wish to set more restrictive permissions for this directory, such as 0700 (in *nix systems), so that only the user under which the validator-engine is running would have access to this directory.
• ${DB_ROOT}/error -- A directory where the validator-engine copies files related to severe error conditions (e.g. invalid block candidates) for further study. It is normally empty and the validator-engine never deletes files from this directory.
• ${DB_ROOT}/archive -- A directory where old and rarely used blocks are kept until their storage period expires. You can mount a larger but slower disk partition at this directory, or make this directory a symlink to a directory in such a partition. We recommend locating the remainder of ${DB_ROOT} in a fast storage device such as an SSD.
• ${DB_ROOT}/etc -- (Non-automatic) configuration files may be kept here, or in any other directory read-accessible to the validator-engine.
• Other subdirectories of ${DB_ROOT} are used to keep ADNL cache data, recent blocks and states, and so on. They are not relevant to the purposes of this document.

4. Global configuration of TOS Blockchain

In order to set up your Full Node, you'll need a special JSON file called the "global configuration (file)". It is called this because it is the same for all full nodes and even nodes participating in different instances of TOS Blockchain (e.g. "testnet" vs. "mainnet") share an almost identical global configuration.

The "mainnet" global configuration can be downloaded at https://tos.network/global-config.json as follows:

$ wget https://tos.network/global-config.json

You may wish to put this file into /var/tos-work/etc/global-config.json .

We'll discuss the structure of this file later in more detail. For now, let us remark that the bulk of this file consists of a list of known TOS DHT nodes required for the bootstrapping of the TOS Network. A smaller section near the end of this file describes the particular instance of TOS Blockchain that we wish to connect to.

All instances of TOS Blockchain use the same "global" TOS Network (i.e. the TOS Network is not fragmented into several instances for each blockchain instance). While the global network configuration is therefore independent of the particular the TOS Blockchain instance chosen, the Full Nodes belonging to different instances will later connect to different overlay subnetworks inside the TOS Network.

It is important to distinguish this "global configuration file", used for setting up a TOS Blockchain Full Node and the "local" or "automatic configuration file" which is automatically updated by the validator-engine and usually stored in ${DB_ROOT}/config.json. The global configuration is used to generate the initial automatic configuration file, which is thereafter updated by the validator-engine itself (e.g. by adding new DHT nodes or storing hashes of newer masterchain blocks).

5. Initializing the local configuration

Once the global configuration file is downloaded, it can be used to create the initial local configuration in ${DB_ROOT}/config.json. To do this, the execute the validator-engine once:

$ validator-engine -C /var/tos-work/etc/global-config.json --db /var/tos-work/db/ --ip <IP>:<PORT> -l /var/tos-work/log

Here /var/tos-work/log is the log directory of the validator-engine, where it will create its log files. The argument to the -C command-line option is the global configuration file downloaded from https://tos.network/ as explained above, and /var/tos-work/db/ is the working directory ${DB_ROOT}. Finally, <IP>:<PORT> are the global IP address of this full node (you need to indicate a public IPv4 address here) and the UDP port used to run TOS Network protocols such as ADNL and RLDP. Make sure that your firewall is configured to pass UDP packets with source or destination <IP>:<PORT> at least for the validator-engine binary.

When the validator-engine is invoked as above, and ${DB_ROOT}/config.json does not exist, it creates a new local configuration file ${DB_ROOT}/config.json using the information from the global configuration file and from the command-line options such as --ip, and then exits. If ${DB_ROOT}/config.json already exists, it is not rewritten; instead the validator-engine starts up as a daemon using both the local and the global configuration.

If you need to change the local configuration afterwards, you'll need to either delete this file and regenerate it from the global configuration (potentially forgetting other important information accumulated in the local configuration) or edit the local configuration in a text editor (when the validator-engine is not running).

6. Setting up remote control CLI

You will almost certainly want to enable validator-engine-console in the local configuration to be able to control your Full Node (i.e. validator-engine daemon) when it is running. For this, you'll need to generate two keypairs, one for the server (validator-engine) and one for the client (validator-engine-console). In the examples below we assume that validator-engine-console runs on the same machine and connects to the validator-engine through the loopback network interface. (This is not necessarily so; you can use validator-engine-console for remote control as well.)

As a first step, use the generate-random-id executable to create two keypairs, one for the server (on the machine running validator-engine) and one for the client (on the machine running validator-engine-console):

$ ./generate-random-id -m keys -n server
6E9FD109F76E08B5710445C72D2C5FEDE04A96357DAA4EC0DDAEA025ED3AC3F7 bp/RCfduCLVxBEXHLSxf7eBKljV9qk7A3a6gJe06w/c=

This utility generates a new keypair and saves the private key into file server and the public key into server.pub. The hexadecimal (6E9F...F7) and the base64 (bp/RC...6wc/=) representations of the public key are displayed in the standard output and are used henceforth to identify this key.

We have to install the private key server into the keyring of the Full Node (validator-engine):

$ mv server /var/tos-work/db/keyring/6E9FD109F76E08B5710445C72D2C5FEDE04A96357DAA4EC0DDAEA025ED3AC3F7

Notice that the file name to store this private key inside the keyring equals the hexadecimal identifier (which essentially is a hash of the public key) of this key.

Next, we generate the client keypair:

$ ./generate-random-id -m keys -n client
8BBA4F8FCD7CC4EF573B9FF48DC63B212A8E9292B81FC0359B5DBB8719C44656 i7pPj818xO9XO5/0jcY7ISqOkpK4H8A1m127hxnERlY=

We obtain a client keypair, saved into files client and client.pub. This second operation should be run in the working directory of the client (validator-engine-console) possibly on another machine.

Now we have to list the client's public key in the server's local configuration file ${DB_ROOT}/config.json. To do this, open the local configuration file in a text editor (after terminating the validator-engine if it was running) and find the empty control section:

"control": [
]

Replace it with the following:

"control" : [
  { "id" : "bp/RCfduCLVxBEXHLSxf7eBKljV9qk7A3a6gJe06w/c=",
    "port" : <CONSOLE-PORT>,
    "allowed" : [
      { "id" : "i7pPj818xO9XO5/0jcY7ISqOkpK4H8A1m127hxnERlY=",
        "permissions" : 15
      }
    ]
  }
],

control.0.id is set to the base64 identifier of the server's public key and control.0.allowed.0.id is the base64 identifier of the client's public key. <CONSOLE-PORT> is the TCP port the server will listen to for console commands.

7. Running the Full Node

To run the full node, simply run the validator-engine binary in a console:

$ validator-engine --db ${DB_ROOT} -C /var/tos-work/etc/global-config.json  -l /var/tos-work/log

It will read the global configuration from /var/tos-work/etc/global-config.json, the local configuration from ${DB_ROOT}/config.json and continue running silently. You should write suitable scripts for invoking the validator-engine as a daemon (so that it does not terminate when the console is closed), but we'll skip these considerations for simplicity. (The command-line option -d of the validator-engine should be sufficient for this on most *nix systems.)

If the configuration is invalid, the validator-engine will terminate immediately and, in most cases, output nothing. You'll have to study the log files under /var/tos-work/log to find out what went wrong. Otherwise, validator-engine will keep working silently. Again, you can understand what's going on by inspecting the log files, and by looking into subdirectories of the ${DB_ROOT} directory.

If everything works as expected, validator-engine will locate other full nodes participating in the same instance of TOS Blockchain and download recent blocks of the masterchain and all shardchains. (You can actually control the number of recent blocks to be downloaded, or even download them all starting from the zerostate---but this topic is outside the scope of this document; try running the validator-engine with command-line option -h to find out the list of all available options with brief explanations).

8. Using the Console CLI

If the validator-engine-console has been set up as explained in Section 6., you can use it to connect to the running validator-engine (i.e. your Full Node) and run simple status and key management queries:

$ ./validator-engine-console -k client -p server.pub -a <IP>:<CLIENT-PORT>

connecting to [<IP>:<CLIENT-PORT>]
local key: 8BBA4F8FCD7CC4EF573B9FF48DC63B212A8E9292B81FC0359B5DBB8719C44656
remote key: 6E9FD109F76E08B5710445C72D2C5FEDE04A96357DAA4EC0DDAEA025ED3AC3F7
conn ready
> gettime
received validator time: time=1566568904

The gettime command obtains the current Unix time at the validator. If everything has been configured properly, you'll see an output similar to the one above. Notice that you need both the client's private key (client) and the server's public key (server.pub) for the console to work. You might wish to move them (especially the client's private key) into a separate directory with suitable permissions.

Other console commands are available in the validator-engine-console. For instance, help displays a list of all console commands with short descriptions. In particular, they are used to set up the Full Node as a Validator for TOS Blockchain, as explained in the separate Validator page.

9. Setting up the Full Node as a Lite Server

You can set up your Full Node to function as a Lite Server, so that you can use the Lite Client to connect to it from the same or a remote host. For instance, sending the command last in a Lite Client connected to your Full Node will display the identifier of the most recent masterchain block known to your Full Node, so that you can inspect the progress of block downloading. You can also inspect the state of all smart contracts, send external messages (e.g. wallet queries) and so on as explained in the Step-by-Step page.

In order to set up your Full Node as a Lite Server, you have to generate another keypair and install the private key into the server's keyring, it is similar to what we did to enable the remote console:

$ utils/generate-random-id -m keys -n liteserver
BDFEA84525ADB3B16D0192488837C04883C10FF1F4031BB6DEECDD17544F5347 vf6oRSWts7FtAZJIiDfASIPBD/H0Axu23uzdF1RPU0c=

mv liteserver ${DB_ROOT}/keyring/BDFEA84525ADB3B16D0192488837C04883C10FF1F4031BB6DEECDD17544F5347

After that, stop the validator-engine if it is running and open the local configuration file ${TOS_DB}/config.json in a text editor. Find the empty section

"liteservers" : [
]

and replace it with a record containing the TCP port for listening to inbound Lite Client connections and the lite server's public key:

"liteservers" : [
{
"id" : "vf6oRSWts7FtAZJIiDfASIPBD/H0Axu23uzdF1RPU0c=",
"port" : <TCP-PORT>
}
],

Now start validator-engine again. If it does not terminate immediately, it is likely that you have re-configured it properly. Now you can use the lite-client binary (usually located as lite-client/lite-client with respect to the build directory) to connect to the Lite Server running as a part of your Full Node:

$ lite-client -a <IP>:<TCP-PORT> -p liteserver.pub

Again, help lists all commands available in the Lite Client. The Step-by-Step page contains some examples of what can be done with the Lite Client.