Full-nodes' Cache - A Faster Way to Retrieve Users' Output Coins

Introduction to Privacy V2 ▸

Problem Statement

In the previous post about Privacy V2, we have spotted the improvements of Privacy V2, explained Incognito keys, introduced a new key, and explained how the public key of an output coin V2 is different from that of an output coin V1. This difference leads to the impossibility to retrieve a user’s output coins in the old fashion.

In Privacy V1, all of a user’s output coins have the same public key, which is also the user’s public key. A full-node database can seamlessly aggregate these output coins. When a user requests all of his or her output coins, the full-node only has to look for those with the same public key.

This is no longer true in Privacy V2 because we used one-time addresses to strengthen receiver anonymity. Each of a user’s output coins will now be assigned a unique public key. As a result, the full-node cannot determine whether two output coins belong to the same or different users. And thus, the full-node will be unable to respond to a user’s request for output coins.

Solution

One approach is for full-nodes to cache a user’s output coins once the user submits his or her OTA key, allowing for quick retrieval. The full-node may then scan each output coin and identify if it belongs to this user or not. However, determining whether a coin belongs to a user is costly. That’s why we allow a full-node to run with two modes of operation.

  • Default mode. After the user’s keys are submitted, the full-node only caches the user’s output coins. As a result, any output coins V2 received prior to the submission of the OTA key will not be stored in the full-node’s cache. If a user wishes to use this mode, he/she must submit his/her key before the first output coin arrives.

  • Enhanced mode. the full-node will re-scan the database and add all output coins of a user into its cache.

Next, we describe in detail each of these modes of operation and gives a brief comparison between the two.


Figure. For the passive caching approach, any output coins received before your OTAKey is submitted will be “lost” in the view of the full-node. Instead, in the later approach, they will all be recoverable no matter when you submit your OTAKey.

The Default Mode

In this mode of operation, the full-node only starts caching a user’s output coin after the user submits the OTA key.

Consider the following example, suppose that Bob sends a UTXO (say UTXO A) to Alice at block height 10, and at this time, Alice has not submitted her OTA key to the full-node. As a result, the full-node has no idea that these coins belong to Alice. 10 blocks later, i.e, block 20, Alice tells the full node to cache her coins. And right after that, David sends another UTXO (say B) to Alice. Now, if Alice queries her output coins from the full-node, the returned result only acknowledges the UTXO B (since the UTXO A has been “lost” in the view of the full-node).

Here is a summary of this approach.

  • The full-node only does its work after Alice submits her key.
  • Every time a new block arrives, the full-node will try to check if any output coins (of this block) belong to Alice. If yes, it caches these coins for Alice. This caching process is called “passive caching” and it does not take much effort of the full node.
  • All output coins arriving before the key is submitted will be “lost”.
  • To avoid losing UTXOs, users are RECOMMENDED to submit their key before any UTXO arrives.

RPC Example

  • Method: submitkey.
  • Params: the only parameter is a base58-encoded OTAKey.
  • Errors: the following are some of the error messages that might be returned by this query.
Error Message Description
OTAKey has been submitted and status = v The OTAKey has been submitted before and has status. If status = 1, the indexing process is in progress, if status = 2, the regular indexing process has finished, if status = 3, the enhanced indexing process has finished.
OTA key submission not supported by this node configuration The current node does not have a cache layer.

The Enhanced Mode

As we can see, the previous mode does not provide much flexibility and creates a poor user experience. That is, if a user fails to submit the key, he/she will be unable to retrieve the balance through the full-node. As a result, we introduce the enhanced mode to assist the full-node owner or anyone else authorized in retrieving their total balance.

Consider the previous Alice example. In this case, she will be able to retrieve both UTXOs A and B if she utilizes the enhanced mode.

Here is the summary of this mode.

  • After a key is submitted, the full-node will try to check and cache all of the output coins of this key from the beginning.
  • Because the full-node has to re-scan from the beginning, this mode is very expensive and takes quite a long time. During the time the full-node is re-indexing output coins, any operations to check balance, retrieve output coins will be STALLED. The longer the blockchain, the more expensive this process is. That’s why it is called “active caching”. Therefore, we recommend you run this mode for the full-node ONLY.
  • Authorization is required since the full-node is easily DDoS’ed. As a result, only a limited number of users should be allowed to use this mode.
  • We estimate that with more than 300 requests at the same time, the full-node will be out of order. Therefore, DO NOT share the access token if it is not necessary. Additionally, connections with HTTPS are recommended to make sure the access token is not stolen.
  • Also, if the authorization fails, the basic mode will be employed.

RPC Example

{
        "Jsonrpc": "1.0",
        "Method": "authorizedsubmitkey",
        "Params": [
                "14y8spKEPrqLndpwjrQsfdX4y8VWrSwAhLPmKF2GpLocEh3pvuaDoug5T7gEgifV8amh9RBs1MKa4fSvXLwL4iAovHQPLwbGcEjJ3A2",
                "0c3d46946bbf9339c8213dd7f6c640ed643003bdc056a5b68e7e80f5525310ca",
                0,
                false
        ],
        "Id": 1
}
  • Method: authorizedsubmitkey.
  • Params: There are 4 parameters needed
    • The first parameter is a base58-encoded OTAKey.
    • The second is an access token. This access token is generated by the full-node’s owner, and is compulsory in this RPC.
    • The third one is the block height at which the full-node will re-scan from. If this parameter is set to 0, the full-node will rescan from the beginning.
    • The final parameter is a boolean indicating the flag reset, and it is optional. In case the privateOTA key has been indexed before and this flag is set to true, the full-node will re-index all output coins for this key.
  • Errors: the following are some error messages that might be returned by this query.
Error Message Description
OTAKey has been submitted and status = v The OTAKey has been submitted before and has status. If status = 1, the indexing process is in progress, if status = 2, the regular indexing process has finished, if status = 3, the enhanced indexing process has finished.
OTA key submission not supported by this node configuration The current node does not have a cache layer.
enhanced caching not supported by this node configuration The current node only operates with the basic mode.
the current authorized queue is full, please check back later The cache layer only supports a limited number of users at the same time to reduce the risk of being DDoS’ed.
fromHeight is larger than the current shard height The third parameter is larger than the current block height on the blockchain.

Run a Node with Cache

There are two ways to run an Incognito node with the cache, from the flag command or from the config file.

From flag

Add the following flags to the command (remember to add the indexeraccesstoken flag if you want to run the cache with the enhanced mode).

  • usecoindata: if this flag is present, the node will operate with cache.
  • coindatapre: the folder prefix in which the cache is stored.
  • numindexerworkers: the number of workers for the cache. If this number is 0, the full-node’s cache will ONLY operate in BASIC mode. Otherwise, it has an enhanced cache layer with numindexerworkers workers running in the background.
  • indexeraccesstoken: a 64-character long hex string.
  • Example command:
./incognito --usecoindata --coindatapre="__coins__" --numindexerworkers=100 --indexeraccesstoken="0c3d46946bbf99c8213dd7f6c640ed6433bdc056a5b68e7e80f5525311b0ca11" --discoverpeersaddress "127.0.0.1:9330" GETH_NAME="http://127.0.0.1:8545" GETH_PORT="" GETH_PROTOCOL="" --relayshards "all" --datadir "data" --listen "0.0.0.0:9433" --externaladdress "0.0.0.0:9433" --norpcauth --rpclisten "0.0.0.0:8334" --rpcwslisten 0.0.0.0:18334 --txpoolmaxtx 100000

From config files

In addition to the flagged way, from this commit, it has become much easier to run a full-node within the Incognito network, simply with a make command

make ENVIRONMENT

For example, if you want to run a full-node within the main-net environment, run the following command

make mainnet

The configurations and parameters for each environment are located inside the config package. For each folder, there are a config.yaml and a param.yaml files.

Now, to run a full-node with a cache, add the following entries to the file config.yaml in the environment you want to run the node.

coin_data_pre: "__coins__"
use_coin_data:
 - true
num_indexer_workers: 0
indexer_access_token: "0c3d46946bbf99c8213dd7f6c640ed6433bdc056a5b68e7e80f5525311b0ca11"

Finally, to retrieve the output coins from a full-node after submitting an OTA key, we use the RPC listoutputcoinsfromcache, instead of the old listoutputcoins.

Conclusion

We have shown how a cache can help a remote node retrieve output coins on receiving users’ requests. There are two methods a full-node available depending on the node’s performance. We summarize the differences between the two methods in the following table.

Property Passive Caching Active Caching
When to cache coins AFTER the key is submitted From the BEGINNING
UX Bad Better
Full-node’s load Low Very high
#Users Unlimited Limited, only a few
Access token Not required Required
Node-friendly Validators/Shard/Full-nodes Full-nodes

In the next post, we will discuss about conversion transactions, which are special transactions that convert UTXOs from version 1 to version 2.

Note

For active caching, you can use the community full-node with the following information:

Conversion Transactions ▸

pDEX Trade with Privacy V2 ▸

11 Likes

Hey @daniel,

How can I delete the cache for a submitted key via submitkey RPC? Is it possible? via RPC or filesystem?

Thanks.

No, we don’t have that option just yet.

1 Like

Is after submitting the OTA key and access key I can get the balance of my node??

Because currently, I am unable to get it.

Hi @Gaurav_Saini,

Did any error occur to you or did you get zero balances? Usually, if you configure your node correctly and the submission is error-free, it will take a while (~10-20m) for the output coins to be fully cached. After that, you are able to retrieve your balance.

1 Like

Hello @daniel @khanhj @Jared
I have a Query
So in order to run the “getbalancebyprivatekey” RPC call, we have to submit the OTA Key.

To submit key there are two modes
Default Mode (Passive caching)
Enhanced Mode (Active caching)

Default Mode only requires OTA Key
While Enhanced Mode Requires OTA Key & Access Token

Can I retrieve my balance by submitting just the OTA Key? Do I really need the access token? If I submit just the OTA Key will I be able to access my balance?
and from where do I get the access token? I know where can I find the OTA Keys but I have no idea what access tokens are and how do I generate them? @sid @Gaurav_Saini

So here’s the description of the key submission. If you only submit OTAKey (using the default mode), then you’ll only be able to retrieve output coins from this point. All other previous output coins are not retrievable. Therefore, if you run an own fullnode, try to configure the full-node with an enhanced cache and use the authorized submission method.

It is the token that you define when you set up your own full-node.

The instruction on how to set-up the access token is presented above.

1 Like

Hey @daniel,

I’ve seen func (ci *CoinIndexer) RemoveOTAKey(otaKey privacy.OTAKey) method while examining the code. Will it be enabled via a RPC soon? Or isn’t it implemented correctly yet?

Thanks.

It is only used for internal calls at the moment. But if there’s a demand for it, we will enable an RPC call soon.

Indeed, I need it so much.

Hello Daniel, Thank you for replying. I had a few queries.
If I am not running a full Node how do I get an access token? It seems like to run enhanced mode ie active caching I have to run a full node. Is there a way I can get an access token without running a full node?
In the comparison chart, I see Passive caching is for Validators/Shard/Full Node. While active caching is only for Full Node owners. Does that Mean validators do not get access to Active Caching? @sid @Gaurav_Saini

Hi @anirudh.chauhan,

You can use the community fullnode (https://beta-fullnode.incognito.org/fullnode) with this access token: 0ec910a54ffbf2a0bdfc0c8b05e8b5445e51a2ae54f659a35ac7ad9980e4fd2c

You can totally set up a validate node with the active caching. However, it is NOT RECOMMENDED due to the following reasons:

  • Validator nodes often have data of a single shard at a time, so active caching does not make sure you always have the correct balance for a given private key.
  • Active caching requires an amount of CPU to do the job, so it might affect the nodes’ performance.
1 Like

Hi @daniel
This link is not working can you please share the community full-node or any other link for check it.

Hi @Gaurav_Saini,
The above is the community full-node, and I have just made a quick check, it’s still fully functioning.

1 Like

Hi @daniel

I am submitted OTA Key but still i am not able to get the balance please check my screen shots.

image_2021_09_29T07_18_12_465Z image_2021_09_29T07_20_32_961Z

@sid @anirudh.chauhan

1 Like

Give it some time, the error says that the syncing is in progress.

Again, sending your private key via an RPC call is NOT SECURE, and thus NOT RECOMMENDED. If you don’t want to use the mobile wallet, consider using the CLI tool (or the go-sdk) for better security.

1 Like