peer-to-peer connection string

gwar

After creating a test db with litesync for p2p use does the connection substring (&connect=....) always have to be in pseudo code:

connection A: n/a
connection B: tcp://A
connection C: tcp://A,tcp://B
connection D: tcp://A,tcp://B,tcp://C
connection E: tcp://A,tcp://B,tcp://C,tcp://D

where each subsequent connection includes all prior primary nodes?

What is the strategy if I wanted to add or remove a node? Obviously, the total_primary_nodes would have to increase/decrease on all nodes but not sure if all nodes have to be disconnected beforehand and then a new connection w/ the updated total_primary_nodes established.

Thanks

Bernardo

Well, it depends.

The requirement is that every node has a connection to each of the other nodes. The above method you mentioned is an example of how to achieve this.

If some node is behind a firewall and cannot receive connections, then it can just make connections to other nodes without binding to an address.

Currently LiteSync does not have automatic handling of connections between peers (no node discovery). And one of the reasons is exactly around the total_primary_nodes config. It is hard to automate it (increasing is easy, decreasing is difficult - how to know that a node will never come back?).

Yes, all nodes must be disconnected to modify the total_primary_nodes. Modify it on all the nodes, then reopen them. This is to avoid problems related to consensus.

gwar

Thanks again for the response Bernardo. My question was more around the connection string to each node. In my tests they are running in a single container where each node is running on a different port. if I explicitly connect each node to each peer they may initially see each other as peers but they soon forget each other

node 1:
file:litesql-test1.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9331&connect=tcp://127.0.0.1:9332,tcp://127.0.0.1:9333,tcp://127.0.0.1:9334,tcp://127.0.0.1:9335

node 2:
file:litesql-test2.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9332&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9333,tcp://127.0.0.1:9334,tcp://127.0.0.1:9335

node 3:
file:litesql-test3.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9333&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332,tcp://127.0.0.1:9334,tcp://127.0.0.1:9335

node 4:
litesql-test4.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9334&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332,tcp://127.0.0.1:9333,tcp://127.0.0.1:9335

node 5:
file:litesql-test5.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9335&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332,tcp://127.0.0.1:9333,tcp://127.0.0.1:9334

From node 5's perspective sql status initially shows:

{
"use_litesync": true,
"node_type": "primary",
"node_id": 7,
"db_is_ready": true,
"sync_down_state": "in sync",
"sync_up_state": "in sync",
"last_tid": 7161970867421961677,
"log_storage": "keep",
"peers": [{
"node_type": "primary",
"node_id": 1,
"conn_type": "outgoing",
"address": "127.0.0.1:9331",
"conn_state": "connected",
"db_state": "unknown",
"log_storage": "keep"
},{
"node_type": "primary",
"node_id": 2,
"conn_type": "outgoing",
"address": "127.0.0.1:9332",
"conn_state": "connected",
"db_state": "unknown",
"log_storage": "keep"
},{
"node_type": "primary",
"node_id": 4,
"conn_type": "outgoing",
"address": "127.0.0.1:9333",
"conn_state": "connected",
"db_state": "unknown",
"log_storage": "keep"
},{
"node_type": "primary",
"node_id": 6,
"conn_type": "outgoing",
"address": "127.0.0.1:9334",
"conn_state": "connected",
"db_state": "unknown",
"log_storage": "keep"
}]
}

however, after about 15 seconds:

{
"use_litesync": true,
"node_type": "primary",
"node_id": 7,
"db_is_ready": true,
"sync_down_state": "unknown",
"sync_up_state": "with local changes",
"last_tid": 7161970867421961677,
"log_storage": "keep",
"peers": []
}

The same behavior is observed on all nodes. Things only seem to work if I connect them implicitly to other peers:

node 1:
file:litesql-test1.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9331

node 2:
ile:litesql-test2.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9332&connect=tcp://127.0.0.1:9331

node 3:
file:litesql-test3.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9333&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332

node 4:
file:litesql-test4.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9334&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332,tcp://127.0.0.1:9333

node 5:
file:litesql-test5.db?node=primary&total_primary_nodes=5&bind=tcp://0.0.0.0:9335&connect=tcp://127.0.0.1:9331,tcp://127.0.0.1:9332,tcp://127.0.0.1:9333,tcp://127.0.0.1:9334

Is there some reason why each node can't explicitly connect to all peers?

Bernardo

Yes, you should not repeat the connections. It should be defined just once. Otherwise they will detect double connections and will close both.

gwar

Is there a way to view the sync status externally from the opened db connection?

Bernardo

No, but you can make your app serve some kind of RPC request.
When a request arrives, execute the pragma sync_status command on the db connection and return the JSON string

gwar

Ok, got it. Thanks.