Relayer

Search…
Relayer
This section contains instructions on how to setup the rust relayer "hermes" and relay IBC packets between bitsong-2b and other IBC-enabled networks.
Official BitSong IBC Channels
source chain-id
source channel
source denom
destination chain-id
destinaion channel
IBC token-address on destinaion chain
bitsong-2b
channel-0
ubtsg
osmosis-1
channel-73
IBC/4E5444C35610CC76FC94E7F7886B93121175C28262DDFDDE6F84E82BF2425452
bitsong-2b
channel-1
ubtsg
cosmoshub-4
channel-229
IBC/E7D5E9D0E9BF8B7354929A817DD28D4D017E745F638954764AA88522A7A409EC
osmosis-1
channel-73
uosmo
bitsong-2b
channel-0
IBC/ED07A3391A112B175915CD8FAF43A2DA8E4790EDE12566649D0C2F97716B8518
cosmoshub-4
channel-229
uatom
bitsong-2b
channel-1
IBC/C4CFF46FD6DE35CA4CF4CE031E643C8FDC9BA4B99AE598E9B0ED98FE3A2319F9
Relayer Tutorial
Hardware specs:
16+ vCPUs or Intel or AMD 16 core CPU
at least 64GB RAM
4TB+ nVME drives
To assist operators in setting up relayers, Bitsong provides tutorials for the following IBC relayers:
Hermes (rust)
​https://hermes.informal.systems/​
Pre-requisites:
latest go-version https://golang.org/doc/install​
Fresh Rust installation: For instructions on how to install Rust on your machine please follow the official Notes about Rust installation at https://www.rust-lang.org/tools/install​
build-essential, git
openssl for rust. The OpenSSL library with its headers is required. Refer to https://docs.rs/openssl/0.10.38/openssl/​
It is recommended to always build binaries on dedicated machine (dev-box), as dev dependencies (rust & go) shouldn't be on your production machine
1
sudo apt install librust-openssl-dev build-essential git
Copied!
Setup full nodes & configure seeds, peers and endpoints
To successfully relay IBC packets you need to run private full nodes (custom pruning or archive node) on all networks you want to support. Since relaying-success highly depends on latency and disk-IO-rate it is currently recommended to service these full/archive nodes on the same machine as the relayer process.
Because the relaying process needs to be able to query the chain back in height for at least 2/3 of the unstaking period ("trusting period") it is recommended to use pruning settings that will keep the full chain-state for a longer period of time than the unstaking period:
edit app.toml - note: at an average block time of 6.5sec pruning-keep-recent=400000 will result in a retained chainstate of ~30d. This will suffice for most cosmos-sdk chains with an unstaking period < 30d
1
pruning="custom"
2
pruning-keep-recent=400000 
3
pruning-keep-every=0 
4
pruning-interval=100
Copied!
hermes needs to be able to query the RPC- and gRPC-endpoints of your nodes, you will need to maintain a well-organized port-setup.
edit app.toml & config.toml, choose a unique port-config for each chain, write down your port-config
app.toml
1
[grpc]
2
​
3
# Enable defines if the gRPC server should be enabled.
4
enable = true
5
​
6
# Address defines the gRPC server address to bind to.
7
address = "0.0.0.0:7012"
Copied!
config.toml - choose unique pprof_laddr port
1
# pprof listen address (https://golang.org/pkg/net/http/pprof)
2
pprof_laddr = "localhost:7019"
Copied!
1
[rpc]
2
​
3
# TCP or UNIX socket address for the RPC server to listen on
4
laddr = "tcp://127.0.0.1:7013"
Copied!
1
[p2p]
2
​
3
# Address to listen for incoming connections
4
laddr = "tcp://0.0.0.0:7010"
Copied!
config.toml - set persistent-peers & seeds for each chain
bitsong-2b seeds:
1
 [email protected]:26656,[email protected]:26656
Copied!
bitsong-2b persistent-peers:
1
[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:15631,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:36656
Copied!
osmosis-1 seeds:
1
[email protected]st-2.compute.amazonaws.com:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656
Copied!
osmosis-1 persistent-peers:
1
[email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:26656, [email protected]:36657, [email protected]:26656, [email protected]:26656, [email protected]:26656
Copied!
cosmoshub-4 seeds:
1
[email protected]it.com:26656,[email protected]:26656,[email protected]:26656,[email protected]hub.certus.one:26656,[email protected]:26656
Copied!
cosmoshub-4 persistent-peers:
1
[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656
Copied!
please reference https://github.com/cosmos/chain-registry for a maintained list of peers & seeds.
To simplify the config process you can use Environment-Variables in the systemd file: sudo vim /etc/systemd/system/bitsongd.service
1
[Unit]
2
Description=Bitsong Daemon
3
​
4
[Service]
5
User=relay
6
Environment=BITSONGD_P2P_LADDR=tcp://0.0.0.0:7010
7
Environment=BITSONGD_RPC_LADDR=tcp://0.0.0.0:7011
8
Environment=BITSONGD_GRPC_ADDRESS=127.0.0.1:7012
9
Environment=BITSONGD_PPROF_LADDR=localhost:7019
10
Environment=BITSONGD_P2P_PERSISTENT_PEERS="[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:15631,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:36656"
11
Environment=BITSONGD_P2P_SEEDS="[email protected]:26656,[email protected]:26656"
12
Environment=BITSONGD_SNAPSHOT_INTERVAL=1000
13
Environment=BITSONGD_P2P_MAX_NUM_INBOUND_PEERS=100
14
Environment=BITSONGD_P2P_MAX_NUM_OUTBOUND_PEERS=100
15
LimitNOFILE=500000
16
ExecStart=/usr/local/bin/bitsongd start --pruning custom --pruning-keep-recent 400000 --pruning-keep-every=0 --pruning-interval 100 --home --x-crisis-skip-assert-invariants
17
Environment=BITSONGD_LOG_LEVEL=info
18
​
19
Restart=always
20
RestartSec=3
21
​
22
[Install]
23
WantedBy=multi-user.target
Copied!
Build & setup Hermes
Beware that for security reasons this step should be done 'on some remote pc'
Make the directory where you'll place the source, clone the hermes source repository and build it using the latest release. Optional: copy binary to /usr/bin (or preferred directory for systemd execution)
1
mkdir -p $HOME/hermes
2
git clone https://github.com/informalsystems/ibc-rs.git hermes
3
cd hermes
4
git checkout v0.13.0
5
cargo install ibc-relayer-cli --bin hermes --locked
6
sudo cp ~/.cargo/bin/hermes /usr/bin
Copied!
If you have built your binary on a remote machine, move the binary to your producion environment
Make hermes config & keys directory, copy config-template to config directory:
1
mkdir -p $HOME/.hermes
2
mkdir -p $HOME/.hermes/keys
3
cp config.toml $HOME/.hermes
Copied!
Check hermes version & config dir setup
1
hermes version
2
hermes 0.13.0
Copied!
Edit hermes config (use ports according to your port config, set filter=true to filter channels you don't relay) vim ~/.hermes/config.toml
1
# The global section has parameters that apply globally to the relayer operation.
2
[global]
3
​
4
# Specify the strategy to be used by the relayer. Default: 'packets'
5
# Two options are currently supported:
6
#   - 'all': Relay packets and perform channel and connection handshakes.
7
#   - 'packets': Relay packets only.
8
strategy = 'packets'
9
​
10
# Enable or disable the filtering mechanism. Default: 'false'
11
# Valid options are 'true', 'false'.
12
# Currently Hermes supports two filters:
13
# 1. Packet filtering on a per-chain basis; see the chain-specific
14
#   filter specification below in [chains.packet_filter].
15
# 2. Filter for all activities based on client state trust threshold; this filter
16
#   is parametrized with (numerator = 1, denominator = 3), so that clients with
17
#   thresholds different than this will be ignored.
18
# If set to 'true', both of the above filters will be enabled.
19
#filter = true # breaking in v0.11.0 where filter = true by default
20
​
21
# Specify the verbosity for the relayer logging output. Default: 'info'
22
# Valid options are 'error', 'warn', 'info', 'debug', 'trace'.
23
log_level = 'info'
24
​
25
# Parametrize the periodic packet clearing feature.
26
# Interval (in number of blocks) at which pending packets
27
# should be eagerly cleared. A value of '0' will disable
28
# periodic packet clearing. Default: 100
29
clear_packets_interval = 25
30
​
31
# Toggle the transaction confirmation mechanism.
32
# The tx confirmation mechanism periodically queries the `/tx_search` RPC
33
# endpoint to check that previously-submitted transactions
34
# (to any chain in this config file) have delivered successfully.
35
# Experimental feature. Affects telemetry if set to false.
36
# Default: true.
37
tx_confirmation = true
38
​
39
​
40
# The REST section defines parameters for Hermes' built-in RESTful API.
41
# https://hermes.informal.systems/rest.html
42
[rest]
43
​
44
# Whether or not to enable the REST service. Default: false
45
enabled = true
46
​
47
# Specify the IPv4/6 host over which the built-in HTTP server will serve the RESTful
48
# API requests. Default: 127.0.0.1
49
host = '127.0.0.1'
50
​
51
# Specify the port over which the built-in HTTP server will serve the restful API
52
# requests. Default: 3000
53
port = 3000
54
​
55
​
56
# The telemetry section defines parameters for Hermes' built-in telemetry capabilities.
57
# https://hermes.informal.systems/telemetry.html
58
[telemetry]
59
​
60
# Whether or not to enable the telemetry service. Default: false
61
enabled = true
62
​
63
# Specify the IPv4/6 host over which the built-in HTTP server will serve the metrics
64
# gathered by the telemetry service. Default: 127.0.0.1
65
host = '127.0.0.1'
66
​
67
# Specify the port over which the built-in HTTP server will serve the metrics gathered
68
# by the telemetry service. Default: 3001
69
port = 3001
70
​
71
[[chains]]
72
id = 'osmosis-1'
73
rpc_addr = 'http://localhost:7001'
74
grpc_addr = 'http://localhost:7002'
75
websocket_addr = 'ws://localhost:7001/websocket'
76
rpc_timeout = '10s'
77
account_prefix = 'osmo'
78
key_name = 'osmosis'
79
address_type = { derivation = 'osmosis' }
80
store_prefix = 'ibc'
81
default_gas = 5000000
82
max_gas = 15000000
83
gas_price = { price = 0.000, denom = 'uosmo' }
84
gas_adjustment = 0.1
85
max_msg_num = 20
86
max_tx_size = 2097152
87
clock_drift = '20s'
88
max_block_time = '10s'
89
trusting_period = '10days'
90
memo_prefix = ''
91
trust_threshold = { numerator = '1', denominator = '3' }
92
[chains.packet_filter]
93
policy = 'allow'
94
list = [
95
  ['transfer', 'channel-73']
96
]
97
​
98
[[chains]]
99
id = 'bitsong-2b'
100
rpc_addr = 'http://127.0.0.1:7011'
101
grpc_addr = 'http://127.0.0.1:7012'
102
websocket_addr = 'ws://127.0.0.1:7011/websocket'
103
rpc_timeout = '10s'
104
account_prefix = 'bitsong'
105
key_name = 'bitsong'
106
address_type = { derivation = 'bitsong' }
107
store_prefix = 'ibc'
108
default_gas = 2000000
109
max_gas = 4000000
110
gas_price = { price = 0.026, denom = 'ubtsg' }
111
gas_adjustment = 0.1
112
max_msg_num = 25
113
max_tx_size = 1800000
114
clock_drift = '10s'
115
max_block_time = '10s'
116
trusting_period = '14d'
117
memo_prefix = ''
118
trust_threshold = { numerator = '1', denominator = '3' }
119
[chains.packet_filter]
120
policy = 'allow'
121
list = [
122
 ['transfer', 'channel-0'],
123
 ['transfer', 'channel-1']
124
 ]
125
​
126
[[chains]]
127
id = 'cosmoshub-4'
128
rpc_addr = 'http://127.0.0.1:7021'
129
grpc_addr = 'http://127.0.0.1:7022'
130
websocket_addr = 'ws://127.0.0.1:7021/websocket'
131
rpc_timeout = '10s'
132
account_prefix = 'cosmos'
133
key_name = 'cosmos'
134
address_type = { derivation = 'cosmos' }
135
store_prefix = 'ibc'
136
default_gas = 2000000
137
max_gas = 3000000
138
gas_price = { price = 0.001, denom = 'uatom' }
139
gas_adjustment = 0.1
140
max_msg_num = 25
141
max_tx_size = 180000
142
clock_drift = '10s'
143
max_block_time = '10s'
144
trusting_period = '14days'
145
memo_prefix = ''
146
trust_threshold = { numerator = '1', denominator = '3' }
147
[chains.packet_filter]
148
policy = 'allow'
149
list = [
150
   ['transfer', 'channel-229']
151
 ]
Copied!
Add your relaying-wallets to hermes' keyring (located in $HOME/.hermes/keys)
Best practice is to use the same mnemonic over all networks, do not use your relaying-addresses for anything else because it might lead to mismatched account sequence errors.
1
hermes keys restore bitsong-2b -m "24-word mnemonic seed" --hd-path m/44'/639'/0'/0/0 
2
hermes keys restore osmosis-1 -m "24-word mnemonic seed"
3
hermes keys restore cosmoshub-4 -m "24-word mnemonic seed"
Copied!
You can validate your hermes configuration file:
1
hermes config validate
2
INFO ThreadId(01) using default configuration from '/home/relay/.hermes/config.toml'
3
Success: "validation passed successfully"
Copied!
Create hermes service file:
1
[Unit]
2
Description=hermes
3
​
4
[Service]
5
User=relay
6
ExecStart=/usr/bin/hermes start
7
LimitNOFILE=500000
8
​
9
Restart=always
10
RestartSec=3
11
​
12
[Install]
13
WantedBy=multi-user.target
Copied!
Refresh service files, enable hermes on system-startup, enable all node-daemons on system-startup, start node-daemons, sync, start hermes.
Tipp: use chainlayer quicksync to bootstrap your nodes faster: https://quicksync.io/networks/osmosis.html​
1
sudo systemctl daemon-reload
2
sudo systemctl enable hermes.service bitsongd.service osmosisd.service gaiad.service
3
sudo systemctl start bitsongd osmosisd gaiad
Copied!
Watch node-daemon output to check if your nodes are syncing:
1
journaltctl -u bitsongd -f
Copied!
When your nodes are fully synced you can start the hermes daemon:
1
sudo systemctl start hermes && journalctl -u hermes -f
Copied!
Hermes does a chain-health-check at startup. Watch the output to check if all connected nodes are up and synced
1
INFO ThreadId(01) using default configuration from '/home/relay/.hermes/config.toml'
2
INFO ThreadId(01) telemetry service running, exposing metrics at http://0.0.0.0:3001/metrics
3
INFO ThreadId(01) starting REST API server listening at http://127.0.0.1:3000
4
INFO ThreadId(01) [osmosis-1] chain is healthy
5
INFO ThreadId(01) [cosmoshub-4] chain is healthy
6
INFO ThreadId(01) [bitsong-2b] chain is healthy
7
...
8
INFO ThreadId(01) Hermes has started
Copied!
Hermes will try & clear any unreceived packets after startup has completed.
Snippets
Query Hermes for unreceived packets & acknowledgements (check if channels are "clear")
1
hermes query packet unreceived-packets bitsong-2b transfer channel-0
2
hermes query packet unreceived-acks bitsong-2b transfer channel-0
Copied!
1
hermes query packet unreceived-packets osmosis-1 transfer channel-73
2
hermes query packet unreceived-acks bitsong-1 transfer channel-73
Copied!
Query Hermes for packet commitments:
1
hermes query packet commitments osmosis-1 transfer channel-73
2
hermes query packet commitments bitsong-2b transfer channel-0
Copied!
Clear channel (only works on hermes v0.12.0 and higher)
1
hermes clear packets omniflixhub-1 transfer channel-1
2
hermes clear packets osmosis-1 transfer channel-199
Copied!
Clear unreceived packets manually. Experimental: you'll need to stop your hermes daemon for it not to get confused with account sequences.
1
hermes tx raw packet-recv osmosis-1 bitsong-2b transfer channel-0
2
hermes tx raw packet-recv bitsong-2b osmosis-1 transfer channel-73
Copied!
Thanks to @ccclaimens from @crypto_crew****
Validator Security
FAQ
Last modified 28d ago
Copy link
Contents
Official BitSong IBC Channels
Relayer Tutorial
Hermes (rust)