Admin procedures

Uploading new mainnet contracts

  1. Get the Git hash of the revision you want to upload. Let's pretend it's 3961e46752bbe7c0a2560e0fe22f538e0ad75fc9.
  2. Locally check out the correct revisions of the perps repo.
  3. Run the full test suite with just cargo-full-check.
  4. Build the WASM contracts with just build-contracts. The resulting files will be in wasm/artifacts.
  5. Store the contract code on the destination chain and get the code ID. See sections below for details.
  6. Store information on the uploaded code in the YAML config files by running cargo run --bin perps-deploy mainnet store-perps-contracts --network injective-mainnet --to-upload market --code-id CODE_ID. You can upload market, factory, liquidity-token, and position-token contracts this way.
  7. Generate the exports for the docs repo with .ci/make-source-tarball.sh 3961e46752bbe7c0a2560e0fe22f538e0ad75fc9. This will produce two new files in the source-tarballs` directory: levana-perps-3961e46752bbe7c0a2560e0fe22f538e0ad75fc9.tar.gz and levana-perps-3961e46752bbe7c0a2560e0fe22f538e0ad75fc9-checksums.txt.
  8. Upload the two generated files to s3://static.meteors.levana.finance/perps-source/.
    • You can use the .ci/upload-source-tarball.sh script to automate this step and the following one.
  9. In the docs repo, add a new line to the src/source-code.md file with the new revision.

At this point, you can now migrate to new contracts.

Uploading to Sei

Uploading to Sei is simple: it's a permissionless chain and does not have gas or size limits that impede us. However, the gRPC endpoints do seem to have some trouble, so we use a cosmjs/TypeScript/RPC approach instead.

  1. Within the perps repo, go to the subdirectory packages/perps-exes/cosmjs
  2. Run yarn upload:sei-mainnet (optionally running yarn first to install dependencies)

You should get back the code ID.

Uploading to Injective

Injective is a permissioned chain that we cannot upload to. Instead, we need to ask for help from Achilleas in our shared Injective Slack channel. Send him a ZIP file with the WASM files and he'll upload them and give you a code ID.

Uploading to Osmosis

This is by far the trickiest. Currently, the default config for all nodes will disallow a store code transaction from entering mempools, and therefore broadcasting to a node won't work. We've asked the Osmosis team to change defaults, but haven't gotten positive responses yet. For now, our very broken process is:

  1. CrosNest runs the Levana validator node. Reach out to them on Telegram and tell them we need to upload a new contract. They will need to:
    1. Expose an RPC endpoint (https://rpc-levana.cros-nest.com)
    2. They turn off the extra high availability nodes in the validator set
  2. Once they've done that, we can broadcast our store code message directly to that node. However, because Osmosis is permissioned, we need to use an approved uploader, which also means we need to use a MsgAuth. This is all handled by yarn upload:osmosis-mainnet within packages/perps-exes/cosmjs. See granting store code permissions for details.
  3. Confirm that the transaction has landed in the /unconfirmed_txs for the RPC node.
  4. Wait for our validator to propose a block with our transaction. Easiest way to do this is to check Mintscan for the wallet you used for broadcasting that transaction.
  5. When the transaction lands, ask CrosNest to return their cluster to its normal state, and then get the code ID from the transaction.

Migrate to new contracts

Once the code is uploading to the chain, we migrate using the multisig contracts. Run a command like the following (replacing code IDs, adding additional ones if needed -- use --help to get all args):

cargo run --bin perps-deploy mainnet migrate --factory injmainnet1 --market-code-id 585

This will print out information on the multisig contracts and messages that need to be placed against them. Follow normal multisig procedures for that.

Granting store code permissions

Levana has a single multisig wallet which is approved for upload: osmo1lqyn9ncwkcqj8e0pnugu72tyyfehe2tre98c5qfzjg4d3vdw7n5q5a0x37. However, it isn't feasible to upload directly via a multisig. Instead, we have the multisig pass a proposal to grant a hot wallet upload permissions via Cosmos's authz system.

The cosmos CLI tool from our levana-cosmos-rs repo provides a built in command for generating cw3 grant message, which looks like this:

cosmos authz cw3-grant --granter osmo1lqyn9ncwkcqj8e0pnugu72tyyfehe2tre98c5qfzjg4d3vdw7n5q5a0x37 --grantee osmo1e2r98hpf3eer8pfpcrsprmrx5vpfq8jp5we88a --duration 90d store-code

Then make a normal multisig proposal with the output.

Manual proposal execution

Generally the process for a multisig proposal happens inside either Apollo Safe or Smart Contract GUI: make the proposal, vote on it, and execute it. Doing these steps from the command line is difficult and/or impossible, since we generally use Ledgers for performing the signing, and our CLI tooling doesn't (currently) support Ledger.

However, there are two cases where this requirement is relaxed:

  1. Executing a proposal that has already passed can be done with any wallet, not just one of the signers. So executing with a hot wallet is possible.
  2. On Injective, our multisigs use hot wallets since there isn't (or at least wasn't) support for Ledger on Injective.

If you run into trouble using one of the web UIs for executing a proposal, you may need to execute it from the command line. This happens most often with Injective, where the gas amounts chosen in the Apollo Safe UI are usually wrong.

To manually execute a proposal, follow these steps:

  1. Identify the proposal ID and multisig contract address. This can often be determined from the Apollo Safe URLs. For example, the proposal https://safe.apollo.farm/injective-1/inj1499cv9umkqt87ajyakr0k3ymjdf5kpg8nks6er/proposals/21 is proposal ID 21 on multisig contract address inj1499cv9umkqt87ajyakr0k3ymjdf5kpg8nks6er.
  2. Install the Cosmos CLI tool.
  3. Run a command like the following:
COSMOS_NETWORK=injective-mainnet COSMOS_WALLET="some seed phrase" cosmos execute-contract inj1499cv9umkqt87ajyakr0k3ymjdf5kpg8nks6er '{"execute":{"proposal_id":21}}'

When I run the command above (with a valid seed phrase), I get the following error message:

Error: On connection to https://sentry.chain.grpc.injective.network, while performing:
simulating transaction: Message 0: inj12jn8c3gxf9gf3p4xczqeh4xn2k55zcj87gpsv9 executing contract inj1499cv9umkqt87ajyakr0k3ymjdf5kpg8nks6er with message: {"execute":{"proposal_id":21}}
Status { code: Unknown, message: "failed to execute message; message index: 0: Proposal must have passed and not yet been executed: execute wasm contract failed [!injective!labs/wasmd@v0.45.0-inj/x/wasm/keeper/keeper.go:401] With gas wanted: '50000000' and gas used: '120055' ", details: b"\x08\x02\x12\xf2\x01failed to execute message; message index: 0: Proposal must have passed and not yet been executed: execute wasm contract failed [!injective!labs/wasmd@v0.45.0-inj/x/wasm/keeper/keeper.go:401] With gas wanted: '50000000' and gas used: '120055' \x1al\n(type.googleapis.com/google.rpc.ErrorInfo\x12@\n\x1cexecute wasm contract failed\x1a\r\n\x08ABCICode\x12\x015\x1a\x11\n\tCodespace\x12\x04wasm", metadata: MetadataMap { headers: {"alt-svc": "h3=\":443\"; ma=2592000", "content-type": "application/grpc", "server": "Caddy", "set-cookie": "lb=061ead0bb10c2fa962e89cc3b6e5ee864437639a384a0eca6005ece9502eb9a1; Path=/", "x-cosmos-block-height": "65626049", "date": "Tue, 09 Apr 2024 07:32:34 GMT"} }, source: None }
Height set to: None
Health report for https://sentry.chain.grpc.injective.network. Fallback: false. Healthy: true. No errors. First request: 2024-04-09 07:32:34.171102793 UTC (Since 0 minutes). Total queries: 2 (RPM: since is 0)

Note that the error messages from the CLI tend to be much more meaningful than what you get from Apollo Safe. And in this case, the message makes perfect sense: since this proposal was already executed, we can't execute it a second time.

Update Multisig addresses

You might have to update the MultiSig addresses if you would want to remove or add a specific member. These are the steps:

  1. Get the wallet address of member that needs to be added/removed. Let's assume we are going to add neutron1cpy2gpwc8lphzyczderwma2rt5nqdmvtyyl26f and remove neutron17s76s29nyuu5p6xaqz0sndw3sgl78hy4gqm342.
  2. Find out which multisig address that you want to modify. Let's assume we are going to do it for kill switch multisig. In that case, find the CW4 group contract address for Kill switch. Let's assume in our case that is neutron19539madyfvjla97j4mc86a3p7t78g66pm2cat6ep7ug4mvcnkscsluagmj.
  3. Create the multisig message using the cosmos cli:
$ cosmos cw3 update-members-message --add neutron1cpy2gpwc8lphzyczderwma2rt5nqdmvtyyl26f --remove neutron17s76s29nyuu5p6xaqz0sndw3sgl78hy4gqm342 --group neutron19539madyfvjla97j4mc86a3p7t78g66pm2cat6ep7ug4mvcnkscsluagmj
{"wasm":{"execute":{"contract_addr":"neutron19539madyfvjla97j4mc86a3p7t78g66pm2cat6ep7ug4mvcnkscsluagmj","msg":"eyJ1cGRhdGVfbWVtYmVycyI6eyJyZW1vdmUiOlsibmV1dHJvbjE3czc2czI5bnl1dTVwNnhhcXowc25kdzNzZ2w3OGh5NGdxbTM0MiJdLCJhZGQiOlt7ImFkZHIiOiJuZXV0cm9uMWNweTJncHdjOGxwaHp5Y3pkZXJ3bWEycnQ1bnFkbXZ0eXlsMjZmIiwid2VpZ2h0IjoxfV19fQ==","funds":[]}}}
  1. Find out the CW3 Flexible Multisig contract address which has been deployed for the kill switch (These details are usually present in Spreadsheet that Michael and others have access to). This is the contract address that you should use to send the above message to.