Managing backend services

We maintain three sets of backend services:

  1. Testnet ECS: where the testnet services run
  2. Mainnet ECS: where the real mainnet services are run

One final complication to mention: the services in testnet ECS often run for both testnet and mainnet, while the mainnet/ECS services are exclusively for mainnet.

ECS mainnet

ECS is Amazon's Elastic Container Service.

The only two people on the team who can deploy ECS mainnet changes right now are Michael and Sibi. Deployments are managed in the devops repo, in the terraform/mainnet subdirectory. All services are managed via Terraform. There are docs in the repo, and justfiles for simplifying common tasks.

Team members that want to review the health and logs of ECS mainnet services should follow these steps:

  • Log in at https://levanafinance.awsapps.com/start/
  • Choose one of the roles under "Levana Perps Mainnet". Different team members will have different roles available, but "PowerUserAccess" and "ReadOnlyAccess" will both give access.
  • Choose "Elastic Container Service."
  • Choose the "bots" cluster.
  • Choose the service you want to review, e.g. querier.
  • From this area, you can select health reviews, look at logs, inspect individual tasks, etc:

ECS

ECS testnet

This is pretty much similar to the above section with the following differences:

  • Nearly all team members have full access to do deployment.
  • The ECS cluster is named testnet
  • For details on how to deployment, refer the documentation here.