Bytebeam
IoT platform managing data streaming, updates, and data visualization of tens of thousands of devices continuously streaming data.
The stack consists of multiple services, written primarily in Rust and Python with a react frontend. Some services have consistent loads, while some services, like the analytics service, have spiky loads based on usage.
The Bytebeam application deployments are complex as they span multiple AWS and GCP accounts across multiple customers.
This system deals with terabytes of data managed in Kubernetes clusters and has very high reliability and uptime requirements.
Argonaut manages all of Bytebeam’s environments across all customers and helped them scale from 5 to 40 engineers and 5x customers over 2 years.
For our IoT platform, uptime for our infra is crucial. Using Argonaut, we could easily set up and manage our entire cloud, which can scale with our workloads with minimal effort. Our deployments to multiple multi-cloud accounts for our customers is also seamlessly handled from one place. Without it, we would have had to invest in massive tooling in-house instead of focusing on product development and growth. -- Gautam BT, CEO of Bytebeam
Automatic Kubernetes upgrades
Over the 2 years that Bytebeam has been using Argonaut, there have been multiple Kubernetes versions that have been released. The very first set of deployments was done using EKS v1.18. However, over time, Argonaut automatically upgraded Kubernetes versions automatically without the Bytebeam team ever having to deal with the underlying Kubernetes layer. This eliminates the need for Kubernetes operations and management, further enabling the Bytebeam team to work efficiently.
Setting up new environments
Setting up new environments for development and staging was an easy process taking just a couple of hours. Each new environment is set up with Terraform for cloud infrastructure and GitOps for applications on Kubernetes.
The cost estimate of cloud-managed infrastructure, like Kubernetes clusters and managed databases (RDS and CloudSQL), is displayed before the resources are created. This has helped the team immensely in being cost-aware from the get-go without having to do a separate round of rightsizing for cost.
Self-hosting OSS
For maximum portability and enabling deployments cost-effectively across multiple customer accounts, Bytebeam self-hosts and manages large multi-terabyte clickhouse clusters and an observability stack. While the clickhouse cluster is managed using Argonaut’s capability to run stateful applications, the observability stack is a popular OSS tool setup using the official helm chart.
Argonaut is capable of leveraging the full helm chart and Kubernetes ecosystem.
“On-prem” deployments onto customer cloud accounts
Onboarding new enterprise-tier customers requiring the deployment of Bytebeam services onto their cloud accounts was made much easier.
This was enabled by one-click deployments of services along with customizable pipelines using GitHub Actions.
Migrating from AWS to GCP and saving $100,000
The Bytebeam team was initially on AWS. However, they were using Argonaut and had consciously chosen to keep the cloud-specific dependencies to a minimum (only managed Postgresql via RDS). When they received $100,000 in credits from GCP, it was trivial for them to migrate all their applications and infrastructure to GCP within a week. The primary challenge was in migrating the data contained in the self-managed clickhouse instance on the Kubernetes cluster managed by Argonaut.
Value - DevOps efficiency
Bytebeam has no infrastructure or platform engineers. They now have a DevOps engineer to administer Argonaut and monitor all the services.
The Bytebeam team has been able to set up automated deployment pipelines, administer cloud infrastructure across multiple clouds and accounts, and manage multiple customer and internal environments via Argonaut, all the while focusing on application development.