A little code is better than a little infrastructure
By Viktor Stanchev
There’s a famous Go language proverb coined in a talk by Rob Pike from 2015 that says, “A little copying is better than a little dependency.” He points out that dependencies can bloat applications and they should be avoided in cases where copying a little code solves the same problem. I think this idea should be extended beyond package dependencies to infrastructure dependencies. Infrastructure dependencies are even more expensive than code dependencies because infrastructure is hard to deploy, monitor, upgrade, etc.
I like to say that “A little code is better than a little infrastructure.” and I hope that this idea catches on. Let’s take a look at some examples explaining why.
Prometheus
Prometheus is a popular tool for monitoring performance of backend server applications. When deploying your own infrastructure in your own datacenter, it makes a lot of sense to deploy Prometheus to monitor your systems. It works by running one or more daemons that make requests to the backend services to collect metrics periodically. However, when running in the cloud, there are many products available for reporting metrics. These products generally require the application to send metrics out to the monitoring system. This is much easier to do because it’s a few lines of code and some wiring to set up authentication to the monitoring system. When running Prometheus, it’s also necessary to run a whole suite of services just to collect and query metrics. Let’s not even talk about monitoring your monitoring tools. There’s an entire system to help deploy Prometheus and related tools to Kubernetes called Prometheus Operator. This is a big improvement over deploying Prometheus manually, but it’s a downgrade from using a hosted metrics service. The libraries for collecting and uploading metrics to a hosted service are bigger and more complex as dependencies, but the overhead of running custom infrastructure is much greater. At Anchorage Digital, currently, we primarily use the OpenCensus libraries to push metrics to Google Cloud’s monitoring system and are in the process of migrating to OpenTelemetry.
Google Cloud SQL connections
As another example, consider how connecting to Google Cloud’s hosted SQL databases works. For many years, the standard was to deploy a sidecar container called Cloud SQL Auth Proxy. It runs as another process on the same machine as the backend service connecting to the database. The backend service connects to the proxy without any TLS or Google Cloud credentials and gets access to the database. This is simple until you realize that you need to synchronize the service’s startup and shutdown behavior to avoid having the service trying to connect to the database while the proxy is not ready. On Kubernetes, this requires some obscure configs so, should something go wrong, very few engineers are able to identify the problem. With this old model, however, it’s hard to determine when connection issues are caused by the service or the proxy.
Because of this, today,the preferred method to connect to Cloud SQL is using Cloud SQL Language Connectors which are just code. They act as database “drivers” that your language of choice uses transparently to make database requests. Their lifecycle is managed by the application and if something goes wrong, the backend service owner can dive into the code and figure it out. Cloud SQL Auth Proxy is written in Go, so if the service owner knows Go, they can debug issues in the proxy too, but the language connectors are guaranteed to be written in a language that the service owner knows, which improves the chances of them being able to help themselves when things go wrong.
Docker containers for tests
For a third example, let’s look at Go application testing. To run a Go test that isn’t a pure unit test, it’s usually necessary to connect to a database running somewhere. A popular solution is to install docker compose, write a docker-compose.yaml file, install, Make, write a Makefile, and rely on anyone running the tests to find and execute the right Make target before running the tests. If the tests are run without this, they generally fail with an unhelpful error message. All these additional tools are essentially “infrastructure”. There is a different, more developer friendly way to do this. Using a Go library dependency, testcontainers, (also available for many other languages), it’s possible to interact directly with Docker from inside the test to create the necessary database containers. That reduces the infrastructure dependencies down to just Docker itself. Any changes to the test’s dependencies can be done in the test code itself, so there’s no need for additional documentation. You can put this in your readme: “Make sure you have Docker running and then use go test to run the tests”!
When to use infrastructure
There are cases when “infrastructure” makes sense, of course. The obvious one is when multiple languages must be supported. It’s much easier to implement Cloud SQL Auth Proxy once as a proxy rather than to implement a connector for every relevant language. It’s also not realistic to modify every application to add new capabilities such as connecting to a database through a new authentication system because sometimes it’s necessary to run third party applications which are hard to modify or closed source.
There could also be cases where the “infrastructure” is implemented in a more efficient language than the application code and there are performance benefits to using it. In those cases, watch out for premature optimization. Usually simplicity and speed of iteration provide more business value than raw computational performance.
Conclusion
The next time you decide to hand off responsibilities from your application to “the infrastructure”, consider the overall cost of that decision. Maybe “bloating” your application with a dependency is cheaper than the continuous overhead of running infrastructure. Remember that infrastructure is code, too, and it’s usually orders of magnitude bigger compared to having your application doing the same job.
Interested in working with engineers like Viktor on our team? Please review our open positions here.
This post is intended for informational purposes only. It is not to be construed as and does not constitute an offer to sell or a solicitation of an offer to purchase any securities in Anchor Labs, Inc., or any of its subsidiaries, and should not be relied upon to make any investment decisions. Furthermore, nothing within this announcement is intended to provide tax, legal, or investment advice and its contents should not be construed as a recommendation to buy, sell, or hold any security or digital asset or to engage in any transaction therein.
