Logs, Metrics, and Feature Flags

In the early days of my career, my error-handling approach consisted of catching errors and then sending myself individual emails containing the error details. This was OK initially; but, it didn't scale well. Which I learned the hard way one morning when I arrived at my desk and found over 47,000 errors in my Inbox (due to a failing database connection).

Eventually, I learned about error logging and log aggregation services. And, suddenly, the idea of emailing myself an error message felt so naive and antiquated. The more I used logs, the more I came to realize just how powerful they were. And, soon thereafter, I started to view logging as a critical component in my application architecture.

Over time, I realized that I could use logs for more than just error messages. In fact, I could write anything I wanted in a log message. And so, I started to use logging as a way to record key user actions. When a user signed-up for an account, I logged it. When a user upgraded to a paid-plan, I logged it. When a user sent an invitation for collaboration, I logged it. I could then query the logs and get a sense of how the system was being consumed.

Then, I attended a web development meet-up group and learned about something called StatsD (created by Etsy). StatsD allowed an application to emit numeric values at a high rate without any added latency (due to the use of UDP as the underlying transport layer). These values could then be aggregated and graphed on a timeline, creating a visual heartbeat for any workflow within an application.

And, suddenly, the idea of recording events in my logs felt so naive and antiquated. The more I used metrics, the more I came to realize just how powerful they were. And, soon thereafter, I started to view metrics as a critical component in my application architecture.

This pattern—of doing something naively and then learning how to do it the "right way"—that's what a career path is. It's how we go from novice to journeyman to master. Doing something naively isn't a misstep—it's an important first step that creates the fertile conditions in which we can achieve a breakthrough in understanding.

Some of these breakthroughs become "best practices". And, some of them become transformational moments that change the entire landscape of work. For me, learning about logs and metrics represent two such moments. Logs and metrics went from being a thing I used to becoming a fundamentally important non-negotiable aspect of web development.

Feature flags have followed the same path. There are many naive ways in which to conditionally activate code in an application—I know this because I've used them personally (see The Status Quo). But, feature flags are the "right way". And, just as with logs and metrics, I now view feature flags as a critical component in my application architecture.

Going forward, every product that I work on will include logs, metrics, and feature flags. These technologies are no longer a point of curiosity—they are the proven building blocks of success.

Have questions? Let's discuss this chapter: https://bennadel.com/go/4557