The Hidden Cost of Feature Flags

Programmers know the benefits of everything and the trade-offs of nothing.

— Rich Hickey (creator of Clojure)

I believe that feature flags are an essential part of modern product development. But, every architectural choice represents a trade-off; and, so do feature flags. While they do create a lot of value, that value comes at a cost—both literal and figurative.

Whether you're building your own feature flags implementation or you're using a third-party vendor, feature flags cost money. You're either paying for development time and hosting if you build your own implementation; or, you're paying for usage and seats if you buy a managed solution.

We'll talk more about the financial trade-offs later (see Build vs. Buy). For now, I want to focus on the more hidden cost of feature flags.

In terms of developer ergonomics, feature flags create additional complexity in your code. By definition, they are spreading the business logic across two different systems: the application code and the runtime configuration. This makes it much more difficult to understand why the application is exhibiting a certain behavior. And, there's no possible way to know—at a glance—which conditional branch in the code should be executing.

Add this to the fact that feature flag configurations are unique across environments and it's completely understandable why engineers fall down accidental rabbit holes. I can't tell you how many times I've seen a good developer lose multiple hours of productive time, not realizing that a feature flag was enabled in production, but disabled in the development environment (or vice versa).

Feature flags clearly reduce cognitive load by allowing large tasks to be split up into many smaller tasks. But, from a planning perspective, feature flags incur slightly higher leading and trailing costs. When you build a product using feature flags, you're forced to think more deeply about the process of development (in a good way). And, once the development is done and a feature has been released, you're required to be much more diligent about your clean-up.

Of course, I say "required" here in the moral sense, not in the business sense. Because, at the end of the day, no one is going to step in and actually force you to clean-up after yourself. In fact, your manager(s) may actively encourage you to skip the clean-up process so that you can move onto other "higher priority" items.

Embracing the importance of clean-up and then advocating for—and, perhaps, even fighting for—such work is uncomfortable. And, it will likely necessitate a higher degree of grit and conviction than you're accustomed to. Because, after all, the system works even when you leave the dead code in place.

But, death does not exist in isolation. And, as the dead code sits there and rots, sepsis begins to set in. Over time, the application becomes increasingly complex and ambiguous. In turn, developers become ever more afraid of deleting code because they are increasingly unsure as to what the code actually does. This, in turn, compounds the overall complexity problem, slowly and steadily grinding the application development to a halt and undoing all of the advantages that feature flags gave us in the first place.

It is on your shoulders, as the developer, to keep the application code in a healthy, predictable state. Which means, you must wake up every day and recommit to doing the right thing. Over and over again.

This includes:

• Creating tickets to remove feature flags so that you don't forget—even if you know that these ticket won't be relevant for weeks or even months in the future.
• Actually executing on said tickets—even if management doesn't think it's worth the effort.
• Negotiating with other teams to enforce a finite timeline for feature flags.
• Removing all outdated control flow logic.
• Removing any deeper dead code that is no longer referenced by the outdated control flow logic.
• Performing the previous step, recursively, until no more deletions are possible.
• Removing feature flag definitions from the administrative system.
• Doing all of this for other developers who were too lazy to do it for themselves (this is a sad but inevitable truth of collaborating on a system with other people).

Depending on your company's culture, it can take grit and dedication to use feature flags in your product development life-cycle. But, I promise you that the value-add is worth the price of admission.

Have questions? Let's discuss this chapter: https://bennadel.com/go/4553