Step Up Your DevOps Game - Webinar Questions Answered (Feature Flags)

[#1]

On October 21st, we hosted a webinar titled, Step Up You DevOps Game with 4 Key Integrations for Jira and Bitbucket. I covered 4 key DevOps practices - agile testing, DevSecOps, feature flagging, and observability - and examples of integrations in each area. If you haven’t see it, you can view an on-demand version of the presentation here.

We received far too many, albeit good, questions during the Q&A portion, so we’d like to answer them here. Read Q&As for the other three DevOps practices, as well:

• Webinar Questions Answered (agile testing)
• Webinar Questions Answered (DevSecOps and observability)

Clarifying questions about DevOps and feature flags

How do you handle features with database changes? For instance, changes in the database model once it was turned on? How do you guarantee that you still have valid data, as there might be other data in your database once users worked with the application.

For our own products, we accept there are no guarantees that a change to the database schema (or data) and a corresponding code change will be deployed at the same time. Therefore, we write “new code” to be capable to detecting which version of the database it is accessing and be able to handle it correctly. And then we deploy code changes ahead of database changes. This technique and others are more fully explained in Evolutionary Database Design.

How do you deal with potential dependencies in features with feature flagging?

I characterized feature flags as “fancy if statements”. You can get as complicated as you want in your code. For example, check the boolean AND for 2 feature flags. However, complexity of the conditions may undermine your ability to manage the feature flags externally. Generally, I recommend looking for ways to factor the features for independence and using complex conditions sparingly.

Do you test the different scenarios of what combination of features are turned on just in case having some combination of the features on causes a potential issue?

Taking Jira as an example, there can be dozens of active feature flags at any given time. That’s over a million possible combinations. We don’t explicitly test for all those combinations. Instead, we focus on keeping features independent and thoroughly testing each one. We test across the whole application regularly. And, as we roll out a new feature, we use monitoring and analytics to proactively check that we aren’t losing daily active usage or other key performance indicators.

Is feature flagging appropriate for monolithic applications? How can it be incorporated in an application code base that is monstrous in size?

Even before introducing the code necessary for feature flagging, I would focus on separating deployment from release with a load balancer, a la blue-green deployment. If you have that mastered, you’ll have a solid fallback strategy so that you can start implementing feature flags safely. In my experience, the size of the code base is less important than the architectural structuring. If the monolith is big ball of mud feature flags will be hard because it will be hard to “surround” the code paths with simple logic blocks. This may require some significant rearchitecture until at least the UI code can be isolated as “features”.

How do you apply the blue-green deployment to mobile apps when you need the app store approval before a customer release?

The “toggle” in blue-green deployment is a network or application router. That isn’t “in front” of the mobile app; hence, it is not the right mechanism for separating deployment from release for mobile apps. Indeed, the architecture of mobile apps forces coupling of deployment and release, at the very least for the mobile app as a client. Either the app will need to delegate features to back-end services that can be continuously deployed, or the continuous deployment is more about automating everything about the deployment to the app store, even if it has to wait for final approval from the app store.

I have experienced that feature flags tend to clutter code over time unless there is a strict governance to ensure clean-up tasks & prioritization of these. Do you have experiences and recommendations on how to manage this issue?

So true! Feature flags aren’t “magic” so teams need to plan for clean-up. I’ve seen teams track feature flags at the level of Jira Epics, where they put a “time bomb” task into the epic at the start so they remember to clean up at the end. I have also seen teams open 2 Git branches when they start a feature flag: first one that adds the “fancy if statements”, then another based on that initial state that removes the code blocks. In that case, the branches are reminders to do the clean up later. With either tactic, it still takes team discipline to remember that active feature flags are a kind of “work in progress” and they have to find a balance between the feeling of “safety” that comes from keeping feature flags around, and the overhead they introduce.

Is it common practice to delete the code when the feature is available and used by users? How do you make the cleanup of legacy code?

Yes! This is a tough new habit to build. In a lot of ways, it challenges the definition of done for teams because they have to remember to clean up after a feature flag is no longer needed. There are no simple answers right now; however, in my work with these vendors, we’ve talked about how their tools might automatically submit pull requests to help remove unused feature flag blocks. It feels like the market is ripe for innovation.

What’s next?

Reference Architectures

Now that you know how to step up your DevOps game with key DevOps practices to consider, you may find our new DevOps guides helpful. Whether you’re a beginner, intermediate, or advanced in your DevOps, find recommendations and inspiration for how to combine your tools and DevOps practices.

In-depth Demos

To see more in-depth demos or explainer videos of the integrations highlighted in the webinar by visiting our curated DevOps marketplace page of integrations in the four categories covered. Not only do partner pages have additional content available in the Marketplace, but most of our Marketplace is full of additional videos, screenshots, and documentation so you can discover these integrations as you see fit.

More on feature flags

Learn how to progressively expose features with feature flags and how it works with Jira so you can test and measure your feature releases with confidence.

[M Amine]

Great article to read