Measuring the Flow of Value Using Flow Metrics

[#1]

In the age of digital transformations, IT organizations and large enterprises are embarking on Agile transformations to stay competitive in their rapidly evolving markets and industries. Many large organizations adopt an Agile scaling framework such as Scaled Agile Framework (SAFe), Large Scale Scrum (LeSS), Scrum at Scale, Spotify model or even create their own way of scaling Agile with the goals of reducing waste, increasing efficiency and predictability, becoming more innovative, and getting products faster to market. Anyone that has had the experience of going through an Agile transformation knows how challenging and complex an undertaking it is and being able to change the way a large organization is structured and operates is a huge accomplishment in an of itself.

For a large enterprise, an Agile transformation is a significant investment and the question that often comes up is: How effective has the transition to Agile been? Which can lead to: How do you measure that? Some Agile scaling frameworks like SAFe provide recommendations on metrics while other frameworks provide very little in terms of guidance on metrics leaving it up to the organization to figure it out. As a result, this can lead organizations to focus on output metrics like story point completion, number of user stories complete, and team velocity. But what do these metrics really tell you about the impact of your Agile transformation?

A common aspect among Agile frameworks is the focus on outcomes which emphasizes a product and value-oriented approach. This contrasts with the traditional project-oriented approach that is output, budget and milestone focused that most large enterprises are commonly transitioning from. In Mik Kersten’s book, Project to Product, he presents his concept of Value Stream Metrics as a way to correlate the flow of business value through a value stream to the business outcomes. Kersten defines two sets of Value Stream Metrics: Flow Metrics and Business Results. Flow Metrics are quantitative measures of the flow of value through a value stream. While Business Results are both quantitative and qualitative measures of the value delivered to the business or customers and the happiness of teams in the value stream. By measuring the flow of value in relation to business outcomes, it can provide insight on how efficiently value is flowing to the business and whether the value delivered is meaningful and impactful to the business. They can also highlight potential bottlenecks within value streams and stimulate a dialogue between development teams and the business on driving investment in improvements to accelerate desired business results.

In this article, we will focus on the types of Flow Metrics, what they measure, and how Jira Align can be used capture them for features (or epics). Flow Metrics consist of four types of measures: Flow Velocity, Flow Time, Flow Load and Flow Efficiency.

Flow Velocity

Flow Velocity measures the number of units of value that are completed in a specified time period (ex. Program Increment, quarter, year) within a value stream. Flow Velocity is not as granular a metric as team velocity which measures story points. Instead, Flow Velocity measures completed increments of value like features that used in SAFe. By gaining visibility into number of delivered units of value, it can help raise questions about improvements required to increase the Flow Velocity in order to meet the desired business goals of an organization. The Work Tree report within Jira Align provides metrics on the work item acceptance at the Epic, Feature, Story and Task level within a Program Increment, release or quarter. The red squares in the example below highlight where to find the Flow Velocity for features completed by the Mobile program across three program increments: PI-3, P-4, and PI-5. The example illustrates a program with a decreasing Flow Velocity over three Program Increments.

Flow Time

Flow Time measures the time elapsed when work is started on a unit of value within a value stream to when it is released to a customer. Extensive Flow Times are indicators that impediments and inefficiencies within a value stream are present and improvements are required. Flow Time differs from Lead Time in that Lead Time measure the time it takes value to flow through the end-to-end process while Flow Time is measured when work has actively started to when work is final completed (including any waiting time) and value is released to the business. In Jira Align, Flow Time can be derived using the Process Step Cycle Time report which tracks the average cycle time in days that features are in pre-defined developmental process steps within a value stream. In the example below, a value stream consisting of a single program, Jira Align Training Program, was set up with four developmental process steps for features: New, Active, Waiting and Done. Features are moved into the developmental process steps on the Feature Backlog - Kanban Process Step View based on the current state of the features. Flow Time is calculated by adding the Average Active Time with the Average Wait Time which is 4.67 days. In order to view the Process Step Cycle Time report and Backlog - Kanban Process Step View in Jira Align, a value stream must be created for a program.

Flow Load

Flow Load measures the number of units of value that are in progress also known as work in progress or WIP. It is a leading indicator that can be correlated to inefficiencies within a value stream. A high Flow Load may indicate that too many units of value are being worked on in parallel thus reducing output. Flow Load can be gathered in Jira Align through the Backlog - Kanban State View. In the example below, the red square highlights the number of features in the In Progress state on Feature Backlog - Kanban State View for a program. The Flow Load can be tracked for each sprint over the course of a Program Increment or quarter to gain perspective into trends within a program.

Flow Efficiency

Flow Efficiency measures the proportion of time that units of value are actively worked on compared to the total Flow Time. A low Flow Efficiency may indicate that the flow of value is stagnating in the waiting step due to bottlenecks or inefficiencies which result in larger queues and more WIP. The Process Step Cycle Time report in Jira Align can be used to derive the data required to calculate the Flow Efficiency of a value stream. In the example below, the Flow Efficiency is calculated for a value stream consisting of a single program in Jira Align. As in the Flow Time example, the value stream was set up with four developmental steps for managing features on the Backlog - Kanban Process Step View.

Flow Metrics can help organizations gain valuable insights with understanding how efficiently value is flowing through their Agile value streams. They also can highlight the effectiveness of an Agile transformation and the progression from a project to product-centric approach for an organization.

[Mike C]

Hi, thank you for this article.

Is there a way to view the trend of flow efficiency and flow time over time, so we can see the effect of process improvement changes over time?

[Sam Tsubota]

@Mike C There isn't a way to directly view a trend of flow efficiency and flow time over time in Jira Align. You could create an Objective and associate Key Results in Jira Align to track these metrics overtime. Then use the Check-In functionality of Key Results to track the flow efficiency and flow time over a specified time period. The trend can be viewed on the Key Results metrics dashboard.  If you have Jira Align Enterprise Insights, you could create a report or dashboard to show the trend in a Business Intelligence tool like Tableau or Power BI.

[amy schneider]

How about flow distribution and its importance?  Does that cover only the issues that have been created during that time, or all issues in all states? How do you calculate this for a program incement?

[Saima Siddiqui]

@Sam Tsubota Any pointers on 'Backlog Health' and second, anything on 'work readiness - meeting acceptance criteria at Epic, Feature, Story level' reports in Jira Align?

In Jira SW, I was able to do query on if 'story points = 0 or empty' then some calculations on multiply by two sprints worth of work in the backlog based on average velocity.  Is there any metrics for Backlog Health of how much work is refined, sized and ready to be picked up before next sprint planning or PI?

[Sam Tsubota]

@amy schneider you can get data to measure flow distribution (features, defects, risks and debts) by program increment from the grid views in Jira Align. Before navigating to the grid views in Jira Align make sure to select the program and program increment in the Configuration Bar.

For features and risks, navigate to the feature and risk grid views and scroll down to the bottom of the page. On the bottom left, you will see the total number of records in the view which will provide you with the total features and risks for the program increment. Feature counts are also displayed in the feature progress dial chart on the Work Tree.

For Defects and Debt, probably the easiest way to do this is to add story types for Defects and Debts. You Jira Align administrator can add a story type for Debt in the Platform settings. Defect is available out-of-the-box. Also if you are using Jira, you can map Jira bugs and a Tech Debt custom issue in the Jira Connector to a Jira Align story work item with type of Defect or Debt. There are instructions on the Jira Align knowledge base on how to do this. Below is an example of how you story type values appear in the Jira Align story work item.

After this is all configured, you can navigate to the story grid view in Jira Align (with the Program and Program Increment selected in the Configuration Bar) and then apply a filter to the view by Type = Defect or Debt. Then scroll to the bottom of the page to get the record count.

This data can then be added to an Excel spreadsheet to create charts or tables for reporting purposes.

If want a more automated way to report on this I recommend getting Jira Align Enterprise Insights. With Enterprise Insights all of the Jira Align data is available for you to create custom reports in a BI tool like Microsoft Power BI or Tableau.  You will be able to create a dashboard to calculate flow metrics and flow distribution for programs and program increments. If you are interested in Enterprise Insights, I recommend reaching out to your Atlassian Enterprise Advocate and they can provide you with more info.

[Sam Tsubota]

@Saima Siddiqui to manage the readiness of epics and features in Jira Align we typically recommending using the Backlog Kanban Process Flow View in the Backlog. This enables you to set up a custom Kanban process flow with process steps you define. You can also set up Exit Criteria for each process step. Your portfolio and program leadership teams would use the Backlog Kanban Process Flow View to manage the delivery of value across and can move it to the next step in the process flow when the Exit Criteria (ex. All Acceptance Criteria is met) is met. Process steps can be mapped to Jira Align work item states and Jira status (if the work item syncs with a Jira issue). Process flows are associated to specific work item types in Jira Align and can be assigned to specific programs. 

To view work items that don't have estimates, you can view that in the Backlog. Click on the Orphan Objects button next to the Columns Shown button. The select the Program and Program Increment/Release you want to view. You can see all the work items (depends which Backlog you have selected to view) that don't have estimates and for stories you can view them by team. Below is screenshot of stories with no estimate.

If you have a specific metrics for measuring backlog health beyond this, I recommend looking into Jira Align Enterprise Insights which will enable you to create custom reports with Jira Align data in a data visualization tool like Microsoft Power BI or Tableau. Reach out to your Atlassian Enterprise Advocate to find out more info. 

[Saima Siddiqui]

@Sam Tsubota Can you recommend how to report on Product Line (and associated products) in Jira Align if we link them to Theme issue type?

[Sam Tsubota]

@Saima Siddiqui When you refer to product line is that terminology that is specific to your company and does that represent an object in Jira Align like a Portfolio, Program, etc.? 

[Saima Siddiqui]

In Jira Align, there is a concept of creating Product and then create multiple products and Declare one product as Hierarchy or a parent product.  that capability is present today.  The language 'Product Line' is something I used.  Question is that are there any recommended reports to be able to see work related to specific products and its allocated budget?

[Sam Tsubota]

There isn't very much in terms of product reports in Jira Align that show the allocated budget. The only report that I can think of is the Progress By Product report. Unfortunately, there isn't a Jira Align help page the explains the calculation of values in the report that I can refer you to. Some of the other reports can be filter by Product in the Configuration Bar like the Investment by Feature report.

[lou_rodriguez]

Hi @Sam Tsubota This article is great!  Thanks for all the work you put into this. 

I have a question for you.  Do you know how to pull these flow metrics through the Align REST API?

Any idea on what combination of GET endpoints I can use?

[Sam Tsubota]

Hi @lou_rodriguez , the Jira Align REST APIs are not intended for reporting and you may get timed out if you attempt to pull a high volume of data. Also the API JSON schemasdo not have all the fields you may need for your reports. If you want to build custom reports using a data visualization tool like Power BI or Tableau, I recommend that you get Jira Align Enterprise Insights (EI). EI is the Jira Align data warehouse product that is intended for custom reporting. With EI you pretty much have access to all the data in the Jira Align database. Data in EI is updated once an hour from JA. If you are interested in EI, I suggest reaching out to your Atlassian Enterprise Advocate. You can even ask for a 45 day trial of EI to check it out.

[lou_rodriguez]

Hi @Sam Tsubota 

Thanks for responding so quickly.  I dont plan on using any BI tools but would like to pull some JSON's into a data lake so  I can run my own calculations.

How can I use Enterprise Insights to get data into a data lake?  Is their a wizard or REST API that I would use?

If there is a Enterprise Insights API, where can I find documentation for all the endpoints in Jira Align and Enterprise Insights?

[Sam Tsubota]

@lou_rodriguez There are no APIs for Enterprise Insights. It is a SQL server database hosted on Azure. You will be provided with login credentials for the EI instance which allows you to connect to the database. This enables you to set up an ETL to export the data from EI to your own data lake or your own preferred method for exporting the data.  

[lou_rodriguez]

Hello @Sam 

My company is creating an application to pull Flow Metrics from Jira Align.  For us to do this, we will need to connect to our customers Jira Align instance. 

Would our customers need a subscription to Enterprise Insights in order for our application to pull flow metrics from their instance of Jira Align?

I am hoping we can do this without our customers needing to shed out extra money for EI. Cant we do it by using the Jira Align Swagger /rest/align/api/docs/2/swagger.json endpoint.

Our application will allow our customers to measure:

• Flow distribution
• Flow velocity
• Flow time
• Flow load
• Flow efficiency
• Flow Predictability

Thanks and happy holidays Sam

[Sam Tsubota]

@lou_rodriguez The Jira Align APIs are not intended for reporting and you are limited to query 100 items at a time when you call the GET APIs. In addition, you will not get historical data and the APIs do not have all the data available in Enterprise Insights such as timestamp data which you will need to calculate some of the Flow Metrics. Enterprise Insights is a separate SQL server data warehouse which can be updated as quick as once an hour. Because it is a separate DB it will not impact the performance of the Jira Align instance. Also the data can be exported to an internal data warehouse or combined with data in a data visualization software like Tableau or Power BI.

[Brian M_ Smith]

I am interested in discussing the art of the possible for measuring flow distribution of issues with JIRA software (data center) or JIRA cloud without JIRA align or additional plugins.  Before we invest in a solution, does anyone have any solutions for this?