Six experts sound off on story points + the evolution of agile estimation

[#1]

Hello there! 

I am a Product Marketer at Atlassian and an agile enthusiast. I am passionate about continuous improvement and helping teams work better together. Before starting at Atlassian, I was responsible for leading agile adoption and transformation efforts across Marketing teams at the world's largest children's healthcare charity.

We know that estimation is HARD. It's an ongoing challenge to reliably predict when a team will deliver software so we’ve joined up leaders in the industry to unpack story points and the evolution of agile estimation. 

Estimation using story points has become the most popular way for agile teams to rate the relative effort of work, instead of using traditional time-based estimates. Unfortunately, story points are often misused. Sometimes, story points even encourage agile anti-patterns!

To improve estimation practices and avoid the pitfalls of story points, I hosted a round table discussion with Mike Cohn, John Cutler, Andrea Fryrear, Troy Magennis, and Dave West. 

Read on for a short summary of key takeaways and watch the full discussion for a comprehensive view of how story points fare in modern agile teams.

What are story points and where did they come from?
Story points are units of measure for expressing an estimate of the overall effort required to fully implement a product backlog item or any other piece of work. Teams assign story points to work, relative to work complexity, the amount of work, and risk or uncertainty. Values are assigned to more effectively break down work into smaller pieces, so they can address uncertainty. Over time, this helps teams understand how much they can achieve in a period of time and builds consensus and commitment to the solution. Story points are generally credited to XP (Extreme Programming).

When misused, how do story points encourage agile anti-patterns?

Story points go wrong when they’re used to judge people, assign detailed timelines and resources, and when they’re mistaken for a measure of productivity.

What are common pitfalls, and how can teams avoid them?

Story points are useless when teams default to filling them in, without considering the internal and external forces that might influence the work priorities. Teams need to focus more on the size of the work and the prioritization of the work, rather than treating story points as a bar to enforce and measure how much work is delivered.

What are some best practices to improve estimation?

Teams need to iterate on the estimation practices by promoting curiosity, hosting reviews of past estimates and how they achieved or missed their timelines, and aligning near-term tasks with the long-term goals of their project, team, and company.

Practical methods include:

• Triangulation: Humans are much better at estimating relative size than absolute size. When teams use triangulation, they compare a user story they want to estimate with previously-estimated user stories. Then, they decide whether the current story is smaller, the same, or bigger than past stories.

• Unpacking: Teams take a project of moderate size and break it down to list the sub-stories within it. They only identify the sub-stories but don't estimate them individually. Once they understand the sub-stories, they re-estimate the parent story. This has been shown to produce better estimates because the team has identified more information about the nature of the work.

• Planning Poker: This technique reduces anchoring bias by asking teammates to use playing cards to submit their story points estimate. Cards are available for different values, and each person “plays” a card for their project. Once all team members have played their estimate card, they discuss the differences and align on the estimation.

• Estimation units: This is a unit of measurement for relative sizing techniques. Many teams use T-shirt sizes (S, M, L, XL), but you can also use the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, and so on). Teams also create their own units, such a gummy bears, NUTS, or foot-pounds.

• Affinity estimation: This is a quick way for teams to visualize the product backlog into groupings of relative sizes. Affinity estimation leverages many types of relative scales, including T-shirt sizes (e.g. XS, S, M, L, XL, XXL), story points based on the Fibonacci sequence scale (e.g. 1, 2, 3, 5, 8, 13, 21...), or other scales (e.g. dog sizes, gummy bears).

• Nesting Teams catalog many small things but ensure they don’t lose sight of the bigger picture and how all the elements fit together.

• Taxonomy: The practice of teams co-designing a reference guide or “taxonomy of the work” helps break down what a team must know about the work to make good decisions and understand when work falls outside of that taxonomy.

Where do you land in the great debate of story points and estimation in agile? We'd love to hear your thoughts!

Meet the experts!

Mike Cohn is the founder of Mountain Goat Software and co-founder of the Scrum Alliance and the Agile Alliance. In addition to authoring three popular agile books (User Stories Applied, Agile Estimating and Planning, and Succeeding with Agile), Mike helps companies adopt and improve their use of agile processes and techniques to build high-performance development organizations. 

www.linkedin.com/in/mikewcohn/

@mikewcohn

John Cutler is the Head of Product Research & Education at Amplitude. As a former UX researcher at AppFolio, a product manager at Zendesk,Pendo.io, AdKeeper, and RichFX, a startup founder, and a product team coach, John has a perspective that spans individual roles, domains, and product.

www.linkedin.com/in/johnpcutler/

@johncutlefish

Andrea Fryrear is a leading authority on Agile marketing. She is a co-founder of AgileSherpas and oversees training, coaching, and consulting efforts for enterprise Agile marketing transformations. Andrea co-authors of the ICAgile Marketing Agility curriculum and has written two books on marketing agility: Mastering Marketing Agility and Death of a Marketer. 

@AndreaFryrear

www.linkedin.com/in/afryrear/

Troy Magennis has helped deliver valuable software to customers at scale for over 25 years. In 2011 Troy founded Focused Objective, a leader in Agile metrics and probabilistic forecasting. Eager to share his passion for using data in better ways to improve business outcomes, he regularly keynotes, consults, and trains organizations wanting to improve decision making for IT through Agile and Lean thinking and tools.

www.linkedin.com/in/troymagennis/

@t_magennis

Dave West is the Product Owner and CEO at Scrum.org. He is a frequent keynote speaker and is a widely published author of articles, along with his acclaimed book: Head First Object-Oriented Analysis and Design. He led the development of the Rational Unified Process (RUP) and then worked with Ivar Jacobson running the North American business for IJI, managed the software delivery practice at Forrester research, and was Chief Product Officer at Tasktop.

www.linkedin.com/in/davidjustinwest

@DavidJWest

Kelly Drozd (me!) I am a Product Marketer at Atlassian and an agile enthusiast. I am passionate about continuous improvement and helping teams work better together. Before starting at Atlassian, I was responsible for leading agile adoption and transformation efforts across Marketing teams at the world's largest children's healthcare charity.

https://www.linkedin.com/in/kellydrozd/

@kellycdrozd

[Ashley Faus]

Loved this conversation! In particular:

• "Velocity of value, not velocity of work" and how it doesn't matter if the goodness is delivered in a burst of focused energy or in spurts throughout the day, just deliver value to your customers
• "Too many jobs to be done". YES, and this is not just the story points, we see this with content all the time. Loved Andrea's comments about how this applies to marketing teams as well :) 
• The general discussion about using metrics and measurement to improve outcomes, not shame teams or employees for "not doing enough".

Thanks for hosting this!

[Jimi Wikman]

The most interesting is that none of the points to support the use of story points are invalid for proper time estimations. At the end of the day the only difference between story points and time estimations is that story points are arbitrary guesstimations that is useless outside the team and that unlike time estimation there is no learning curve where the team become better at making correct estimations.

You only get a rough understanding on how wrong you are and since the estimates are basically big chunks of time you could just as easily just say that the estimate is between 16 and 100 hours.

"Velocity of value, not velocity of work" is incorrect. Estimated value has nothing to do with the work done because there is no correlation between value and time spent to create value. Velocity is also a completely useless measurement and anyone actually using it for any purpose have a problem with trust, not the lack of value being produced.

"Too many jobs to be done." This is because you do not limit the jobs to be done based on actual values. No one can add more time than they have if you do the right estimations and measure that against time available. Not the happy estimates here, but the actual estimate based on time to completion and your efficiency.

Using metrics to measure outcome is pretty much what estimates are for. Time, Quality and Resources are the metrics always used and your estimates are based on those metrics. If you have those metrics then you can never be shamed for anything because numbers don't lie. You only have this problem when you don't know how to estimate or know your own availability.

Using Story Points because you can not make accurate estimates so you need a way to deflect responsibility is a sure way to never learn to estimate. It also locks you out from any large organization with portfolio and project management, even if you go for lean budgets, because you will constantly fail in your predictions and with all teams measuring arbitrarily there is no way to manage this on a financial level.

Story Points are fine if you are a single team working in isolation where you have financial freedom to do whatever you want within your budget. Still, you are just doing guestimates all the time, so why not learn to make proper estimations that are actually useful anywhere you work?

Here is a starting point for you:

https://jimiwikman.se/articles/professional/development/how-to-make-realistic-estimates-when-working-with-software-development-r194/

[Eoin]

Cheers all for a fun discussion :)

I believe estimation is a great way to gain a shared understanding in a team and it can bear fruit to many productive conversations that can help a team deliver value more efficiently. 

However it's unfortunate it can also cause anxiety for people when it's mis-used / over-used. The latter is a heavy burden (spoiler - I'm product manager working on Jira Software) as there are often requests to extend estimation within the product that will deliver great value to specific users, but there is always the potential that they could be used in ways that could potentially have an adverse impact on individuals or teams.

An area interest to me is how estimation more broadly can continuously ladder from a team level, to a team of teams level, to an org level (bi-directional). And in doing helping to positively support real time impact analysis (and facilitate productive conversations) when estimates do change - as they absolutely will. Then throw in how we are increasingly seeing teams take ownership of both planned and unplanned work - and things start getting real spicey.

I'm confident agile ways of working are set to have a bright future as many software and non-software teams become more agile, and as an agile enthusiast content like this is great way to just maybe make the future lives of these teams a little bit better. Thanks!

[Rajesh Viswanathan]

super conversation , lot of ideas to do estimation ...Thanks all

[nempafrancis59]

Very interesting discussions. Thanks

[Marshall Walker Lee]

This is great, thanks for sharing @Kelly Drozd 

[Mykenna Cepek]

Something new I learned from this great conversation:  a new estimation approach, which I'll call "timescale". Mike Cohn very briefly talks about this starting at 18:45 in the video.

Instead of using Fibonacci numbers or T-shirt sizes, the timescale approach just focuses on the unit of time that the estimator has in mind: hours, days, weeks, months, years (or even longer). What's magical is that the number isn't important, only the units.

To put it another way: "I don't care if your estimate is 2 days or 4 days - I hear that you're estimating 'days' as opposed to 'hours' or 'weeks'. " This is enough information, for example, for a Product Owner to prioritize several PBIs based on net value and team effort required.

This seems great for items in the backlog. I'd take a different approach to estimate within a sprint -- either hours to enable capacity planning, or Story Points for team velocity, or #noEstimates for Kanban or even Scrum.

Some crude math suggests that Fibonacci is about 4 times more precise than this timeframe approach (62% ratio vs 15% ratio on average). I like that the timeframe units have some basis in reality, and thus are less abstract compared to Story Points or T-Shirt Sizes. Just another option to experiment with!

[Kelly Drozd]

@Mykenna Cepek this is so great! Thank you for checking out the discussion & for taking time to comment back with something new you learned!!! 

[Robert Manuel Garcia Rivas]

En realidad, no hace falta calidad al momento de intercambiar ideas y finalizar con una respuesta efectiva o con un agradable perspectiva  y que allá cambios en el diario de cada uno , mi sinceros  aprecio por el tiempo y intercambio de palabras sinceras  a la familia de atlassian     Community

[Rajesh Viswanathan]

Very interesting .

[ahmad dashoush]

this is very up lifting and the team work is out standing! never seen anything like this before! thumbs up

[ahmad dashoush]

this is really an experience like no other

[ahmad dashoush]

i really really hope im doing this right!

[Lakshan Fernando]

An interesting discussion even for beginners to have an idea about the story point estimation.  

[Aneeqa Khan]

How can one keep the story estimation and its respective tasks effort allocation aligned within a sprint. Can someone point me to the right article that shares more insight? thanks

[Chowtapalli]

Every story takes up a set of sub-tasks (Design, Development, Code Review, Testing, BUV signoff and so on) for its completion. If we can come up with a set of defined subtasks (template) to realize a specific kind of story then it will lead us to tasks effort allocation as pointed out in above comment.