How is everyone measuring Impact?

[Heather Ford]

I'm relatively new to JPD and loving it so far as it gives us a holistic view of our product roadmap across multiple project teams. I now have all our 'ideas' on the board and looking for guidance on how to define "Impact." I watched a great video on how you can create your own impact score using insights, however, I like the out-of-the-box option that plots Impact vs. Effort on a visual map to help identify how to prioritize roadmap items. Effort was pretty straightforward based on swag estimates/hours. Impact is proving to be more difficult as there are so many factors to consider as it relates to impact...impact to the business (e.g. new revenue), impact to customers (e.g. better UX, stickiness). I'm thinking about using this in terms of 'customer impact'.  

Two questions.

1. How would you recommend measuring Impact?
2. How are you using Impact and Effort to prioritize work?

[Tim F · Accepted solution]

I think there is a lot of nuances here you still have to take into consideration and what kind of project or phase you are in.

For my line of work, I normally collect feedback from Members and then add it to this project discovery board.

From there my team and I work on adding some impact ranks to it.

We like:

• Development Ease ( How easy is it for our Dev team to work on this, Rough estimate 1= hardest , 5= easiest)
• Marketability ( Can we market this new feature or showcase it on our website to increase revenue)
• User Impact ( Do we think current users will enjoy this and increase quality of life even if we cannot market the upgrade)
• User Requests ( everytime a member reports wanting this feature we increase the ranking by 1)

If not collecting user feedback and taking on other projects, the answers from the others on this thread will probably help as well.

[Amina Bouabdallah · Accepted solution]

Hi @Heather Ford, thank you for writing! The out-of-the-box impact field can be used qualitatively within the team to denote impact of the work on the business, while remembering that customer satisfaction impacts the business. 

One way to go: you go through the list as a team, assign 5 to the most impactful features, 1 to the least impactful. In a second round, you could relatively assign a number to the other ideas by comparing them to one another.

E.g. if Feature B is less impactful than Feature C which has impact=5, but more impactful than Feature A (Feature A < Feature B < Feature C) then assign impact=4 to Feature B and impact=3 to Feature. 

Another option would be to have a convention between an impact score and a range of value of a business metric. E.g. if being impactful means increasing revenue, then you could have a mapping like this:

- Impact= 5 --> idea requested by $1M ARR+ customers

- Impact=4 --> requested by $[500k,1M] customers 

etc

Hope this helps! can't wait to hear from the community on their Impact stories!

Happy PMing :) 

Amina

[Chris Timms · Accepted solution]

Hello @Heather Ford ,

If you got back a step and think about what you are trying to impact, it will typically be:

1. A single North Star Metric like CSAT or failure rates. A single metric is pretty straightforward to define, you'd just use whatever scale you were are measure that metric on to measure impact on that metric. (e.g. we think this idea would impact CSAT positively by 2% net because of X, Y and Z reasons)

2. Multiple Metrics. Impact on customers is a little general for my uses, since you aren't being precise enough on what would improve if you did the work. I'd map out all the possible "initiatives" you could have (e.g. stickiness, new revenue, transaction value, faster order times) in a seperate field, stack rank those with the rest of the business and then assess ideas impact against it's assigned initiative. 

So in this model where you're impact field is genericised, I use the below definitions which I shamelessly stole and tweaked from the BitBucket demos here

1. Zero impact
2. Moderate impact
3. Significant impact
4. Prevents SOME users from completing a transaction
5. Prevents MOST users from completing a transaction

This does a good job of appropriately escalating big functionality issues while also giving clear impact values.

Hopefully some of that is helpful.

Regards,

- C

[lbrewer]

You can also tie Impact score to other metrics.  Like Impact score could look at a calculation to include Reach, Sizing and Goal impact.  

Like this: 

40% Goal Impact

30% Reach

30% Sizing

[Tom Haapanen]

In my experience revenue impact of a major feature is really difficult to estimate with any level of accuracy, and for smaller items it's practically impossible. One option is to use something like a five-point scale, as suggested by a few other posters; this is coarse enough to not be a total guess.

We ended up not using the impact score at all, and only using the voting capability. As we want the voting to be done on an organizational level (each one of the global regions votes) we removed the voting function from almost everyone, and then assigned it to just a few people per region, 50 votes each. The regions were able to assign up to 10 votes to any individual idea, and we then use the total number of votes as the proxy for the impact of the idea.

[Gergely Fehér]

Hi,

My approach here:

• 5: serious legal or security or other issue, the stops our software if we don't act in time
• 4: affects all/nearly all users
• 3: affects large user groups
• 2: affects small user groups
• 1: no impact on users at all