Team Metrics Notes

DORA

4 DORA metrics — time elapsed between merge and deploy, frequency of deploy, time to recover from outages, duration of outages — as well as how often someone is paged outside of business hours. These track pretty closely to engineering productivity and efficiency.

Martin Fowler on Measuring Productivity

Principles for Team Metrics

• Assume that the people we are looking at the metrics are not prics - metrics do not give you context or allow you to do comparisons between teams. It does however give you indicators on whether a team may need help, etc.
• Prefer leading metrics to lagging metrics - metrics that show things before the fact are more useful than metrics that show things after the fact.
• Keep things simple - metrics that are simple to collect and understand are preferred to metrics that are complex to collect and/or understand.
• Keep things convenient - metrics that offer slightly less value but require no major team process adjustments are preferred to metrics that require major team process adjustments.

Proposed metrics for the team

How are metrics collected

• Because we value human interactions we currently prefer physical team boards to digital team boards. This means that collecting metrics will require more effort than for teams who use digital boards.
• Most metrics will be taken from artifacts that the team uses (board, cards, etc) HOWEVER, we understand that people make software and so understanding how the people that make our software are feeling is also extremely important.
• Because we work

Questions asked per team member each week on a Friday

• As a team this last week we made measurable progress towards our strategic work as well as improving our system

  I strongly agree / I’m not sure / We definitely didn’t

• This last week I strived to do my best work

  I’m Distracted / I’m tried, but not sure if I was successful / Yes, I did

Progress is what ultimately motivates us all and makes us happy—working on a team where we feel we’re making progress, working alongside people we trust and connect with, creating something really important. Lessons from Agile - Happier teams are more productive

• Do you feel as a team we are meeting external expectations on ...

Metrics gathered each week

• Cycle time on Red, Yellow, Orange & Green Cards

  • Red card (POBT Items or bug that have been reported to us of things that are already in production that need to be fixed)
  • Orange card (Misc or Ops cards - work that does not meet our event line strategy or adhoc ops tasks / investigations)
  • Yellow card (Work that helps us meet our strategy eventlines - typically this relates to a user story on JIRA)
  • Green card (Strategic technical improvement cards - technical improvements we feel necessary to help us get legacy systems to meet platform principles)

  What is Cycle Time for us? For us cycle time is the measure of the elapsed time from the moment we “start’ working on an item until it is released into production. For us start is when we have pulled a card from our backlog into ‘Dev’ column. Why is it important to us? Cycle time reveals how smoothly work is flowing through our development process. It helps us spot bottlenecks and see the effects they have on your delivery.

• How long does it currently take us to get a single line of code from ‘ready for release’ to ‘released’ - measured in hours

  • We are looking to focus on this area going forward and at some point measuring actual time to release, for now we are just going an a rough estimate.

Notes

It is important to distinguish between metrics that are important to the team (and immediate group) vs. what stakeholders outside of the group:

• How confident are we that we are meeting our estimates on Epics? (ie. When will we be done?)
• How happy is there team?
• Anything else?

When tracking velocity please make sure you do not track it on tasks, but rather stories.

• Epics: Large feature or sets of features
• Stories: A single discrete feature, increment of a feature, or feature iteration (or spike)
• Task: Analysis, investigation, design, development, test, research, document, etc.

Present data:

• along with a story,
• with values over time,
• in a way that if it is compared it makes sense or in a way that it cannot be compared.

Probing Questions

• I would recommend this aas a great place to work
• I am proud to work in my team
• We have a good level of collaboration within our group
• We are reducing technical debt
• We deliver customer value often
• I have all the tools necessary to do my job
• I receive appropriate recognition from leaders
• I have a development plan that supports my career objectives
• I understand the purpose of the work my team is doing
• I have access to the learning & development I need to do my job
• My leader has created a great working environment
• I believe there are good career opportunities for me
• I have fun at work
• We have an embeded culture of innovation
• How long does it take you to deploy one line of code to production.

Other

Quantitative data on how the team is improving Progress tracking on delivery to give us a feel for when we will be done and how we can project future work Impact of wasteful activities

I acknowledge that some teams are still learning how to track progress themselves and decided to include a brain dump of different types of metrics that I can think of (without debating the merit of each);

Performance Delivery rate on estimate hours, story points or story counts (velocity) Performance of actual delivery vs estimated delivery over time. Delivery speed such as lead times, cycle times Utilisation, capacity, etc. Number of releases per day or week Flow and Waste “Value add” time vs “Non-value add time” Wait time and Hand-offs Task switching and interruptions Day stories been blocked or not moving Ad-hoc non-backlog work, meetings, etc. Rate of unplanned work added to backlogs Work in flight and queues Defect rates / Change request rate Support calls / incidents General Team / customer / product management happiness NPS Test coverage Build success rate Value Impact on revenue by features Cost of delay NPS Improvement The effect of improvements on the above Number of experiments per month

Some metrics that may be useful

Event Line -> Milestones Estimates -> Predicatbility

Peformance?

• Why do they need the data?

Visibility?

ROI vs Performance

Should Event Lines?

• Goals

Establishing Pulse of team

• Social Connection
• Brand & Company Reputation
• Rewards & benefits
• Career opportunities
• Values
• Service & Quality
• Feedback & recognition

Enablement

• I have access to the things i need to do my job well
• The process for managing performance here enables me ot perform at my best

Feedback & recognition

• I receive appropriate recognition when I do good work, beyond my pay

Leadership

• The leaders have communicated a vision that motivates me
• The leaders demonstrate that people are important to the company’s success
• I have confidence in the leaders

Learning & Development

• I have access ot the learning & development i need to do my job well

Management

• My manager gives me useful feedback on how well I am performing

Values

• I know what our company values are…

Service & Quality

• I believe we learn from client feedback and make improvements accordingly