Velocity
DevOps Metrics We Love – IBM
The key goal was metrics that would help teams improve their agility and efficiency, while maintaining their quality.
Speaking at the DevOps Enterprise Summit IBM’s Craig Cook, DevOps Coach and Ann Marie Fred explored ‘DevOps Metrics We Love’.
Their goal was to define how to incentivise the right behaviours without encouraging people to game the system.
This was prompted by a request from their VP who wanted to develop a Velocity Dashboard, which would show the data reporting on development squad performance. Their initial challenge was that:
- Story point numbers are not consistent across squads.
- If you ask teams to increase velocity they’ll probably just start padding their story board estimates.
So the key goal was metrics that would help teams improve their agility and efficiency, while maintaining their quality. They decided to focus on code reviews and the code delivery process.
Following an extensive discovery process, they defined the following as their core metrics to monitor:
- Availability vs SLO Attainment: A score of 100 is given for 100% uptime, decreasing to 80 for meeting the SLO, and zero at four times below the SLO. Goal: Never have the site go down.
- Deployment Frequency: The number of successful deployments over the last 30 days, with 0-1 deployments: Red, 2-3: Yellow and 4+: Green. Goals: Make small, frequent, low risk changes. Use continuous delivery, automate tests and leverage Github.
- Deployment Stability: The percentage of time when the most recent build was successful: 0-50%: Red, 50-90%: Yellow, 90%+: Green. Goals: Shift left to find errors earlier, fix long-running build and deployment issues.
- Deployment Speed: The time to deploy changes to production. A perfect score for build times under 20 minutes, decreasing to zero for above 90 minutes. Goals: Judicious use of build parallelization to speed up builds, improve the Mean Time to Recovery and increase Deployment Frequency.
- Repository Speed: The time from submission to merge of Github pull requests, with a perfect score for 0-2 workdays, decreasing to zero for 5 workdays. Goals: Eliminate neglect of pull requests, reduce Work in Progress.
- Repository Efficiency: Added lines of code in Github pull requests over last 30 days, with a perfect score for less than 150 lines, decreasing to zero for 500 lines. Goals: Keep changes small and low risk, make code reviews easier and faster.
