DevOps Transformation: Metrics That Show Business Value

In this video, David Rizzo, Vice president of Product Engineering and David Kennedy, Solutions Architect at Compuware talks about implementing DevOps and committing to continuous integration/continuous deployment.

During this process, it is very important to baseline, collect as well as analyze the valid metrics. This will help in the identification of bottlenecks in the value stream. This video will also provide details on how a company enhanced the time spent on innovation, improved MTTR (Mean Time To Resolution) and reduced escaped defects.

At 2:26, Mr.David Rizzo states that agile and DevOps were implemented at Compuware, thereby transforming the company in a very short period of time. He also quoted that continuous improvement is better than delayed perfection.

Compuware has value streams for enhancement work and bug fixing. The speaker adds that the metrics dashboard is monitored at the end of every quarter. The metrics help in evaluating the performance of the different processes that took place in a brief period of time.

At 6:16, Mr.David Kennedy briefs on the metrics that show business value. He mentions that the right KPIs (Key Performance Indicators) are extremely important for DevOps success. The continual improvements on the metrics can enhance the organization’s DevOps process and development outcomes.

Key Value Stream Metrics

At 8:06, Mr.David Kennedy briefs about a key metric, ‘Flow velocity’. Flow velocity metric can be defined as the absolute count of each type of work that gets closed in a specific period of time.

The measure of this metric is helpful in answering questions like ‘How productive is the team’ and ‘What is the throughout of the team’. This metric provides an actual count of the type of work that was delivered to the customers. This means that the new update went though the whole value stream, software was updated and value was delivered to the customers. In addition, this metric also provides a number that indicates the number of issues that were not delivered.

At 10:00, the speaker talks about another significant value stream metric ‘MTTR’ (Mean time to resolution). This metric can be defined as the average time elapsed from when external defects are created until they are closed. Basically this metric is a measure of time it would take to get a work completed.

This metric plays a huge role when it comes to providing customers with well-educated estimates on when the defect would be resolved. Determining the rate of complexity will help in the estimation to a great extent. The speaker emphasizes that MTTR applies only to external bugs. In case of no bugs, MTTR would be reported as 0. At 11:02, ‘Flow time’ metric is discussed.

This metric can be defined as the average time elapsed from when work items get created until they are closed. This metric is very helpful in knowing how long it takes for the work items to be coded, get through development, pass the code review phase and to be shipped. He adds that ‘Flow Cycles’ is the number of days spent in each phase of development.

At 14:16, the speaker talks about the ‘Innovation’ metric. This metric can be defined as the distribution of time spent on non-customer defects, including but never limited to just user-stories, epics, technical debt and enhancements. This metric helps in gauging the balance of defects that are being actively worked on and new works.

The measure of this innovation metric will help the company know how consistent they are delivering new functionality.

At 17:10, the speakers illustrated a sample dashboard containing all the key metrics. The dashboard has filters for work type, agile team, product and family. People can choose a specific work type or agile team to know the metrics that are applicable only to the performance of their team or applicable for a specific work type. The flow velocity, flow time and MTTR metric numbers are highlighted with precise details. The flow distribution metric is displayed in the form of a circular graph.

Conclusion

At 26:11, the speaker adds that at Compuware, data inputs are retrieved from Compuware tool usage information and DevOps metrics. This is followed by monitoring KPIs (Key Performance Index) that measure productivity and compare against the existing benchmark. Machine learning shows behavior patterns to enhance the DevOps process.

The next step is to identify prescriptive options for improvement. At Compuware, continual monitoring and quality improvement is of highest priority. Baselining, collecting these value stream flow metrics has played a huge role in identifying different bottlenecks. Identifying these bottlenecks at an early stage has helped the company to make quick adjustments quickly and continuous thereby enhancing the productivity level.