Agile metrics were always a hot topic, and as more and more organizations are making a move toward business agility, investing millions of dollars into transforming their business, more leaders wonder how the success of such transformations could be measured.
Below is my take on how you can put a number value on agile transformation progress or measure it qualitatively.
Product Delivery Flow & Process Engineering
First and foremost, agility is not about working in sprints and creating burn-down charts. It is about the business’ ability to respond to the changing demand and timely discover and realize opportunities (new products, services, features of an existing product, etc.). With this in mind, we should apply lean practices to measure the delivery pipeline and establish metrics that will help to answer relevant questions such as:
Quantitative:
1.How many opportunities have we identified? How many have we prioritized and actioned upon? How many were delivered to the market? How is that trending over time?
2.How long does it take for us to deliver an idea once it has been identified (Time-to-Market)? How is it trending month over month or year over year?
3.How long does it take to go through the steps in the delivery pipeline? How long are the delays between steps?
4.How much time and money do we save per released idea compared to the initial benchmark?
5.How frequently do we need to stop the “production line” or move ideas back to previous steps in the pipeline?
6.What is the predictability of delivery happening?
Qualitative:
1.How well can we predict the outcome of benefit hypotheses?
2.How well aware is leadership of what the organization is working on for each product? How accessible is this information?
3.Are all the necessary stakeholders and SMEs engaged during each step in the life-cycle of an opportunity?
4.Do we enable everyone within the organization to voice their idea?
Product, Quality, and Customer Experience Metrics
Simply delivering upon an opportunity does not guarantee success. An agile organization is not only organized around value production, but is focused on its customers. That is why it is equally as important to measure the quality of the outcomes.
Quantitative:
1.How many times do we achieve the goals set by the benefit hypotheses? How many iterations does it take to get to the expected results?
2.What percentage of product ideas have failed? How many are discarded before delivery?
3.What is the impact on the customer retention, acquisition, and conversion rates?
4.How do our Customer Satisfaction rate and Net Promoter Score change over time?
5.How does product usage telemetry change over time?
6.What percentage of the market prefers our competitors’ product or service?
7.How do the number of quality-related support requests trend over time?
Qualitative:
1.What do our customers say about our company and products? How is this trending over time?
2.How well does our product or service align with the needs and expectations of different customer personas?
Employee Experience Metrics
Business Agility practices do not only focus on the value delivery speed and quality of outcomes, but also on the quality of life of the employees.
Traditionally, employee surveys are conducted on a yearly basis, yet modern Agile leadership practices promote a real-time 360-degree feedback approach. Following this approach, employees are enabled to provide immediate anonymous feedback to the organization day over day, or after major events and announcements. Similarly, leaders are encouraged to provide feedback to employees with no delay enabling a “work together, grow together” mentality.
Quantitative:
1.Daily employee response trends
1.Team Morale / Happiness assessment results
2.Employee retention rate, turnover rate
Qualitative:
2.Employee feedback sentiment analysis
Team Metrics
You may be wondering why I haven’t yet mentioned popular Agile metrics like Velocity, Throughput, and Cycle Time. While these metrics make perfect sense from the perspective of team performance improvement, they lose their magic powers when applied at scale.
In my practice I often roll up Cycle Time, Lead Time, and Predictability metrics to a ‘team of teams’ level to enable a quick “temperature check” across multiple teams of teams. However, these metrics should not be aggregated for teams that don’t deliver together, nor aggregated to the organizational level, as they then correspond to an “average temperature in the hospital”.
Two things that organizations can choose to assess in relation to teams is the:
1.Percentage of people engaged in delivery by cross-functional teams
2.Percentage of teams that are following agile principles or a corporate agile playbook
Financial Metrics
Agilization of the Product and Delivery practices will inevitably impact the bottom line. While it is not a straightforward exercise to track the impact of an agile transformation progress on PnL, we could observe the correlation of its progress and the revenue growth as business agility tends to reduce the cost of delay. However, this is the case of a lagging indicator that could help keep the board happy, but is not particularly useful from an operational perspective.
Many practitioners are recommending to calculate ROI, yet the question is how much of it can be associated with agility?
To address this question, we can refer to the following metrics that can be collected given a reliable pre-transformational benchmark:
1.Comparative analysis of the project costs and timelines between projects of a similar size and complexity before and after the transformation has started.
2.Risk Mitigation Costs: Assess the financial impact of Agile practices in identifying and mitigating risks early in the development process.
3.Cost of Work in Progress: Assess the financial impact of maintaining work in progress, aiming to minimize the cost associated with incomplete or partially completed work.
4.Defects Cost: Calculate the cost of defects and rework as agile emphasizes the importance of building quality into the development process.
5.Cost of Delay: Assess the trends of costs or savings associated with delaying the release, or releasing a product or feature to the market earlier.
In short, assessing the progress of an agile transformation is not an easy task. Given the complexity, it is relatively easy to fall victim to KPIs and metrics that do not adequately reflect reality. As you can see, I am not suggesting any particularly special or novel Agile metrics to assess the success of an agile transformation. Instead, I recommend assessing the ability of an organization to realize the expected outcomes of business agility, focusing on the customer and employee experience, responsiveness and value delivery speed, as well as the product and financial results.