CEB Blogs


IT Applications

4 Ways to Measure Agile Software Development Productivity

One insurer in our network uses four metrics to measure Agile teams' productivity and to spot any problems with their development process

Agile software development principles are used widely in many of the world’s firms, and for good reason.

Hardware, and the software which operates it, have changed radically since the “Agile Manifesto” was published in 2001, but most of those changes only serve to make the manifesto more relevant to the way software is now developed and used. Ideas which those early pioneers aimed to show as futile – such as creating a separate paper-based manual to accompany a new piece of software, or treating a piece of software as a “finished” product which could never be changed or improved upon – would be considered laughable in 2014.

Much like the software development process it describes, however, the concept of Agile development is constantly being refined and improved. Recently the “Agile community” (as software and applications developers that use the principles are called) has debated the value of using “velocity” as measure of a developers’ productivity. Critics note that velocity suffers from many of the same challenges as other attempts to measure productivity, like counting the lines of code produced by a team or developer: without a way to accurately quantify the development work being undertaken, productivity metrics can’t truly be compared from one team or developer to another.

How to Improve Agile Productivity Measurement

Instead of chasing the red herring of productivity metrics, leading applications teams have changed their approach to compare the performance of each team to its historical performance rather than comparing different teams working on different applications to one another. The dashboards also track leading indicators of teams’ chances of delivering the right application on time.

Dashboards like this allow senior applications managers to spot and remove obstacles to team’s productivity before they upset the project. One US insurance firm in the CEB Applications network includes four specific metrics in its Agile development dashboard.

  1. Team adherence to Agile standards: The applications team identified a set of core Agile practices for successful project delivery across five areas: requirements and analysis, release and iteration management, development and integration, testing, and organization. Each development team is assessed on their adherence to these standards every six months through a peer assessment by another Agile team.

    The standards are written as a description of expected behavior, but allow flexibility in how teams accomplish them as long as they are adhering to the intent of the standard. The insurance company then treats the set of standards as the basis for continuous improvement, and the peer assessments are also used to identify innovations and best practices that should be applied across teams.

  2. Team-specific productivity levels: Like most applications functions, the CEB member tracks team productivity using industry-standard metrics including lines of code, function points, time, and cost. But these are not used to compare productivity levels across teams. Recognizing that differences in system type, programming language, team make-up and tenure, the insurance company tracks productivity on each team over time.

    The productivity level of each team is expected to improve over time as team members collaborate more effectively, so any productivity dip is an alert that the team has encountered a roadblock and may need external support to overcome it.

  3. Staffing health of critical team roles: Instead of having a single team leader, the firm separates applications team leadership into four roles to develop expertise in each of the disciplines core to Agile project delivery.

    This metric ensures that all four “keys” – “Scrum Master,” “Requirements Lead,” “Test Lead,” and “Technical Lead” – are correctly staffed as they have proven critical to keeping a project on track. Monitoring the staffing level of these roles across teams enables senior applications managers to prioritize open roles and reallocate “keys” to projects with the greatest need.

  4. Level of business input: A core tenet of Agile methodologies, business input measures whether the product owner role is staffed by someone with the necessary knowledge and decision-making authority to effectively contribute to the project. The ideal product owner is a subject-matter expert, to whom the business partner is comfortable delegating decision-making authority.

    When the business partner cannot delegate authority, or the absence of a suitable business SME requires an IT staff-member to act as proxy, the insurance company introduces extra rigor or checkpoints to mitigate the risk of the project failing to meet desired business objectives.


More On…

One Response

Leave a Reply to Lana Boltneva



Recommended For You

IT Applications: Help the Business Let Go of Legacy

Despite complaints they may make about an application during its lifetime, business partners are often...