LOG-IN
FREE REGISTRATION
SEARCH
EMAIL

Continuous Iterative Development (Agile) Measurement Framework

This report provides recommendations for the measurement of continuous iterative developments (CID). The report includes a Practical Software and Systems Measurement (PSM) CID measurement framework detailing common information needs and measures that are effective for evaluating CID approaches. The information needs address the team, product, and enterprise perspectives to provide insight and drive decision-making. The framework also identifies and specifies an initial set of measures identified as practical measures to address these information needs.

This guidance is intended to be used by team, program, and enterprise personnel who are implementing CID approaches, as a reference for common, practical measures that can be utilized.  The measures a program or enterprise chooses to implement and collect will be tailored based on alignment with its information needs and objectives, so they may differ from those described here. The measures presented are intended to be tailored and adapted to the development approach and environment.

Version 1.05 detailed potential information needs and measures that are common to CID approaches, and an initial set of ten measurement specifications that were prioritized by user surveys as highest value. This Version 2.1 includes added material that has been researched and developed by the CID working group. The new materials include information on measuring:

• Product value (Part 2, section 8.11)

• Enterprise measurement (Part 2, section 9)

• Software Assurance (Part 3, section 10)

• Technical debt (Part 3, section 11)

CID Measurement Specs from the White Paper (updated 15 April 2021)
AUTOMATED TEST COVERAGE (PRODUCT OR ENTERPRISE MEASURE)
BURNDOWN (TEAM, PRODUCT, OR ENTERPRISE MEASURE)
COMMITTED VS COMPLETED (TEAM, PRODUCT, OR ENTERPRISE MEASURE)
CUMULATIVE FLOW (TEAM, PRODUCT, OR ENTERPRISE MEASURE)
CYCLE TIME/ LEAD TIME (TEAM OR PRODUCT MEASURE)
DEFECT DETECTION (TEAM, PRODUCT, OR ENTERPRISE MEASURE)
DEFECT RESOLUTION (TEAM OR PRODUCT MEASURE)
MEAN TIME TO RESTORE (MTTR)/ MEAN TIME TO DETECT (MTTD)
RELEASE (OR DEPLOYMENT) FREQUENCY (PRODUCT OF ENTERPRISE MEASURE)
TEAM VELOCITY (TEAM MEASURE)

PRODUCT VALUE (TEAM, PRODUCT, OR ENTERPRISE MEASURE)

Background:

A collaborative working group was established between Practical Software and Systems Measurement (PSM), the National Defense Industrial Association (NDIA) Systems Engineering Division, and the International Council on Systems Engineering (INCOSE) to develop a PSM CID measurement framework for Continuous Iterative Development (CID) in response to recommendations of the strategic Defense Science Board (DSB) (Design and Acquisition of Software for Defense Systems, Defense Science Board (DSB) Task Force on Design and Acquisition of Software for Defense Systems, 2018) and Defense Innovation Board (DIB) (Software Is Never Done: Refactoring the Acquisition Code for Competitive Advantage, 2019) Software Acquisition and Practices (SWAP) study.

Additionally, the U.S. Department of Defense (DoD) is making a transformational change in acquisition policy by redesigning the Defense Acquisition System, including the addition of a new Software Acquisition Pathway (Software Acquisition Pathway Interim Policy and Procedures, 2020). The general guidelines for this new acquisition policy are established in Section 800 of the 2020 National Defense Authorization Act which became law on December 20, 2019. The new pathway will promote Agile and DevSecOps and allow for upgradeable and timely fielding of software in a way that aligns with this CID approach. The measurement recommendations in this paper provide a good framework to measure the Execution Phase of the Software Acquisition Pathway.

The most critical information needs and measures have been prioritized, based on a series of surveys with members of relevant NDIA, PSM, and INCOSE working groups.  Additional measures will be specified, and revisions to the information needs will be included, as additional input is provided.  This framework will be improved over time. We welcome your recommendations and comments.