A simple measure of effectiveness
In the Lean manufacturing world there’s a measurement called First-Time-Through (FTT), which monitors whether a cell is making products right the first time. It’s a measure of the effectiveness of the cell’s standardized work and shows the percentage of product made without any need for rework or scrap.
FTT = ( Total units processed – Rejects or Reworks ) / Total units processed
If the standardized work is adhered to, the product will be made right first time and FTT will be 100%. However, flawed materials, faulty components and operator error all contribute to rework and scrap.
Who cares about parallels between manufacturing and software development? I was just interested to read about FTT because I’ve been thinking for a while now about the effectiveness of software teams … at an operational level, let’s say. I’ve long considered an effective team as one that is able to sustain throughput (i.e. the number of cards released to production that deliver value) while fixing defects immediately and repaying technical debt to keep the amount of rework small.
I consider technical debt and defects to be rework, and technical debt to be a natural byproduct of software development. It stems from earlier decisions, based on what we knew at the time, and requires attention later when the system has outgrown the outcomes of those decisions. It is necessary rework that keeps the emerging design relevant and the software healthy and habitable, reducing risks and medium to long-term costs. Defects are basically mistakes. They happen. How we create software determines whether we have a small and manageable amount of rework or a crippling amount of rework. If we’re responsible, skilled and bake quality into code we can minimize rework to technical debt and occasional defects. If we’re irresponsible and cut corners, or we’re rubbish and write crap code, then rework can become so large that the only viable option is to cancel or start again.
Technical debt requires careful management and continuous investment while defects should be fixed as soon as they are found. A proportion of a team’s capacity is therefore always expended doing an amount of rework. That’s a good thing providing:
- the completed rework is small compared to the throughput so that capacity mostly focuses on value demand, and
- the completed rework is enough to keep the remaining rework small compared to the throughput, thus minimizing further failure demand.
(Throughput excludes repaid technical debt and fixed defects that went live).
On a weekly basis then, the throughput in relation to the remaining technical debt and defects might be a useful measure of a team’s effectiveness.
Effectiveness = ( Throughput – Rework ) / Throughput
Throughput = Number of cards released to production that deliver value
Rework = Number of technical debt and defect cards in inventory and work-in-process
I’ve pushed various teams’ data through and the charts seem to correlate with the events described in my historical notes. Here’s a chart based on a small, experienced team working on a small project for 3 months.
You can see there wasn’t any throughput in the first 4 weeks as completed cards queued up in inventory. In week 5 that inventory was flushed to became throughput as the first cut was released. Effectiveness then varied with the weekly releases until week 10, which saw the team 100% effective with no rework cards in inventory or work-in-process. In week 12, however, effectiveness dropped to -33% because 1 technical debt card was work-in-process and 3 fixed defects were queued in inventory while only 3 cards were released.
Although it’s perhaps a simplistic indicator do you think it’s useful as a measure for effectiveness (i.e. a team’s ability to deliver value and stay healthy)? Or is it utter tosh? Can it be refined (without complicating it)?