A structured approach to metrics in Agile has a tendency to generate a lot of interesting data points. Having access to better data sooner is at the same time enabling and sobering. But we need to exercise some caution in what we do with that data. A team works in a certain context – and every team’s context is different. Metrics can easily gloss over this point. If you reward (or not) teams based on metrics alone you can easily build a system which is fine tuned for delivering great metrics but not necessarily delivering value.
An example of this happens when teams get the message that the org is using story points to assess team performance. Suppose team A’s typical velocity is 80 points per iteration. Also suppose that they get told that if they can make that number 100 they will get a big bonus. Them getting more done is certainly possible. But it is also possible that their estimates will become subject to inflation. It is also possible that they renegotiate their definition of done to make it a little easier to get stories through the system more easily. It is also possible that they start choosing very conservative options (let’s go with what we know will work so we don’t have a trip we can’t recover from). These are all undesirable outcomes aren’t they?
If you don’t buy the point on context – let me illustrate with a non-software example. If I were to ask you to estimate how long it would take to change four tires on your car what would be your number? Perhaps an hour? In Nascar racing the drivers tend to get a lot of the attention and glamour. But it remains a team sport even during the race itself. Well-executed pit stops are crucial to a team’s overall strategy and no driver will succeed if the team’s pit stops do not go well.
Watch the pitstop of the No. 27 car in the clip below.
Amazing no? If you were the manager of this team you might easily find yourself handing out bonuses to the entire team. Measured at about 16 seconds “those kids are fast” might well be your assessment similar to the voice at the end of the clip.
And so you attend a cocktail party that very evening (because that’s all we software professionals do) and you proudly share your story of your team who executed a pit stop in only 16 seconds. Cue Freda who manages an F1 team. She has a similar story to share.
This is Freda’s F1 story:
That was a 1.92 second pit stop shown from several angles. Suddenly you might feel like you want back all those bonuses.
But hang on a minute. The context isn’t the same!
1. In F1 there is only one lug nut on a wheel. Nascar has five.
2. F1 has no refueling mid race (they once did). Nascar refuels.
3. The F1 team has close to twenty people involved. (four people operating jacks alone – one front, one rear and each has a backup). Nascar rules allow for only seven team members to go over the wall.
This third point is worth discussing. You might be tempted to conclude that the Nascar team should still take no longer than approximately three times the duration of the F1 crew since they are approximately one third the size. But you would be wrong and metrics are misleading you. The overhead to switch to a second task is there to be viewed. And take a fresh look at how the Nascar team were set up to begin with – they move the teams to the work (mmm hmmm). In short the nature of what the teams are asked to do is different. It sure looks similar though doesn’t it? And in fairness it is similar. But there is sufficient difference to make a direct comparison meaningless.
The temptation to make direct comparisons between teams in the workplace is strong. I would recommend you resist it though. You run the risk of demoralizing teams and making yourself look out of touch.
This is clearly an extreme example to illustrate. But mind well that your teams are set up differently, have different contexts and are often engaging according to different rules. Making easy comparisons between them is not advisable.
Is it okay to aspire to a 1.92 second pit stop? Of course – but that comes at a price and requires a certain team set up. Whether you really need pit stops performed in 1.92 seconds is really driven by the business/sport context of course. Funny trivia: Juan Manuel Fangio won the 1957 German Grand prix with an extended break for a drink and a smoke.