Measuring to Fail
4 minute read
Are you leading teams and want to make sure they are productive? There are many ways to do that. Many are doing it incorrectly.
It seems like I’m always talking about metrics. When I was first exploring how to grow continuous delivery as a team capability, I needed to find ways to find out we were doing it correctly. I mostly did it wrong at the start, but I sought out better information and then tried to verify if it was better.
Continuous delivery is not using Jenkins to automate build and deploy and hoping things don’t break. CD is the process of optimizing the flow to delivery, from request to the end-user, to improve quality feedback, continuously improve security and quality gates, and get user feedback a rapidly as possible. To know if you’re doing this well you need to understand metrics. However, even if you aren’t focusing on CD you need to understand how to measure because you cannot improve what you cannot measure.
Measuring Productivity
A common question is, “how productive are we?”. There are many ways to measure productivity, so let’s start with the bad ones.
Utilization
I had a conversation recently with a friend who told me their manager wanted to measure the utilization of the team to make sure they were fully utilized. In fact, the manager’s opinion was that if the teams weren’t sometimes failing to meet their commitments, they probably weren’t working hard enough. This attitude is something I hear frequently from many companies. I even hear it from manufacturing companies. That’s just astounding to me. No one in manufacturing would want 100% utilization of the assembly line. However, let’s ponder this in terms most engineering managers should understand.
This is a highway
This is a fully utilized highway
If you had a high-value delivery to get to your customer, which would you prefer to use?
Committed vs Completed
We want to make sure that teams are “meeting their commitments”. They promised they could deliver and we need to hold them accountable, right? So let’s focus on how much they failed to deliver. In the past five sprints, the team committed to 100 story points and delivered, on average, 50 story points. They really must not be very productive, right? So we will use this ratio to make them do better. Look! it worked! It just 1 sprint that lazy team decided to meet their commitments! they doubled their velocity! How’d they do that? We need all teams doing that?
They could reduce their commitment and inflate the story points or they could skip quality processes. We definitely want to encourage that because the metrics look better. Now everyone is happy but the customer.
Doing it better
Productivity is our ability to deliver what the customer needs and to continuously improve our ability to be more efficient and more effective at doing that in a sustainable way that reduces neither the health of the code nor the health of the team. We need to measure the things the customer cares about so we can make them happy. If the customer is unhappy, it really doesn’t matter how management feels about the metrics they are using. What does the customer care about? They want to reliably receive quality work when they need it. We can measure delivery frequency, change fail rates, bug reports, customer feedback, and team feedback. Productivity is not about how many features we can deliver. It can be more productive to deliver less because we are delivering the right features instead of racing to produce things that no one wants. Remember, activity is not work. If you add energy to an inefficient system, you mostly generate heat, not work. That is only valuable if the customer is cold.
Written on January 14, 2021 by Bryan Finster.
Originally published on Medium