Constructive Pessimism
5 minute read
Hope creep is the plague of value delivery. We assume that since something worked in the past it will work again. We forget how much pain we’ve had in the past. “How hard can it be to build that? Looks pretty simple to me!” This is a big-bang, waterfall delivery mindset. We need a mindset that embraces reality. We need to internalize that everything is probably wrong and broken so we can prioritize our actions correctly.
There are many technical and process practices required for executing continuous delivery. It’s not easy, some are counterintuitive, and it requires a high degree of team and organization discipline. However, the rewards for doing it for the customers, the organization, and the individual developers are so great that baffles me that it’s not the standard industry practice. It does require that we modify our priorities though.
Priority 0: Can we get to production or, minimally, a production-like environment? This is feature 0 and should block all other feature work. Nothing we do matters if we can’t deliver.
Priority 1: KEEP THE PIPELINE GREEN! If anything blocks delivery, stop and fix it.
The Reality of Development
I’ve recently joined an organization where security is taken far more seriously than most organizations. Delivery pipelines are secure as we know how to make them and known attack vectors that can be detected with the pipeline will block builds. My first task is to deliver delivery pipeline observability to help any team using our tools find and resolve constraints in the flow of value delivery. This is one of my passions and there aren’t very many options available in the industry today for solving this problem, so it’s very cool.
We are building the application using React and other standard tools because we want to stay as close as we can to patterns we know will pass the pipeline security gates. As an advocate and mentor on CD practices, the first task we needed to accomplish was to deliver the pipeline. Should be simple
npx create-react-app <dashboard-name>
60 seconds later we have something we can build on!
For any new application, feature 0 is “can we deploy the application?” So, now we just need to deploy this to meet that requirement so we can start on feature 1. How hard could it be?
Going through the internal CD platform documentation to deploy a React application yields some simple configurations that are required. So, armed with confidence, I trigger the pipeline and await success. When I go to the newly deployed site, nothing is there. Blank page. But it worked on my desktop? No errors? WTF? Pulling the logs shows Content Security Policy violations. Going back through the docs shows I missed a step. To avoid CSP issues, React needs the INLINE_RUNTIME_CHUNK environment variable set to false. Doing that and rerunning the pipeline resolves the issue and we are off to the races.
We start working on the first story. We need a header, a collapsable navigation menu and to display something the first route. We mob the requirements and a few hours later we are looking good!
This is going to be easy! We have a few things to clean up. We refactor some tests and refactor some code. We only need to deploy the newest change and send it out for people to give feedback. We believe in rapid feedback, so we aren’t going to wait for “done” to show this off.!
After triggering the deployment, all of the tests pass. there is no indication anywhere that we have a problem. However, what appears in production is…
And the hair rending that is software development begins again. Inspecting the logs shows dozens of “Refused to apply inline style because it violates the following Content Security Policy…” errors
We’ve run afoul of CSP again after adding MaterialUI. We have a backlog of critical features that we need to deliver quickly. However, it doesn’t matter. We cannot deliver. Our highest priority is solving the reason we cannot deliver. We begin mobbing the issue…
It Could Be Worse
Imagine for a second we executed the way a majority of teams do.
“Pipelines are for delivery. We aren’t ready for delivery until we get signoff on v1.0 so that’s not the priority. When we finish all of the work we need to do for the first version, then we’ll focus on how to deliver. I mean, it works on our desktops so it’s probably fine. We hope we can deliver! We hope our technical decision will work in our environment! We hope it’s what the end-user actually needs!”
I’ll never again work on a team that works that way. Some people reading this are still working this way. I have empathy for you, but no sympathy. We don’t need to work this way anymore. We shouldn’t pretend we can deliver on time if we don’t know we can deliver at all. We need confidence. We need to assume things are broken until we can prove they are not. We need to assume that the ideas or requirements we have for features are wrong until we prove they are not. Building features on hopes and dreams of delivery and correctness is just a waste of time.
Ship first, verify, and keep shipping as fast as possible to keep confidence high that you can deliver and are delivering the right thing. Be pessimistic. Don’t give in to hope.
Written on April 30, 2021 by Bryan Finster.
Originally published on Medium