Neither waterfall nor endless sprints: the canal-lock method, for whoever has to choose

Friday, 17:42. You're closing your laptop when Dr Brambilla's email lands. Subject: AGILE TRANSFORMATION — URGENT. The body is short: "I watched a webinar. From next quarter we're going Agile. I want your proposal on Monday at 9. P.S. The new client wants the delivery date, price and features fixed in the contract. Prepare the Gantt chart too."

You read the last line three times. Agile, with a Gantt chart. Date, price and features fixed, but "we're going Agile". You're the team lead of seven people, and you're the only thing standing between that email and their next quarter.

You have a weekend. On Monday morning you could present the usual textbook Scrum, which Brambilla will turn into something else within two weeks. Or you could try to understand what's really going on and propose something that works for the project and for the people on it. You pick the second. You open an empty file and start with the symptoms.

What they call Agile

You don't have to look far: you've seen them all, at this company and the one before.

• The fifteen-minute daily that lasts forty-five. One by one, everyone tells the boss what they did yesterday. It doesn't unblock anybody: it's a status report delivered standing up.
• The sprint taken literally. Two weeks of running, with the second Thursday evening spent in the office. Then straight into the next one, no break. Sprint after sprint, forever: no athlete would do that.
• Velocity as a KPI. Management compares points across teams and wants them up 20% a quarter. Teams, not being stupid, inflate their estimates. Velocity goes up; the product doesn't.
• Story points converted into hours, so the Gantt chart can be drawn, which is what everyone wanted to do in the first place.
• "Agile" as a synonym for "no planning". Priorities change every Tuesday, nobody decides the architecture, the documentation is "the code", and technical debt never makes it into a sprint.

The result is a hybrid, yes, but the worst one possible: all the rigidity of waterfall (fixed date, price and features) with all the anxiety of Agile (constant change, a deadline every two weeks). The technical term, coined by Ron Jeffries, one of the signatories of the Agile Manifesto, is Dark Scrum. Martin Fowler, another signatory, calls it the Agile Industrial Complex: an industry of certifications and consultancy selling ceremonies to people who have never read the original text. Dave Thomas, a third signatory, titled a 2014 post Agile is Dead (Long Live Agility). Jeffries, who probably invented story points, wrote in 2019: "I may have invented story points, and if I did, I'm sorry now."

So you do the thing nobody at the company has done: you go and read the original text.

Two documents nobody has read

The Agile Manifesto was written in February 2001 by seventeen developers at a ski resort in Utah. Four values and twelve principles, a five-minute read. The values say we prefer working software over comprehensive documentation, customer collaboration over contract negotiation, responding to change over following a plan. And they add a phrase everyone forgets: "while there is value in the items on the right". Plans aren't banned. They matter less than working software.

Then you reach the eighth principle, and stop:

"Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely."

Indefinitely. The Manifesto doesn't talk about sprints. It talks about a pace you can keep for years. Thursday nights in the office, every two weeks, are the exact opposite of what's written there.

The second document is even more surprising. The waterfall model is usually credited to a 1970 paper by Winston Royce, Managing the Development of Large Software Systems. You open it and find that Royce draws the famous cascade (requirements, analysis, design, coding, testing, operations) and right underneath writes that this way of working "is risky and invites failure". The rest of the paper proposes fixes: step back a phase when needed, involve the customer and, above all, "do it twice", building a throwaway prototype first to find out where the real problems are. The diagram became famous; the text didn't. In 1985 the US military standard DOD-STD-2167 turned that diagram into a mandatory procedure for defence software, and from there it spread everywhere. Pure waterfall, the one everybody criticises, had already been criticised by its supposed inventor.

It's eleven at night, and you realise that "Agile or waterfall?" is the wrong question. You don't need to pick a religion. You need to work out, for each decision in the project, how much it costs to change your mind.

How much does changing your mind cost?

Saturday morning, over coffee, you take the list of things to build for the new client and sort them with two questions. How sure are we about what's needed? And how much would it cost to change it later?

• Uncertain and cheap to change: the UI, the copy, the sign-up flow, features whose real value nobody knows. Here Agile wins hands down: build a little, show it, correct.
• Clear and expensive to change: the contract with the payment gateway, GDPR requirements, the integration with the client's ERP that has a fixed go-live date. Here you need a plan, written down and agreed. Changing your mind halfway costs weeks.
• Uncertain and expensive to change: the data model, the architecture, choosing between a relational database and a document store. This is the danger zone. Here you need Royce's advice: a throwaway prototype, built precisely to get things wrong quickly, before making the final decision.
• Clear and cheap: bugs, small improvements. A continuous flow is enough, no ceremonies.

Jeff Bezos, in Amazon's 2015 letter to shareholders, distinguished between "one-way door" and "two-way door" decisions. The first should be made slowly and carefully, the second quickly, by whoever has the information, because you can always walk back through. The problem with large companies, the letter argued, is that they end up treating every decision as a one-way door. The problem with Dark Scrum is the opposite: it treats everything as a two-way door, including the schema of a database holding ten years of data.

Why your team is tired: the maths

Saturday afternoon you look at the quarterly plan Brambilla prepared. Everyone is allocated at 100%, and he's proud of it: "No wasted resources." That's when you remember an operations research course from university.

A team receiving work behaves like a queue: requests arrive at irregular intervals, and each one takes a variable amount of time. In the simplest model, the M/M/1 queue, the average wait before someone starts working on a request, measured in multiples of the time the work itself takes, is ρ/(1−ρ), where ρ is utilisation. Try it:

# M/M/1 queue: average wait in queue, in multiples of the service time
for rho in (0.5, 0.7, 0.8, 0.9, 0.95, 0.99):
    print(f"utilisation {rho:4.0%}   wait ≈ {rho / (1 - rho):5.1f}× the work")

# utilisation  50%   wait ≈   1.0× the work
# utilisation  70%   wait ≈   2.3× the work
# utilisation  80%   wait ≈   4.0× the work
# utilisation  90%   wait ≈   9.0× the work
# utilisation  95%   wait ≈  19.0× the work
# utilisation  99%   wait ≈  99.0× the work

The curve isn't linear: it explodes. Going from 80% to 95% utilisation doesn't make the team 15% more productive. It multiplies waiting times by almost five. Every surprise (a production bug, a colleague off sick, an urgent client request) finds no room, and queues up behind everything else. The feeling people have, "we're always late and I don't know why", isn't a commitment problem. It's maths.

The second formula is even simpler, Little's law: average lead time = work in progress / throughput. If your team finishes ten tasks a week and has forty open, each one takes four weeks on average. Keep fifteen open and each takes a week and a half, without anyone working a minute longer. Fewer things started, more things finished. And fewer switches between tasks, which are the leading cause of mental fatigue for anyone who writes code.

You jot down two numbers for Monday: plan at most 80% of capacity, and put a limit on work in progress.

The canal-lock method

On Sunday afternoon you go for a walk along Milan's Navigli canals. Under a bridge there's an old lock: two wooden gates closing to a point, a chamber in between. Boats don't tumble down a waterfall, and they don't drift wherever the current takes them either. They sail freely along the stretch between one lock and the next, and every so often they pass through a chamber where the water level changes in a controlled way, once, with the gates shut. That gate shape, the sign says, is owed to Leonardo da Vinci, who studied Milan's canal locks. Historians are more cautious (Leonardo was one of several engineers who improved them), but in Milan they're still called "porte vinciane", Leonardo's gates.

You go home and write the proposal. You call it the canal-lock method. It fits on one page.

1. The first lock (2–4 weeks). Before setting off, only the one-way-door decisions are made: core architecture, data model, contracts with external systems, regulatory constraints. For the uncertain ones you build a throwaway prototype, as Royce suggested. Each decision goes into a one-page Architecture Decision Record: context, decision, consequences. This is the bit of waterfall you actually need, and it takes weeks, not months.
2. Six-week navigable stretches. Time and budget are fixed, scope is variable: at the start you decide how much time a problem is worth, not how much it will take, and you ship the best version that fits in that time. It's the "appetite" idea from Shape Up, the method Basecamp published in 2019. To the client who wants fixed date, price and features you answer with the iron triangle: you can fix two out of three. We fix date and price, and decide together, every six weeks, what goes in.
3. Two weeks of still water after every stretch. No new features: bugs, technical debt, small experiments, documentation, training. And breathing room. It's the difference between interval training and a never-ending sprint.
4. Within the stretch, continuous flow. A Kanban board with a work-in-progress limit, no sprints inside the sprint. The daily lasts fifteen minutes and answers one question: who's blocked, and who can help? Status updates are written, asynchronously, and the boss reads them whenever he likes.
5. A demo every two weeks, with real users. This is the heart of Agile, and it stays intact: the short feedback loop that lets you discover "that's not what I meant" after a fortnight instead of after eighteen months.
6. Measure the flow, not the people. Instead of velocity, the four DORA metrics: deployment frequency, lead time from commit to production, change failure rate, time to restore service. They measure the system, not effort, and they're never compared across teams.
7. Sustainable pace, by contract. Planned capacity at 80%. Overtime is treated as an incident: if it happens, you run a retrospective to find out why, just as you would when a server goes down. The Manifesto's eighth principle, finally taken seriously.

You reread the page. There's nothing new in it, really. A bit of Royce, a bit of the Manifesto, a bit of Kanban, a bit of Shape Up and some queueing theory. But together they answer both of Brambilla's requests: the client gets a date, a price and locks where decisions are made; the team gets a pace it can keep for years.

Monday, 9am

In the meeting room the projector is on and the coffee hasn't run out yet. You present the page. Brambilla listens in silence, which has never happened before. At the end he asks two questions.

The first: "What about the Gantt chart?" You show him the locks: three dates, in red, matching the one-way-door decisions and the contractual deliveries. Between one lock and the next, no coloured bars: a scope decided every six weeks by looking at working software. He thinks for a moment, then says: "The client will like it. The dates are there."

The second: "Two weeks with no new features? Isn't that waste?" You show him the M/M/1 table. "At 99% utilisation," you tell him, "every urgent client request waits, on average, ninety-nine times as long as it takes to do. The two weeks of still water are the reason there'll be someone free to handle the urgent requests."

Brambilla stays silent for a few seconds. Then he asks if he can keep the table, to show the board. You take that as a yes.

Outside the room, one of the developers on your team asks how it went. You tell her that from October, Thursday evenings are for going home. She looks at you as if you'd announced a pay rise. In a way, you have.

Moo. 🐄