Reflections on the Agile Manifesto

On the Occasion of Its 20th Year

I had the good fortune to come of age professionally at the dawning of the Agile movement. As it began to gain traction in the mid-2000s, there was a hunger for new ways of thinking and doing software. I found myself working at a consulting company (pre-Nortal) that was heavily involved in Agile transformation, eXtreme Programming, and working with large corporations and government agencies as they grappled with this promising new way of doing business. These were heady times!

But in recent years, I’ve seen many companies reverting to old practices, their Agile disciplines deteriorating, and developers operating without even a basic understanding of the core principles of Agile. Worse, nothing better has replaced it. In fact, organizations are struggling with the same problems Agile sought to address and they are turning to the wrong people and processes looking for answers. But the answers were there all along in the Agile Manifesto, a seminal document from 2001, printed here in its entirety:

The genesis of the Manifesto and its impact on the industry is ably covered in an excellent article from The Atlantic – you should read it for the history and anecdotes. In this post, however, I’d like to offer my own observations on the Manifesto and its practical implications.

Individuals and Interactions over processes and tools

First of all, we should establish that a process is a tool, and it’s always possible to use the wrong tool for a given job. It may even work for a while but eventually, tool misuse will always come with a cost. The writers of the Manifesto seemed opposed to the idea of promoting a particular process as the way to do Agile. Rather, what’s truly agile is to concentrate on people and how they work together.

“Individuals and Interactions over processes and tools” can be summarized in one word: Autonomy. An empowered team with a healthy agile culture will choose the processes and tools that serve their needs rather than bowing to the demands of their tools and processes. This means taking into account individuals on the team rather than treating them as pawns in a process or conforming to a particular tool’s constraints. Autonomy demands trust and rewards it with flexibility that leads to productivity. Aside: Dear Reader – please, for the love of all that is holy – never call a person a “resource”.

Working Software over comprehensive documentation

I’d like to make a couple of assertions:

• Documentation is useless if it isn’t read
• Documentation is actively harmful if it is out of date

I am, therefore, a proponent of the idea that the right amount of documentation is the least amount necessary. It is disrespectful of people’s time to provide more documentation than they need to do what they need to do. It’s also tremendously expensive to produce and maintain. There’s really not much more to say on the topic.

As for “Working Software”, I would define it as software that doesn’t contain bugs and that does what it’s supposed to. There is an implication, then, that the developers know what the software is supposed to do. This requires effective product ownership and communication, which are fundamental to Agile. And how does one prevent bugs? I would argue that comprehensive test coverage and automation are the best routes to keeping quality high. TDD (Test-Driven Development), in particular, can be an effective way to write software with clear contracts between components and with a high degree of test coverage.

Customer Collaboration over contract negotiation

I was once involved in salvaging a consulting project in which the assigned team’s skill composition was ill-suited to the client’s technical needs. The team gamely, but ultimately unsuccessfully, tried to implement what was requested of them. Management was late to adjust and the failed project nearly resulted in legal action. Through a commitment to redoing the work at our own cost, we were slowly able to regain the trust of the client. To my surprise, upon the conclusion of the project, they gave us two additional projects, and our sponsor at the client became a big booster of our company. We could have quibbled over contract details and tried to do as little as possible to meet contractual obligations and get out of a difficult deal but sticking with the relationship and serving the customer’s needs ultimately benefited both parties.

It should be noted that this clause doesn’t only apply to consulting relationships. Any development team has customers. If all of their work, however, is pushed from on high and they have no say in scope, timelines, etc. – this is not Collaboration. Such a project will fail. Developers are stakeholders in the success of any project they work on and they should be trusted as partners. As well, insular developers who do not listen to experts on their product will inevitably write the wrong software. It won’t meet the customer’s need, or it will prioritize the wrong features or will otherwise confound Customer Collaboration.

Responding to Change by following a plan

This post was partially inspired by my own observations that companies are regressing in their understanding of Agile and its purpose, practices, and precepts. In particular, gimmicks such as the Scaled Agile Framework (SAFe) mark the Revenge of Waterfall without addressing the immutable realities of software development. It’s entirely understandable that Management doesn’t want to hear that they cannot have a given product with a static team on a fixed timeline but it’s a law much older than Agile that asserts that they cannot. Agile simply accepts this reality and prioritizes doing the most important thing next.

There are many variations of the aphorism, “no plan survives contact with the enemy,” but the essence with respect to Agile is that plans are, by definition, incapable of dealing with the unknown. Therefore, it is incumbent on any project subject to unknowns (which is to say, any of more than trivial scale) to be designed to change rather than to hew to a rigid plan. All plans of any consequence are doomed. This is an immutable law of the universe. Change accordingly.

So, what’s next?

Recently, Nortal’s global staff had the privilege of attending a private virtual presentation from Kent Beck, one of the Agile Manifesto signatories. In the Q&A portion of his presentation, I asked the following, admittedly leading, question:

Are we in a post-agile world? Did we get excited and try it out and convince corporations to convert to it and then all this heavy process crept back in and is hiding behind agile rituals and we’ve lost the magic?

And he offered this delightfully unexpected response (lightly edited for readability):

No! No! We are in a pre-agile world!

We haven’t…Most people didn’t even experience the magic. I think the basic…what didn’t change was the power relationships. And if the power relationships don’t change then the development doesn’t change. So if I can say, “hmmm, the fastest I think I can do this is four months” and someone else can say, “that’s not the right answer, the right answer is two months,” and I say, “oh, ok, two months…” That’s…that’s–we’ve just seen a power relationship play out. And as long as that power is in place, for someone to say–for a technical person to say something–which is true from their perspective, and to have it be contradicted and that to be the end of the conversation…if that power relationship remains in place, nothing else we do changes. Oh, now you can do TDD. So? We’re doomed. We already know we’re doomed. So, we’re not going to be energized, we’re not going to be learning, maybe we’ll panic, our relationships inside the team will be strained because we’ll just be wondering who to blame… bleeggghhh!!!

eXtreme Programming…There was a point in eXtreme Programming…I was in one of the early conferences, on a panel, and someone raised their hand and they said, “if we just do a better job of software development won’t other people look at that and say, “hey I wanna do that too,” and I stopped for a second and I said, “I can count five teams represented in this room and each of those teams operated unbelievably better than the other teams in organization. It was common for teams to have zero reported defects, which is something we need to get back to. No defects, happy developers, happy customers, making deadlines in organizations where deadlines had never been made before, and all five of these teams had been fired. So no, it is not enough to do a better job. That feels really good to programmers for a while but if the power relationships haven’t changed nothing changes.

I’m struggling with this now. I’m being asked for deadlines, for milestones… And I’m like, “well, hang on, we have a 1,000 things to do and I say, ‘well we can do 5 things this month’” and someone says, “well that’s not the right answer.” and I’m like, ”You asked how many we can get done this month and the answer is five and if the answer changes I’ll tell you but you better plan on five. The fact that you want 50 it’s just…It’s not my problem.” Part of me wants to be that kind of…you know, I feel my… back it up and that is a power relationship at play. And, I need to–and this is a skill I’m definitely still working on–I need to figure out how to have those conversations without it turning into me arguing with my father. And…personal growth….bleeggghhh…that’s the work of a lifetime. But, we need to work out a new set of power relationships and then we’ll have new software development. And it will be magic! It CAN be magic! If everybody’s on the same page. If authority and responsibility align. It’s such a fundamental principle and people ignore it all.the.time. So, yeah, agile…we had the…we NEVER had the magic. People STILL haven’t experienced it. Software development can be so much more than it is today. And that’s such a real frustration for me. That people choose not to have this experience. And they could spend less time, less energy, and have a better experience and they choose not to. But, you know, we’re all embedded in these power relationships and maybe it’s true that if that doesn’t change then the rest of it doesn’t change, but…God dammit I want it to…

I heartily concur.

The Agile Manifesto doesn’t promise that your product will be finished on time and under budget. What it does promise is that you will build the right solution for your customer as efficiently as possible and that your people will be happier doing it. It is difficult to conceive of exceeding that kind of success with any other process. Those who suggest you can are almost certainly ignoring the interdependence of Scope, Cost, and Time – the infamous Iron Triangle. But that’s a topic for another post.