Many companies who have had formal Software Development Life Cycle processes are now making the transition to Agile software development methods. Because these organizations are used to a great deal of process formality, they may be tempted to define a rigid process for Agile, but to do that is to miss the point. Agile is inherently a learning process where experimenting and improving process is a core principle. Being agile vs following a rigidly prescribed Agile Process will enable a company to be be more resilient in the face of change, and in today’s environment change is happening at a very rapid pace.

What Does it Mean to be Agile?

Let’s start with the Wiktionary definition:

Having the faculty of quick motion in the limbs; apt or ready to move; nimble; active; as, an agile boy; an agile tongue.

So what’s an agile organization?  First, there is the ability to quickly notice and respond to something, be it an opportunity or a threat.  But I think think the definition of organizational agility needs a little more, because a reactive organization can respond quickly, but often inappropriately.  So how about this?

Organizational agility is the systemic ability to rapidly notice and effectively respond to emerging ideas and situations.

I added the word “systemic” because I think an agile organization, is agile at all levels of the organization, from front-line worker to CEO.  Consider the Ritz-Carlton hotel.

The Baldridge Award-winning Ritz-Carlton hotel chain empowers (and even encourages) any employee at any of its properties to “fix” any guest problem on the spot. Employees are empowered to implement or create any customer satisfaction solution that will cost under $2,000. Ever stayed at a Ritz-Carlton? The commitment to customer service radiates from every employee.

Northern, Mark (2004, November). Everyone Loves Their Own Ideas.
Fast Company, Issue 88

The top management team can’t be everywhere, seeing everything and responding to everything.  It takes the whole organization to be agile.

Why Do Companies Want to Become Agile?

Companies often become interested in becoming more agile when they notice that they aren’t getting what they want out of their current processes.  They find that it takes too long to get an new idea funded and too long to get a product to market.  They are over budget and under delivering.

Agile product development practices are seen as a way to become more agile, so they are brought into the company, usually using a pilot project to see how they work. Often the pilot will be quite successful. The team is excited, the market folks are pleased with the fast turnaround, silos are broken down and collaboration rises.

And then someone says “this is great”, let’s write the process document for exactly how Agile works, and have everyone follow it”. Unfortunately, that way of thinking can take the company away from the agility they were trying to achieve by bringing Agile into the company in the first place.

The Inherent Conflict Between Agile and Rigid Process

Human beings design processes based on their experience.

“Hmm.  The last guy that went into the woods over there got mauled by a bear.  Maybe I won’t go that way.”

“We need to keep people from getting killed by the bear.  Let’s put up a sign to keep people from going on that path.”

We figure out things that work and don’t work and try to make use of that knowledge productively in the future.  Nothing wrong with that. Particularly if we keep using new evidence to modify the process over time.

“OK, so Bill, Mary and Yoko all sneak down the path every day, ignoring the sign.  It has been five years since someone got mauled by a bear going that way.  Maybe the bear died or went somewhere else.  The path is, frankly, the fastest way to go.  It is probably OK to take down the sign and let people walk that way again.”

When we start to believe in a process and no longer pay attention to reality and adjust the process when it needs adjusting, we get into trouble.   Most of the stifling processes we force our employes to follow originally had a good reason for being created, but the processes weren’t updated when there was new information.

So the intent of building a rigid process – to reliably and repeatably get a good result – is fine, but in reality, it doesn’t always work.  In fact, Meg Wheatley and Myron Kellner-Rogers argue that stability arises when an organization has the freedom to regularly experiment to find what works.

If we do not exercise that freedom to change, the organization cannot maintain its stability.  … Stability is found in freedom – not in conformity and compliance.  We may have thought that our organization’s survival was guaranteed by finding the right form and insisting that everyone fits into it.  But sameness is not stability.  It is individual freedom that creates stables systems.  It is differentness that enables us to thrive.

This need for flexibility might be particularly true today, with the incredible rate of change brought about by the severe recession and the rapid introduction of new legislation intended to combat that recession.

Why do People Follow Bad or Out-of-Date Processes?

The simplest reason is because the organization they are a part of reenforces and rewards following the process.  People are asked to write status reports based on the process, show process documents to get approval to the next stage of funding. Additionally there may be formal process audits.  Doing something other than following the process is difficult.

Note: people sometimes take actions that make it seem like they are following the process, while following a different, often more effective, process in the background.  This is obviously wasteful.  But when processes don’t work, or work very poorly, it is often the only way to get things done.

Is this Empowerment Thing Real?

Having been managed to a set of formal processes for years, people are usually hesitant to try new behaviors.  After years of being to what to do, the idea of being a self-organizing Agile team is pretty unusual.  People will be watching the management team to see if this newly-granted empowerment is real.  The slightest hint that it isn’t for real is likely to drive people back to the familiar ground of following instructions.

If the agile culture change is successful, you’ll notice that people being agile – observing what’s going on and rapidly adapting to the changing circumstances.

Advice for Companies who Like Formal Processes

My advice is to do the least amount of standardization of the Agile practices as possible, trust your people, and to adapt the existing organization to support Agile instead of having Agile adapt to the organization.  If you really want an agile organization, you’ll have to let go of some of the formality that you’ve had in the past.  If you try to standardize too much, you risk losing the benefits that had you adopt Agile practices in the first place.

There are a few areas that commonly require a little adjustment, status reporting and project funding are two common ones.

Take status reporting as an example.  Organizations that follow a waterfall methodology will typically report progress against each of the phases of the project (requirements, design, development, test etc.).  To ask an Agile project to report against these milestones doesn’t make sense, and it sends a very strange message to the team.  ”We want you to be curious and resilient if the face of change.  We have no plans, however, to do that ourselves”.

Let Agile teams report using standard Agile means (e.g. a burndown chart) and waterfall teams report using their normal means.  It may take a bit of explaining for a company president or product manager to understand what they are looking at, but only a bit.  Once people understand Agile, they’ll realize they have very high-quality information on the status of projects.

Funding is another place where there may be a need for new process.  Some organizations require approved documents to move from one stage of funding to another.  Instead, consider having your unit of funding be for three-month releases, where something is delivered into production.  That’s really pay for performance.

Summary

If you’ve decided to become an agile organization and adopt Agile practices, give the new culture the care it needs to flourish.  Live with the discomfort and uncertainty of looser processes than you are used to.  Standardize on the least number of new processes you possibly can.  You’ve hired great people.  Give them room to breathe.  Let them be great.

If you’ve made it this far, thank you!  If you have a moment would please leave a comment?  I’d like my blog to be as helpful as possible.

In some companies it’s possible to try Agile software development on a small out-of-the way project.  This is one of the easiest ways to get started.  You try something, show results and then move up to a larger-scale project.  But in some companies, there isn’t anyplace out-of-the way, and every project must engage with complex organizational structures and processes.  In these companies the best strategy for success may be to choose the opposite kind of project: one that is critical to the company and visible.

Why?

Your first Agile project will probably be one of your most difficult.

• A dedicated, full-time team is critical for success.  Most organizations who are not structured for Agile, have their people spread across many projects.  This will be one of the most difficult obstacles to overcome.
• A strong Product Owner, with deep understanding of what a customer needs, is critical for the project’s success, yet it may be difficult to find someone to fill this role.
• A dedicated team room works best for Agile projects.  This might be difficult to secure.
• The role of testers changes dramatically in Agile.  Testers are part of the Agile team.  Instead of testing at the end of the project, they are testing regularly, alongside the developers.
• The team needs the ability to regularly test new software in an integrated environment.  In many companies that is difficult to arrange or the controls around the test environment make it very difficult to test multiple times per day.
• It is likely that the management team will have a format for status reports intended for waterfall projects.  Agile teams usually can’t complete these reports since their measures of progress are entirely different.  The management team will have to learn a new status reporting methodology.
• There is documentation, but it will most likely not be the same documentation that has been produced in the past.  It will take a while for the rest of the organization to get used to the changes.
• Functional managers have been accustomed to moving team members between projects like interchangeable parts, but this will really hamper progress on an Agile team.  This will be a challenge for the functional managers to accommodate.
• People are probably used to having a comprehensive set of requirements up-front.  Not having this will likely make them uncomfortable.
• Even though project plans are often wildly inaccurate, people are used to having them and will be may be uncomfortable without one until they learn how things are managed on an Agile project.

With all of these challenges you’ll benefit from strong senior management support.  You won’t get that level of support if you’re working on a low-visibility project; the burden for solving most the the challenges outlined will fall to the team.

My recommendation is to find a project that is critical to the company and get the management team on board.  Let them know what the challenges will be and enlist their support.  If possible, have them communicate what you are doing and why you are doing it to the rest of the organization.  Use the management support when you need it.

It takes work to change an organization, but if you choose the right project and get the right support, you are more likely to be successful.

In the book “a simpler way” by Margaret J. Wheatley and Myron Kellner-Rogers, there’s a paragraph that got my attention.

“Life is intent on finding what works, not what’s “right.” It is the ability to keep finding solutions that is important; any one solution is temporary.  There are no permanent right answers.  The capacity to keep changing, to find what works now, is what keeps any organism alive.”

In software development, teams are often asked to follow practices that seem to impede effective work.  I’ve seen them respond by pretending to follow the formal process while doing what works behind the scenes.  Instead of being open about what they are doing, they conceal it.  And they incur the additional work of making it seem like they are following the official process by filling out the approved project templates.

How much more productive would it be if teams could talk openly about what they were doing, and other teams could learn from their experimentation?  How much more engaged would an organization be where experimentation was encouraged?  Where people talked openly about what they learned?

Have you been part of an organization that has encouraged experimentation with process and open discussion about the results?  How did it work?  What was it like being part of that culture?