Navigating the methodology maze

Project managers and team leaders have an extensive array of development methodologies at their disposal. Over the past 20 years methodologies to fill every conceivable development need have evolved. Rapid development techniques, long-range waterfall or spiral management models, “extreme” programming and iterative methodologies only name a few. Each one targets a different perceived project need, with some focusing on getting the job to market as quickly as possible while others emphasize reliability and reproducibility. From the Rational Unified Process and XP to waterfall and SCRUM, how can you decide what methodology is “right?” And when is it right? Where is the methodology for choosing the right methodology?

Standardizing on a single methodology is often not the right decision, either. A single technique is often not adequate for the myriad variables of software development. Organizations frequently make the mistake of selecting a single methodology and applying it rigidly to all technology projects. We wouldn’t choose a single vehicle for all conceivable transportation needs — such as taking out the sports car to haul furniture on Friday and jetting to the office in it on Saturday. Yet this happens all the time as organizations choose a single development methodology and employ it in all projects.

Compounding this difficult decision is the complexity involved in many aspects of methodology implementation. How can the plethora of options be evaluated? What is important for your project? How can the interaction of your team’s maturity with different methodologies be evaluated, and what impact should it have on your decision? Many so-called “best” methodologies are rejected out of hand for being too invasive, requiring too much artifact production or being too constraining. More often than not these beliefs stem from an incomplete understanding of the methodology and sometimes even its core principles. At Bold Development, where I directed evaluation and selection of methodology standards, I meet three different individuals who had “practiced the Rational Unified Process” and yet had negative experiences, laden with heavy process, excessive documentation and artifact production, as well as very poor results. Such indicators usually mean that the process is poorly understood and, sure enough, each person I interviewed had misconceptions about the Rational Unified Process. In the end, it wasn’t the Rational Unified Process that failed, but rather that the execution was in many regards misguided:

“A process should not be followed blindly, generating useless work and producing artifacts that are of little added value. Instead, the process must be made as lean as possible while still fulfilling its mission to help developers rapidly produce predictably high-quality software.” —Kroll, Kruchten (The Rational Unified Process Made Easy: A Practitioner’s Guide to Rational Unified Process)

Such problems are not limited to the Rational Unified Process (RUP). Extreme Programming (XP) horror stories abound where projects spin out of control, seeming to operate in chaos mode with no roadmap, schedule or forethought. At SeeBeyond, where I directed world-wide support operations, it was clear that standardization on Extreme Programming had resulted in over-reaching the bounds of what it could accommodate. The recurring problems of misunderstood goals, inability to plan for the future and missed delivery dates clearly indicated that the methodology was not working. Perhaps not surprisingly, this most often stems from a poor understanding of XP itself — and that includes understanding what the creators of the process originally intended.

Choosing the right tools to get the job done is time consuming. Development process is not a simple, one-size-fits-all equation. Project teams have a wide array of techniques available to them — but it’s important to remember that its the project, not the manager, that chooses the methodology. That is to say, the parameters of the project as a whole must be taken into consideration when deciding what methodology to use. These parameters include project complexity, scope, planned change, stability of the project, expected duration, team makeup and team maturity and other elements of your organization. Understanding the sometimes subtle and not so subtle variations between methodologies is critical. Choosing a methodology that works for the team and accommodates project needs is, at best, tricky.

Many characteristics of the project team itself can and should influence the choice in methodology. Yet these criteria are often overlooked as committees organize to create The Next Great Software Development Plan. Misunderstandings erupt around the perceived value versus effort of following the chosen technique, ultimately resulting in a failure that stems directly from inception. Sometimes selection is based on the prior experience of a few committee members. Other times, committees resort to whatever practice was recently highlighted in Business Week. Often a mixture of project management techniques are combined to create a new, hybrid methodology that is somehow expected to perform better than methodologies that have been evolving for the past 20 years. Whatever the source, most “design by committee” efforts result in less than optimal decisions. Making unilateral project guidance decisions without regard to the dynamics or background of the people who will be following it quickly leads to poor adoption or even outright ridicule of the methodology.

One frequently discounted consideration is team maturity. Inexperienced team members tend to reject formal process, not having seen the benefit of it. On the other hand, taking an incremental approach to process adoption can often calm fears regarding overly complex or procedural methods. Choosing a methodology that supports a lightweight, flexible starting point is critical in such a situation. Agile methodologies such as Crystal Clear and Extreme Programming provide such a starting point, as does the Rational Unified Process. But the latter is seldom selected because of a common misconception that RUP is incompatible as a valid lightweight starting point. In fact, it provides for very simple, iterative processes that can grow over time — offering the best of both worlds in flexibility and a quick start paired with very powerful, full-scale practices for larger projects. A “bare bones” introductory process can be implemented that feels like Extreme Programming, yet is completely compatible with the more advanced topics of the Rational Unified Process. Team maturity and its implications in methodology selection are explored in-depth later in this book.

Team size is another incredibly important factor that is frequently discounted when choosing a methodology. SeeBeyond attempted Extreme Programming in an organization of nearly 300 people. Bold Development, a mere 10 person organization, attempted to adopt over-weighted variations of the Rational Unified Process. Both situations are extremes, and both are incorrectly implemented. These situations result in overall failure and the mistaken opinion that the chosen process is “no good.” In fact, it is not the fault of the process, which has its place in the spectrum of development techniques, but the fault of the approach taken to implement it.

Smaller teams tend to call for smaller, lighter-weight processes. This is where methodologies such as Extreme Programming and SCRUM come in, particularly with mature teams. The danger with lightweight processes comes not so much from project complexity, but from team maturity. A team that is not well versed in the full lifecycle of a project and its impact on the organization as a whole needs more support. Combining an inexperienced team with Extreme Programming is seldom a wise idea; conversely a mature team can often employ Extreme Programming to sustain surprisingly large-scale projects. This implies an almost counter-intuitive and very important conclusion: The Rational Unified Process tends to support immature development teams better than Extreme Programming, while mature teams often can achieve great success with Extreme Programming even in complex projects. It is very common to conclude the opposite, as immature teams tend to favor lightweight, easy to use processes (the “let’s just code it” attitude), and mature teams tend to favor “complete” processes that better represent a full product lifecycle. Yet inexperienced teams use Extreme Programming all the time with the justification that an inexperienced team lacks the maturity or capability of working with what is perceived as a more complicated process. Unfortunately, this lack of complexity carries with it a high price-tag: Lack of adequate structure and control to meet reasonable, predictable goals. We’ll explore team size in greater depth later in this book.

Understanding the implications of different methodologies on your team, your project and your company is the first, crucial step toward a successful project. The myriad choices and considerations make it impossible to make the right choices under pressure, especially if you are inexperienced in a wide array of methodologies. This book provides a roadmap through the jungle of development methodologies, comparing and contrasting a number of techniques, and mapping the land mines and bridges along the way. You’ll find it an invaluable aid in deciding which methodologies are right for your team and organizations, and how best to employ them. Perhaps most important, you’ll attain an understanding of the capabilities, strengths and weaknesses of today’s widely accepted “best practice” methodologies.

This has been an excerpt from my upcoming book Finding Your Way: Navigating the methodology maze, a roadmap to successfully adopting process in your organization.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s