Making Scrum work: Common failings in adopting Scrum

Scrum has been the happy recipient of a great deal of hype in the past few years. In fact, it’s been hyped so much that it’s becoming the “next best thing,” a phenomenon that can sometimes end up being detrimental to the general market perception of industry focus: As more and more people jump on the bandwagon “Scrum” becomes the hot topic of the day. The problem is, many of the people using Scrum have only a vague understanding of its principles and place in the software development life cycle.

Scrum can be remarkably beneficial in many kinds of software projects. But, as with any process, methodology or management technique, when used inappropriately it can cause more problems that it solves.

In this article I’ll discuss some of the common misconceptions and “lessons learned” as related to Scrum.

You don’t need training to use Scrum

This is the most important one in my mind: Training is just as necessary with Scrum as it is with any other process. I’ve run into many teams that eschew formal training in favor of simply “reading the book” or sending one lead engineer to a quick Scrum certification program.

Let’s look at it another way: For upwards of 30 years, the software development industry has followed a largely industrial process (sometimes called “waterfall” or “spiral,” depending on your particular experience). Across that time, many professionals have struggled with the complexity of software development, striving to create a process that is simple, reliable and repeatable. More often than not, to varying degrees, the industry has met with failure: Nobody has turned software development into a wholly reliable, repeatable procedure that always hits its goals dead-on.

Scrum is a process that has evolved out of this 30 year history. While the process itself focuses on simplicity and emphasizes a set of easy to follow practices, this does not make it inherently intuitive or trivial. In some regards Scrum is so contrary to established process that many team members just “won’t get it” unless they experience it in a learning-enabled setting. Training is critical, and I highly advocate sending your entire team to a Scrum workshop — or, better yet, bring the workshop to your team. Good programs exist that will turn your training session into a guided working session, which means your productivity will not be hurt by sending people off-site for training. Just the contrary — your productivity will leap ahead in just a week or two as training, learning absorption, experience and actual hands-on work all combine to create a Ken Schwaber’s original goal for Scrum: A hyperproductive team.

Scrum is all we need

Often sung to a Beatles inspired tune, I’m afraid it’s not true: Scrum is not all you need, and any idyllic fantasies otherwise will be met with dashed and broken hopes on the rocks of failure. Scrum is not a complete methodology, it’s a management control process — and that implies that you still need methodology. Don’t take my word for it, take Ken Schwaber’s, the creator of Scrum: “Scrum is superimposed on top of and wraps existing engineering practices, development methodologies and standards.” (Schwaber & Beedle, 2002; Agile Development with Scrum, p. 2).

Scrum is an excellent control process that helps us cut through the red tape of more traditional, bulkier control processes. It helps us to focus in on what’s important. It enables accountability to an extreme degree and, through accountability, achieves greater productivity. Its empirically driven measurement technique is well suited to the often experimental development environment of new products, such as inventing new software.

Nowhere in the Scrum process itself do we discuss development methodology. This means we must bring our own methodology to the table and wrap Scrum around it, enhancing processes that have not been hyperproductive and increasing their productivity. In other words: Software quality assurance, formal testing, configuration management, risk analysis, budget planning and project scope planning, verification, requirements management and specific methodologies such as the Rational Unified Process, Extreme Programming or Spiral programming must also be part of the equation.

The backlog replaces the Project Design Document

Here I’m essentially belaboring the point that Scrum is not all you need. Different development methodologies employ different kinds of design strategies and documentation strategies, but one thing is common across almost every single methodology: There is always a Project Design Document (PDD) in one form or another. The project backlog is not a complete design document. A well-implemented project backlog is going to provide enormous technical detail and will probably fulfill all the requirements for a technical specification — but this is only part of the PDD. Team collaboration is not a substitute for someone putting the product design specification in writing.

The Project Design Document addresses a host of project objectives and requirements that simply won’t show up in the project backlog, at least not in an easily interpreted, cohesive fashion. Some of these include: Overall project scope; market objectives; user interface models; business models; user acceptance criteria; original customer requirement statements or vision statements; a project baseline; key risks and mitigation strategies; user interface style guides; software architecture and software integration procedures; and the list goes on. Many of these artifacts will, in time, turn into very specific project backlog items, but many will not. It’s important to document not only the specific, day-to-day deliverables of the project, but also the overall goals, architecture, and design guidelines.

One of my favorite product suites for managing both the product backlog and project documentation is Atlassian’s JIRA and Confluence. Combined, the tools provide fantastic support for managing tasks and deliverables (the project backlog) and a collaborative documentation environment. The ability to seamlessly create links between the JIRA tickets and more in-depth, collaborative Confluence documentation is an excellent tool. It allows the team to expand on technical detail and cross reference extended documentation, all within the context of the product backlog.

The 15 minute daily scrum is unnecessary

I’ve run into a lot of resistance to the daily scrum, especially very early in the process of introducing Scrum to a new team. Nevertheless, within a matter of a month or two, the value of the daily scrum has been realized and wholly adopted by all of the teams I’ve led.

One of the worst situations you can get into is a “fuzzy scrum,” one that doesn’t start on time or that exceeds the boundaries of the meeting on a regular basis. If the Scrum Master shows up five minutes late, then politely waits ten minutes for everyone to gather, a focused 15 minute briefing turns into a 30 or 40 minute distraction that everyone resents. Stay focused on the daily scrum rules of conduct — specifically, start on-time every day, in the same meeting place. The Scrum Master must arrive early and be ready to kick off precisely on time. If someone is late, don’t hold up the meeting. Use a timer to casually ensure that everyone gets a turn, and nobody goes significantly over their time limit.

A well-structured 15 minute daily scrum will benefit the team enormously. It keeps the entire team apprised of progress and current goals. It helps to focus the team, avoiding situations where one person gets derailed, instead giving them support from the team. Remote workers and partner-vendors will stay in-touch with current goals.

I can’t count the number of times something has popped up in a daily scrum that resulted in days, potentially even weeks of gained productivity. I’ve seen team leaders that sit next to their developers react with surprise when they hear what an individual is actually working on. In situations like this, enormous effort has been saved when the team leader corrects the objective right then and there.

I’m not allowed to work outside of the sprint goals

Another very frequent misconception is that Scrum introduces artificial roadblocks; for example, forcing the team to work on current sprint goals and nothing else. This is only partially true: It is correct that Scrum focuses the team on the current sprint. The objective is to complete the sprint and then move ahead to the next set of objectives. This means that an individual team member must finish their sprint goals and, ideally, try to help out other team members with other goals, if possible.

However, once the team member is done with their sprint goals they are free to “look ahead.” There is nothing that says a team member can’t start working on the next sprint ahead of schedule or — if the next sprint doesn’t have enough definition yet — even draw directly from the product backlog.

Scrum is about productivity. It will never require the team to stop work because of an arbitrary set of procedures or policies. Quite the opposite: Well implemented, Scrum enables the team to become far more productive than previously possible.

We don’t have to adopt all of Scrum

This issue has come up with every single client and, I fully expect, will continue to do so. Technically, it’s true — but not initially.

As with any process or methodology, it’s critical to understand how the process works properly before customizing it. Scrum should be followed entirely and rigorously for at least the first month or two of a project. Ken Schwaber’s recommendation is to do so until “several sprints” have been successfully delivered and the goals of achieving a hyperproductive team are met. As Clara Ko writes in Jeff Sutherland Talks Shock Therapy For Scrum, “Perhaps there are good reasons for excluding some parts of Scrum, but without truly understanding the reason behind a Scrum practice or the implications of skipping it, agile teams struggle to become hyperproductive, or mistakenly, struggle with Scrum itself.”

In other words, before changing Scrum, become an expert in it. Once the Scrum process has become second-nature it becomes much easier to begin customizing it or cutting pieces of it out. It will also become clear why something stopped working smoothly. Past experience will give the team the awareness to know the process broke because it was changed, and the motivation to move back to a working process.

With proper training and guidance, teams uniformly experience incredible productivity gains with the Scrum process. They key is adopting Scrum correctly, and that means doing it with the right level of training and commitment, especially early on. Process change is never easy, and while Scrum may appear to be a simple process it is not an intuitive one.

So you think you’re following Scrum?

I have a prediction. If you take the Nokia “Scrum Test” you are going to score somewhere less than 7. That means you aren’t doing Scrum, you’re doing “ScrumButt:”

A ScrumButt is a sort of like Scrum implementation… but some changes that were too painful have been left out… Companies in this category tend to only experience moderate success with Scrum, i.e revenues up 0-35%. This is very different from the design goals Jeff Sutherland had for Scrum, i.e. to create hyper productive teams and hyper-profitable companies.

In 2005 Bas Vodde, agile pioneer and coach within Nokia Networks, created the original test. In 2008, Scrum co-creator Jeff Sutherland extended the test by introducing new questions and a scoring model. Today, Jeff works with openview venture partners to help investors make money by only investing in companies with state of the art software development capabilities.

The test itself is incredibly informative — and the results usually much more so. Merely answering the questions posed in the Nokia Test will reveal some obvious shortfalls in most Scrum implementations. But take care to answer honestly and objectively. It’s not a bad idea to make the exercise a team activity, reaching serious consensus on the answer to each question.

The Weekly Review as an OmniFocus Project

Automation can be a wonderful thing when used right. With information overload at an all-time high (see Is Google Making Us Stupid) it’s a challenge to reap the benefits while avoiding the pitfalls. Systems such as OmniFocus and other GTD-oriented solutions can be hugely successful when adopted — and consistently used. Fraser Speirs has documented a handy OmniFocus tip to help “guide us” through the mundane wool-gathering tasks that we hate to bother over.

Why projects fail 101

90% of projects do not meet time/cost/quality targets. Only 9% of large, 16% of medium and 28% of small company projects were completed on time, within budget and delivered measurable business and stakeholder benefits. [Standish Group Chaos Report, 1995] There are many reasons for such failures. As per a KPMG survey of 252 organizations, technology is not the most critical factor. Inadequate project management implementation constitutes 32% of project failures, lack of communication constitutes 20% and unfamiliarity with scope and complexity constitutes 17%. Accordingly 69% of project failures are due to lack and/or improper implementation of project management methodologies. [citation]