You’re Testing It Wrong…

I really like test driven development. It lets me be lazy. I don’t have to worry about my software quality, or that something I did broke some other thing. And with good dependency injection to make sure every component is working right, “it just works.” Now I code using TDD (writing my tests first, then coding to fulfill them), and I focus our QA efforts on making sure we have great test plans, and great coverage.

Closed System
A closed system wants to be tested.

So, when one of my project teams kept telling me they couldn’t write tests because the database wasn’t ready I got worried. Our team had been immersed in TDD for months — and every single engineer had nodded vigorously when I set expectations. The team leader recited the definition of “dependency injection,” just to drive home how ready they were to embrace it!

But when I asked to see the what was wrong, I knew we had a problem. The team’s tests were not injecting mock objects the right way. The idea behind dependency injection is to replace the smallest component possible in a closed system with another object, a “mock.” That mock can then monitor the system around it, inject different behaviors, and create desired results.

For example, let’s say we have a program that connects to a gizmo — your home thermostat. The thermostat itself is a separate component that lives outside your program. We can expect the thermostat to behave like a thermostat should… reporting the current temperature, and letting the home owner enter a desired room temperature. Pretty straight forward.

So the first step is to write a program that talks to the thermostat. We can wire up a real thermostat, but we’ve got a problem right off the bat. We want to know how our program behaves as the ambient temperature changes — 65 degrees, 32 degrees, or 100 degrees. But a real thermostat is only going to report the actual room temperature, and making the room frigid or boiling just isn’t going to be very comfortable or practical.

Not Mocking
Faking is not mocking.

This is where dependency injection comes in — wouldn’t it be great if we could inject a new gizmo, one that behaves according to our test plan?

It turns out that my team had been taking the wrong approach — but one that is pretty easy to make if you’re new to the idea of mocking and dependency injection. Unfortunately, it meant that they weren’t really testing the application. The were testing something else entirely.

Once we walked through the system, the mistake was clear. During application start up, it created a connection to a database. My team’s approach had been to add a “mocking” variable to the application. In effect, it created a test condition; if the application was in “mocking mode” it would only simulate database calls. If the application was not in “mocking mode” it sent real requests to a real database. Sounds great, right?

But it’s all wrong. Here’s the problem: The application was faking real world behavior. That is, throughout the program there were dozens of little tests, in effect asking, “if we are mocking, then don’t check the database for a user record, instead just return this fake user record.”

This meant that the real program — the actual application logic that would be deployed to the real world — was never tested. Instead, an alternate branch of logic was tested — a fake program, if you will. So two things happened:

  1. We weren’t testing the real program, we were testing something else altogether.
  2. The program itself became terribly complicated because of all the checks to find out “are we mocking?” and the subsequent code to do something else entirely.

And all of that is why my team said they couldn’t really test the system, because the database wasn’t up and running.

So what does real dependency injection look like? It’s simple: You want to change the actual gizmo, but change it in the most subtle way possible — and then you want to put that actual gizmo right back into your program.

Mocking
Real mocking doesn’t affect the original program flow.

Getting back to the thermostat example, an ideal solution would be to modify a real thermostat. You could crack it open, remove the temperature sensor, and add a little dial to it that lets you change the reported temperature. Then you plug the “mock thermostat” into your program, and you change the temperature manually! A potentially better approach would be to change the software that talks to your thermostat, and instrument it so that you can override the actual reported temperature. Your program would still think it’s talking to a real thermostat, and the connecting software could change the actual temperature before handing it off to your program.

In our case, the right solution could be injecting a simple mock component at the just the right point in our program.

For example, lets say our application uses an Authenticator object to log in users. The Authenticator checks the validity of a user in the database, and then returns a properly constructed User object. We can use dependency injection to substitute our own test data by overriding the single function we care about:

object fakeAuthenticator extends Authenticator {
    override def getUser(id: Int): Option[User] = {
        Some(User(id: -1, name: "Fake User"))
    }
}

On line 2, we replace the real Authenticator’s getUser function. The overridden method returns a hard-wired User object (in this case, one that clearly doesn’t represent a valid user account). By overriding the Authenticator in the test package only, the original program is not altered — all that’s left is to inject our altered Authenticator into the program.

The old fashioned way of doing injection is still reliable: Don’t tell, ask. Use a factory object to ask for the Authenticator. Given a factory in the application (let’s call it the AuthenticatorFactory) we can override what the factory actually returns in our test case only:

AuthenticatorFactory.setAuthenticatorInstance(fakeAuthenticator)

A slightly more modern approach is to use a dependency injection framework, but the underlying principle is exactly the same.

Likewise we can take the concept of mock objects further by using frameworks such as Mockito (a framework that works wonderfully with specs2). Mockito makes it easy to instrument real objects with test driven behavior. For example, Mockito will produce a mock object that acts just like a real object, but fulfills expectations (such as testing to make sure that a specific function is called a certain number of times).

Whatever tools and frameworks you use, test driven development has proven itself over the past decade. My own experience is the same: Every TDD project has produced more predictable results, has better velocity, and has been more reliable overall. It’s why I don’t do any coding without following TDD.

How Leadership Differs Around The World

British linguist Richard D. Lewis has explored in depth how leadership style differs across cultures and countries. His diagrams of Leadership Styles, published in When Cultures Collide, offer wonderful insight into why so many multinational efforts run into problems. Anyone doing business across borders needs to understand these differences and adapt their own style accordingly.

Different Culture, Different Leadership

Leadership Styles
Leadership Styles

The variation of styles — from structured individualism of America, to consensus rule of Asia — fit  preconceptions about foreign culture. Even so, it’s important to understand the meaning behind each model, and also be aware of individual variation. The models are not unilaterally true across a country. Every individual will have their own blended style of leadership.

See Lewis’ Leadership Styles diagrams, inset, but also be aware that stereotyping is risky, as Lewis himself warns: “Determining national characteristics is treading a minefield of inaccurate assessment and surprising exception. There is, however, such a thing as a national norm.”

Lewis also argues that these cultural characteristics won’t change anytime soon. He writes, “Deeply rooted attitudes and beliefs will resist a sudden transformation of values when pressured by reformists, governments or multinational conglomerates.” While the “Westernization” of many Eastern countries gets a lot of press, most of these changes are superficial. Cultural preferences are deeply rooted. We learn about our culture from birth. Especially in countries with thousands-years-old history and culture, changes are slow to emerge. Stated more directly: Individuals may jump at the chance to adopt foreign practices, such as capital investment, but this doesn’t mean they are also adopting Western culture.

Management gurus have time and again tried to quantify and distill the secret of successful management into an easily followed formula. Peter Drucker, James Champy, Frederick Taylor, Henri Foyal, Frederick Brooks, and Mike Hammer have all put down their thoughts on the topic. But each has placed a Western emphasis on their particular management magic (and, except for Foyal, a very American emphasis).

As I’ve pointed out many times, cultural conflict is common across multinational organizations. Learning how to avoid the conflict — misunderstandings, misinterpretation, and direct cultural incompatibility — is the first step.

Multinational Leadership Success

There is no single management tool that can work in the global landscape. The cultural intricacies that define how people interact, both in a business setting and a social setting, run far too deep. And, just as management styles depend on environment, so do our relationship-building tools. Creating a successful International business relationship depends so strongly on cross-cultural awareness, in fact, that without extensive exposure to foreign culture most efforts are rife with failure.

Check out this short six-part series that talks about how business cultural preferences affect 27 project management disciplines.

Power Distance And International Success

If you missed the first part of this six-part series, see: Part 1 of the series, Creating An International Culture Of Success, or see the entire series right here.

I’ve posted in depth on power distance and how it varies from one culture to another. To recap, power distance, or “PDI,” is the degree of inequality in society and the emotional distance that separates subordinates from superiors.

Many Western cultures thrive on very low power distance principles. Since most of today’s modern management theory has come directly from the West, this means these theories work great in Western cultures but tend to have problems in the East.

Most modern management expects employees to think independently, be honest and critical, question the status quo, and openly voice disagreement. Many recent management tools, such as Scrum and Agile methods, empower the employee so much that the line between “boss” and “employee” becomes blurred and — sometimes — almost eradicated.

Across much of the Middle East and Asia, this approach fails miserably. Traditional organizational structures don’t tolerate this approach. Direct criticism and questioning tends to be viewed as dissent. Respect for seniority, wisdom, and age play into it. Decision making happens at higher levels in the company, and decisions flow downward. Employees are expected to act in unison, provide information when requested, and respond like a well-oiled machine to the strategic decisions of their senior management.

The Global Project Compass™ (introduced previously) covers 27 project management disciplines. It identifies the following management disciplines as being most directly affected by power distance:

  1. Goal Setting
  2. Organizational Structure & Policy Setting
  3. Standard Compliance
  4. Business Case Validation
  5. Positive Assurance of Compliance

Goal Setting

As pointed out in the introduction, different cultures have different expectations about where their goals come from. Employees that are used to low power distance will feel slighted if they are not closely involved in setting goals, or if their voice is not heard. On the other end of the spectrum, those accustomed to being told what to do may conclude that their boss doesn’t really know what’s going on if too many questions are asked or if the boss seems to rely on subordinate opinions.

Organizational Structure & Policy Setting

Closely related to goal setting is policy setting, and that includes the hierarchy (or lack of hierarchy) in an organization itself. Employees from high power distance cultures tend to feel far more comfortable in an environment that provides clearly defined roles. That translates into greater hierarchy, and more clearly responsibilities. As Honeywell learned, without adequate training and management programs, their Chinese R&D department really had no idea how to go about inventing truly new products.

Standards Compliance

Compliance is an interesting topic to explore, because it shows off a reversal of competence along the power distance spectrum. Employees accustomed to high power distance and being given clear guidelines tend to flourish when it comes to compliance. Such standards provide a clear set of instructions, set boundaries, and make the job an easy one to follow (at least, when the standards are well documented).

Unfortunately, with poorly defined or conflicting standards, problems occur. Poorly written rote instructions rapidly lead to chaos when those instructions are in conflict — and high power distance cultures also tend to demonstrate little critical thinking or problem solving here.

On the other hand, with a team that is used to low power distance, standards can become a “thorn in the side.” These teams — trained to think critically and voice their opinions — often struggle to see the rationale or validity of standards. They might push back against them, although when the standards themselves are questionable this can be a boon: Those same teams will quickly point out flaws (and perhaps push to have the standards disqualified).

Business Case Validation

Critical thinking, scenario planning, and a talent for seeing the future are traits needed when validating a business case. These skills tend to flourish at the executive level in high power distance cultures, while the critical thinking of low power distance teams can be incredibly valuable to examine every aspect of a business model.

Positive Assurance of Compliance

Making sure that you are complying with standards is often the responsibility of the quality assurance department or a separate standards body. Power distance and organizational structure play a huge role. Assurance of compliance carries with it a need for authority. Failure in compliance means, potentially, putting a stop to project activities, and challenging the organization and the team (at least, in so far as ensuring products conform to agreed standards). Organizational structure is important, but often the standards compliance body is not set up with adequate authority — in some cases, being subordinate to conflicting objectives (such as project management). Ensuring that the right structure exists; that there is a separation of concerns; and there is authority to act, is critical, and very dependent on the cultural biases at play.

Cover graphic attribution: The artist and visual designer Yang Liu was born in China and lives in Germany since she was 14. By growing up in two very different places with very different traditions she was able to experience the differences between the two cultures first-hand.

Dealing With Timezone Differences

Working in the global economy means spending lots of time connecting with clients and colleagues on the other side of the world. But multinational teams also face “multi-timezone” management problems. What seems like an obvious, potential problem can cause management nightmares for multinational leaders. Here are few tips on how to deal with time zone differences and build smoothly functioning, multinational teams.

Hi, I am Zacharias Beckman, president of Hyrax International, and today I want to talk about dealing with timezone differences. In my work, I’m frequently fixing problems with projects that have gone off the rails. That often means a lot of travel — going to international partners, finding out what’s wrong and fixing it. And when I’m traveling, then — that means being able to collaborate with my team, back here in the United States, is also a problem.

Timezone Challenges For Teams

Focus on finding a method for seamless communication, throughout your entire company, worldwide. You want your teams to break down barriers. You don’t want a team here to be thinking “Oh, I just cant call the other team because they are in different time zone, they’re half way around the world and I can’t bother them.” You do want them to pick up the phone and call or use Skype or whatever it is. The teams need to get to know each other. One way to do that is through co-location. Bring the foreign team home for a while. Or, send some of your team members there, so that you can build a tighter relationship.

But when co-location isn’t an option, you can turn to frequent short meetings — by phone, by Skype, it doesn’t matter. It’s the frequent contact that helps. It breaks down barriers so that the teams starts to operate as a single team, not as a bunch of different team separated by distance and culture. You don’t want your teams to feel distant, because then they are going to act distant.

The other thing you want to do is work on implementing collaboration tools that work really well with remotely located teams. So, project management systems that are easy to access, information radiators and easy to use communication tools. Plan your work days to overlap a little bit. It wont do to have your team in India working from 10 to 6pm and your team in the United States working from 9 to 5 because there is no overlap, there’s no communication. Instead adjust schedules a little bit on each side and try to have about an hour or so of overlap, so that your team can then have a daily or semi-daily stand up meeting. The idea is just to get everyone on the phone and in the virtual room together, so that they can find out what happened on the other side of the world and the they can ask the other team for what they need in order to move forward. The frequent contact and the direct connection is going to go a long way towards breaking down barriers between the team and making them more efficient.

But the meetings are short. They are just to touch base. They’re there for one team to let the other one know what happened and what they need so that they can move on and make progress, the next day.

Improving Multinational Team Collaboration

Power Distance in the Global Context: How business cultural preferences can directly impact your team’s ability to work together and communicate effectively.

When we talk about our own culture, it won’t seem like culture at all. It’s just the way things are. For example, if you’re American you believe everyone has a right to free speech and you think any thing else is a human rights violation. You also enjoy about two or three weeks of vacation each year. And you’re very prompt. You feel like your entire day is interrupted if someone’s late.

If you are Turkish, you know about free speech, but you probably don’t dare talk about it much. You are blissfully unaware that you have more official and not so official vacations than anyone else in Europe. And for you, time is fluid. Being half an hour late is fine, you don’t expect anybody else to be on time, so you plan on doing several things at once. These are examples of culture, and businesses have culture too.

Phil, a CEO at a US company, came to me telling me that they had outsourced all of their product development to India. But, they had a problem. They hadn’t met a single deadline in over a year. Product development was stalled. When I asked Phil what was wrong, he said the team in India was terrible at communication. They never raised a red flag if something didn’t make sense.

The problem is Phil was completely wrong. He was ascribing Western expectations to an Eastern team. The real problem was more complex, but the root of it was power distance. Power distance is the degree to which a boss and an employee are separated by society. And in the East power distance is very important. Employees can’t simply tell their boss that they think he is wrong. But Phil’s employees are rewarded for questioning and challenging and thinking outside of the box. He had no idea how unfamiliar this was for his India team.

We worked with Phil to develop a cross cultural training program. A year later, Phil’s company is doing great. Their new product just shipped and Phil told me that the team has hit every milestone perfectly. This is just a small part of how business culture can affect a global business. It’s why we created the business synergy compass; to guide businesses to success in the new global economy.

Why we can’t just burndown everything

Prashad is the project manager for a new product. After reviewing the project scope, Prashad’s team gives it an initial estimate of about three to four months to complete, which is good enough for management. The project kicks off, and using Scrum along with a little bit of project management, the project gets underway. The first month goes quickly, there is great progress, and everyone loves what they see. But in the second month, things start to slow down — so, Prashad adds more resources to the project to compensate. But by the third month, there are real problems and it looks like the project will be late. Prashad adds even more resources. Now the team is almost double the original size. He adds a lot more quality assurance and testing support for the team, too, because a lot of problems are cropping up. The team cranks away, and they finish the project in the fifth month, and claim “success” because they almost hit the deadline (it was, after all, just an “estimate“).

But what about all those additional resources? Is it really success if the project cost twice as much as originally estimated? The problem is, that cost was never formally stated. It was never tracked as a metric, so the fact that the project went horribly over budget is quietly swept under the rug — at least this time. There are plenty of cases where the budget isn’t there, or the resources aren’t there, or the product just can’t ship late.

So, if so many projects are missing their mark, what’s causing the problem?

The fallacy of so called Agile methods

According to studies conducted by KPMG, as much as 70% of projects fail to meet their goals. In this case, “goals” mean quality, schedule, and function (or a combination of those). Clearly, being able to accurately estimate a project’s delivery date is important. Without knowing when a project is done, you can’t predict cost, plan business operations, or dovetail releases with marketing, training, and customer service.

Agile methods like Extreme Programming and Scrum make promises that, in my opinion, they can’t keep. For well over a decade we’ve seen an increasing trend in project failures, and a large number of those failures are the result of “unending projects.” Projects that go on and on, continuing to deliver improvement but slowly creeping over budget and never reaching an end state. Part of the problem is the lack of formal estimation and metrics.

Both Scrum and Extreme Programming dodge the entire issue of project life cycle estimation. They focus on the near term, providing estimates for the next one or two sprints. This works great if you don’t have budget or time constraints, but in the real world that’s rarely the case.

Burndown just doesn’t work

Most Agile methods don’t explicitly define how you estimate progress, but the most common method is burndown — the measurement of completed effort against the planned sprint goals. This is fine for a quick-and-dirty project but as a strategy it utterly fails to pinpoint problems with budget and timeline. For anything larger than a few months effort, it doesn’t do the job.

And the problem is, few of today’s engineers seem to be getting the formal education and training needed to use proper estimation methods.

There are better solutions than burndown

Earned Value Management, or EVM, is nearing it’s 40th year of practice. Throughout this 40 year history, EVM has repeatedly evolved to meet new demands as technology and innovation made new leaps forward. It’s a proven method for measuring progress and it’s been proven time and again.

Of the dozen or so software engineers I spoke with about this article, none of them had even heard of EVM.

Earned Value Management, as well as a variation known as Performance Based Earned Value® (PBEV), are my favorite choices for estimating a project. The PBEV approach provides an incredibly robust method for measuring progress and staying on top of your project. It’s only marginally more difficult than other techniques of measurement. And most project managers are already capturing the metrics needed to make it work.

PBEV is an improvement over Earned Value Management Systems’ (EVMS) national standard. It supplements EVM with guidelines for the integration of project cost, schedule, and technical performance — thus created a comprehensive and highly accurate method for measuring true progress in a project. The key here is integration of effort with schedule and performance.

Had Prashad been using PBEV, he would have had a very different experience. The early estimates would have been more accurate, since PBEV estimates the entire project. It would have measured not only resources and cost, but also the actual performance — the rate at which work is being done. Had the project gone off track, Prashad would have known much more quickly, and would have had a more precise picture of how to correct the project’s course.

There will definitely be situations in which PBEV is not warranted, and a skilled project manager will need to know when to use other methods, and why. EVMS, for instance, addresses only the quantity of work completed, ignoring cost and schedule metrics. The implication here is that a quick burndown or EVMS-based evaluation of a project might look A-OK, when in fact the only reason it’s on track is because resources are being thrown at it — and costs are skyrocketing.

Burndown versus critical path thinking

Burndown is a very simple method for estimating work effort over a short term. It explicitly avoids timeline, cost and performance metrics. That’s fine for projects that are not constrained by time, budget, or resources. The problem is, burndown is used as a defacto standard too often, on the promise that using an Agile method will deliver a project faster or cheaper than its alternatives. I don’t disagree on this front. Agile methods generally are faster and cheaper — but the question is, faster and cheaper than what?

Most business need to be concerned with time, money, and staffing. Projects that have those constraints need to think about the critical path — the sum of all activities to reach the end of the project. Having a solid understanding of the fundamentals of critical path management is important to managing any project schedule. Depending on the nature of your project, you may be able to rely on casual effort estimates — or, you may be required to carefully identify, analyze, and attack critical paths to shorten your project lifecycle and stay within stringent guidelines. Without understanding the principles, and knowing where to turn for more information, neither is possible.

I’ll revisit the topic of PBEV in a future article and provide more depth into how it can be applied to a project. If you’re eager for more information, check out Performance-Based Earned Value®, Paul J. Solomon, Ralph R. Young, IEEE Computer Society, Wiley Interscience, 2007.

What you won’t get out of your certification

When it comes to project management certification, there’s no doubt quite a few options available. The real question is, do you know what you’re getting with a shiny new certification (such as PMI’s PMP, or IPMA’s Level A through D)?

Certification programs are more about demonstrating your competency than about learning how to manage. Consider this: PMP and IPMA certification takes you through a process guide and an examination that you can easily enough prepare for in a few weeks. The process guide itself is a valuable reference, a great way to organize all the possible areas of knowledge in a project — but it’s just that. A process guide is largely a checklist, giving you a tool to make sure all the right pieces of a project are in motion.

The other part of the certification process is the exercise of documenting your experience, as a project manager, and having the certifying organization vet your experience (although the vetting process is often cursory). It’s supposed to show the world that you have a certain level of project management competence. It is explicitly not going to teach you that competency. The real theory behind PMP certification is that by achieving it, you have demonstrated relevant competency as a project management professional. There’s a good bit of debate regarding the value of this certification: How well does the PMI do in vetting experience? Is there any qualitative distinction in the evaluation? I’ve seen a few terrible managers get their PMP by documenting their management of projects that were miserable failures.

For the most part, deciding whether a candidate’s PMP or IPMA certification (or lack thereof) is valuable lies with the employer. For the individual, I always recommend learning the bodies of knowledge, so long as you are fully aware of what you’re getting. Here are a few of the things that either program won’t prepare you for:

  1. Project management is more about management, and less about process. Most of the certification programs out there tend to emphasize the latter, the process, and spend little time on the “soft and fuzzy” bits: People. The fact is, this is where you’ll succeed or fail. It often comes down to how good you are at picking up on subtle (and not so subtle) cues between team members, sponsors, and stakeholders. There is no certification program in the world that will teach you how to be a great manager — that requires experience, more than anything else (but if you have a degree from a top management school it’s bound to help).
  2. The rosy project you just inherited is actually completely out of touch with reality. More often than not businesses will commit to a project that can’t be met (often establishing budgets and schedules in the process). Studies by Standish and KPMG bear this out, pointing at 70% of projects failing to meet cost, schedule, and quality goals.
  3. Identifying all the right pieces of a project is just the beginning. For example, knowing who the stakeholders are (an important step in the PMBOK) is the easy part. You’ll spend far more time motivating them, coordinating their schedules, dealing with stakeholders that want your project to fail (or just don’t care about it), responding to impossible demands, and figuring out what everyone’s secret agenda is all about.
  4. You’ll need to be good at managing other people’s anger and frustration. Part of being a strong manager is knowing how to evaluate the project dynamics, and then make the right decision. Most of the time, you won’t find a rosy world where everyone agrees about what has to be done. You’ll be stepping inside someone else’s world, and messing with it. You’ll have to tell people to do things they don’t agree with, or don’t want to do. You’ll have to step in and change everything, and people don’t like change.
  5. Your certification didn’t warn you about some of the important bits. There is no such thing as a comprehensive process guide or methodology that gives you all the answers. Some guides are really good in some areas, and horrible in others. The PMBOK, for example, spends precious little time talking about quality assurance and risk management — so little, in fact, that without turning elsewhere you won’t have any idea what these two things are, how important they are, or what to do about them.
  6. You are actually not ready to manage a large scale project simply because you’ve earned your PMP/IPMA/PRINCE2 certification. These certification programs document your past experience, and your basic knowledge of the relevant process guides. Unfortunately, splashy advertising sells certification, so I’m sure we’ll keep seeing claims such as “everything you need to manage complex projects!” It’s a lie.

Deciding whether obtaining a certification is worthwhile or not is a personal decision. I definitely recommend learning the reference material — and after all, if you learn the PMBOK, it’s not much more work to get a PMP certification. Just be honest with yourself about what it means. Being a good project manager means having the experience to guide an organization toward success, not just recite the process guide. I always recommend starting small and finding out what your personal aptitude for management is.

Your key to success as a project manager is going to hinge on your ability to listen to others, learn from others, and always be open and ready to support your team. You’ll need to turn to other people and other sources of information. Be humble, never let obstacles derail you, and make sure your team knows they can rely on you for support. These are the things you don’t find in the process guides.