You’re Testing It Wrong…

I really like test driven development. It lets me be lazy. I don’t have to worry about my software quality, or that something I did broke some other thing. And with good dependency injection to make sure every component is working right, “it just works.” Now I code using TDD (writing my tests first, then coding to fulfill them), and I focus our QA efforts on making sure we have great test plans, and great coverage.

Closed System
A closed system wants to be tested.

So, when one of my project teams kept telling me they couldn’t write tests because the database wasn’t ready I got worried. Our team had been immersed in TDD for months — and every single engineer had nodded vigorously when I set expectations. The team leader recited the definition of “dependency injection,” just to drive home how ready they were to embrace it!

But when I asked to see the what was wrong, I knew we had a problem. The team’s tests were not injecting mock objects the right way. The idea behind dependency injection is to replace the smallest component possible in a closed system with another object, a “mock.” That mock can then monitor the system around it, inject different behaviors, and create desired results.

For example, let’s say we have a program that connects to a gizmo — your home thermostat. The thermostat itself is a separate component that lives outside your program. We can expect the thermostat to behave like a thermostat should… reporting the current temperature, and letting the home owner enter a desired room temperature. Pretty straight forward.

So the first step is to write a program that talks to the thermostat. We can wire up a real thermostat, but we’ve got a problem right off the bat. We want to know how our program behaves as the ambient temperature changes — 65 degrees, 32 degrees, or 100 degrees. But a real thermostat is only going to report the actual room temperature, and making the room frigid or boiling just isn’t going to be very comfortable or practical.

Not Mocking
Faking is not mocking.

This is where dependency injection comes in — wouldn’t it be great if we could inject a new gizmo, one that behaves according to our test plan?

It turns out that my team had been taking the wrong approach — but one that is pretty easy to make if you’re new to the idea of mocking and dependency injection. Unfortunately, it meant that they weren’t really testing the application. The were testing something else entirely.

Once we walked through the system, the mistake was clear. During application start up, it created a connection to a database. My team’s approach had been to add a “mocking” variable to the application. In effect, it created a test condition; if the application was in “mocking mode” it would only simulate database calls. If the application was not in “mocking mode” it sent real requests to a real database. Sounds great, right?

But it’s all wrong. Here’s the problem: The application was faking real world behavior. That is, throughout the program there were dozens of little tests, in effect asking, “if we are mocking, then don’t check the database for a user record, instead just return this fake user record.”

This meant that the real program — the actual application logic that would be deployed to the real world — was never tested. Instead, an alternate branch of logic was tested — a fake program, if you will. So two things happened:

  1. We weren’t testing the real program, we were testing something else altogether.
  2. The program itself became terribly complicated because of all the checks to find out “are we mocking?” and the subsequent code to do something else entirely.

And all of that is why my team said they couldn’t really test the system, because the database wasn’t up and running.

So what does real dependency injection look like? It’s simple: You want to change the actual gizmo, but change it in the most subtle way possible — and then you want to put that actual gizmo right back into your program.

Mocking
Real mocking doesn’t affect the original program flow.

Getting back to the thermostat example, an ideal solution would be to modify a real thermostat. You could crack it open, remove the temperature sensor, and add a little dial to it that lets you change the reported temperature. Then you plug the “mock thermostat” into your program, and you change the temperature manually! A potentially better approach would be to change the software that talks to your thermostat, and instrument it so that you can override the actual reported temperature. Your program would still think it’s talking to a real thermostat, and the connecting software could change the actual temperature before handing it off to your program.

In our case, the right solution could be injecting a simple mock component at the just the right point in our program.

For example, lets say our application uses an Authenticator object to log in users. The Authenticator checks the validity of a user in the database, and then returns a properly constructed User object. We can use dependency injection to substitute our own test data by overriding the single function we care about:

object fakeAuthenticator extends Authenticator {
    override def getUser(id: Int): Option[User] = {
        Some(User(id: -1, name: "Fake User"))
    }
}

On line 2, we replace the real Authenticator’s getUser function. The overridden method returns a hard-wired User object (in this case, one that clearly doesn’t represent a valid user account). By overriding the Authenticator in the test package only, the original program is not altered — all that’s left is to inject our altered Authenticator into the program.

The old fashioned way of doing injection is still reliable: Don’t tell, ask. Use a factory object to ask for the Authenticator. Given a factory in the application (let’s call it the AuthenticatorFactory) we can override what the factory actually returns in our test case only:

AuthenticatorFactory.setAuthenticatorInstance(fakeAuthenticator)

A slightly more modern approach is to use a dependency injection framework, but the underlying principle is exactly the same.

Likewise we can take the concept of mock objects further by using frameworks such as Mockito (a framework that works wonderfully with specs2). Mockito makes it easy to instrument real objects with test driven behavior. For example, Mockito will produce a mock object that acts just like a real object, but fulfills expectations (such as testing to make sure that a specific function is called a certain number of times).

Whatever tools and frameworks you use, test driven development has proven itself over the past decade. My own experience is the same: Every TDD project has produced more predictable results, has better velocity, and has been more reliable overall. It’s why I don’t do any coding without following TDD.

Don’t Use Basecamp

Hi, I’m Zacharias Beckman, President of Hyrax International. Managing a multinational team can be really challenging. You have to make sure that each team, around the globe, is coordinating their efforts. That means overcoming the communication problems between them, tracking task dependency from one team to the other, and staying on top of projects — so that one team doesn’t get held up waiting for another one. So, the real challenge is figuring out to stay on top of each team and how do you make sure that all of their activities are coordinated well.

Get The Right Toolbox

Part of the answer is making sure that you have the right tools for the job. A good project management tool is going to help coordinate team activities and tasks around the globe. With the wrong tools in place, project teams suffer. They don’t have good communication, they don’t understand what the other team is doing, and task dependencies — the handoff of activities from one team to the other team — is not coordinated well. You end up with project chaos.

And when you’ve got team members that are separated by 8 or 10 or more hours in a day, this can derail the entire project for more than a day at a time.

We have two “go-to” tools that we use all the time very successfully. And we also have one that is a project nemesis. The one that we don’t like is Basecamp. We tell our customers, “Never use Basecamp.” The thing with Basecamp is, it’s super easy to use. You can sign up and start using it, literally, in a few minutes. People who don’t like process love Basecamp, because there is no process. You just put your tasks in. It’s easy, it’s unstructured, you can attach documents, and it all goes into Basecamp — and it kind of vanishes into Basecamp. That’s the problem, Basecamp doesn’t drive the process. We need tools that drive the process.

Driving The Process

A great project management tool is going to remind people of what they need to be working on, it’s going to track the interdependencies between tasks, and it’s going to make sure that someone doesn’t get hung up waiting on somebody else. This is particularly important for multinational teams where communication is already an issue. We need tools that will fill that gap and work hard to coordinate these teams, because they are already distributed, they are already having a hard time coordinating.

For small companies we recommend Teamwork PM. Teamwork PM is a good step towards an enterprise grade project management tool but it doesn’t have a lot of overhead. Your team can be using it in no time at all. It does coordinate tasks really well between team members, and it tracks dependencies, and it notifies team members of what they need to be working on. Which is one of the key ingredients to success.

For mid-sized and large teams, we recommend Atlassian Jira. It’s an enterprise grade solution. It’s completely customizable workflow system means that you can build out really elaborate, powerful processes to support your team and to support your entire organization. Jira can be customized to go all the way from requirements management and development through to customer support and care.

There are lot of great tools out there, Teamwork PM and Jira are two of them. But the most important thing to remember when selecting your project management tool — make sure it drives the process, make sure that the Project Manager is going to be able to easily get the information they need out of the systems so that they can stay on top of the project — before problems start to crop up.

Be sure to check our website. On this blog post we’ve listed a number of other project management tools that we have used in the past in addition to Atlassian Jira and Teamwork PM.

Our Project Manager’s Toolbox

Here are a few of the project management platforms that we feel are worth taking a look at. We couldn’t recommend Atlassian JIRA (for mid- to large-size organizations) and Teamwork PM (for small-size organizations) more highly, but they clearly aren’t the only solutions on the block. We track close to fifty different project management tools — these are the ones that have risen to the top, in our opinion.

  1. Atlassian — JIRA, Confluence, ServiceDesk for a complete enterprise solution
  2. Genius Project — Traditional, full cycle portfolio and project management platform
  3. Herogami — Agile development with Kanban
  4. Jama Software — Full lifecycle project management
  5. Liquid Planner — Traditional team and project planning
  6. Rally Software — Agile project management platform
  7. Teamwork PM — Full featured, easy to deploy task and project management

Time Orientation And International Success

If you missed the first part of this six-part series, see: Part 1 of the series, Creating An International Culture Of Success, or see the entire series right here.

How we think of time is a tricky subject, and one that varies from one culture to another, as I’ve talked about before. Does your culture view time as more fluid, a resource that is infinite? Or is timeliness and meeting deadlines of critical importance?

Time, Projects, And Business

In the project context, time becomes very meaningful. To the business, meeting a product delivery date can be the difference between success and failure — but, at the same time, different cultures will view the importance of meeting that date relative to other priorities. In strongly relationship-driven cultures, for example, the date is subordinated to relationship building. Customer happiness may be more important than shipping before the Christmas buying season. It can also imply an expectation of tolerance and understanding when dates slip.

Time can have a dramatic impact on our business relationships as well. When Japan and Australia entered into a sugarcane export agreement, conditions where beneficial for both parties. As time changed market conditions, Japan ended up with the “dirty end of the stick.” But the relationship-centered business model of Japan led to a huge misunderstanding when Australia refused to renegotiate business terms — in essence, Australia felt the timing of the deal was good fortune for them, while Japan expected that business terms would adjust as time went on. A very fixed versus fluid perspective (and one that resulted in a long and nasty dispute).

The American phrase “time is money” indicates how the typical American prioritizes time, but this approach never works in a culture that prioritizes the relationship (meaning, most of the Middle East, South America, and Asia). To these cultures, it’s more important to get to know each other, to build a trusting relationship, and then begin talking about business. There will always be time to make money together — in the future. Anyone that rushes the process is probably going to be viewed as impetuous, unreliable, or even untrustworthy.

The Global Project Compass™ identifies the following management disciplines as being most directly affected by time orientation:

  1. Project Time Estimation
  2. Quality Assurance Plan
  3. Requirements Management
  4. Testing Plan
  5. Acceptance Plan
  6. Performance Measurement

Project Time Estimation

Probably one of the most obvious consequences of viewing time differently is how we estimate time. Is that estimate a “drop dead” date that we absolutely will meet, no matter what? Or is it an average of where we’ll end up if all goes reasonably according to plan? Might it merely be a hopeful guess at what could be possible?

Depending on your culture, any of these options will be true. Understanding how your partner’s culture views time is crucial to knowing what a project estimate means.

Quality Assurance Plan

Planning the successful — and problem free — launch of any product demands forethought. It demands awareness and convergence of many different plans: Research, development, supply, construction, testing, marketing, customer support, distribution, and more. In a multinational situation, supply chain logistics and regional conditions ranging from weather, product availability, and local holidays play into it.

Assuming a quality assurance organization that is timely and schedule driven, it’s not hard to imagine how difficult their job must be. Consider a global team, where different offices have different notions about the priority and meaning of “time.”

And finally, ask yourself: How does our quality assurance organization, itself, think about time? Is being on time important? Is it one of the quality metrics they are watching out for?

Requirements Management

Are the requirements known at the outset of your project? Or are they vague and fuzzy, with new features “popping up” here and there? Scope creep, or the unending addition of new requirements, is one of the most dramatic influencers on a project.

If your business cares about setting a clear end-point for a project, the team needs to understand that. In cultures where time is fluid, the idea that a product is set in stone and cannot change will seem irrationally rigid and short-sighted. At the same time, projects that seem to shift like a sand dune under someone’s feet will drive a sequential, time-oriented person crazy.

Setting the right expectations is part of the solution, but also knowing how to leverage the strengths of each perspective is key.

Testing And Acceptance

Different products take different approaches to testing. Software can begin testing early in the product life cycle, while manufactured goods need to be tested once they come off the production line. In all cases, though, testing and acceptance is critical and needs to happen at the right time, and in the most effective way.

Both are “critical paths,” too. This means that someone, somewhere, is waiting on the results of testing or acceptance.

Will your testing team be ready to go at the right time? Will the right urgency be applied to the process — or will testing be run like like a fluid project, adding new requirements on the fly?

Time Orientation: Fixed Or Fluid?

Understanding time orientation means knowing how to build a healthy organization — one that supports the time orientation of its employees, without sacrificing necessary business goals. It’s a tough topic to master, because how we think about time is so deeply ingrained in our subconscious. It’s a part of who we are, and changing that doesn’t come naturally.

Think about how you feel, when kept waiting in the conference room for the other team. Are they late, rudely wasting your time — or are they instead thoughtfully giving you a few extra minutes to prepare, while they respectfully and unhurriedly wrap up another meeting?

Think about how hard it will be to change that initial, first reaction, the next time someone is “late,” or seems offended that you are not “prompt.”

Cover graphic attribution: The artist and visual designer Yang Liu was born in China and lives in Germany since she was 14. By growing up in two very different places with very different traditions she was able to experience the differences between the two cultures first-hand.

Creating An International Culture Of Success

The International Business Dimension

Multinational teams present new challenges for the International manager. There are logistics problems: How do you coordinate teams that work in different time zones? What kind of collaboration can you create in a team that rarely sees one another?

As well as the logistic problems come cultural problems. For example, successfully creating a culture of innovation can be a challenge. Honeywell experienced this, according to a November, 2013, Time article, when Rameshbabu Songukrishnasamy began working as general manager of the company’s R&D centers in Shanghai and Beijing. He found his employees were not innovating. They weren’t tinkering or inventing on their own — not a positive sign in an R&D lab! “They were happy just doing what they were asked to do,” Rameshbabu says. The problem is, R&D is about doing something new.

A project manager for a large corporation in Brazil recently told me that the PMI Book of Knowledge is used infrequently at best inside Brazilian projects. He also warned against assuming that someone with a PMI certification has extensive experience, as is the case in the US. — Moore, Brandi, The Little BRIC Book.

Rameshbabu found that his Chinese workers had a fear of failure. They worried that the company would be upset if their work did not yield positive results, so they didn’t experiment. Another problem is that some Chinese engineers “tend to shy away from critical questioning,” a process that is fundamental in R&D. “The reason they are able to make so much innovation in Silicon Valley is that people question the status quo and find alternative ways,” says Rameshbabu. But he found that Chinese culture and education focused on rote learning, not critical thinking.

Creating A Culture Of Success

Creating successful International programs requires understanding and adapting to different business cultures. Applying Western management practices in Asia will fail, just as surely as transplanting Western employees into an Eastern environment. Imagine an independent, critical thinker from Silicon Valley landing in Foxconn, Shenzhen — where challenging the status quo is forbidden.

Team dynamics play a huge factor in management style, objectives, and capabilities. Building a culture of innovation is just one example of where these dynamics become complicated. Power distance will affect everything from goal setting to how problems are socialized. Communication style can quickly lead to misunderstandings. Differences on the fluidity of time can mean completely missing the mark with customer deadlines. And differences in identity and engagement style can lead to initial confusion, bad first impressions, or distrust.

This is why understanding business cultural practices is so important. Hyrax International LLC has a program that explores each of these five preferences. The program examines each of 27 different management disciplines, such as goal setting, risk management, change management, and assessing outcomes. The affect of business culture on each discipline is explored and explained, providing a road map to success on the International management scene. The company also offers many free resources to explain and explore International project management, and is also sponsoring Successful International Project Management, an in depth book that maps project management processes to cultural preferences.

We’ll be posting five more parts to this article (read Part 2, or see the entire series right here) in the coming couple of weeks. Each post will look at one of the five business cultural preferences, and briefly introduce how that preference impacts and affects the 27 management disciplines.

Hyrax International LLC’s Global Project Compass™ is the only visual map that clearly shows the connection between business culture and business process. This is what makes Cross Cultural Management™ so much more effective than traditional management.

The Compass maps 27 project management disciplines directly to business cultural preferences, and shows how these preferences affect business. The goal of the Global Project Compass, and Hyrax International’s associated management program, is to show how culture affects businesses worldwide — and to provide a clear map on how businesses can adapt successfully.

Did you know India has a little more risk today?

Euler Hermes, the 100-year-old trade credit insurance firm, has a fantastic little tool for assessing credit risk around the globe. Euler Hermes monitors country risks in 241 countries and territories. Their country risk map aims to assess the risk of non-payment by companies in a given country.

In other words, if you’re thinking of developing a new market, you can use this little tool to see how likely (or unlikely) it is you’ll get paid. The map is interactive, and updated periodically to reflect changes in the global economy.

As many of you know, we do a lot of business in India — so, I was pretty surprised to see that Euler Hermes upgraded India from “low risk” to “medium risk” this year. In retrospect, while it took me by surprise, it probably shouldn’t have. India still suffers from an immature business market. I can see how this can translate into higher credit risks.

Country Risk Map
Country Risk Map, Euler Hermes (click to open)

It also underscores how important it is to work with a trusted source, and an experienced partner, no matter what country you go into. Euler Hermes is one such resource when it comes to insuring your revenue stream — whether its import/export, manufacturing, or even R&D. If you’re doing International business, trade credit insurance is a must, and these guys know how to do it well.

Getting back to India: Yes, maybe it is a little bit more risky today, at least when it comes to credit risk. But that’s no reason to stay away, it just means you need to account for those risks. Engage with a partner that really knows how to work with India, take the right precautionary steps (such as using trade credit insurance), and move forward with your eyes open. India is an incredible market opportunity, and not one to shy away from!

Improving Multinational Team Collaboration

Power Distance in the Global Context: How business cultural preferences can directly impact your team’s ability to work together and communicate effectively.

When we talk about our own culture, it won’t seem like culture at all. It’s just the way things are. For example, if you’re American you believe everyone has a right to free speech and you think any thing else is a human rights violation. You also enjoy about two or three weeks of vacation each year. And you’re very prompt. You feel like your entire day is interrupted if someone’s late.

If you are Turkish, you know about free speech, but you probably don’t dare talk about it much. You are blissfully unaware that you have more official and not so official vacations than anyone else in Europe. And for you, time is fluid. Being half an hour late is fine, you don’t expect anybody else to be on time, so you plan on doing several things at once. These are examples of culture, and businesses have culture too.

Phil, a CEO at a US company, came to me telling me that they had outsourced all of their product development to India. But, they had a problem. They hadn’t met a single deadline in over a year. Product development was stalled. When I asked Phil what was wrong, he said the team in India was terrible at communication. They never raised a red flag if something didn’t make sense.

The problem is Phil was completely wrong. He was ascribing Western expectations to an Eastern team. The real problem was more complex, but the root of it was power distance. Power distance is the degree to which a boss and an employee are separated by society. And in the East power distance is very important. Employees can’t simply tell their boss that they think he is wrong. But Phil’s employees are rewarded for questioning and challenging and thinking outside of the box. He had no idea how unfamiliar this was for his India team.

We worked with Phil to develop a cross cultural training program. A year later, Phil’s company is doing great. Their new product just shipped and Phil told me that the team has hit every milestone perfectly. This is just a small part of how business culture can affect a global business. It’s why we created the business synergy compass; to guide businesses to success in the new global economy.

Why we can’t just burndown everything

Prashad is the project manager for a new product. After reviewing the project scope, Prashad’s team gives it an initial estimate of about three to four months to complete, which is good enough for management. The project kicks off, and using Scrum along with a little bit of project management, the project gets underway. The first month goes quickly, there is great progress, and everyone loves what they see. But in the second month, things start to slow down — so, Prashad adds more resources to the project to compensate. But by the third month, there are real problems and it looks like the project will be late. Prashad adds even more resources. Now the team is almost double the original size. He adds a lot more quality assurance and testing support for the team, too, because a lot of problems are cropping up. The team cranks away, and they finish the project in the fifth month, and claim “success” because they almost hit the deadline (it was, after all, just an “estimate“).

But what about all those additional resources? Is it really success if the project cost twice as much as originally estimated? The problem is, that cost was never formally stated. It was never tracked as a metric, so the fact that the project went horribly over budget is quietly swept under the rug — at least this time. There are plenty of cases where the budget isn’t there, or the resources aren’t there, or the product just can’t ship late.

So, if so many projects are missing their mark, what’s causing the problem?

The fallacy of so called Agile methods

According to studies conducted by KPMG, as much as 70% of projects fail to meet their goals. In this case, “goals” mean quality, schedule, and function (or a combination of those). Clearly, being able to accurately estimate a project’s delivery date is important. Without knowing when a project is done, you can’t predict cost, plan business operations, or dovetail releases with marketing, training, and customer service.

Agile methods like Extreme Programming and Scrum make promises that, in my opinion, they can’t keep. For well over a decade we’ve seen an increasing trend in project failures, and a large number of those failures are the result of “unending projects.” Projects that go on and on, continuing to deliver improvement but slowly creeping over budget and never reaching an end state. Part of the problem is the lack of formal estimation and metrics.

Both Scrum and Extreme Programming dodge the entire issue of project life cycle estimation. They focus on the near term, providing estimates for the next one or two sprints. This works great if you don’t have budget or time constraints, but in the real world that’s rarely the case.

Burndown just doesn’t work

Most Agile methods don’t explicitly define how you estimate progress, but the most common method is burndown — the measurement of completed effort against the planned sprint goals. This is fine for a quick-and-dirty project but as a strategy it utterly fails to pinpoint problems with budget and timeline. For anything larger than a few months effort, it doesn’t do the job.

And the problem is, few of today’s engineers seem to be getting the formal education and training needed to use proper estimation methods.

There are better solutions than burndown

Earned Value Management, or EVM, is nearing it’s 40th year of practice. Throughout this 40 year history, EVM has repeatedly evolved to meet new demands as technology and innovation made new leaps forward. It’s a proven method for measuring progress and it’s been proven time and again.

Of the dozen or so software engineers I spoke with about this article, none of them had even heard of EVM.

Earned Value Management, as well as a variation known as Performance Based Earned Value® (PBEV), are my favorite choices for estimating a project. The PBEV approach provides an incredibly robust method for measuring progress and staying on top of your project. It’s only marginally more difficult than other techniques of measurement. And most project managers are already capturing the metrics needed to make it work.

PBEV is an improvement over Earned Value Management Systems’ (EVMS) national standard. It supplements EVM with guidelines for the integration of project cost, schedule, and technical performance — thus created a comprehensive and highly accurate method for measuring true progress in a project. The key here is integration of effort with schedule and performance.

Had Prashad been using PBEV, he would have had a very different experience. The early estimates would have been more accurate, since PBEV estimates the entire project. It would have measured not only resources and cost, but also the actual performance — the rate at which work is being done. Had the project gone off track, Prashad would have known much more quickly, and would have had a more precise picture of how to correct the project’s course.

There will definitely be situations in which PBEV is not warranted, and a skilled project manager will need to know when to use other methods, and why. EVMS, for instance, addresses only the quantity of work completed, ignoring cost and schedule metrics. The implication here is that a quick burndown or EVMS-based evaluation of a project might look A-OK, when in fact the only reason it’s on track is because resources are being thrown at it — and costs are skyrocketing.

Burndown versus critical path thinking

Burndown is a very simple method for estimating work effort over a short term. It explicitly avoids timeline, cost and performance metrics. That’s fine for projects that are not constrained by time, budget, or resources. The problem is, burndown is used as a defacto standard too often, on the promise that using an Agile method will deliver a project faster or cheaper than its alternatives. I don’t disagree on this front. Agile methods generally are faster and cheaper — but the question is, faster and cheaper than what?

Most business need to be concerned with time, money, and staffing. Projects that have those constraints need to think about the critical path — the sum of all activities to reach the end of the project. Having a solid understanding of the fundamentals of critical path management is important to managing any project schedule. Depending on the nature of your project, you may be able to rely on casual effort estimates — or, you may be required to carefully identify, analyze, and attack critical paths to shorten your project lifecycle and stay within stringent guidelines. Without understanding the principles, and knowing where to turn for more information, neither is possible.

I’ll revisit the topic of PBEV in a future article and provide more depth into how it can be applied to a project. If you’re eager for more information, check out Performance-Based Earned Value®, Paul J. Solomon, Ralph R. Young, IEEE Computer Society, Wiley Interscience, 2007.