Elegance Can Be the Enemy of Efficiency

I’ve been auditing some of the advanced Scala courses offered by École Polytechnique Fédérale de Lausanne, and as always am curious about my own solutions compared against what other people come up with.

One of the things I love about Scala is its elegance. Once you understand the language, you can achieve some really complex things quickly and easily — and, elegantly. So, a problem posed in the coursework is to write a function that, given a character in a game field, returns the first index of that character in the field.

After writing my own solution, I Googled a few other solutions. Here’s one:

def findChar(c: Char, levelVector: Vector[Vector[Char]]): Pos = {
   val ys = levelVector map (_ indexOf c)
   val x = ys indexWhere (_ >= 0)
   Pos(x, ys(x))
}

This is pretty elegant. In fact, I was first taken by how functional it looked — more functional than my own code. But there’s a problem. This elegant code is potentially very inefficient.

Let’s assume the game field is big. It consists of a Vector of Vectors, and the above code searches the entire game field. That is, even if the character matches the very first element in the field, all the rest of the elements will be scanned:

  1. ys maps over the entire Vector of Vector scanning for the index of c.
  2. x then scans the vector ys for a positive hit.
  3. Then the coordinate is returned from the points described by x and ys(x).

So the obvious problem here is our elegant, functional, good looking code could be terribly inefficient. Here’s my implementation:

def findChar(c: Char, levelVector: Vector[Vector[Char]]): Pos = { 
   @tailrec def iterate(x: Int): Pos = levelVector(x).indexOf(c) match { 
      case y if y > -1 => Pos(x, y) 
      case _ => iterate(x + 1) 
   } 
   iterate(0) 
}

Ok, it’s not as pretty, it’s longer, but — it’s still functional by design. More important, it will look for the character using a linear search, and stops when it finds the character. It’s efficient:

  1. We start in the first Vector, checking for the index of the character.
  2. As soon as we find it, the function returns. Otherwise, it recurses and moves to the next Vector.

It’s not as pretty but it preserves efficient design. Just because we can write pretty, elegant code doesn’t mean we always should.

Incidentally, you could also achieve nearly the same efficiency using something like this:

val row = levelVector.indexWhere(_.contains(c))
val col = levelVector(row).indexOf(c)

Personally, I prefer to avoid any potential inefficiency — but, over-optimizing can also be a problem. It’s probably best to find a happy balance. Don’t ignore efficiency, but don’t spend all your time optimizing situations that, quite likely, don’t really need the effort.

You’re Testing It Wrong…

I really like test driven development. It lets me be lazy. I don’t have to worry about my software quality, or that something I did broke some other thing. And with good dependency injection to make sure every component is working right, “it just works.” Now I code using TDD (writing my tests first, then coding to fulfill them), and I focus our QA efforts on making sure we have great test plans, and great coverage.

Closed System
A closed system wants to be tested.

So, when one of my project teams kept telling me they couldn’t write tests because the database wasn’t ready I got worried. Our team had been immersed in TDD for months — and every single engineer had nodded vigorously when I set expectations. The team leader recited the definition of “dependency injection,” just to drive home how ready they were to embrace it!

But when I asked to see the what was wrong, I knew we had a problem. The team’s tests were not injecting mock objects the right way. The idea behind dependency injection is to replace the smallest component possible in a closed system with another object, a “mock.” That mock can then monitor the system around it, inject different behaviors, and create desired results.

For example, let’s say we have a program that connects to a gizmo — your home thermostat. The thermostat itself is a separate component that lives outside your program. We can expect the thermostat to behave like a thermostat should… reporting the current temperature, and letting the home owner enter a desired room temperature. Pretty straight forward.

So the first step is to write a program that talks to the thermostat. We can wire up a real thermostat, but we’ve got a problem right off the bat. We want to know how our program behaves as the ambient temperature changes — 65 degrees, 32 degrees, or 100 degrees. But a real thermostat is only going to report the actual room temperature, and making the room frigid or boiling just isn’t going to be very comfortable or practical.

Not Mocking
Faking is not mocking.

This is where dependency injection comes in — wouldn’t it be great if we could inject a new gizmo, one that behaves according to our test plan?

It turns out that my team had been taking the wrong approach — but one that is pretty easy to make if you’re new to the idea of mocking and dependency injection. Unfortunately, it meant that they weren’t really testing the application. The were testing something else entirely.

Once we walked through the system, the mistake was clear. During application start up, it created a connection to a database. My team’s approach had been to add a “mocking” variable to the application. In effect, it created a test condition; if the application was in “mocking mode” it would only simulate database calls. If the application was not in “mocking mode” it sent real requests to a real database. Sounds great, right?

But it’s all wrong. Here’s the problem: The application was faking real world behavior. That is, throughout the program there were dozens of little tests, in effect asking, “if we are mocking, then don’t check the database for a user record, instead just return this fake user record.”

This meant that the real program — the actual application logic that would be deployed to the real world — was never tested. Instead, an alternate branch of logic was tested — a fake program, if you will. So two things happened:

  1. We weren’t testing the real program, we were testing something else altogether.
  2. The program itself became terribly complicated because of all the checks to find out “are we mocking?” and the subsequent code to do something else entirely.

And all of that is why my team said they couldn’t really test the system, because the database wasn’t up and running.

So what does real dependency injection look like? It’s simple: You want to change the actual gizmo, but change it in the most subtle way possible — and then you want to put that actual gizmo right back into your program.

Mocking
Real mocking doesn’t affect the original program flow.

Getting back to the thermostat example, an ideal solution would be to modify a real thermostat. You could crack it open, remove the temperature sensor, and add a little dial to it that lets you change the reported temperature. Then you plug the “mock thermostat” into your program, and you change the temperature manually! A potentially better approach would be to change the software that talks to your thermostat, and instrument it so that you can override the actual reported temperature. Your program would still think it’s talking to a real thermostat, and the connecting software could change the actual temperature before handing it off to your program.

In our case, the right solution could be injecting a simple mock component at the just the right point in our program.

For example, lets say our application uses an Authenticator object to log in users. The Authenticator checks the validity of a user in the database, and then returns a properly constructed User object. We can use dependency injection to substitute our own test data by overriding the single function we care about:

object fakeAuthenticator extends Authenticator {
    override def getUser(id: Int): Option[User] = {
        Some(User(id: -1, name: "Fake User"))
    }
}

On line 2, we replace the real Authenticator’s getUser function. The overridden method returns a hard-wired User object (in this case, one that clearly doesn’t represent a valid user account). By overriding the Authenticator in the test package only, the original program is not altered — all that’s left is to inject our altered Authenticator into the program.

The old fashioned way of doing injection is still reliable: Don’t tell, ask. Use a factory object to ask for the Authenticator. Given a factory in the application (let’s call it the AuthenticatorFactory) we can override what the factory actually returns in our test case only:

AuthenticatorFactory.setAuthenticatorInstance(fakeAuthenticator)

A slightly more modern approach is to use a dependency injection framework, but the underlying principle is exactly the same.

Likewise we can take the concept of mock objects further by using frameworks such as Mockito (a framework that works wonderfully with specs2). Mockito makes it easy to instrument real objects with test driven behavior. For example, Mockito will produce a mock object that acts just like a real object, but fulfills expectations (such as testing to make sure that a specific function is called a certain number of times).

Whatever tools and frameworks you use, test driven development has proven itself over the past decade. My own experience is the same: Every TDD project has produced more predictable results, has better velocity, and has been more reliable overall. It’s why I don’t do any coding without following TDD.

Don’t Use Basecamp

Hi, I’m Zacharias Beckman, President of Hyrax International. Managing a multinational team can be really challenging. You have to make sure that each team, around the globe, is coordinating their efforts. That means overcoming the communication problems between them, tracking task dependency from one team to the other, and staying on top of projects — so that one team doesn’t get held up waiting for another one. So, the real challenge is figuring out to stay on top of each team and how do you make sure that all of their activities are coordinated well.

Get The Right Toolbox

Part of the answer is making sure that you have the right tools for the job. A good project management tool is going to help coordinate team activities and tasks around the globe. With the wrong tools in place, project teams suffer. They don’t have good communication, they don’t understand what the other team is doing, and task dependencies — the handoff of activities from one team to the other team — is not coordinated well. You end up with project chaos.

And when you’ve got team members that are separated by 8 or 10 or more hours in a day, this can derail the entire project for more than a day at a time.

We have two “go-to” tools that we use all the time very successfully. And we also have one that is a project nemesis. The one that we don’t like is Basecamp. We tell our customers, “Never use Basecamp.” The thing with Basecamp is, it’s super easy to use. You can sign up and start using it, literally, in a few minutes. People who don’t like process love Basecamp, because there is no process. You just put your tasks in. It’s easy, it’s unstructured, you can attach documents, and it all goes into Basecamp — and it kind of vanishes into Basecamp. That’s the problem, Basecamp doesn’t drive the process. We need tools that drive the process.

Driving The Process

A great project management tool is going to remind people of what they need to be working on, it’s going to track the interdependencies between tasks, and it’s going to make sure that someone doesn’t get hung up waiting on somebody else. This is particularly important for multinational teams where communication is already an issue. We need tools that will fill that gap and work hard to coordinate these teams, because they are already distributed, they are already having a hard time coordinating.

For small companies we recommend Teamwork PM. Teamwork PM is a good step towards an enterprise grade project management tool but it doesn’t have a lot of overhead. Your team can be using it in no time at all. It does coordinate tasks really well between team members, and it tracks dependencies, and it notifies team members of what they need to be working on. Which is one of the key ingredients to success.

For mid-sized and large teams, we recommend Atlassian Jira. It’s an enterprise grade solution. It’s completely customizable workflow system means that you can build out really elaborate, powerful processes to support your team and to support your entire organization. Jira can be customized to go all the way from requirements management and development through to customer support and care.

There are lot of great tools out there, Teamwork PM and Jira are two of them. But the most important thing to remember when selecting your project management tool — make sure it drives the process, make sure that the Project Manager is going to be able to easily get the information they need out of the systems so that they can stay on top of the project — before problems start to crop up.

Be sure to check our website. On this blog post we’ve listed a number of other project management tools that we have used in the past in addition to Atlassian Jira and Teamwork PM.

Our Project Manager’s Toolbox

Here are a few of the project management platforms that we feel are worth taking a look at. We couldn’t recommend Atlassian JIRA (for mid- to large-size organizations) and Teamwork PM (for small-size organizations) more highly, but they clearly aren’t the only solutions on the block. We track close to fifty different project management tools — these are the ones that have risen to the top, in our opinion.

  1. Atlassian — JIRA, Confluence, ServiceDesk for a complete enterprise solution
  2. Genius Project — Traditional, full cycle portfolio and project management platform
  3. Herogami — Agile development with Kanban
  4. Jama Software — Full lifecycle project management
  5. Liquid Planner — Traditional team and project planning
  6. Rally Software — Agile project management platform
  7. Teamwork PM — Full featured, easy to deploy task and project management

How To Go Global

How do you learn to “go global” and take your product, and your company, to an International scale?

Hi, I’m Zacharias Beckman, President of Hyrax International. I founded this company because I believe that American businesses, in particular, are really embracing the global economy. That means learning how to adapt products to different cultures around the world; changing a business’s internal culture in order to be compatible; changing how projects are managed, because the traditional Western management style that most of these American businesses employ, those styles are not going to work in Asia or South America or the Middle East. So, American businesses need to start to adapt.

Adapting To “Global”

A new level of business cultural awareness is needed. We need to understand how to communicate with each other, how to adapt to each other’s way of thinking about time, how to manage people with a different concept of power distance or the separation between a boss and an employee. It’s a complex landscape.

Taking a product into another country means adapting that product so that it fits well with the culture there. For example — lets say you wanted to take a product that was a four pack of golf balls, here in the United States, and sell it in Japan. It’s probably not going to work — because the word for “four” in Japan sounds very much like the word for “death.” So taking that product and simply slapping a new label on it in Japanese and shipping it over there — that’s not going to work.

This is why we created the Global Project Compass (see our six-part article on the Compass). It’s a map that takes 27 different project management verticals, things like quality assurance; time estimation; acceptance testing, and it maps them to business cultural preferences. And we see how, for example, communication is going to change each one of these 27 different project management disciplines.

We are global project experts. We understand the technical execution and we understand the cultural implication. Our programs will make sure that you succeed taking your products overseas and building a multinational organization.

How Business Culture Affects Your Business

Hi, I’m Zacharias Beckman, President of Hyrax International. We get a lot of questions about how business culture affects business on a day to day basis.

Sarah, a project manager here in America, is very successful at what she does. She’s got a lot of successful projects under her belt. But right now she’s having problems. It looks like the project that she is working on is going to ship late. There are lot of quality problems with it.  It’s over budget, it’s behind schedule, and Sarah is very frustrated. All of the techniques that she’s been using in the past aren’t working for her now. She feels like she is  not getting the feedback from her team that she’s used to getting. For example, she proposed some changes to quality assurance system and instead of getting feedbacks, there’re silence, delays and  then finally the team agreed to implement what she had proposed.

In a typical Western style, she is expecting very clear communication from her team.  Direct, critical feedback on the project and on the proposed changes that she is making. The problem is, her team is in Asia and they don’t think  that she knows she is doing. She asks too many questions. She doesn’t demonstrate the authority and the wisdom necessary for the team to feel like she’s in charge of the project.  They are not accustomed to this kind of management.

Sarah’s run into a couple of business cultural preferences. She’s experiencing power distance, which is the separation between a boss  and a subordinate, and how they ‘re allowed to communicate because of cultural constraints. And she’s also experiencing differences in communication style — the low context, direct communication of the West versus the very high context, rich and subtle communication of the East.

Sarah’s solution in this case is to get business cultural training and understand how her management style differs from what her team is used to and then adapt  her management program, her project management technique, to work well with her team is Asia.

Check our website for more information on both of these business cultural preferences. I’ve blogged about them quite a bit in the past.

And if you are managing an International multicultural team, it’s really important to understand how much business cultural preferences will affect your project and your management style. You need to be sure that the project management methods you’re putting into play, are going to work with your multinational team.

Communication Style And International Success

If you missed the first part of this six-part series, see: Part 1 of the series, Creating An International Culture Of Success, or see the entire series right here.

I’ve posted a lot about how communication style varies dramatically from one culture to another (including this great infographic on how different cultures negotiate). It’s both very obvious, a clear variation in how we interact, and at the same time deviously subtle in how quickly it can derail an otherwise healthy team and project.

Different Styles Of Communication

Low context cultures, most often associated with Western, industrialized countries, pride themselves on a directness that is unparalleled in other cultures. Lack of subtlety and being “honest and straight-shooting” is the norm. But these cultures end up missing most of the conversation when confronted with high context, rich communication styles.

High context cultures (especially Middle Eastern and Asian, but also South American, some European, and African countries) don’t know how to communicate in this simple, direct style. Confronted with direct, low-context partners, it’s as if 90% of their vocabulary is stripped away. The rich subtlety conveyed in circumstance, timing, silence, body language, story telling, deference, saving face, and tone are missing — leaving behind nothing but blunt, inelegant words (often, to make matters worse, in a second or third language on top of it).

Communication Style: Expressing Opinions (East vs. West)
Communication Style: Expressing Opinions (East vs. West)

Understanding one another’s communication style and being able to adapt, and interpret signals from both cultures accurately, is critical.

The Global Project Compass™ (introduced in Part 1)  identifies the following management disciplines as being most directly affected by communication style:

  1. Continuous Improvement Plan
  2. Segregation of Duties
  3. Project Management Plan
  4. Project Monitoring, Execution, & Control
  5. Change Control & Management
  6. Communications Plan
  7. Performance Measurement

Continuous Improvement Plan

Your continuous improvement plan is absolutely affected by other business cultural preferences, but communication style has a huge impact. Continuous improvement relies on understanding each other without ambiguity. Anything that stands in the way will throw sand into a delicately working machine. Processes like CMMI (the Capability Maturity Model) rely on putting complex, integrated processes into action. Everyone has to understand the process, support it, and pursue it’s objectives.

Segregation of Duties

The Compass also identifies segregation of duties as highly affected by communication style. Clearly defined roles are important in any organization. Segregation of duties is intended to create checks and balances to enforce standards or, in some cases, prevent fraud or malfeasance. One way of looking at this is whether control is unchecked in one person (or one office). A common reason to separate quality assurance, giving it authority on its own, is to support a separate office that has the authority to enforce quality (or at least, stop a project that is not going well).

For this to work, communication lines must be clear. How can quality assurance know how the project is going if there is limited, inaccurate, or unclear communication?

It’s important to note that power distance also deeply affects segregation of duties. The political alignments and often muddied visibility of some organizations create complex, co-dependent relationships. These relationships interfere with the goals of segregating duties.

Project Management, Monitoring, and Change Control

Excellent project management relies on clear communication as well. Across a culturally diverse organization, “clear communication” can mean many things. How does the American manager interpret his Chinese subordinate’s silence, when critical feedback is expected? How will an Indian employee react to the direct communication of a German boss causes him to lose face? Building a project management plan that works well within the multiple, diverse cultural environments of a multinational organization is a challenge.

Performance Measurement

Knowing how your team, and your company, is doing demands no ambiguity. You’ve got to be able to assess performance accurately. For business performance, that means getting accurate, timely information. To assess your team, you need to understand and assess your team member’s contribution. That means understanding what everyone has to say, in their own subtle or not-so-subtle communication style.

Western-style “360 evaluations,” where employees critically evaluate their peers, subordinates, and superiors, rely on American-style direct communication. When used in other cultural settings the 360 evaluation completely fails. When compared to typical American feedback, French and German respondents more easily criticize, but hold back compliments — so evaluations appear much less positive. In many Asian cultures, the idea of openly criticizing is taboo. Here, evaluations come back with seemingly perfect marks — and that can lead to incorrectly concluding that the Asian office is perfect.

Communication: The Tip Of The Iceberg

Most often, problems between multinational teams get put down to bad communication. It’s true that communication is important. It’s also true that most cross cultural situations have communication barriers (and often serious problems). How well people communicate — or, how poorly your team is communicating — is a very visual indication that there are problems.

Just like an iceberg floating in the ocean, this visual indicator usually means there is more going on beneath the surface. When your team isn’t communicating, it’s time to look for other problems too.

Cover graphic attribution: The artist and visual designer Yang Liu was born in China and lives in Germany since she was 14. By growing up in two very different places with very different traditions she was able to experience the differences between the two cultures first-hand.

What Is “Saving Face” In Other Cultures?

What does “saving face” really mean? Westerners tend to think “face” means preserving one’s reputation… but that’s not right. It’s particularly important in high-context cultures, including most of Asia and the Middle East, where tradition is highly valued and the interests of the group outweigh the interests of the individual.

Hi, I’m Zacharias Beckman, President of Hyrax International and I wanted to speak briefly about “what is saving face.” Face is a collectivist notion. It’s something that applies in many Eastern cultures and as such it’s an extremely foreign idea to Western culture.

Misunderstanding “Saving Face”

So, here’s an example of how not understanding face can go wrong with Western and Eastern interaction. Let’s say you are a Western Manager, applying western management theory. So, if one person does a particularly good job, the natural thing to do is to reward that person, to call them out and tell them they did a better job, possibly give them a raise or some kind of a reward within the firm.

But, in Asian society, this actually sends the wrong message. What you’re doing is saying that the individual failed in their responsibility, to their group, to their fellow employees, because that person did not show those individuals how to perform well. So, the net result is you tell one person that they didn’t do a good job, and you tell the entire group that they also failed to do a good job, in this respect. It backfires terribly when Western managers do that with Eastern cultures. And this is a great example on why it is so important to really understand what face is whenever you are doing business with the East or the Middle East.

What is Face?

It was first defined by David Ho, a social scientist working in Hong Kong. He basically defines saving face as saying that face is lost when an individual, or someone who is closely related or connected to that individual, act in a way that fails to meet the social obligations that are set up for that person. In other words, if they don’t meet their social responsibility with family, with work, with their friends, then they loose face.

In Asia and the Middle East, having face is a very bankable notion. It is a literal translation, or a literal representation, of your status in society, of your reputation and your abilities to fulfill your obligation within that social network. Because collectivist societies are so tightly integrated and tightly social, there is only one face. Social, work, family, it’s all integrated into a single representation of who that person is. That means that your face at work and your face at home can be damaged in the same way.

If you’d like to see another take on saving face, check out this short video (the bit on saving face is in the latter half of the video).