We all want to be Apple. We want to have their reputation, at any rate. A zealous customer base, fantastic products that seemingly flow out of design and into production without a hitch, and a virtually zero record of recalls or product delays.
But it’s the part about the customer that really grabs our attention. So the question is, how do they do it? If we put the right people in a room together will they just “get it,” and execute a flawless vision?
That’s likely a key part of it, at least in so far as it takes the right people to make the right decisions. But how do we execute our vision with such precision? And if we look at other successful companies, will we find some theme that’s in common with Apple? Absolutely. That common theme isn’t just one thing — But every single successful company has one common element in their strategy: A mechanism for avoiding undue risk.
Risk management has become mainstream. It’s no longer the domain of rocket scientists and actuaries. In fact, it’s become so mainstream that formal risk management practices are showing up everywhere we look. Most of the time, we’ll see the word Enterprise included in the definition — a way of letting us know “this is for the whole firm.” Enterprise Risk Management (ERM), Business Continuity Planning (BCP) and Governance and Risk Compliance (GRC) are just a few of the different names risk management flies its flag under.
Is More Attention A Good Thing?
But is all this sudden attention to risk management going in the right direction? To answer that, we need to look at the specifics of different risk management techniques.
For example, the Project Management Institute (PMI) and National Institute of Standards and Technology (NIST) have both put forward standards that devote significant space to the topic of risk management. The PMI standard of risk management (PMI-RMP®, or Risk Management Professional) includes some pretty extensive methods for identifying, quantifying and mitigating risk.
Much of the PMI-RMP standard can be considered a brief introduction to risk management. It doesn’t introduce quantitative analysis or provide any background of Judgement and Decision Making (JDM) theory. It does, however, provide a starting point, some kind of a baseline that we can use to at least make sure that our projects, programs and organizations are addressing risk management — at some level.
This is good, at least at first blush. But, unfortunately, when we dig deeper there could be a more subtle problem here: The practices advocated by PMI and NIST standards are, quite simply, apt to cause more harm than good.
Worse Than Nothing
There are decades of remarkable research in JDM and risk management theory. The research that has gone into this kind of theory has produced an invaluable treasure trove of tools, processes and techniques that we can leverage to learn how to accurately and effective assess risk across our organization.
This same research has also largely debunked “crackpot” risk management theory and poor decision making practices. For instance, Harvard Business Review led a study of over 200 popular management tools, like TQM, ERP and so on. Independent external reviews of the degree of implementation of each of these various tools was compared to stakeholder return on investment over a five year period. The resounding conclusion from this in-depth study, as reported by HBR, was that: “Our findings took us quite by surprise. Most of the management tools and techniques we studied had no direct causal relationship to superior business performance.”
But this shouldn’t be a surprise, at least not to anyone familiar with formal risk management and JDM theory. In research conducted over many decades, such as that of Brunswik, Kahneman, Hubbard and others, most of these recently introduced management practices have been exposed as ineffective and often even harmful.
Consider, for example, the principle method for quantifying risk in the PMI standard is a matrix-based weighted scoring system. This system advocates highly subjective risk assessment practices, such as relying on risk assessment almost entirely from subject matter experts. Studies have shown that even well trained experts — let alone the people that often serve as experts on review boards — tend to provide highly inconsistent and spotty assessment results. One study by Hubbard tested a group of experts in their ability to assess risk across a portfolio of projects. Unbeknownst to the participants, two of the assessed projects were identical — and, hence, we should expect identical risk assessment of the two projects. But that’s not what the study shows: Participants only agreed with their own risk assessment 22% of the time. The rest of the time, risk assessment varied widely, sometimes as much as 35% by the same individual.
Of all the professions that practice risk management, actuaries are the only ones that can claim a real profession. Actuaries, much like accountants, doctors and scientists, must demonstrate their ability to assess risk using scientifically proven methods. And, like other formal professions, an actuary puts her license on the line when certifying a Statement of Actuarial Opinion. As with doctors and lawyers, if she loses her license she can’t just get another job next door. The industry of risk managers, modelers and assessors outside of the insurance industry would be greatly served by this level of professional standards.
Likewise, organizations such as PMI and NIST should stop promulgating what amounts to crackpot risk management practices. Decades of extensive study have shown that the core principles of risk management integrated into the PMI and NIST standards simply do not work. Worse, in many cases these practices actually cause more harm than good. Scoring methods should be disposed of. Instead, standards should rely on existing bodies of proven risk management and JDM practices.
But in the meantime, attaining a greater awareness of the risks associated with bad risk management practice is our responsibility. Understanding what to look for in risk management, and consulting trained professionals that can employ statistical risk methods is a good starting point. At the very least, firms should consult with formally trained professionals — and look for empirical, statistics-based methods. Anyone proposing a weighted scoring system should be shown the door!
If you would like to learn more about risk management theory and practical methods of assessing and avoiding risk, see Hyrax International’s seminars on these topics. Attendees are welcome at public presentations. If you are interested in hosting a presentation at your firm, contact Hyrax International directly. Introductory seminars are offered at no cost.