OUTRAGEOUS COMPLIANCE; RCSA

Risk & Control Self-Assessments (RCSA) have become a handy tool to communicate to management, regulators and others that an organization has conducted an analysis of their risks to understand both the severity and likelihood of the occurrence of an event.  Each risk category is highlighted with its own color-coordinated assessment based on a “Table Top” exercise wherein subject matter experts participate in a facilitated session to list these risks and assign severity and probability based on nothing more than memory.

Of course, we all understand that RCSAs are objectively subjective. The “risk” in RCSA tend to sometimes give a false sense of security we place in believing these exercises are really a representation of entire and accurate risk exposures in an organization.  This is not totally correct and here is why.

Statistically speaking, risks tend to have a shape.  In some cases, the shape of risk is a normal curve.  In other cases, the shape may be skewed to the right or left.  But in an RCSA, the shape of risk is uniform.  Each risk is evaluated on a scale that looks exactly like this chart below which is a bit uniform.

Intuitively, we understand that risks are not uniform, but we never question charts and graphs that look like this. Additionally, these charts lack the benefit of the law of large numbers. You might be surprised to learn that risk management is based on scientific laws of statistical analysis. The RCSA is flawed because it’s based on a small sampling of data (your memory) that is usually inherently biased by recent events that are easy to recall, not representative of frequencies found in a large stochastic database of risk events.  What does stochastic mean?  Stochastic is a process involving a randomly determined sequence of observations, each of which is considered as a sample of one element from a probability distribution. In other words, if you are not using a stochastic process for measuring risk, you are totally objective – may be a reliance on native intelligence kind of!

RCSAs have some value as a tool for understanding the risks subject matter experts deal with on a daily basis.  These tools are a great starting point, from which you should begin to develop a stochastic database of risk events — they should not be the conclusion.  This brings us to the last point about Outrageous Compliance: the risk repository.

A risk repository represents a third flaw in thinking about risks. Capturing risks in a risk repository is called a deterministic model.  A deterministic model is one in which every set of variable states is uniquely determined by parameters in the model and by sets of previous states of these variables; therefore, a deterministic model always performs the same way for a given set of initial conditions. Conversely, in a stochastic model—usually called a “statistical model”—randomness is present, and variable states are not described by unique values, but rather by probability distributions.

Why is this wrong? When developing deterministic models (risk repository), you predetermine the outcome. Lots of organizations make this mistake. The reality is that all not all models are wrong, neither are they all correct. Understanding how to develop useful risk assessment models takes time and patience, but knowing the difference means being able to avoid Outrageous Compliance.