The madness with metrics

2016 A Status of Madness in Metrics

In the next 4 minutes you will learn:

1)    Lessons from key areas of voter logic and emotional effects on behavior and metrics

2)  The questions to ask to reduce risk errors in strategy

3)  Natural cognitive bias and effects on decisions

4)  Factors to consider for algorithm development

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.
— Donald Rumsfeld

Measured with error

Polling errors

Polling errors

What a year 2016 has proven to be so far within the political arenas of the West.

First the UK public delivered Brexit despite metrics from polling to say otherwise, and then we had the Trump presidential win in the US elections despite the polls also indicating otherwise. In such circumstances it is fair to ask ourselves;-

When can we trust such numbers and what does this mean in other measures we take in life and business?

Why bother with all this research if they end up being wrong, isn’t it just a waste of money and time?

HRC Clinton

HRC Clinton

One of the interesting points surrounding both of these election cycles has been how close both opponents have been to winning.

Within such a tight race a small margin of polling error may deliver the incorrect insights, confidence and strategic shifts.


What Clinton got wrong, the Brexiters got right, what Trump got right the remainders got wrong. This crisscross of none connected positions tell a story of something hidden behind the numbers. They offer not a clear insight to a 2% error margin but rather the fact that the polling scale and weighting systems were wrong.

Steak or salad metrics:

Steak or salad.

Steak or salad.

If you ask me in a poll, would I choose a chicken salad over steak and chips, I might be tempted to declare my love for chicken salad.

I do this to express my intention of eating healthy, showing you I care about health and knowing that the right thing to do for the environment is to indulge the salad.

However, when surrounded by a group of friends and family at the restaurant and enjoying great company with a great atmosphere my behavior will often swing to the steak and chips.

I know it’s the wrong logical decision however I’m having a nice time and love steak and chips.

Key Lesson: Do not ask if you would choose chicken salad or steak but rather ask “Considering how much you enjoy yourself when out with friends and your previous ordering habits, how much likely are you to order steak or salad at your next evening out”.

First Principles and brute facts:

Tesla dashboard

Tesla dashboard

One of the biggest lessons within business life is understanding how to use basic first principles instead of analogies. Elon Musk is a great advocate for first principles business strategy.

If he measures the numbers of conventional wisdom, then Tesla and Space X would probably not exist and if it did then it certainly would be very different than the business models and scales we see them operate on today.

First principles use in metrics and decision making is important, without making assumptions decisions are placed on absolute facts and then built upon.

Great data viz with brute facts.

Great data viz with brute facts.

Brute facts are points that have no explanation in the context of many polls and analytics models there are increasing amounts of use of this term.

This is applied incorrectly to try to magically explain away a more sophisticated nuanced level of required understanding and measure of a system.

There may be certain reasons that people have voted or behaved in ways that require deeper understanding than labeling or model exceptionalism allows. For this case, we need to take the role of an investigator and question motives, environment and scales of measure.

Key Lesson: When reviewing and making decisions based on metrics, KPI’s understand fully and question the fullness of the facts presented. What else is missing from the data? What assumptions have been made to direct and influence the trend, methods of measurement and control systems?

Silver bullet success:

Some of the more popular articles we publish offer within the headline a short list or single action to gain a large result. This is a natural cognitive bias people have to seek any short cut advantage to gain success or their goal aim. What silver bullet success does not do is explain the reality for the majority of people.

The plank challenge

The plank challenge

Although it is easy to lose weight by taking a simple “30 day plank challenge” the truth is most people will never start or even get beyond day 11.

Even the very few that succeed the completion of 30 days will fail to retain their newly developed habit over a sustained 6 month or 2 year period. The same applies to metrics results. Managers will seek meaning and evidence for a short term solution as opposed to taking the long hard but more sustainable decision of redesigning a process to solve a problem.`

Lessons Learned: Despite facts and logic, people have a bias to be persuaded by emotional bias, over time this reality may bring behaviors and actions that appear to others wrong or misjudged.

Reality and wise excuses

There are many explanations for the results in both elections and many more to be published in the future. What is a stark reality is how and when and why the polls got it wrong. Even Nate Silver from the highest respected polling company FiveThirtyEight can be heard back tracking and wriggling around with excuses on how his model got the USA election so wrong (or in his view so close to right).

Lessons Learned: It is natural to try and defend a position, especially when there is a reputation to protect. The need for critical thinking should over ride at all times any potential for bias.

AI and Machine Learning and Future Decision Making



What is important from the lessons from these 2 elections in 2016 is how to balance and consider outlier opinions, influencers and their weighting value on decisions.

This becomes ever more important when considering AI and machine learning algorithms.

The algorithms are improving every day with their ability to predict the predictable within safe and logic biased controlled environments, however the pure power of random influence and influencers are harder to program into AI systems with success.

This important point is needed to be understood from a political perspective as much as a business strategic perspective.

Frankly put, things can be totally misunderstood on an increasingly higher level of sophistication.

  • I would love to here your thoughts on the polls this year. Please drop me a comment below or on social media.

  • How many metrics in your business are at risk to some of these factors raised in this article?

Please share this article across any  one of your social media platforms. This helps JAMSO extend our reach to others who can benefit with our free information. 

James Doyle - JAMSO

James Doyle - JAMSO

 JAMSO supports people and business. For goal setting, goal management, metrics, key performance indicators, business intelligence and analytics. Our clients are personal and business to business partners.

Further Reading

What went wrong with the 2016 polls - By The Atlantic

The UK Polling Report 

Take a read into this Professor who has called most elections correctly .....