“In God we trust; all others bring data.”
― W. Edwards Deming
Recently, the Walk Free Foundation released their inaugural annual report on global slavery – the Global Slavery Index. It is a detailed report defining what constitutes slavery, estimated number of slaves in the world, a ranking of countries based on estimated slaves, and detailed socio-economic analyses.
What is relevant here is not the content of the report (which is not to say that it is unimportant), but the process and value of measurement and data. Peopleware, an all-time classic for knowledge workers, lays out a two-stage theory of measuring things:
1 – What cannot be measured, cannot be improved
2 – Any measurement is better than no measurement at all
Our decisions are generally based on heuristics, rules of thumb, hunches, gut feelings, personal experiences, cultural norms and our opinions about things. While all of these have their place, they more often than not are developed by circumstances and experiences, rather than data that represents reality. That is why most interventions, whether at policy, strategy or tactical level fail. That is one of the reasons Change Sucks.
Any effective intervention, planning and decision-making has to be based on solid data that is statistically valid. Having valid data available may not be sufficient, but it is certainly necessary.
Lets get back to the Global Slavery Index report. It’s an excellent example of the process and value of having data. You can think about it the process having 4 stages:
1 – Realize there is a need for Measurement
Gina Dafalia, the research manager for Walk Free Foundation, said in an interview:
“When we started working in this area we realized that we didn’t have a good understanding of what exactly the situation of slavery is in the world. We needed that information before we started doing any interventions.”
Appreciating the need to gather data that represents reality, and which can be used to make decisions is the starting point. Without that mind, you are in the gut feeling mode!
2 – Define what needs to be Measured
It is important to be very specific and laser-focused to define upfront what is being measured. Being vague or too generic will result in meaningless measurements. This becomes more important for terms like Slavery, which are overloaded and have different interpretations. This is what the Walk Free Foundation defined:
‘Slavery’ refers to the condition of treating another person as if they were property – something to be bought, sold, traded or even destroyed.
3 – Make Statistically Valid Measurements
Ad-hoc and random measurements will be ineffective. The measurement metrics, methods and sources need to be statistically valid and scientifically defined. For example, focusing more on just one region or specific types of slavery would have made the scope of the report limited and not global. Opinion polls or election surveys are good examples of measurement that are very careful.
The index, which draws on 10 years of research into slavery and was produced by a team of 4 authors supported by 22 other experts. Additionally, it had a margin of error of between 5-10%. We always erred on the conservative side.
— Source CNN report
4 – Interpret the Measurements
Collecting basic data is useful. For example, the report estimated that the total number of slaves worldwide is 30 million. But it is also about collating and deriving useful information from the base data. The real value comes from logical interpretations based on which concrete actions can be taken.
For example, the Global Slavery Index measured that 10 countries of the world accounted for over 75% of slavery worldwide. This helps clearly define where the focus on major interventions should be. It helps us appreciate the Power of the Power Law.
It was also interpreted that the reasons and how slavery manifested itself varied from country to country. Sometimes it varied even within a country. Hence, there cannot be one-size-fit-all solution. You need to understand regional causes and actors to customize your interventions.
Think about any change or improvement you are making – whether at professional or personal level. It may be entering into a new market, determining need for a product, improving your time management or changing a process in your company. Working without data is like shooting in the dark, hoping you hit the target.
“In God we trust; all others bring data.”
― W. Edwards Deming
Very nice thoughts Ather and I loved your last point of ‘Interpreting the measurement’. Lot of times we gather data but fail to get a meaningful information out of it. For example on software development projects, I get to hear a lot of testing reports that are full of numbers. Where as what need in those reports are stories or news so as what that data means.
Thanks Majd! You are right – this tier 2 and 3 level derivative information is where the value lies! Thanks for reading and the feedback.
Good Article Ather. Would like to raise one point. Many a times, “data scientists” are plagued with “confirmation bias”. They are so invested in the idea, problem, task, project or whatever that they fail to see its uselessness sometimes. One of the factors of confirmation bias is definitely the lack of data – and measurements can really improve that situation. But like all matters in life, there is no silver bullet. You can even err towards the confirmation bias with data to support it (which is worse than doing it without data – because then the onus is on your gut feeling). To eleviate that, one needs to wear different hats and act as a devil’s advocate. To always take the data with a grain of salt. To always counter argue the assumptions, focus grouping, hypothesis etc.
Very interesting Zaki. I totally agree that the biases can override the value that data can bring. Being biased or stereotypical will lead you to a conclusion that you want (this is true in your general thought processes too). I was assuming a neutrality and unbiased approach, but I should have stated it explicitly. If you read Malcolm Gladwell’s ‘Blink’, while he argues in favor of an instantaneous and gut feeling decisions (which is not really all hunches, rather rooted in experience and processed by our subconscious processor instantaneously), the only thing he mentions to be mindful about is stereotyping and being driven by biases.
very well written.. and point well taken.
Pingback: Happy Birthday Thinking Spirits! | Thinking Spirits ...