Big Impact from Small Data – Part 2
In this, the second part of the latest in a series exclusive to Data Makes Possible, Dr. Kirk Borne, Principal Data Scientist for Booz Allen Hamilton, adds onto his explanation of the value proposition of small data points with a set of real-world examples. If you missed Part 1, read it here.
When we are looking at instances where small data can have a big impact, we need look no further than the far-reaching consequences of such cases in scientific research. We can build beautiful theories to explain the universe, but those theories must stand up against all evidence and new data, no matter how small. Here is an example that I can personally relate to: my academic lineage includes the 19th-century scientist and philosopher Thomas Huxley (who was my academic great-great-great-great-great-great-great grandfather) – he summarized the above scientific dogma in this colorful way:
“There is nothing so tragic as a beautiful theory destroyed by an ugly fact.”
Below are a few cases where small data had a big impact:
Hyper-personalizing Customer Experience
In digital marketing, one of the most important current trends is hyper-personalization and one-to-one marketing to the individual customer. A large collection of customer data is important to find the one thing that will most delight the customer: the right offer, at the right time, in the right context, at the moment of impact. That one small thing (evidence of a specific customer’s unique preference) is the golden ticket to successful marketing campaigns. Small data yields big value!
A Zero Data Tragedy
Sometimes we forget that the smallest unit of data is not one byte. The smallest unit is zero data! My second example is a case where zero data was enormously significant. The tragic story of the explosion of the Space Shuttle Challenger on an extremely cold morning in January 1986 has been used in statistics, decision science, and business management classes for three decades.
One reason for this extraordinary level of academic attention is one of the primary precursors that led to the tragic deaths of all seven astronauts aboard. On the night before launch, the engineers convinced the launch decision-makers that a cold-weather launch should not affect the key engineering component of this enormously complex system: the O-rings that sealed the hot explosive gas propellants within the shuttle propulsion systems.
What they said was that there was no evidence of a failure of the O-rings in cold weather conditions. What they didn’t say was that they never tested the O-rings in cold weather conditions. In this case, the absence of evidence (of O-ring failure in freezing conditions) was misconstrued as evidence (proof) of the absence of failure in such conditions. Zero data had huge consequences!
My third example is from NASA’s deep space probe Voyager 1. The Voyager I spacecraft was launched in 1977 and subsequently conducted fly-by studies (i.e., scientific data collection) at both Jupiter and Saturn from 1979 through 1980. The Voyager I spacecraft trajectory was intentionally programmed to align with those massive planets in such a way that it received a strong gravitational boost – or “slingshot effect” – during the flybys that ultimately sent the probe on its way at the speed of 38,000 miles per hour into deep space.
NASA scientists have monitored data coming from Voyager 1 for over four decades now. Specifically, there is a very low-bandwidth transceiver on the probe that sends back measurements of the high-energy particle flux that the spacecraft measures in its space environment several times each day. The data flow from that magnetometer instrument is very small, probably much less than one kilobyte per day. In August 2012, those tiny data revealed that Voyager I’s situation had suddenly changed. The magnetometer readings dropped dramatically from approximately 25 energetic particles detected per second (that had been the norm for nearly 35 years) to just one or two particles detected per second!
What happened? Well, one of the most significant events in the history of technology and perhaps in all of human history! The sudden decline in particle flux (albeit from very small data) proved without a doubt that the Voyager I probe has exited the heliopause, the region whose plasma physics interactions are dominated by our Sun. The probe has entered true interstellar space, the region between the stars where the Sun has little, if any, influence.
This is not quite as dramatic as “Elvis has left the building”, but it is far more significant – a minuscule stream of data has proven that something of human design and invention has left our Sun’s sphere of influence entirely and is now truly traveling through interstellar space. Small data had a significant impact on our understanding of our place in the universe!
Data makes possible countless innovations, insightful discoveries, and impactful actions (usually beneficial, though not always). The value associated with those results is not dependent on the size of the data. Small data, even zero data in some cases, may represent the long tail of your big data distribution, but it can a leave a long trail of impact that is amplified across organizations through the years. The real key to success from data of any size is to ask the right questions. That’s the essence of cognitive analytics – generating new questions from your data. So here is my response when I am asked about where is big data’s biggest impact:
“When it comes to big impact, Small Data is the new Big Data.”
Now, take your thoughts on the Big Impact from Small Data conversation to Twitter! Agree or disagree with Kirk’s ideas and examples? Want to tell us your story? Tweet @KirkDBorne using the hashtag #datamakespossible right now!
Certain blog and other posts on this website may contain forward-looking statements, including statements relating to expectations in the market for our products and applications of our products. These forward-looking statements are subject to risks and uncertainties that could cause actual results to differ materially from those expressed in the forward-looking statements, including development challenges or delays, changes in markets, demand, global economic conditions and other risks and uncertainties listed in Western Digital Corporation’s most recent quarterly and annual reports filed with the Securities and Exchange Commission, to which your attention is directed. Readers are cautioned not to place undue reliance on these forward-looking statements and we undertake no obligation to update these forward-looking statements to reflect subsequent events or circumstances.