dark data is helping diagnose diabetic retinopathy

Dark Data is Helping Diabetics Retain Their Sight

At any moment, if they aren’t receiving proper treatment, a person with diabetes could go blind. Leveraging medical dark data could save them from losing their eye sight.

Diabetic retinopathy is a condition that causes progressive damage to the retina. It’s also the most common causeof vision impairment and blindness in working-age adults in the U.S. Anyone with diabetes—a disease that afflicts about one in tenAmericans—can develop it.

dark data is helping diagnose diabetic retinopathy

Ophthalmologists are the first line of defense against diabetic retinopathy. These eye doctors observe images of the eye hoping to spot visible hemorrhages. But it’s a process prone to human error and limited by human speed. Thousands of images are required for a truly thorough inspection. So even with hours of review, a doctor could still miss the tiny nuances within a given set of images that indicate a hemorrhage.

Add to this that ophthalmologists see 5,000 patients a yearon average. If a computer could handle and analyze even a small amount of the volume of dark data that these doctors collect and look at themselves, it would make a big difference.

“We rely on data all the time to improve patient outcomes, surgical technique, the flow of the patient journey to help enhance, record and minimize [patient time] in order to run an efficient practice,” says Dr. Cynthia Matossian, an ophthalmologist and founder of Matossian Eye Associates.

What is Dark Data?

The images these doctors examine are part of a growing set of data points available to healthcare professionals known as dark data because they produce information that goes uncategorized. In fact, research shows that 80 percentof health data is uncategorized, or “dark data,” and includes anything from medical staff notes, to scanned documents, to images of the eye.

dark data is helping diagnose diabetic retinopathy

Researchers believe there’s a wide breadth of untapped potential in unstructured dark data and are working hard to figure out how to corral and analyze it effectively. Dr. Matossian and her team are constantly inputting data into their electronic medical records, but she says their ability to extract information from all this data is primitive.

“There is a disconnect between what we can get out that’s customizable, that is real time, that we can use, versus the amount of time and effort and costs that we spend to input data,” she says.

But she and other doctors have a new tool in the fight: artificial intelligence (AI).

AI is at the forefront of emerging technologies that can make sense of unstructured dark data points, and spot patterns that humans might miss. In the case of diabetic retinopathy, these advancements could mean the difference for a diabetic patient on the verge of vision impairment.

Dark Data and AI Team Up to Shorten Diagnosis Time

In 2016, a health technology company in Nashville, Tenn. tested its artificial intelligence software’s ability to diagnose diseases, including the use of dark data on diabetic retinopathy.4

dark data is helping diagnose diabetic retinopathyThe company trained its computers to read eye images and diagnose different stages of diabetic retinopathy. They approached it as a five-class identification problem—four disease states and one disease-free state—based on the industry standard for ranking a patient’s risk.

The software analyzed dark data from half a million eye images, then compared those results with physician diagnoses of the same images. The result was a notable improvement in the quality and consistency of diagnosis.

Perhaps even more significantly, the computers made incredible time—processing and categorizing five years of clinical study data in just 24 hours.

Dark Data Has Many Healthcare Applications

Training computers to analyze and categorize images is just one way medical experts hope to transform unstructured dark data into something more useful for patients, doctors and researchers.

Emerging technologies like machine learning and deep learning—subsets of artificial intelligence—can, for example, analyze dark datasets of conversations between a doctor and a patient to look for patterns that could lead to more comprehensive treatment options.

Researchers can also train software to detect patterns present in drug molecules to understand how they react with body chemistry—helping get drugs to market faster than ever before.

Using Dark Data to Combat Prevalent Disease

Focusing on specific diseases is one way the industry is tackling the massive amounts of dark data in healthcare. Another healthcare company, Medtronic, based in Minneapolis, is also looking at how hidden information in dark data can help diabetic patients.

“Diabetes is a complex disease where the majority of the management is done by the patient in between doctor’s visits. People who live with diabetes spend a lot of mental energy tracking and thinking about their actions and how it will impact their diabetes,” says Pratik Agrawal, Director of Data Science and Informatics Innovation at Medtronic Diabetes.

In 2016, the company launched a cognitive app, Sugar.IQ, that uses analytics to detect patterns in data gleaned from patients’ insulin pumps and glucose sensors. This continuous monitoring of personal data can provide insights to patients and their doctors, such as whether someone is in danger of hypoglycemia.

“The Sugar.IQ assistant continually analyzes how an individual’s glucose levels respond to their food intake, insulin dosages, daily routines, and other factors,” says Agrawal.

The product became commercially available this summer but testing ahead of the launch revealed that those using the app increased their time in a healthy glucose range by an average of 36 minutes per day.Patients were also able to prevent high and low glucose levels, rather than reacting to them when they happen.

dark data is helping diagnose diabetic retinopathy

We have more health data, and access to it, than ever before. But doctors can’t unlock its full potential without tools to tap into the myriad images, notes, and other unstructured data formats hidden in our electronic records. These companies are making promising inroads into showing how technology can be used to illuminate dark data for a healthier population in the future.

how is artificial intelligence solving medical data problem

This content is produced by WIRED Brand Lab in collaboration with Western Digital Corporation.


  1. Common cause of blindness in diabetics
  2. About one in ten patients
  3. 5000 patients a year
  4. 80% healthcare data goes unanalyzed
  5. Nashville, TN health company tests AI
  6. Increased healthy glucose range by 36 minutes per day