Eye Scans to Detect Cancer and

Alzheimers Disease
By Megan Scudellari
Posted 31 Aug 2017 | 18:30 GMT

Photo: Dennis Wise/University of Washington

It is said that eyes are the window to the soul. Well, what about the window to our health?
We recently reported on two AI systems trained to detect eye diseases, specifically
diabetic retinopathy and congenital cataracts. Now, weve found groups extending that
concept to illnesses beyond the eye. Two new projects pair imaging systems with
advanced software in an effort to catch early symptoms of pancreatic cancer and
Alzheimers disease.
At the University of Washington, a team led by computer scientist Shwetak Patel created
a smartphone app to screen for pancreatic cancer with a quick selfie. Developed over the
last year and a half, the team recently tested their system in a clinical study of 70 people.
They were able to identify cases of concern with 89.7 percent sensitivity and 96.8 percent
accuracy.
While the app is not yet ready to be used as a diagnostic tool based only a single small
trial, it might soon provide a tool for doctors monitor disease progression in patients
undergoing treatment with a simple photograph, rather than a blood test, says first author
Alex Mariakakis, a UW graduate student.
The app, called BiliScreen, monitors levels of bilirubin, a metabolic byproduct that builds
up in cases of jaundice, causing a yellowing of the skin and whites of the eyes. Increased
bilirubin in the blood is one of the earliest symptoms of pancreatic cancer, but it is
currently detected by a blood test in a doctors office and done only for patients at high-
risk or with other symptoms.

Photo: Dennis Wise/University of Washington The BiliScreen app detects levels of

bilirubin in the whites of the eyes.
To create an easy-to-use, non-invasive early screen for pancreatic cancerwhich claimed
the life of study co-author James Taylors fatherthe researchers designed a three-step
system. First, users take a selfie with their smartphone using one of two accessories to
control for environmental conditions: either a cardboard box to block out the light, or
colored glasses to give a color reference. In the study, the team found that the box was
slightly better at controlling light conditions than the glasses.
Once an image is captured, the software relies on computer vision algorithms to isolate
the sclera, or white of the eye, from the skin and pupil. The clinical trial version of the
software was based on an algorithm from Microsoft Research called GrabCut, says
Mariakakis. A newer iteration, which works even better to isolate the sclera, uses a fully
convolutional neural network to identify the most important local features in each image.
You give the algorithm enough cases of labeled imageswith people of different skin
tones, eye colors, and orientation, and it can figure out where the sclera is, he says.
Next, that image information is matched to levels of bilirubin taken from a blood draw
and fed into a machine-learning algorithm to train it to detect images of concern. In the
initial 70-person study, roughly half the participants had elevated levels of bilirubin and
half did not. BiliScreen was able to detect those individuals with high levels with good
accuracy and sensitivity, but both measures could be improved with more data, adds
Mariakakis. The team is now seeking funding for additional clinical trials.
At Cedars-Sinai and NeuroVision Imaging LLC in California, researchers have
developed a sophisticated camera and retinal imaging appraoch to detect early signs of
Alzheimers disease (AD). This system, recently detailed in a proof-of-concept trial
published in the journal JCI Insight, relies on a specialized ophthalmic camera that is not
yet available on a smartphone.
In June 2010, the Cedars-Sinai team, led by Maya Koronyo-Hamaoui and Yosef
Koronyo, made a novel discovery: Beta-amyloid protein deposits, a neurotoxic protein
that builds up in the brains of Alzheimers patients, are also present in the postmortem
retina of such patients and even in early-stage cases. Further investigation has shown that
beta-amyloid deposits in the eye appear in clusters and seem to be more abundant in
certain regions then others in the human retina, says Koronyo-Hamaoui.
She and collaborators at Cedars-Sinai founded a company, NeuroVision, to develop a
way to detect and quantify those plaques in the eye, in the hopes of being able to see early
signs of the disease before cognitive decline becomes obvious. Today, their system
consists of a modified scanning laser ophthalmoscope with a special filter to capture a
fluorescence signal emitted by the plaques when tagged with a marker, and advanced
software to process the images.
In the JCI Insight study, the team used their imaging system on 16 live patients and the
donated eyes and brains of 37 deceased patients, 23 of whom were confirmed with
Alzheimers disease. Overall, they found a 4.7-fold increase in the post-mortem retinas of
patients with AD compared to controls.
The next step is larger scale studies to see if the method is practical for diagnosing
AD. The group is conducting additional clinical trials in the U.S., Europe, and Australia,
Koronyo-Hamaoui told IEEE Spectrum in an email. We are testing larger cohorts of
living patients for the possible relationship between retinal amyloid index and the gold
standard amyloid-PET brain imaging and other AD biomarkers.
She believes that the imaging could someday be adapted to less expensive cameras, but
currently a doctors office would require a high-definition camera and image processing
and quantification tools in order to use the system.
AI Predicts Heart Attacks and Strokes
More Accurately Than Standard Doctor's
Method
By Eliza Strickland
Posted 1 May 2017 | 17:00 GMT

Photo-illustration: iStockphoto
Here at The Human OS, we are slightly obsessed with matchups between artificial
intelligence and doctors.
In many experiments (though not yet in many clinics), AI systems are showing great
promise in diagnosing diseases, analyzing medical images, and predicting health
outcomes. Theyve even performed better than human doctors in certain tasks like
surgical stitching and diagnosing autism in infants.
Now, in the latest win for AI medicine, researchers at the University of Nottingham in
the UK created a system that scanned patients routine medical data and predicted which
of them would have heart attacks or strokes within 10 years. When compared to the
standard method of prediction, the AI system correctly predicted the fates of 355 more
patients.
Predicting these cardiovascular events is a notoriously difficult task. In a recent paper,
published in the journal PLOS One, the researchers note that about half of all heart attacks
and strokes occur in people who havent been flagged as at risk.
The leap from research studies to applications in clinical care will happen over the next
five years. Stephen Weng, University of Nottingham
Currently, the standard way of assessing a patients risk relies on guidelines developed
by the American Heart Association and American College of Cardiology. Doctors use
these guidelines, which focus on well-established risk factors such as high blood pressure,
cholesterol, age, smoking, and diabetes, to shape their counsel and treatment for their
patients.
To make a system that could do better, researcher Stephen Weng and his colleagues tested
several different machine learning tools on medical records from 378,256 patients across
the UK. These records tracked the patients and their health outcomes from 2005 to 2015,
and contained information on demographics, medical conditions, prescription drugs,
hospital visits, lab results, and more.
The researchers took 75 percent of the medical records and fed them into their machine
learning models, which set out to find the distinguishing characteristics of those patients
who experienced heart attacks or strokes within the 10-year span. Then Wengs group
tested the models on the other 25 percent of the records to see how accurately theyd
predict heart attacks and strokes. They also tested the standard guidelines on that subset
of records.
Using a statistic in which a score of 1.0 signifies 100 percent accuracy, the standard
guidelines got a score of 0.728. The machine learning models ranged from 0.745 to 0.764,
with the best score coming from a type of machine learning model called a neural
network.
While the machine scores may not sound like a resounding triumph, when translated into
human terms the significance becomes clear: The neural network model predicted 4,998
patients who went on to have a heart attack or stroke out of 7,404 actual cases355 more
than the standard method. With those predictions in hand, doctors could have taken
preventative measures such as prescribing drugs to lower cholesterol.
Weng says the AI medical tools being tested in labs today will soon boost clinicians
accuracy in both diagnosis and prognosis. The leap from research studies to applications
in clinical care will happen over the next five years, he says.
What might that look like in practice? Weng pictures busy primary care doctors using AI
tools that have been trained to recognize patterns. Then the algorithm can look through
the entire patient list, flag this up, and bring this to the attention of the doctor, he says.
This could be done with the patient sitting in front of them during a routine appointment,
or in a systematic screen of the entire list. While Weng notes that similar clinical decision
support software already exists, he says those systems dont make use of AI pattern
recognition, which could provide far more accurate results.
Before AI comes to your doctors office, however, the technology will have to get past
major regulatory hurdles. The key barrier to implementation will be managing privacy
and patient confidentiality issues, with computer algorithms trawling through vast
amounts of patient data which contain confidential and sensitive medical information,
Weng says.
In addition to coping with those privacy concerns, any AI technology will have to deal
with regulators wariness of medical machines that make their own decisions. With all
that red tape looming, one wonders: What would a machine learning tool predict about
its own chances of gaining approval?
Smartphone Heart Monitor Beats Doctors
at Diagnosing Atrial Fibrillation
By Eliza Strickland
Posted 21 Sep 2017 | 13:00 GMT

Photo: AliveCor
Doc, hows my ticker? In olden days (like 5 years ago), patients went to their general
practitioners with such anxious inquiries, looking for cardiac checkups and reassurances.
These days, consumers can buy a heart monitor thats about the size of a piece of gum
and stick it on the back of their smartphone, then check their heart rate as often as they
like.
But how good are the results from such a consumer gadget? A new study that tested the
Kardia Mobile heart monitor, made by AliveCor, found that the device detected more
cases of the dangerous heart condition atrial fibrillation than general practitioners offering
routine care.
For the study, published in the journal Circulation, researchers in Wales enrolled 1000
patients aged 65 and above. Half of the patients received routine medical care from
general practitioners, the other half were instructed to use the Kardia monitor twice a
week to take 30-second recordings of their heart rate. During the 1-year study, the Kardia
diagnosed 19 patients while the doctors diagnosed 5.
To use the Kardia device, the user presses two fingers of each hand against an electrode
pad. The device records an electrocardiogram (ECG or EKG) by detecting the subtle
electrical changes in the skin that reflect the heart muscles electrical pattern. The display
shows users their heart rate, and the device can also upload the results to the cloud for a
doctors review. AliveCor CEO Vic Gundotra notes that the Kardia is FDA cleared: This
is not a Fitbit, this is a clinical grade device, so you can be confident in its results, he
says.

Photo: AliveCor With the Kardia Mobile, users can take 30-second electrocardiograms
to determine whether their heart rate is normal and healthy.
Atrial fibrillation (AFib) is a condition where the heartbeat becomes erratic and the upper
chambers of the heart quiver and shake. If theres a lurking blood clot, this shaking can
break it loose and send it up through the blood vessels to the brain, where it can cause a
stroke. People with AFib have a 5 times greater likelihood of having a stroke, and doctors
typically put patients with the condition on blood-thinning medications that prevent clots.
But the condition can go undiagnosed for years, says AliveCors Gundotra, because about
40 percent of people with AFib dont exhibit any symptoms. Unless you think you have
a problem or your doctor thinks you have a problem, you wont be checked, he says.
In routine medical care, a patient whos suspected of having AFib is referred to a
cardiologist for an ECG recording, in which 12 electrodes are placed on the patients chest
and limbs. But even patients who are thought to be at risk of AFib might only be checked
by a cardiologist once or twice a year. Those infrequent tests might not diagnose patients
who have a form of the disorder called paroxysmal AFib, in which the heart arrhythmia
comes and goes.
Of the 19 patients diagnosed with AFib via the Kardia device in the new study, 12 had
paroxysmal Afib, and 8 had no symptoms at the time of diagnosis. All 19 were started on
blood-thinning medications. The study wasnt intended to compare clinical outcomes
between the two patient groups, but the authors write that a future study should examine
whether remote ECG monitoring leads to fewer strokes.
Photo: AliveCor The Kardia Mobile can be stuck to the back of a cellphone or tablet.
For the study, the twice-weekly ECGs recorded by patients were analyzed automatically
by AliveCors software. But the paper notes that abnormal ECGs were sent to a
cardiologist for review, and also says that the Kardia produced some false positives
(indications of AFib that the reviewing cardiologists found to be false). For now, at least,
there's still a need for a human in the loop.
The study was led by Julian Halcox, chair of cardiology at the Swansea University
Medical School in Wales, who did not respond to a request for comment on the research.
Halcox has no financial relationship with AliveCor.
While the Kardia device got good marks from the studys participants for being easy to
use, even this small device could be rendered obsolete by future gadgetry. Apple has
announced that its Apple Watch will soon sport a better built-in heart rate monitor and
app, and Stanford researchers plan to test the Apple gear soon for AFib detection. Before
long, it may be common for people to wear their hearts on (well, under) their sleeves.