ADESSA:

A Real-Time Decision Support Service for Delivery of Seman:cally Coded Adverse Drug Event Data

Jon D. Duke, MD AMIA Annual Symposium 2010

Overview
Introduc@on to ADESSA Concept and Design Evalua@on Study Results Discussion / Future Work

ADESSA is a web service designed to provide labeled adverse drug event data in coded, structured form

Origins of ADESSA

The Problem

What about the Structured Product Label?

<component> <code code="34084-4" codeSystem="2.16.840.1.113883.6.1" displayName="ADVERSE REACTIONS SECTION/>
<@tle mediaType="text/x-hl7-@tle+xml">ADVERSE REACTIONS</@tle> <text> <paragraph> <content styleCode="bold">Central and Peripheral Nervous System Disorders</content> <content styleCode="italics">Frequent:</content> hypertonia, hypoesthesia; <content styleCode="italics">Infrequent:</content> twitching, confusion, hyperkinesia, ver:go, ataxia, migraine, abnormal coordina:on, hyperesthesia, leg cramps, abnormal gait, nystagmus, hypokinesia; <content styleCode="italics">Rare:</content> dysphonia, coma, dyskinesia, hypotonia, ptosis, choreoathetosis, hyporeexia. </paragraph>

Open Sources of Coded Adverse Event Data

NDFRT
Physiologic eects

SIDER
CO-START Terminology Limited drug set Infrequently updated

Adverse Event Repor@ng System

Spontaneous reports, not labeled data

Op@mal Solu@on
Comprehensive source of labled ADE data MedDRA coded
useful for pharmacovigilance / ADE repor@ng natural phrasings of adverse events, human-readable

SNOMED-CT coded
useful for integra@on with clinical decision support

Web service capable of retrieving data for one or more drugs using standard medica@on iden@ers

System Components
Structured Product Label Informa@on Coder and ExtractoR (SPLICER) Quan@ta@ve Adverse Reac@on Knowledgebase (QuARK) Adverse Drug Event Support Service (ADESSA)

General Overview of SPLICER Architecture

SPL ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~

SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection

ADE Extractor
Processors List MedDRA term identifier Table

MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper

General Overview of SPLICER Architecture

Structured Product Label XML Document HL7 V3 Specica@on Mandated since 2006
SPL
~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~
SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection

ADE Extractor
Processors List MedDRA term identifier Table

MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper

General Overview of how SPLICER Processes SPLs

SPL ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~

SPL Parser
Section Identification XML tag removal Text Normalization Sentence Detection

Cleans and prepares SPL for extrac@on

ADE Extractor
Processors List MedDRA term identifier Table

MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper

General Overview of how SPLICER Processes SPLs

SPL ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~

SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection

ADE Extractor Extracts raw ADEs List from clean SPL Table
Processors MedDRA term sweeper

MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper

General Overview of how SPLICER Processes SPLs

SPL ~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~~

SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection

ADE Extractor
Processors List MedDRA term identifier Table

MedDRA Mapping Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper

Maps raw terms to MedDRA concepts

Database Crea@on
Retrieved 5782 SPLs from DailyMed Used SPLICER to extract and map the ADEs from these labels to MedDRA preferred terms Linked SPLs to RxCUIs using RxNorm
via SPL_SET_ID via NDC codes

Mapped MedDRA terms to SNOMED-CT

via UMLS CUI synonymy

Database Crea@on
Successfully processed 5602 SPLs Extracted 545 602 ADEs Per Label
180 not processed due to XML varia@on 1706 generic medica@ons 22 097 unique raw phrases 8709 unique MedDRA terms Mean ADEs: 70 Median ADEs: 49 Range: 0 to 525

ADESSA Overview
CCD Level 3
PHP/MySQL NuSOAP

ADESSA RxCUI String

Labeled ADEs

RxCUI

MedDRA SNOMED-CT

QuARK

CCD Medica@on Example

<manufacturedMaterial> <code code="312664" codeSystem= "2.16.840.1.113883.6.88" displayName="Promethazine 25 MG Oral Tablet" codeSystemName="RxNorm Concept Unique Iden@er"> <originalText>Promethazine 25 MG Oral Tablet </originalText> <transla@on code="00781183001" codeSystem= "2.16.840.1.113883.6.69" displayName= "PROMETHAZINE HCL" codeSystemName="NDC"> </code> </manufacturedMaterial>

Example XML Response

<Medica@ons> <Medica@on code="312664" codeSystem= "2.16.840.1.113883.6.88 displayName="Promethazine 25 MG Tablet codeSystemName="RxNorm Concept Unique Iden@er"> <LabeledAdverseEvents> <Event code="41931001" codeSystem= "2.16.840.1.113883.6.96" displayName="Abdominal distension" codeSystemName= "SNOMED-CT"/> <Event code="21522001" codeSystem= "2.16.840.1.113883.6.96" displayName="Abdominal pain codeSystemName="SNOMED-CT"/> <Event code="17182001" codeSystem= "2.16.840.1.113883.6.96" displayName="Agranulocytosis codeSystemName="SNOMED-CT/> . . . </LabeledAdverseEvents> </Medica@on> </Medica@ons>

Evalua@on Goals
Assess the accuracy of the adverse event extrac@on process Assess the comprehensiveness and performance of the service using real-world data

NLP Evalua@on
Independent manual review of SPLICER output by a MedDRA specialist For 100 random labels, reviewer marked ADEs
True Posi:ve ADE in both output and label False Posi:ve ADE claimed by SPLICER but either not present in label or not a true adverse event False Nega:ve ADE present in label but not found by SPLICER or not correctly mapped

NLP Evalua@on
Out of 9529 ADEs:
9064 True Posi@ves 465 False Posi@ves 706 False Nega@ves Recall: 92.8% Precision: 95.1% F-Measure: 0.94

Web Service Evalua@on

Passed ADESSA 30 randomly selected anonymized CCDs Measured
accuracy of drug capture (how many drugs were correctly iden@ed and available in QuARK) @me from submission to response

Results: DSS
For 30 CCDs, drug count ranged 4-37, mean 12.2 ADEs successfully returned for 157/181 (88%) of these drugs Mean response @me 620ms Reasons for failure to return results (n=24)
7 due to CCDs without RxCUI codes 17 due to incomplete RxCUI to SPL mappings in QuARK In no case was the target drug missing from the database

Discussion
SPLICER extracted labeled adverse event data with high precision and recall ADESSA returned pa@ent-specic ADE data rapidly upon receipt of a CCD / RxNorm codes Poten@al value in:
adverse event repor@ng (checking against known adverse reac@ons) clinical decision support (associa@ng pa@ent condi@ons with poten@ally causa@ve agents)

Limita@ons
Even with 95% precision, s@ll ~25,000 incorrect asser@ons in database Sta@c set does not account for new and updated labels Encountered some issues with mapping of RxCUIs to SPLs SNOMED concepts may not reect original nuanced ADE concept

Conclusions
Extrac@on via NLP and dissemina@on via DSS can provide coded ADE data for use in CDS Future work includes
Search by specic ADE or ADE category Support for real-@me updates of QuARK using FDAs RSS feed for new / updated labels Broadening of ADESSA for use outside rewall

Thanks! Ques@ons?
Jon Duke jduke@regenstrief.org