MS | Search arXiv

arXiv2020

Assessment of Amazon Comprehend Medical: Medication Information Extraction

In November 27, 2018, Amazon Web Services (AWS) released Amazon Comprehend Medical (ACM), a deep learning based system that automatically extracts clinical concepts (which include anatomy, medical conditions, protected health information (PH)I, test names, treatment names, and medical procedures, and medications) from clinical text notes. Uptake and trust in any new data product relies on independent validation across benchmark datasets and tools to establish and confirm expected quality of results. This work focuses on the medication extraction task, and particularly, ACM was evaluated using the official test sets from the 2009 i2b2 Medication Extraction Challenge and 2018 n2c2 Track 2: Adverse Drug Events and Medication Extraction in EHRs. Overall, ACM achieved F-scores of 0.768 and 0.828. These scores ranked the lowest when compared to the three best systems in the respective challenges. To further establish the generalizability of its medication extraction performance, a set of random internal clinical text notes from NYU Langone Medical Center were also included in this work. And in this corpus, ACM garnered an F-score of 0.753.

cs.CL↗

arXiv2019

Statistical multiscale mapping of IDH1, MGMT, and microvascular proliferation in human brain tumors from multiparametric MR and spatially-registered core biopsy

We propose a statistical multiscale mapping approach to identify microscopic and molecular heterogeneity across a tumor microenvironment using multiparametric MR (mp-MR). Twenty-nine patients underwent pre-surgical mp-MR followed by MR-guided stereotactic core biopsy. The locations of the biopsy cores were identified in the pre-surgical images using stereotactic bitmaps acquired during surgery. Feature matrices mapped the multiparametric voxel values in the vicinity of the biopsy cores to the pathologic outcome variables for each patient and logistic regression tested the individual and collective predictive power of the MR contrasts. A non-parametric weighted k-nearest neighbor classifier evaluated the feature matrices in a leave-one-out cross validation design across patients. Resulting class membership probabilities were converted to chi-square statistics to develop full-brain parametric maps, implementing Gaussian random field theory to estimate inter-voxel dependencies. Corrections for family-wise error rates were performed using Benjamini-Hochberg and random field theory, and the resulting accuracies were compared. The combination of all five image contrasts correlated with outcome (P<.001) for all four microscopic variables. The probabilistic mapping method using Benjamini-Hochberg generated statistically significant results (P<.05) for three of the four dependent variables: 1) IDH1, 2) MGMT, and 3) microvascular proliferation, with an average classification accuracy of 0.984 +/- 0.02 and an average classification sensitivity of 1.567% +/- 0.967. The images corrected by random field theory demonstrated improved classification accuracy (0.989 +/- 0.008) and classification sensitivity (5.967% +/- 2.857) compared with Benjamini-Hochberg. Microscopic and molecular tumor properties can be assessed with statistical confidence across the brain from minimally-invasive, mp-MR.

physics.med-ph↗