Skip to main content

Blog

Learn About Our Meetup

5000+ Members

GO >

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOIN

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

JOBS

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

CONTACT

Category: Global

Error Parer: How AI Could Help Cardiologists Detect Heart Defects Without Skipping a Beat

Written on . Posted in NVIDIA.

Nearly a third of physicians will be sued at least once in their careers — most commonly for an error in diagnosis. Medical errors are also the third-leading cause of death in the United States, according to a study by Johns Hopkins Medicine.

Deep learning has the potential to help doctors cut down on diagnostic errors, said cardiologist Rima Arnaout in a talk at the GPU Technology Conference.

An assistant professor of medicine at the University of California, San Francisco, Arnaout is focusing on the potential of AI to analyze cardiac ultrasounds and detect congenital heart disease from fetal ultrasounds.

“In medicine, a picture is worth more than a thousand words,” she said. “It really can be worth a patient’s life, in some cases.”

What Can AI Do to Help?

Arnaout outlined a few key challenges for humans analyzing medical images. For one, people sometimes make mistakes. There’s also a physical limit to how much data cardiac imaging specialists like cardiologists and radiologists can analyze.

“We cannot allow those kinds of shortcomings,” she said. “We need accuracy, precision, and we need it delivered at scale.”

While AI models are not without their limitations, Arnaout said, they can help clinicians use medical imaging techniques like ultrasound to their full potential.

She turned to echocardiogram data because “it’s balanced in terms of information richness and clinical volume compared to other cardiovascular imaging tools.” Since echocardiograms can be used for the diagnosis and management of almost every cardiovascular disease, she said, there’s very little selection bias in the datasets.

Echocardiograms are a challenging training dataset, however, because one ultrasound study consists of still images and videos captured from over a dozen angles. A study Arnaout’s team published in npj Digital Medicine used deep learning to classify 15 of these standard views, achieving 91.7 percent accuracy on low-resolution images.

Detecting Congenital Heart Disease in Utero

Recent work by Arnaout focused on the detection of congenital heart disease from fetal ultrasounds. While in theory more than 90 percent of complex congenital heart disease cases can be diagnosed through traditional fetal screening ultrasounds, the actual detection rate is under 50 percent.

This gap occurs in part because fetal hearts are small and fast beating. Since the fetus itself is often moving, diagnostic-quality images can be difficult to obtain. And although it’s the most common birth defect, congenital heart disease affects just one percent of live births. Since it’s so rare, the condition can easily go overlooked by human readers.

That’s where an algorithm can help: once trained, it could reliably catch congenital heart disease in perpetuity.

Using hundreds of fetal echocardiograms from 18 to 24 weeks of gestational age, the UCSF researchers developed convolutional neural networks to distinguish two varieties of congenital heart disease. The deep learning model, trained on NVIDIA GPUs hosted on Amazon Web Services, was able to classify the two kinds of defects at well above the average diagnostic rate.

Catching heart defects early can lead to better outcomes for patients after birth. And if certain types of lesions are spotted in a fetal ultrasound, doctors can recommend in-utero therapies that significantly improve the heart’s condition by birth.

Arnaout said, “This has the potential to really affect the natural history of an entire life.”

 

Main image is of an echocardiogram showing a ventricular septal defect, or hole in the heart — a common congenital heart defect. Photo by Kjetil Lenes/Ekko, licensed under public domain on Wikimedia Commons.

The post Error Parer: How AI Could Help Cardiologists Detect Heart Defects Without Skipping a Beat appeared first on The Official NVIDIA Blog.

Seeing Stars: Astronomers Turn to AI to Track Galaxies as New Telescopes Come Online

Written on . Posted in NVIDIA.

Good news: astronomers are getting new tools to let them see further, better than ever before. The bad news: they’ll soon be getting more data than humans can handle.

To turn the vast quantities of data that will be pouring out of these instruments into world-changing scientific discoveries, Brant Robertson, a visiting professor at Princeton’s Institute for Advanced Studies and an associate professor of astronomy at UC Santa Cruz, is turning to AI.

“Astronomy is on the cusp of a new data revolution,” he said told a packed room at this week’s GPU Technology Conference in Silicon Valley.

Better Eyes on the Sky

Within a few years the range of instruments available to the world’s star-gazers will give them once unimaginable capabilities. Measuring an enormous 6.5 meters across, the James Webb Space Telescope — which will be deployed by NASA, the U.S. space agency, will be sensitive enough to give us a peek back at galaxies formed just a few hundred million years after the Big Bang.

The Large Synoptic Survey Telescope gets less press, but it has astronomers equally excited. The telescope, largely funded by the U.S. National Science Foundation and the Department of Energy, and being built on a mountaintop in Chile, will let astronomers survey the entire southern sky every three nights. This will produce a massive amount of data — 10 terabytes a night.

The Large Synoptic Survey Telescope, on Cerro Pachón, in Chile, will give astronomers the ability to survey the entire southern sky every three nights when it is completed in 2020.

Finally, the Wide Field Infrared Survey Telescope puts an enormous digital camera into space. With origins in the U.S. spy satellite program, the satellite’s features will include a 288-megapixel, multi-band, near-infrared camera with a field of view 100x larger than that of the Hubble Space Telescope.

‘Richly Complex’ Data

Together, these three instruments will generate vast quantities of “richly complex” data, Robertson said. “We want to take that information and learn as much as we can,” he said. “Both from individual pixels and by aggregating them together.”

It’s a task far too large for humans alone. To keep up, Robertson is turning to AI. Created by Ryan Hausen, a Ph.D. student in UC Santa Cruz’s computer science department, Morpheus — a deep learning framework classifies astronomical objects, such as galaxies, based on the raw data streaming out of telescopes — such as the Hubble — on a pixel by pixel basis.

“In astronomy, we really do care about the technological advances that people in this room are engineering,” Robertson told his audience at GTC.

Translation: to find new stars in outer space, this prominent astrophysicist is looking, first, to deep learning stars here on Earth for help.

Image credit: NASA. 

The post Seeing Stars: Astronomers Turn to AI to Track Galaxies as New Telescopes Come Online appeared first on The Official NVIDIA Blog.

Developers, start your engines and get ready to race in the 2019 AWS DeepRacer League

Written on . Posted in Amazon.

Get ready to take the pole position in the AWS DeepRacer League. Today, we’re excited to bring you the next stage in the AWS DeepRacer journey – the AWS DeepRacer League 2019.

In November 2018, Jeff Barr announced the launch of AWS DeepRacer on the AWS News Blog as a new way to learn machine learning (ML). With AWS DeepRacer, developers have an opportunity to get hands-on with a fully autonomous 1/18th scale race car driven by reinforcement learning, a 3D racing simulator, and a global racing league.

Thousands of developers joined the live action at the race tracks at AWS re:Invent 2018, racing for glory and prizes, and to win the coveted AWS DeepRacer Championship Cup. The 2018 reigning Champion is Rick Fish, co-founder of UK-based FinTech startup Jigsaw XYZ. It was a close race, but Rick rose victorious with a winning lap time of 50.51 seconds. You can check out the final race and also a recent interview with Rick where he gave us an update on what he and his race team are doing to prepare to defend the title in the 2019 Championship, on deepracerleague.com.

Now it’s your turn to get on the track and learn ML with AWS DeepRacer. There are two ways to participate in the AWS DeepRacer League 2019: the Summit Circuit and the Virtual Circuit. Winners and top point scorers will be heading on an expenses-paid trip to race in the final stages of competition at AWS re:Invent 2019 in Las Vegas.

Summit Circuit – Bringing the race to a city near you

You can join us at any of the 20 AWS Summits, globally, where we help you build and train a model at a workshop, or you can bring a model you have trained at home. You can then put your model to the test and compete on the track in the AWS Summit Expo. There are no limits on how many of the 20 Summit events you can participate in. Simply register for the Summit you want to race at, and we will see you there. Check out the Summit Circuit schedule here.

Virtual Circuit – Compete from anywhere in the world

The Virtual Circuit opens up the race to anyone, anywhere. You can build models and compete online in the league through the AWS DeepRacer console. The virtual races will take place monthly on new, increasingly challenging tracks, and are open to all levels of expertise. Sign up for the preview to get on the list for early access. The first event of the Virtual Circuit will launch soon, stay tuned for updates on deepracerleague.com.

Whether you are new to machine learning or ready to build on your existing skills, we can help you get ready to race. Developers with no prior machine learning experience can get started by watching this AWS DeepRacer Tech Talk to get familiar with the basics of reinforcement learning (a branch of machine learning that’s ideal for training autonomous vehicles) and AWS DeepRacer. For the early adopters out there who are already comfortable with reinforcement learning, you can dive in and build an AWS DeepRacer model using the Amazon SageMaker RL notebook.

Come join us at one of the Summits or on the Virtual Circuit and get in on the action. It could be you lifting the Championship Cup in 2019. You can also follow the action online at deepracer.com and on Instagram: awsdeepracer.

Developers, the race is on!

About the Author

Sally Revell is a Principal Product Marketing Manager for AWS DeepLens. She loves to work on innovative products that have the potential to impact people’s lives in a positive way. In her spare time, she loves to do yoga, horseback riding and being outdoors in the beauty of the Pacific Northwest.

 

 

 

 

AI and Clinicians a ‘Winning Combination,’ Healthcare Luminary Eric Topol Says at GTC

Written on . Posted in NVIDIA.

Deep learning is putting the “care” back in healthcare.

“It used to be we just talked about deep sequencing. Now we talk about deep everything in medicine,” healthcare luminary Eric Topol told a packed audience Tuesday morning at the 10th annual GPU Technology Conference, in San Jose.

With advances in AI and healthcare, he said, medical professionals will need to spend less time entering and looking at data on computers. This will give them “the gift of time” to provide patients with personal care and bring back the strong doctor-patient bond that existed decades ago.

“A common enemy of the patient and the doctors and the nurses is the keyboard, because it interferes with their relationship,” he said. “It makes doctors data clerks.”

The founder and director of the Scripps Research Translational Institute, Topol released last week a new book, “Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again.” His talk outlining how AI will transform everything doctors do was followed by a book signing.

“Every single type of health professional” will be impacted by AI, Topol said. Bringing AI into the medical workflow can help healthcare institutions provide “better, faster, cheaper” care by augmenting what clinicians can do.

After the talk, Topol signed copies of his new book, “Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again.”

“When you put the two together — machine algorithm plus the radiologist — you start getting a really winning combination.”

But humans will always remain integral to healthcare.

“We’re not going to get to the point where all medical diagnosis will not require human backup. Ever,” he said. “But we may get to a point where some of them, routine things like a sore throat or an ear infection or skin rash can be done completely algorithmically — both diagnosis and recommendations for treatment.”

Scripps Translational Institute focuses on genomics, a field that “is starting to go medical mainstream. Finally,” Topol said. NVIDIA and Scripps recently established a center of excellence for AI in genomics and digital sensors.

Topol showed initial results of the joint work, which included deep learning applications for genomics that improve accuracy, decrease costs and produce results faster.

“I said at the time that eventually, eventually, it would markedly improve accuracy, efficiency and workflow,” he said. “But I didn’t realize that just five months later, we’d do that. I thought it was going to take years.”

Healthcare at GTC

GTC features more than 40 healthcare sessions with innovators in AI and medicine, including:

  • Brandon Fornwalt, associate professor and chair of the imaging science and innovation department at Geisinger, and Aalpen Patel, chair of Geisinger System Radiology
  • Sunita Chandrasekaran, assistant professor at the University of Delaware
  • Dima Rekesh, senior distinguished engineer, and Julie Zhu, chief data scientist and distinguished engineer, at Optum — the health services platform of UnitedHealth Group
  • Rima Arnaout, assistant professor, and Christopher Hess, professor and chair of radiology and biomedical imaging, at the University of California, San Francisco
  • Neil Tenenholtz, director of machine learning at the MGH & BWH Center for Clinical Data Science
  • Tessa Cook, assistant professor of radiology at Penn Medicine
  • Richard Tobias, CEO of Cephasonics Ultrasound Solutions
  • Gerald Quon, assistant professor at the University of California, Davis

The full lineup of healthcare speakers at GTC, which runs through March 21, is available here.

The post AI and Clinicians a ‘Winning Combination,’ Healthcare Luminary Eric Topol Says at GTC appeared first on The Official NVIDIA Blog.

Reducing the Need for Labeled Data in Generative Adversarial Networks

Written on . Posted in Google.

Posted by Mario Lučić, Research Scientist and Marvin Ritter, Software Engineer, Google AI Zürich

Generative adversarial networks (GANs) are a powerful class of deep generative models.The main idea behind GANs is to train two neural networks: the generator, which learns how to synthesise data (such as an image), and the discriminator, which learns how to distinguish real data from the ones synthesised by the generator. This approach has been successfully used for high-fidelity natural image synthesis, improving learned image compression, data augmentation, and more.

Evolution of the generated samples as training progresses on ImageNet. The generator network is conditioned on the class (e.g., “great gray owl” or “golden retriever”).

For natural image synthesis, state-of-the-art results are achieved by conditional GANs that, unlike unconditional GANs, use labels (e.g. car, dog, etc.) during training. While this makes the task easier and leads to significant improvements, this approach requires a large amount of labeled data that is rarely available in practice.

In “High-Fidelity Image Generation With Fewer Labels“, we propose a new approach to reduce the amount of labeled data required to train state-of-the-art conditional GANs. When combined with recent advancements on large-scale GANs, we match the state-of-the-art in high-fidelity natural image synthesis using 10x fewer labels. Based on this research, we are also releasing a major update to the Compare GAN library, which contains all the components necessary to train and evaluate modern GANs.

Improvements via Semi-supervision and Self-supervision
In conditional GANs, both the generator and discriminator are typically conditioned on class labels. In this work, we propose to replace the hand-annotated ground truth labels with inferred ones. To infer high-quality labels for a large dataset of mostly unlabeled data, we take a two-step approach: First, we learn a feature representation using only the unlabeled portion of the dataset. To learn the feature representations we make use of self-supervision in the form of a recently introduced approach, in which the unlabeled images are randomly rotated and a deep convolutional neural network is tasked with predicting the rotation angle. The idea is that the models need to be able to recognize the main objects and their shapes in order to be successful on this task.

An unlabeled image is randomly rotated and the network is tasked with predicting the rotation angle. Successful models need to capture semantically meaningful image features which can then be used for other vision tasks.

We then consider the activation pattern of one of the intermediate layers of the trained network as the new feature representation of the input, and train a classifier to recognize the label of that input using the labeled portion of the original data set. As the network was pre-trained to extract semantically meaningful features from the data (on the rotation prediction task), training this classifier is more sample-efficient than training the entire network from scratch. Finally, we use this classifier to label the unlabeled data.

To further improve the model quality and training stability we encourage the discriminator network to learn meaningful feature representations which are not forgotten during training by means of an auxiliary loss we introduced previously. These two advancements, combined with large-scale training lead to state-of-the-art conditional GANs for the task of ImageNet synthesis as measured by the Fréchet Inception Distance.

Given a latent vector the generator network produces an image. In each row, linear interpolation between the latent codes of the leftmost and the rightmost image results in a semantic interpolation in the image space.

Compare GAN: A Library for Training and Evaluating GANs
Cutting-edge research on GANs is heavily dependent on a well-engineered and well-tested codebase, since even replicating prior results and techniques requires a significant effort. In order to foster open science and allow the research community benefit from recent advancements, we are releasing a major update of the Compare GAN library. The library includes loss functions, regularization and normalization schemes, neural architectures, and quantitative metrics commonly used in modern GANs, and now supports:

Conclusions and Future Work
Given the growing gap between labeled and unlabeled data sources, it is becoming increasingly important to be able to learn from only partially labeled data. We have shown that a simple yet powerful combination of self-supervision and semi-supervision can help to close this gap for GANs. We believe that self-supervision is a powerful idea that should be investigated for other generative modeling tasks.

Acknowledgments
Work conducted in collaboration with colleagues on the Google Brain team in Zürich, ETH Zürich and UCLA. We would like to thank our paper co-authors Michael Tschannen, Xiaohua Zhai, Olivier Bachem and Sylvain Gelly for their input and feedback. We would like to thank Alexander Kolesnikov, Lucas Beyer and Avital Oliver for helpful discussion on self-supervised learning and semi-supervised learning. We would like to thank Karol Kurach and Marcin Michalski for their major contributions to the Compare GAN library. We would also like to thank Andy Brock, Jeff Donahue and Karen Simonyan for their insights into training GANs on TPUs. The work described in this post also builds upon our work on “Self-Supervised Generative Adversarial Networks” with Ting Chen and Neil Houlsby.

De-identify medical images with the help of Amazon Comprehend Medical and Amazon Rekognition

Written on . Posted in Amazon.

Medical images are a foundational tool in modern medicine that enable clinicians to visualize critical information about a patient to help diagnose and treat them. The digitization of medical images has vastly improved our ability to reliably store, share, view, search, and curate these images to assist our medical professionals. The number of modalities for medical images has also increased. From CT scans to MRIs, digital pathology to ultrasounds, there are vast amounts of medical data collected in medical image archives.

These medical images are also useful for medical research. Using machine learning, scientists at global medical research institutions can analyze hundreds of thousands or millions of images to deepen their insight into medical issues. The challenge for health professionals is how to use these images while complying with regulatory obligations like the Health Information Portability and Accountability Act (HIPAA). Often, medical images contain Protected Health Information (PHI) stored as text within the image itself. This has historically presented a challenge because the process of removing PHI, called de-identifying, required the manual review and editing of images. This manual process can easily take many minutes per image and makes de-identifying large datasets time consuming and expensive.

In 2017, Amazon Web Services (AWS) announced the ability to easily detect and extract text from images using our machine learning service Amazon Rekognition. In 2018, we announced a new machine learning Natural Language Processing (NLP) service for medical text called Amazon Comprehend Medical that can help customers to detect and identify PHI in a string of text. You can use these two services, plus some Python code, as demonstrated in this blog post, to inexpensively and quickly detect, identify, and redact PHI from within medical images.

De-identification architecture

In this example, we will use the Jupyter Notebooks feature of Amazon SageMaker to create an interactive notebook with Python code. Amazon SageMaker is an end-to-end machine learning platform that enables you to prepare training data and build machine learning models quickly using pre-built Jupyter notebook with pre-built algorithms. In this blog post, for the actual machine learning and prediction, we will be using Amazon Rekognition to extract text from the images and Amazon Comprehend Medical to help us to identify and detect the PHI. All of our image files will be read from and written to a bucket in Amazon Simple Storage Service (Amazon S3), an object storage service that offers industry-leading scalability, data availability, security, and performance.

When using Amazon Comprehend Medical to detect and identify protected health information, note that the service provides confidence scores for each identified entity that indicate the level of confidence in the accuracy of the detected entity. You should take these confidence scores into account and review identified entities to make sure they are correct and appropriate for your use case.  For more information about confidence scores, see the Amazon Comprehend Medical documentation.

Using the notebook

You can download the Jupyter Notebook that supports this blog post from GitHub.

This notebook shows an example chest x-ray image from a dataset made available by the NIH Clinical Center. The dataset can be downloaded from this link. See the NIH Clinical Center’s CVPR 2017 paper for more information.

At the very beginning of the notebook, you will see 5 parameters you can adjust to control the de-identification process outlined in this example.

  • bucket defines the Amazon S3 bucket where images will be read from and written to.
  • object defines the identified image that you want to de-identify. These can be PNG, JPG, or DICOM images. If the object ends with the extension .dcm, then the image is assumed to be a DICOM image, and the ImageMagick utility will be used to convert it to PNG format before processing it.
  • redacted_box_color defines the color that will be used to cover up identified PHI text within the image.
  • dpi defines the dpi setting that will be used in the output image.
  • phi_detection_threshold is the threshold for the confidence score mentioned earlier (between 0.00 and 1.00). Text detected and identified by Amazon Comprehend Medical must meet the minimum confidence score you set to be redacted from the output image. The default value of 0.00 will redact all text that Amazon Comprehend Medical has detected an identified as PHI, regardless of the confidence score.
#Define the S3 bucket and object for the medical image we want to analyze.  Also define the color used for redaction.
bucket='yourbucket'
object='yourimage.dcm'
redacted_box_color='red'
dpi = 72
phi_detection_threshold = 0.00

After these parameters are set, you can execute all of the cells in the Jupyter Notebook. The first cell converts the image file you specified from DICOM format to PNG, if necessary, and then reads the resulting file from S3 into memory.

#If the image is in DICOM format, convert it to PNG
if (object.split(".")[-1:][0] == "dcm"):
    ! aws s3 cp s3://{bucket}/{object} .
    ! mogrify -format png {object.split("/")[-1:][0]} {object.split("/")[-1:][0]}.png
    ! aws s3 cp {object.split("/")[-1:][0]}.png s3://{bucket}/{object}.png
    object=object+'.png'
    print(object)
…
#Download the image from S3 and hold it in memory
img_bucket = s3.Bucket(bucket)
img_object = img_bucket.Object(object)
xray = io.BytesIO()
img_object.download_fileobj(xray)
img = np.array(Image.open(xray), dtype=np.uint8)

From there, the image can be sent to Amazon Rekognition for text detection using the DetectText feature.  Amazon Rekognition returns a JSON object that contains a list of the text blocks detected within the image, as well as a bounding box that defines each text block, letting you know where the text is located within the image.

response=rekognition.detect_text(Image={'Bytes':xray.getvalue()})
textDetections=response['TextDetections']

After we have all of the text detected within the image, we can send that text to Amazon Comprehend Medical to determine which text blocks may contain PHI using the DetectPHI feature. Amazon Comprehend Medical then returns a JSON object containing the entities that may be PHI, a type that defines what kind of information it believes the text to be (name, date, address, ID), and a confidence score for each detection. We can use this to determine which bounding boxes contain PHI.

philist=comprehendmedical.detect_phi(Text = textblock)

After we know which areas of the image might contain PHI text, we can draw redacting boxes over those areas.

for box in phi_boxes_list:
    #The bounding boxes are described as a ratio of the overall image dimensions, so we must multiply them by the total image dimensions to get the exact pixel values for each dimension.
    x = img.shape[0] * box['Left']
    y = img.shape[1] * box['Top']
    width = img.shape[0] * box['Width']
    height = img.shape[1] * box['Height']
    rect = patches.Rectangle((x,y),width,height,linewidth=0,edgecolor=redacted_box_color,facecolor=redacted_box_color)
    ax.add_patch(rect)

The resulting de-identified image is then written to the S3 bucket that you specified, in PNG format, and with the text “de-id-“ prepended to the original file name.

Conclusion

This blog post has shown you the power and agility of using pre-trained machine learning models. It doesn’t take much development effort to combine AI services to accomplish complex tasks. When you use these services within a pre-configured machine learning development environment, such as Amazon SageMaker notebooks, you can execute your entire software development life cycle on fully managed infrastructure.

Running this example within an Amazon SageMaker notebook provides an ideal environment for you to understand the process used here and see the results. In addition, if you want to batch process thousands or millions of images, this same code could be implemented using AWS Lambda or AWS Batch. You could even associate a Lambda function containing this code with an Amazon S3 bucket so that every time a new image is added it would be de-identified.

About the Author

James Wiggins is a senior healthcare solutions architect at AWS. He is passionate about using technology to help organizations positively impact world health. He also loves spending time with his wife and three children.

 

 

 

 

Map clinical notes to the OMOP Common Data Model and healthcare ontologies using Amazon Comprehend Medical

Written on . Posted in Amazon.

Being able to describe the health of patients with observational data is an important aspect of our modern healthcare system. The amount of quantifiable personal health information is vast and constantly growing as new healthcare methods, metrics, and devices are introduced. All of this data allows clinicians and researchers to understand how the health of a patient changes over time and identify opportunities for precision treatment. The aggregate of this data informs epidemiologists about the health of populations and enables the identification of cause and effect patterns.

Unstructured text, commonly in the form of clinical notes, is a rich source of patient observational health data. Often, there is important information written by clinicians into notes that isn’t encoded into a patient’s structured health record. Clinical notes can also be used to help validate the quality of structured health record data. Historically, the challenge with clinical notes has been that they require manual review to extract the contained medical insights, which is time consuming and expensive.

Amazon Comprehend Medical is a natural language processing (NLP) service that uses machine learning to extract insights like medical condition, medication, dosage, and strength quickly and accurately. Customers can use Amazon Comprehend Medical in a pay-as-you-go model and immediately begin extracting insights from medical text without having to develop or train a complex machine learning model themselves.

The Observational Medical Outcomes Partnership (OMOP) Common Data Model, maintained by the Observational Health Data Science and Informatics (OHDSI) community, is an industry standard, open source data model used for health data. OMOP uses standardized medical ontologies, or “vocabularies,” like SNOMED, to store observational health data. From the OHDSI website:

“The OMOP Common Data Model allows for the systematic analysis of disparate observational databases. The concept behind this approach is to transform data contained within those databases into a common format (data model) as well as a common representation (terminologies, vocabularies, coding schemes), and then perform systematic analyses using a library of standard analytic routines that have been written based on the common format.”

A feature of OMOP is the ability to store clinical notes captured from disparate health data sources. In the data model, these notes are linked to an individual patient and a visit, giving context for the note. OMOP also has the ability to store insights inferred from notes by a natural language processing (NLP) engine. In this blog post we’ll explore how you can use Amazon Comprehend Medical to read notes from OMOP, extract medical insights, and write them back into OMOP using SNOMED ontological codes to enhance patient and population observational health data.

OMOP note processing architecture

This example works with clinical notes in the larger context of a full OHDSI architecture. You can learn more about our automated architecture for OHDSI on AWS by visiting this GitHub repository. The code given here specifically reads notes from and writes insights to the OMOP common data model. However, the same general process can be used with other structured data models for observational data that support clinical notes.

A primary feature of OMOP is the ability to consolidate data from many sources, like Electronic Health Record (EHR) systems and administrative claims data, from many geographic regions, into a common data model. In OMOP, the analysis of observational data from disparate sources is enabled by mapping dozens of source vocabularies (sometimes called dictionaries or ontologies) like SNOMED, RxNorm, ICD10, Read, LOINC, and others to an OMOP standard vocabulary. This enables OMOP to act as a kind of Rosetta stone to allow the interpretation of observational data between different sources and from different global regions.

One important aspect of the approach demonstrated here is mapping the entities detected by Amazon Comprehend Medical to the SNOMED ontology. Because observations must be expressed using standard codes in the OMOP data model, we must first map the insights detected by Amazon Comprehend Medical to a standard ontology before we can write them to OMOP. This is accomplished by sending entity text to the SNOMED CT Browser service to receive a SNOMED code for each entity. The following diagram shows the architecture for this process.

 

Using R code for notes processing

You can download the R Code demonstrating this process from GitHub.

This R code is demonstrated inside of the OHDSI on AWS architecture running on an Amazon EC2 instance running RStudio Server Open Source Edition instance. RStudio is a web-based, integrated development environment (IDE) for working with R. It can be licensed either commercially or under AGPLv3.

The code provided here can also be used from an environment outside of AWS to call Amazon Comprehend Medical. If you want to use it outside of AWS, you’ll need to configure a credentials file that allows this code to call the Amazon Comprehend Medical service. You can find instructions for doing that here.

At the very beginning of the notebook, you’ll see the following statements that define your OMOP CDM connection details and schema name. If you are using the OHDSI on AWS architecture, these are provided for you in a file called connectionDetails.R in your home directory. If outside of AWS, populate these lines with the details specific to your architecture.

#Source the DatabaseConnector::connect() call for my OMOP database
	connectionDetails <- DatabaseConnector::createConnectionDetails(
	dbms = "redshift",
	server = "myRedshiftClusterURL.us-east-1.redshift.amazonaws.com/mycdm",
	user = "master",
	password = "password")
cdmDatabaseSchema <- "CMSDESynPUF1k"

Amazon Comprehend Medical returns a confidence score for every entity, trait, and attribute that it detects. Next in the code, you’ll see a variable that defines a minimum confidence score that detections must meet in order to be written to the NOTE_NLP table. This minimum confidence score can be altered as appropriate for your use case.

#Set the minimum confidence score an inference must meet from Amazon Comprehend Medical to be added to the NOTE_NLP table.
min_score <- 0.80

The actual calls to Amazon Comprehend Medical are implemented using the AWS SDK for Python (boto3). An R library called “reticulate” is used to execute this Python code from within R, pass in the note text, and receive back the entity data detected by Amazon Comprehend Medical.

#Reticulate is used to run Python code from within R
library(reticulate)
...
#source the Python code to call Amazon Comprehend Medical
e <- environment()
reticulate::source_python('call_comprehend_medical.py', envir = e)

Every note found in the OMOP NOTE table is read using a SQL query and processed. For each entity detected in each note by Amazon Comprehend Medical, an appropriate SNOMED code is determined by calling the SNOMED CT Browser.

#Pass the detected medical entity to the SNOMED REST interface to get the matching SNOMED code.
snomed_call <- paste(base,endpoint,"?","query","=", entity$Text,"&limit=1&searchMode=partialMatching&lang=english&statusFilter=activeOnly&skipTo=0 &returnLimit=1&normalize=true", sep="")
get_snomed <- GET(snomed_call, type = "basic")

As each note is processed you’ll see output to the R console that shows each record written to the NOTE_NLP table.

[1] "Writing note_nlp record, 441... Standard Concept ID:313217, Lexical Variant: atrial fibrillation, Term Modifiers: CATEGORY: MEDICAL_CONDITION; TYPE: DX_NAME; TRAIT: DIAGNOSIS;"
  |=======================================================================================================================================| 100%
Executing SQL took 0.287 secs
[1] "Writing note_nlp record, 442... Standard Concept ID:4300877, Lexical Variant: left, Term Modifiers: CATEGORY: ANATOMY; TYPE: DIRECTION;"
  |=======================================================================================================================================| 100%
Executing SQL took 0.258 secs
[1] "Writing note_nlp record, 443... Standard Concept ID:4298444, Lexical Variant: breast, Term Modifiers: CATEGORY: ANATOMY; TYPE: SYSTEM_ORGAN_SITE;"
  |=======================================================================================================================================| 100%
Executing SQL took 0.259 secs
[1] "Writing note_nlp record, 444... Standard Concept ID:4112853, Lexical Variant: breast cancer, Term Modifiers: CATEGORY: MEDICAL_CONDITION; TYPE: DX_NAME; TRAIT: DIAGNOSIS;"
  |=======================================================================================================================================| 100%
Executing SQL took 0.45 secs
[1] "Writing note_nlp record, 445... Standard Concept ID:4080761, Lexical Variant: right, Term Modifiers: CATEGORY: ANATOMY; TYPE: DIRECTION;"
  |=======================================================================================================================================| 100%

After the processing is complete, you’ll have a NOTE_NLP table filled with records that represent medical insights extracted from the notes in your NOTE table. Each of these records has an OMOP Standard Concept ID (the NOTE_NLP_CONCEPT_ID field), mapped from a SNOMED code, that represents the primary entity detected. The TERM_MODIFIERS field in the NOTE_NLP table is used to capture the category, type, and any attribute or trait data that Amazon Comprehend Medical detected about each entity.

The following image shows a comparison of some original portions of note text with entities detected by Amazon Comprehend Medical and the corresponding NOTE_NLP record written to OMOP.

Conclusion

This example shows how Amazon Comprehend Medical makes natural language processing for medical text easily accessible. The insights detected by Amazon Comprehend Medical enable data scientists, epidemiologists, and medical researchers to extend patient and population structured observational health records with information that is locked away in unstructured notes. This additional data provides important insight for precision medicine, longitudinal studies, clinical trial candidate assessment, and population health studies. The code given here can help you get started processing notes into the OMOP Common Data Model today, or it can be used as a pattern to process clinical notes into any observational health data model.

About the Author

James Wiggins is a senior healthcare solutions architect at AWS. He is passionate about using technology to help organizations positively impact world health. He also loves spending time with his wife and three children.

 

 

 

 

 

Measuring the Limits of Data Parallel Training for Neural Networks

Written on . Posted in Google.

Posted by Chris Shallue, Senior Software Engineer and George Dahl, Senior Research Scientist, Google AI

Over the past decade, neural networks have achieved state-of-the-art results in a wide variety of prediction tasks, including image classification, machine translation, and speech recognition. These successes have been driven, at least in part, by hardware and software improvements that have significantly accelerated neural network training. Faster training has directly resulted in dramatic improvements to model quality, both by allowing more training data to be processed and by allowing researchers to try new ideas and configurations more rapidly. Today, hardware developments like Cloud TPU Pods are rapidly increasing the amount of computation available for neural network training, which raises the possibility of harnessing additional computation to make neural networks train even faster and facilitate even greater improvements to model quality. But how exactly should we harness this unprecedented amount of computation, and should we always expect more computation to facilitate faster training?

The most common way to utilize massive compute power is to distribute computations between different processors and perform those computations simultaneously. When training neural networks, the primary ways to achieve this are model parallelism, which involves distributing the neural network across different processors, and data parallelism, which involves distributing training examples across different processors and computing updates to the neural network in parallel. While model parallelism makes it possible to train neural networks that are larger than a single processor can support, it usually requires tailoring the model architecture to the available hardware. In contrast, data parallelism is model agnostic and applicable to any neural network architecture – it is the simplest and most widely used technique for parallelizing neural network training. For the most common neural network training algorithms (synchronous stochastic gradient descent and its variants), the scale of data parallelism corresponds to the batch size, the number of training examples used to compute each update to the neural network. But what are the limits of this type of parallelization, and when should we expect to see large speedups?

In “Measuring the Effects of Data Parallelism in Neural Network Training“, we investigate the relationship between batch size and training time by running experiments on six different types of neural networks across seven different datasets using three different optimization algorithms (“optimizers”). In total, we trained over 100K individual models across ~450 workloads, and observed a seemingly universal relationship between batch size and training time across all workloads we tested. We also study how this relationship varies with the dataset, neural network architecture, and optimizer, and found extremely large variation between workloads. Additionally, we are excited to share our raw data for further analysis by the research community. The data includes over 71M model evaluations to make up the training curves of all 100K+ individual models we trained, and can be used to reproduce all 24 plots in our paper.

Universal Relationship Between Batch Size and Training Time
In an idealized data parallel system that spends negligible time synchronizing between processors, training time can be measured in the number of training steps (updates to the neural network’s parameters). Under this assumption, we observed three distinct scaling regimes in the relationship between batch size and training time: a “perfect scaling” regime where doubling the batch size halves the number of training steps required to reach a target out-of-sample error, followed by a regime of “diminishing returns”, and finally a “maximal data parallelism” regime where further increasing the batch size does not reduce training time, even assuming idealized hardware.

For all workloads we tested, we observed a universal relationship between batch size and training speed with three distinct regimes: perfect scaling (following the dashed line), diminishing returns (diverging from the dashed line), and maximal data parallelism (where the trend plateaus). The transition points between the regimes vary dramatically between different workloads.

Although the basic relationship between batch size and training time appears to be universal, we found that the transition points between the different scaling regimes vary dramatically across neural network architectures and datasets. This means that while simple data parallelism can provide large speedups for some workloads at the limits of today’s hardware (e.g. Cloud TPU Pods), and perhaps beyond, some workloads require moving beyond simple data parallelism in order to benefit from the largest scale hardware that exists today, let alone hardware that has yet to be built. For example, in the plot above, ResNet-8 on CIFAR-10 cannot benefit from batch sizes larger than 1,024, whereas ResNet-50 on ImageNet continues to benefit from increasing the batch size up to at least 65,536.

Optimizing Workloads
If one could predict which workloads benefit most from data parallel training, then one could tailor their workloads to make maximal use of the available hardware. However, our results suggest that this will often not be straightforward, because the maximum useful batch size depends, at least somewhat, on every aspect of the workload: the neural network architecture, the dataset, and the optimizer. For example, some neural network architectures can benefit from much larger batch sizes than others, even when trained on the same dataset with the same optimizer. Although this effect sometimes depends on the width and depth of the network, it is inconsistent between different types of network and some networks do not even have obvious notions of “width” and “depth”. And while we found that some datasets can benefit from much larger batch sizes than others, these differences are not always explained by the size of the dataset—sometimes smaller datasets benefit more from larger batch sizes than larger datasets.

Left: A transformer neural network scales to much larger batch sizes than an LSTM neural network on the LM1B dataset. Right: The Common Crawl dataset does not benefit from larger batch sizes than the LM1B dataset, even though it is 1,000 times the size.

Perhaps our most promising finding is that even small changes to the optimization algorithm, such as allowing momentum in stochastic gradient descent, can dramatically improve how well training scales with increasing batch size. This raises the possibility of designing new optimizers, or testing the scaling properties of optimizers that we did not consider, to find optimizers that can make maximal use of increased data parallelism.

Future Work
Utilizing additional data parallelism by increasing the batch size is a simple way to produce valuable speedups across a range of workloads, but, for all the workloads we tried, the benefits diminished within the limits of state-of-the-art hardware. However, our results suggest that some optimization algorithms may be able to consistently extend the perfect scaling regime across many models and data sets. Future work could perform the same measurements with other optimizers, beyond the few closely-related ones we tried, to see if any existing optimizer extends perfect scaling across many problems.

Acknowledgements
The authors of this study were Chris Shallue, Jaehoon Lee, Joe Antognini, Jascha Sohl-Dickstein, Roy Frostig and George Dahl (Chris and Jaehoon contributed equally). Many researchers have done work in this area that we have built on, so please see our paper for a full discussion of related work.