Method and System of Building Hospital-Scale Medical Image Database

Developing computer systems that can recognize and locate image features associated with disease is a challenge for developing fully-automated and high precision computer assisted diagnostics. Joint learning of language tasks in association with vision tasks (association of image features with text annotation) adds an additional level of challenge.  Furthermore, scaling-up approaches from small to large datasets presents additional issues, particularly related to medical images. In this case, identifying such features requires specialized skill for even a human and the text descriptions from trained physicians may be variable, complex, and abstract.  The application of deep learning to medical image feature detection in association with language recognition may aid in the development of precision automated computer-aided diagnostics for a wide range of disease conditions, based on large scale PACS datasets.  


The technology developed by researchers at the National Institutes of Health Clinical Center (NIHCC), applies natural language processing techniques and deep learning methods to mine PACS images and generate large-scale image databases. Diseases can be detected and spatially-located within the dataset generated by this method. The generation of such datasets is an important step toward utilizing PACS informatics and development of fully-automated high precision computer diagnostic systems.

Potential Commercial Applications: Competitive Advantages:
  • Computer Assisted Diagnostics
  • Medical Image Informatics
  • Ability to create large-scale labeled medical image database

Development Stage:
Discovery (Lead Identification)


Xiaosong Wang (CC)  ➽ more inventions...

Yifan Peng (NLM)  ➽ more inventions...

Le Lu (CC)  ➽ more inventions...

Zhiyong Lu (NLM)  ➽ more inventions...

Ronald Summers (CC)  ➽ more inventions...

Intellectual Property:
US Provisional Application No. 62/476,029

Wang X et al. ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases arxiv abstract 1705.02315

Collaboration Opportunity:

Licensing only

Licensing Contact:
John Hewes, Ph.D.
Phone: 240-276-5515

OTT Reference No: E-076-2017
Updated: Apr 4, 2018