Medical imaging is transforming healthcare. Researchers are training deep learning models to detect pneumonia from chest X-rays, estimate cardiac function from echocardiograms, and identify tumors from MRI scans. But before any of these images can be shared with researchers or used to train machine learning models, one critical challenge must be solved.
How Do We Protect Patient Privacy?
Medical images often contain sensitive information such as patient names, dates of birth, hospital identifiers, and accession numbers. Some of this information is stored in DICOM (Digital Imaging and Communications in Medicine) metadata, but much of it is also burned directly into the image pixels.
In this tutorial, you’ll learn how to build an AI-powered de-identification pipeline that removes PHI from both metadata and image pixels. Along the way, we’ll explore OCR (Optical Character Recognition), NER (Named Entity Recognition), and standards-based DICOM processing.
At the end, I’ll show how I combined these ideas into an open-source PyTorch project called Aegis.
What You’ll Build
In this tutorial, you’ll build a custom MONAI (PyTorch) preprocessing pipeline that automatically de-identifies medical images before they are used for clinical research or AI model training.
The pipeline will:
Discover DICOM studies
Load metadata and pixel data
Detect burned-in text using OCR
Classify text as PHI or non-PHI
Redact sensitive pixel regions
Remove PHI from DICOM metadata and pixel data
Save privacy-safe images for downstream AI workflows
By the end, you’ll have a reusable MONAI transform that can be integrated directly into any medical imaging workflow to prepare privacy-safe datasets for research and deep learning.
Prerequisites
To follow this tutorial, you should have:
Intermediate Python experience
Basic understanding of PyTorch
Familiarity with medical imaging concepts
Python 3.10 or later
We’ll use:
MONAI
pydicom
EasyOCR
NumPy
Transformers
Stanford NER
Set Up the Environment
# Create and activate a virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Upgrade pip
pip install --upgrade pip
# Install the core libraries used in this tutorial
pip install \
monai \
pydicom \
easyocr \
numpy \
transformers \
torch
# Download the Stanford medical de-identification model from Hugging Face
python -c "
from transformers import AutoTokenizer, AutoModelForTokenClassification
model_name = 'StanfordAIMI/stanford-deidentifier-base'
AutoTokenizer.from_pretrained(model_name)
AutoModelForTokenClassification.from_pretrained(model_name)
print('Stanford NER model downloaded successfully.')
"
Why Privacy Matters in Medical Imaging
Healthcare organizations generate enormous volumes of imaging data every day. These datasets are invaluable for:
Clinical research
Multi-center collaborations
Regulatory submissions
Artificial intelligence model development
Educational datasets
But privacy regulations such as the HIPAA (Health Insurance Portability and Accountability Act) in the United States require that PHI (Protected Health Information) be removed before data can be shared. This creates a significant bottleneck.
Many hospitals still rely on manual review to inspect thousands of images, searching for patient identifiers hidden in metadata and image annotations. This process is slow, expensive, and prone to human error.
Automated de-identification solves this problem by combining software engineering, computer vision, and natural language processing.
Understanding PHI, HIPAA, and DICOM
What Is PHI?
Protected Health Information (PHI) includes any information that can identify a patient, such as:
Name
Medical record number
Date of birth
Study date
Hospital ID
Accession number
What Is HIPAA?
The Health Insurance Portability and Accountability Act (HIPAA) defines rules for safeguarding patient data. One common approach is the Safe Harbor method, which requires removing specific identifiers before data is shared.
What Is DICOM?
Medical images such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Ultrasound (US) are commonly stored in the DICOM (Digital Imaging and Communications in Medicine) format, the international standard for storing and exchanging medical imaging data.
Unlike ordinary image formats such as JPEG or PNG, a DICOM file contains both the image itself and a rich set of structured metadata that describes the patient, the study, and the imaging procedure.
A typical DICOM file contains two main components:
Pixel Data – the actual medical image, such as a CT slice, MRI volume, or ultrasound frame.
Metadata – structured fields that may include:
Patient name and medical record number
Date of birth
Study and acquisition dates
Imaging modality (CT, MRI, US)
Scanner manufacturer and technical acquisition parameters
This combination makes DICOM far more than just an image format. It serves as a standardized container that allows imaging devices, hospital systems, and research software to exchange data reliably and consistently.
Because DICOM metadata often contains protected health information (PHI), and because identifiers may also be burned directly into the image pixels, particularly in ultrasound studies, both the metadata and the pixel data must be addressed during de-identification before images can be safely shared for clinical research or AI development.
Why Metadata Anonymization Is Not Enough in DICOM format
Many tools remove PHI only from metadata. For example, deleting the PatientName tag may appear sufficient.
But in modalities such as ultrasound, fluoroscopy, and some X-ray workflows, identifying information is often burned directly into the image.
Common examples include:
NAME: JOHN DOE
DOB: 01/01/1980
MRN: 123456
HOSPITAL: ABC
If these annotations remain, privacy is still compromised. This means a complete solution must inspect both:
DICOM metadata
Image pixels
OCR and AI for Identifying PHI
To detect PHI embedded in pixels, we first need to find all visible text.
Step 1: Optical Character Recognition (OCR)
OCR converts image text into machine-readable strings.
import easyocr
reader = easyocr.Reader(['en'])
results = reader.readtext('ultrasound.png')
Each OCR result typically includes:
Bounding box coordinates – where the text appears in the image
Extracted text – the recognized characters
Confidence score – how certain the model is about the result
Example:
[
([[10, 20], [120, 20], [120, 45], [10, 45]], 'JOHN DOE', 0.98)
]
Step 2: Determine Whether the Text Is PHI
Not all detected text should be removed.
Medical images also contain clinically relevant labels such as:
LEFT VENTRICLE
APICAL VIEW
B-MODE
To distinguish PHI from legitimate clinical text, we can combine:
Allowlists of known clinical terms
Regular-expression heuristics
Named Entity Recognition (NER)
Step 3: Named Entity Recognition
NER models identify entities such as:
PERSON
DATE
LOCATION
ID
def contains_phi(text):
if looks_like_date(text):
return True
if looks_like_identifier(text):
return True
return ner_model.predict(text)
This hybrid approach reduces both false positives and false negatives.
Pixel Redaction and DICOM Scrubbing
Pixel Redaction
Once PHI is detected, the corresponding image regions can be masked.
image[y1:y2, x1:x2] = 0
This replaces the sensitive area with black pixels.
DICOM Metadata Scrubbing
Using pydicom, metadata fields can be modified or removed.
import pydicom
ds = pydicom.dcmread('study.dcm')
ds.PatientName = 'ANONYMIZED'
del ds.PatientBirthDate
Additional steps may include:
Removing private tags
Replacing UIDs
Recursively processing nested sequences
Together, metadata scrubbing and pixel redaction provide comprehensive de-identification.
Building the Complete Pipeline
The overall workflow looks like this:
Discover medical image files
Load DICOM metadata and pixel data
Run OCR on annotation regions
Classify text as PHI or non-PHI
Redact sensitive pixel regions
Remove PHI from metadata
Save the de-identified output
Challenges and Lessons Learned
Building a production-ready de-identification system involves many practical challenges.
Clinical Terminology
OCR may detect legitimate labels that should not be removed.
OCR Errors
Low-contrast text and ultrasound overlays can produce inaccurate detections.
Nested DICOM Sequences
PHI may appear in deeply nested metadata structures.
Multi-Frame Studies
Ultrasound cine loops may contain dozens or hundreds of frames.
Deterministic Pseudonymization
Researchers often need the same patient to receive the same replacement identifier across studies.
These challenges require thoughtful engineering rather than a single machine learning model.
How I Built Aegis
While exploring this problem, I developed an open-source MONAI (PyTorch based) project called Aegis.
Aegis combines:
OCR-based text detection
AI-driven PHI classification
Pixel-level redaction
Standards-based DICOM de-identification
Batch processing for research workflows
Key Design Decisions
Standards First
I aligned metadata scrubbing with the DICOM confidentiality profile to follow established healthcare standards.
Hybrid AI + Rules
Clinical allowlists, heuristics, and NER models work together to improve accuracy.
Ultrasound-Specific Optimization
Aegis uses SequenceOfUltrasoundRegions to focus OCR on annotation areas instead of scanning the entire image.
Deterministic Identity Management
Consistent pseudonyms enable longitudinal research while protecting privacy.
Open Source Architecture
The project is modular, testable, and designed to integrate with research pipelines.
You can explore the full implementation in the Aegis GitHub repository:
https://github.com/lakshmi-mahabaleshwara/aegis
Future Directions
Automated de-identification continues to evolve.
Future enhancements may include:
Multilingual OCR
Handwriting recognition
Vision-language models
Human-in-the-loop review
Cloud-native deployment
Integration with AI training pipelines
As healthcare AI expands, privacy-preserving data preparation will become even more important.
Conclusion
Clinical research depends on access to high-quality medical imaging data.
But privacy regulations require that patient identifiers be removed from both DICOM metadata and image pixels.
By combining OCR, named entity recognition, pixel redaction, and standards-based DICOM processing, we can automate this task and dramatically reduce the burden of manual review.
The techniques covered in this tutorial are applicable far beyond a single project.
Whether you’re building a hospital data pipeline, preparing research datasets, or training the next generation of healthcare AI models, automated de-identification is a foundational capability.
To put these ideas into practice, I built Aegis as an open source reference implementation.
More importantly, the underlying concepts can help developers and researchers create privacy-safe workflows that accelerate innovation while respecting patient confidentiality.