Case study

Medical Image Classifier API

Jun 2026 · Personal project

I built an image classification service with PyTorch and FastAPI that returns a prediction, a score for each class and a Grad-CAM heatmap. It is trained on synthetic brain-phantom images and is not for clinical use.

  • PyTorch
  • FastAPI
  • Medical Imaging
  • Grad-CAM
  • Docker
  • CI
synthetic test images classified correctly (94%)
127/135
macro-F1 on the same synthetic test split
0.94
automated tests, run in CI
20
model parameters in a 0.4 MB checkpoint
97,779

Demo only, not a diagnostic tool. The model is trained and tested on procedurally generated images, not real scans, because the scans from the original project (ADNI PET) are access-restricted. The numbers show that the pipeline works. They say nothing about clinical accuracy.

How it works

  1. Generate data

    A seeded generator draws grayscale brain-phantom images in three classes that differ by how much one region is dimmed. The default set is 810 images.

  2. Preprocess

    Grayscale, resize to 128 by 128, percentile scaling, then standardization. Training and serving call the same function.

  3. Train

    A small CNN with four convolution blocks trains on CPU with Adam and early stopping on validation loss.

  4. Package

    The checkpoint stores the weights together with the class order and the normalization statistics, so the server reads them from the file.

  5. Predict

    FastAPI checks each upload, then POST /predict returns the predicted class, a confidence score and a score for every class.

  6. Explain

    A separate POST /explain call returns a Grad-CAM overlay as a PNG, showing which region drove the prediction.

Three panels: a synthetic input image, a bar chart of class scores with Alzheimer's Disease at 96.6 percent, and a Grad-CAM heatmap over the lower region of the image. Opens the full-size picture in a new tab.
Output of the committed model on a bundled synthetic sample, rendered with Matplotlib by a script in the repo. The input is a generated phantom, not a real scan. Tap the picture to see it full size.
Confusion matrix on the synthetic test set. Cognitively Normal: 45 of 45 correct. Mild Cognitive Impairment: 39 correct, 3 predicted as normal, 3 as Alzheimer's Disease. Alzheimer's Disease: 43 correct, 2 predicted as Mild Cognitive Impairment. Opens the full-size picture in a new tab.
Confusion matrix on the 135-image synthetic test split. All 8 errors are between neighboring classes.

The problem

This project grew out of an earlier Alzheimer's PET classification project (also on this site) that trained transfer-learning models and stopped at the notebook stage. It does not reuse that data or those models. It builds the part the notebook lacked: a service someone else can run.

The real scans (ADNI) are access-restricted and cannot be redistributed, so the repo generates its own data. The three classes carry the names Cognitively Normal, Mild Cognitive Impairment and Alzheimer's Disease. In this data they differ only by how much one region of the image is dimmed.

What I built

A small PyTorch CNN with a FastAPI service around it. The part I cared about most is consistency between training and serving. Both call the same preprocessing function, and the normalization statistics and class order travel inside the checkpoint, so the server never hard-codes them.

  • POST /predict returns the predicted class, a confidence score, a score for each of the three classes and a disclaimer.
  • POST /explain returns a Grad-CAM overlay as a PNG. Grad-CAM is implemented with PyTorch hooks, without an external library.
  • Bad uploads are rejected before inference: wrong type (415), empty file (400), over 10 MB (413) or not a readable image (422).
  • The API, a command-line script and a Streamlit demo all go through the same Predictor class.

Results

On the held-out synthetic test split (135 images from the same generator, never used in training) the committed model gets 127 right. That is 94% accuracy and a macro-F1 of 0.94, from a single training run with a fixed seed.

Mild Cognitive Impairment, the middle class, is the hardest, with a recall of 0.87. All 8 errors are between neighboring classes. That is expected, because the generator separates the classes along one dimming factor with random jitter that makes neighbors overlap.

These are generated shapes, not real scans. The numbers show that the pipeline works end to end and say nothing about clinical performance.

Testing and delivery

20 pytest tests cover the API, preprocessing, the model and checkpoint, and the predictor with Grad-CAM. The API and predictor tests run against a small untrained checkpoint, so the suite checks shapes, response contracts and error handling, not accuracy. The training and evaluation code is not covered by tests.

GitHub Actions runs Ruff and pytest on Python 3.10 and 3.11 on every push to main and on pull requests, and a second job builds the Docker image. The image runs as a non-root user and has a health check.

Limits and next steps

  • Not a diagnostic tool. The model is not clinically validated.
  • Trained on synthetic data, so the metrics will not transfer to real scans.
  • The 94% figure comes from one training run on 135 test images, and CI does not re-run training.
  • Softmax scores are not calibrated probabilities.
  • Nothing here handles differences between scanners, sites or populations.
  • Not built yet: transfer-learning backbones, cloud deployment, authentication and rate limiting.

Built with

Model
  • PyTorch
  • Custom CNN
  • Grad-CAM
Data and evaluation
  • NumPy
  • Pillow
  • scikit-learn
  • Matplotlib
API and demo
  • FastAPI
  • Pydantic
  • Uvicorn
  • Streamlit
Tooling
  • pytest
  • Ruff
  • GitHub Actions
  • Docker

Let's build something people love.

I'm open to ML/AI engineering and data science roles, collaborations, and interesting projects. The fastest way to reach me is email.