import matplotlib.pyplot as plt
import torch
from diffdrr.drr import DRR
from diffdrr.data import load_example_ct
from diffdrr.visualization import plot_drr
# Read in the volume
= load_example_ct()
volume, spacing
# Initialize the DRR module for generating synthetic X-rays
= DRR(
drr # The CT volume as a numpy array
volume, # Voxel dimensions of the CT
spacing, =300.0, # Source-to-detector radius (half of the source-to-detector distance)
sdr=200, # Height of the DRR (if width is not seperately provided, the generated image is square)
height=4.0, # Pixel spacing (in mm)
delx=1, # How many batches of parameters will be passed = number of DRRs generated each forward pass
batch_size"cuda" if torch.cuda.is_available() else "cpu")
).to(
# Rotations and translations determine the viewing angle
# They must have the same batch_size as was passed to the DRR constructor
# Rotations are (yaw pitch roll)
# Translations are (bx by bz)
= torch.tensor([[torch.pi, 0.0, torch.pi / 2]])
rotations = torch.tensor(volume.shape) * torch.tensor(spacing) / 2
translations = translations.unsqueeze(0)
translations
# Generate the DRR
drr.move_carm(rotations, translations)= drr().detach() # Only keep the graph if optimizing DRRs
img = plot_drr(img)
ax plt.show()
DiffDRR
Auto-differentiable DRR synthesis and optimization in PyTorch
DiffDRR
is a PyTorch-based digitally reconstructed radiograph (DRR) generator that provides
- Auto-differentiable DRR syntheisis
- GPU-accelerated rendering
- A pure Python implementation
Most importantly, DiffDRR
implements DRR synthesis as a PyTorch module, making it interoperable in deep learning pipelines.
Installation Guide
To install DiffDRR
from PyPI:
pip install diffdrr
Usage
The following minimal example specifies the geometry of the projectional radiograph imaging system and traces rays through a CT volume:
On a single NVIDIA RTX 2080 Ti GPU, producing such an image takes
34.9 ms ± 90.5 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
The full example is available at tutorials/introduction.ipynb
.
Application: 6-DoF Slice-to-Volume Registration
We demonstrate the utility of our auto-differentiable DRR generator by solving a 6-DoF registration problem with gradient-based optimization. Here, we generate two DRRs:
- A fixed DRR from a set of ground truth parameters
- A moving DRR from randomly initialized parameters
To solve the registration problem, we use gradient descent to maximize an image loss similarity metric between the two DRRs. This produces optimization runs like this:
The full example is available at tutorials/optimizers.ipynb
.
How does DiffDRR
work?
DiffDRR
reformulates Siddon’s method (Siddon RL. Fast calculation of the exact radiological path for a three-dimensional CT array. Medical Physics, 2(12):252–5, 1985.), the canonical algorithm for calculating the radiologic path of an X-ray through a volume, as a series of vectorized tensor operations. This version of the algorithm is easily implemented in tensor algebra libraries like PyTorch to achieve a fast auto-differentiable DRR generator.
Citing DiffDRR
If you find DiffDRR
useful in your work, please cite our paper (or the freely accessible arXiv version):
@inproceedings{gopalakrishnanDiffDRR2022,
author = {Gopalakrishnan, Vivek and Golland, Polina},
title = {Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging},
year = {2022},
booktitle = {Clinical Image-based Procedures: 11th International Workshop, CLIP 2022, Held in Conjunction with MICCAI 2022, Singapore, Proceedings},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
doi = {https://doi.org/10.1007/978-3-031-23179-7_1},
}