2D/3D registration functions


The Registration module uses the DRR module to perform differentiable 2D-to-3D registration. Initial guesses for the pose parameters are as stored as nn.Parameters of the module. This allows the pose parameters to be optimized with any PyTorch optimizer. Furthermore, this design choice allows DRR to be used purely as a differentiable renderer.



 Registration (drr:diffdrr.drr.DRR, rotation:torch.Tensor,
               translation:torch.Tensor, parameterization:str,

Perform automatic 2D-to-3D registration using differentiable rendering.

Type Default Details
drr DRR Preinitialized DRR module
rotation Tensor Initial guess for rotations
translation Tensor Initial guess for translations
parameterization str Specifies the representation of the rotation
convention str None If parameterization is euler_angles, specify convention

Pose Regressor

We perform patient-specific X-ray to CT registration by pre-training an encoder/decoder architecture. The encoder, PoseRegressor, is comprised of two networks:

  1. A pretrained backbone (i.e., convolutional or transformer network) that extracts features from an input X-ray image.
  2. A set of two linear layers that decodes these features into camera pose parameters (a rotation and a translation).

The decoder is diffdrr.drr.DRR, which renders a simulated X-ray from the predicted pose parameters. Because our renderer is differentiable, a loss metric on the simulated X-ray and the input X-ray can be backpropogated to the encoder.



 PoseRegressor (model_name, parameterization, convention=None,
                pretrained=False, height=256, **kwargs)

A PoseRegressor is comprised of a pretrained backbone model that extracts features from an input X-ray and two linear layers that decode these features into rotational and translational camera pose parameters, respectively.