X-Gaussian: Radiative Gaussian Splatting
for Efficient X-ray Novel View Synthesis

European Conference on Computer Vision (ECCV), 2024

Yuanhao Cai1, Yixun Liang2, Jiahao Wang1, Yulun Zhang3, Xiaokang Yang3, Zongwei Zhou1, Alan Yuille1
1 Johns Hopkins University   2 HKUST(GZ)   3 Shanghai Jiao Tong University

The first 3D Gaussian Splatting Method for X-ray 3D Reconstruction.

CT Reconstruction During Optimization

Chest

Foot

Head

Abdomen

Pancreas

Pine

Seashell

Walnut

Interactive CT Volume

Drag to rotate · Scroll or pinch to zoom

Abstract

X-ray is widely applied for transmission imaging due to its stronger penetration than natural light. When rendering novel view X-ray projections, existing methods mainly based on NeRF suffer from long training time and slow inference speed. In this paper, we propose a 3D Gaussian splatting-based method, namely X-Gaussian, for X-ray novel view synthesis and direct CT reconstruction. Firstly, we redesign a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging. Our model excludes the influence of view direction when learning to predict the radiation intensity of 3D points. Based on this model, we develop a Differentiable Radiative Rasterization (DRR) with CUDA implementation. Secondly, we formulate a principled Radiative Gaussian Volume Querying (RGVQ) method that defines a continuous attenuation field from the original radiative point attributes and enables direct CT volume reconstruction through CUDA-based parallel querying, extending conventional 3DGS beyond image rendering. Thirdly, we customize an Angle-pose Cuboid Uniform Initialization (ACUI) strategy that directly uses the parameters of the X-ray scanner to compute the camera information and then uniformly samples point positions within a cuboid enclosing the scanned object. On X-ray novel view synthesis, experiments show that our X-Gaussian outperforms state-of-the-art (SOTA) methods by 6.5 dB while enjoying less than 15% training time and over 73× inference speed. For direct CT reconstruction, X-Gaussian outperforms SOTA methods by 0.40 dB and 4.34 dB in average 3D PSNR on synthetic and real scenes, respectively, while achieving 4.30× faster inference of the learned CT volume on average. Code, models, and datasets are at github.com/caiyuanhao1998/X-Gaussian

Method Overview

X-Gaussian comprises three key techniques for efficient X-ray projection rendering and direct CT reconstruction.

Radiative Gaussian Point Clouds and DRR

Our radiative Gaussian point cloud model fits view-independent radiation intensity through a Radiation Intensity Response Function (RIRF). A CUDA-based Differentiable Radiative Rasterization (DRR) renders novel X-ray projections from these points.

Comparison of RGB Gaussian spherical harmonics and the view-independent radiative Gaussian model
The radiative model removes view-dependent color modeling to represent isotropic X-ray penetration.

Radiative Gaussian Volume Querying (RGVQ)

Conventional 3DGS is designed for image synthesis and does not directly define a queryable CT attenuation field. RGVQ derives positive attenuation coefficients from the existing radiative features and opacities, and sums Gaussian contributions into a continuous field. Here, ρi is the product of opacity and positive radiative intensity. CUDA-based parallel querying at voxel centers directly reconstructs the CT volume without additional learnable point attributes.

V(x) = ∑i ρi exp[−½(x − μi)TΣi−1(x − μi)]

Angle-pose Cuboid Uniform Initialization (ACUI)

For X-ray novel view synthesis, ACUI computes camera calibration from scanner geometry and uniformly samples point centers in a cuboid enclosing the scanned object, avoiding structure-from-motion. The direct CT variant uses initialization reconstructed from training projections.

X-Gaussian novel view synthesis pipeline: scanner initialization, radiative Gaussian model, and differentiable radiative rasterization
The original X-ray novel view synthesis pipeline: scanner-based initialization, radiative Gaussian point clouds, and DRR.

X-ray Novel View Synthesis

Compared with NAF, X-Gaussian improves average PSNR by 6.5 dB while achieving 73× inference speed and requiring less than 15% of the training time in the original NVS experiments.

X-ray projection rendering comparisons on Pancreas and Chest
Novel view synthesis on Pancreas (top) and Chest (bottom). Red and blue boxes mark the enlarged regions. Original NVS experiments use an RTX 8000 GPU.

Direct 3D CT Reconstruction

From 50 projections, X-Gaussian reaches 33.25 dB average 3D PSNR on five synthetic scenes and 39.53 dB on three real scans, improving over SAX-NeRF by 0.40 dB and 4.34 dB, respectively. Complete CT volume inference is 4.30× faster on average on the same RTX A5000.

Synthetic ScenesReal Scenes
MethodChestFootHeadAbdomenPancreasAveragePineSeashellWalnutAverage
NeRF33.9110.926630.2060.838731.7600.900730.4350.832618.8080.847129.0240.869136.7660.854736.2250.799930.7790.513334.5900.7226
NAF32.5920.920231.7740.878236.9940.942634.3710.883619.4340.659931.0330.856940.4590.926440.5210.927532.4400.658437.8070.8374
SAX-NeRF34.3590.937832.2460.888339.6990.971434.9950.900422.9490.934932.8500.926637.3030.916837.0180.932131.2450.670135.1880.8396
R²-Gaussian33.6010.942732.0220.879939.9440.976333.4980.893121.8080.857732.1750.909941.3700.945443.4260.961632.1740.708238.9900.8717
X-Gaussian36.0010.939132.1810.881541.4380.976834.2700.894622.3780.879133.2540.914241.7980.946544.0080.963832.7690.716939.5250.8757

PSNR (dB, upper) / SSIM (lower). Best and second-best entries are bold and underlined. Real scans are evaluated against full-view FDK pseudo-references. Inference timing includes coordinate generation, volume querying and CPU transfer, excluding model loading and disk writing.

Direct CT reconstruction comparisons on Head and Seashell
Head (top) and Seashell (bottom). References are original CT for Head and a full-view FDK pseudo-reference for Seashell. All methods use matching slices, display windows, and zoom regions.

BibTeX

@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}