Endo-HDR: Dynamic Endoscopic Reconstruction with Deformable 3D Gaussians and Hierarchical Depth Regularization

Reconstructing endoscopic videos is crucial due to its potential for robot-assisted surgery and intelligent medical. However, existing methods focus on fast reconstruction, and the reliance on low-rank deformation fields and globally aligned depth supervision often results in suboptimal appearance and noisy depth, leading to fused point clouds suffering from texture blur and surface distortion. To address these issues, we propose Endo-HDR, a framework for high-quality dynamic surgical video reconstruction based on 3D Gaussian splatting (3DGS) technology. Specifically, the surgical scene is decoupled into canonical Gaussians and a Gaussian Anchors (GAs)-based deformation field. We leverage time-varying GAs with a consistency constraint to model complex tissue motion, where an MLP predicts the continuous motion of sparse GAs. The deformation of each Gaussian is then interpolated from its neighboring GAs, easing the learning of deformation. Additionally, since discrete Gaussians are sensitive to small errors, we employ hierarchical depth regularization to learn depth information at both window-level and image-level from sub-regions, enabling the model to reconstruct sharp and accurate depth. To further enhance Endo-HDR's performance, we introduce an efficient Gaussian initialization strategy to generate dense geometric priors in scene initialization phase. We also design a Gaussian growth control strategy to limit the number of Gaussians, ensuring the model’s real-time rendering capability. Experimental results on the EndoNeRF, StereoMIS and Hamlyn datasets demonstrate that Endo-HDR outperforms existing methods in both realistic rendering and accurate depth reconstruction.