RHINO: Regularizing the Hash-based Implicit Neural Representation

Abstract

The use of Implicit Neural Representation (INR) through a hash-table has demonstrated impressive effectiveness and efficiency in characterizing intricate signals.However, current state-of-the-art methods exhibit insufficient regularization, often yielding unreliable and noisy results during interpolations. We find that this issue stems from broken gradient flow between input coordinates and indexed hash-keys, where the chain rule attempts to model discrete hash-keys, rather than the continuous coordinates. To tackle this concern, we introduce RHINO, in which a continuous analytical function is incorporated to facilitate regularization by connecting the input coordinate and the network additionally without modifying the architecture of current hash-based INRs. This connection ensures a seamless backpropagation of gradients from the network's output back to the input coordinates, thereby enhancing regularization. Our experimental results not only showcase the broadened regularization capability across different hash-based INRs like DINER and Instant NGP, but also across a variety of tasks such as image fitting, representation of signed distance functions, and optimization of 5D static / 6D dynamic neural radiance 吧 fields. Notably, RHINO outperforms current state-of-the-art techniques in both quality and speed, affirming its superiority.

RHINO

The figure illustrates the workflow of our method. Coordinates are processed through Hash-table encoding to obtain hash code values, and simultaneously fed into our proposed analysis functions—either MLP or PE+MLP. The results from both approaches are concatenated and input into subsequent MLP layers.

Illustrate the effectiveness of our method through the following visualization. Input a one-dimensional signal, output a three-dimensional result. Hash-based INRs (DINER and Instant NGP) provide high accuracy for representing the coordinates appeared in the training set ((c) and (d)). However, both of them produce unreliable results for interpolated coordinates (i.e., the black points in (a)). (g) visualizes the learned function 𝑓𝜃 (H𝑖) of the neural network in DINER. It is noticed that the neighboring pixels (e.g., 𝑥1 and 𝑥2, or 𝑥3 and 𝑥4) become far from each other, and the line between them will pass through several color bands, thus unreliable interpolations ((c) and (d)) are produced.

Results

Image representation

RHINO rebuilds the analytical connection between the input coordinate and the output attribute, as a result, the noisy artifacts could be significantly alleviated when the RHINO is applied to these hash-based INRs.

3D shape representation

The results indicate that our approach outperforms other methods in representing 3D shapes by effectively capturing smoothness, while other methods exhibit noise artifacts.

Neural Radiance Fields

In 'Drums', RHINO interpolate more reasonable textures while other methods yield smoother reconstructions. In 'Ship', RHINO suppresses the artifacts appeared in the DVGO and the Instant NGP, producing more smooth results in the red box.

Dynamic Novel View Synthesis

In DINER, artifacts become apparent due to the broken gradient flow. For instance, observe the bent or missing handle of the spatula, particularly in the zoomed-in yellow boxes found in both the top and bottom subfigures. RHINO effectively rebuilds the gradient flow, resulting in more coherent reconstructions in these problematic regions.

BibTeX

@article{zhu2023rhino,
      title={RHINO: Regularizing the Hash-based Implicit Neural Representation},
      author={Zhu, Hao and Liu, Fengyi and Zhang, Qi and Cao, Xun and Ma, Zhan},
      journal={arXiv preprint arXiv:2309.12642},
      year={2023}
    }