News

2024.12.06 Open source Unicorn Pre (SparsePCGC)!

2024.10.28 Unicorn version 2 has responded to the Call for Proposals for AI-based Point Cloud Coding (m70061 & m70062 in MPEG).

2024.10.05 Initial release of part of the code and results. (The entire source code will be released to the public after the approval from the funding agency.)

2024.09.12 Unicorn version 1 was accepted by TPAMI. ( Part I and Part II )

Abstract

A universal multiscale conditional coding framework, Unicorn, is proposed to compress the geometry and attribute of any given point cloud. Geometry compression is addressed in Part I of this paper, while attribute compression is discussed in Part II.

For geoemtry compression, we construct the multiscale sparse tensors of each voxelized point cloud frame and properly leverage lower-scale priors in the current and (previously processed) temporal reference frames to improve the conditional probability approximation or content-aware predictive reconstruction of geometry occupancy in compression.

For attribute compression, Since attribute components exhibit very different intrinsic characteristics from the geometry element, e.g., 8-bit RGB color versus 1-bit occupancy, we process the attribute residual between lower-scale reconstruction and current-scale data. Similarly, we leverage spatially lower-scale priors in the current frame and (previously processed) temporal reference frame to improve the probability estimation of attribute intensity through conditional residual prediction in lossless mode or enhance the attribute reconstruction through progressive residual refinement in lossy mode for better performance.

The porposed Unicorn is a versatile, learning-based solution capable of compressing static and dynamic point clouds with diverse source characteristics in both lossy and lossless modes. Following the same evaluation criteria, Unicorn significantly outperforms standard-compliant approaches like MPEG G-PCC, V-PCC, and other learning-based solutions, yielding state-of-the-art compression efficiency while presenting affordable complexity for practical implementations.

Contributions

Comprehensive coding metric:  Unicorn is the first, versatile, learning-based PCC solution.
1) It can compress the geometry and attribute information, either separately or jointly, of an input point cloud.
2) It flexibly supports the static and dynamic coding of point clouds in either lossless or lossy mode.
3) It demonstrates the leading performance for diverse types, including solid, dense, and sparse object point clouds, as well as scant LiDAR.
Better compression performance:   Unicorn provides significant performance gains to existing approaches.
Low computation complexity:   Unicorn is a low-complexity approach with comparable runtime measures to the G-PCC codec and variable-rate coding capability using a single neural model.

Method

Data processing in Unicorn. A specific frame Ptk includes the geometry part Otk and attribute intensity Itk; Voxelized Otk is represented using sparse tensor that only contains occupied voxels.



Geometry                                   Attribute


           

Unicorn's Multiscale Sparse Representation. (left) geometry: 1 - Occupied voxel, 0 - Unoccupied voxel; (right) color attribute exemplified using luma or Y intensity. OPU is the Occupancy Processing Unit, and APU is the Attribute Processing Unit.



           

Cross-scale Processing Units. (left) OPU; (right) APU. Spatially or spatiotemporally lower-scale priors are used to support probability approximation in lossless mode or predictive/progressive reconstruction in lossy mode for respective static or dynamic coding.



              

Losslesss Coder in Unicorn. (left) geometry; (right) attribute.


              

Lossy Coder in Unicorn. (left) geometry; (right) attribute.


              

Dynamic Coder in Unicorn. (left) geometry; (right) attribute.


Unified compression of geometry and attribute in Unicorn.

Experiments


    

Point Cloud Examples. We conducted extensive experiments on various point cloud datasets to thoroughly understand the efficiency and generalization of Unicorn. These datasets include static and dynamic samples with diverse contents, densities, resolutions, and other characteristics.


Part I: Geometry


        

R-D comparison for static geometry coding.



        

R-D comparison for dynamic geometry coding.



  

Error map visualization of reconstructed point clouds.




Part II: Attribute


        

R-D comparison for static attribute coding.


        

R-D comparison for lossy compression of geometry & attribute.


        

R-D comparison for dynamic attribute coding.


  

Qualitative results of reconstructed point clouds with lossy attribute coding.


  

Qualitative visualization of reconstructed point clouds with lossy compression mode of both geometry & attribute.

Our Team

Team members contributed to Unicorn.

Ma Zhan

Professor at Nanjing University.

Email: mazhan@nju.edu.cn

Ding Dandan

Associate Professor at Hangzhou Normal University.

Email: DandanDing@hznu.edu.cn

Chen Tong

Associate Researcher at Nanjing University.

Email: chentong@nju.edu.cn

Wang Jianqiang

Ph.D. Candidate at Nanjing University.

Email: wangjq@smail.nju.edu.cn

Xue Ruixiang

Ph.D. Candidate at Nanjing University.

Email: xrxee@smail.nju.edu.cn

Li Jiaxin

Ph.D. Candidate at Nanjing University.

Email: lijiaxin@smail.nju.edu.cn