PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling

2020·arXiv

Abstract

1 Introduction

Three-dimensional (3D) point clouds, as the raw representation of 3D data, are used in a wide range of applications, such as 3D immersive telepresence [2], 3D city reconstruction [3], [4], cultural heritage reconstruction [5], [6], geophysical information systems [7], [8], autonomous driving [9], [10], simultaneous localization and mapping [11], [12], and virtual/augmented reality [13], [14], just to name a few. Though recent

Fig. 1: Illustration of various sampling factors of the Retheur Statue model with 5,000 points. Due to the low-resolution input, the details, such as cloth wrinkles and facial features, are missing. PUGeo-Net can effectively generate up to points to fill in the missing part. See also the accompanying video and results.

years have witnessed great progress on the 3D sensing technology [15], [16], it is still costly and time-consuming to obtain dense and highly detailed point clouds, which are beneficial to the subsequent applications. Therefore, amendment is required to speed up the deployment of such data modality. In this paper, instead of relying on the development of hardware, we are interested in the problem of computational based point cloud upsampling: given a sparse, low-resolution point cloud, generate a uniform and dense point cloud with a typical computational method to faithfully represent the underlying surface. Since the problem is the 3D counterpart of image super-resolution [17], [18], a typical idea is to borrow the powerful techniques from the image processing community. However, due to the unordered and irregular nature of point clouds, such an extension is far from trivial, especially when the underlying surface has complex geometry.

The existing methods for point cloud upsampling can be roughly classified into two categories: optimization-based methods and deep learning based methods. The optimization methods [19], [20], [21], [22], [23] usually fit local geometry and work well for smooth surfaces with less features. However, these methods struggle with multi-scale structure preservation. The deep learning methods can effectively learn structures from data. Representative methods are PU-Net [24], EC-Net [25] and MPU [26]. PUNet extracts multi-scale features using point cloud convolution [27] and then expands the features by replication. With additional edge and surface annotations, EC-Net improves PU-Net by restoring sharp features. Inspired by image super-resolution, MPU upsamples points in a progressive manner, where each step focuses on a different level of detail. PU-Net, EC-Net and MPU operate on patch level, therefore, they can handle high-resolution point sets. Though the deep learning methods produce better results than the optimization based methods, they are heavily motivated by the techniques in the image domain and takes little consideration of the geometries of the input shape. As a result, various artifacts can be observed in their results. It is also worth noting that all the existing deep learning methods generate points only, none of them is able to estimate the normals of the original and generated points.

In this paper, we propose a novel network, called PUGeo-Net, to overcome the limitations in the existing deep learning methods. Our method learns a local parameterization for each point and its normal direction. In contrast to the existing neural network based methods that generate new points in the abstract feature space and map the samples to the surface using decoder, PUGeo-Net performs the sampling operations in a pure geometric way. Specifically, it first generates the samples in the 2D parametric domain and then lifts them to 3D space using a linear transformation. Finally, it

projects the points on the tangent plane onto the curved surface by computing a normal displacement for each generated point via the learned second fundamental form. Through extensive evaluation on commonly used as well as new metrics, we show that PUGeo-Net consistently outperforms the state-of-the-art in terms of accuracy and ef-ficiency for upsampling factors . It is also worth noting that PUGeo-Net is the first neural network that can generate dense point clouds with accurate normals, which are highly desired by the existing surface reconstruction algorithms. We demonstrate the efficacy of PUGeo-Net on both CAD models with sharp features and scanned models with rich geometric details and complex topologies. Fig. 1 demonstrates the effectiveness of PUGeo-Net on the Retheur Statue model. The main contributions of this paper are summarized as follows. 1. We propose PUGeo-Net, a novel geometric-centric neural network, which carries out a sequence of geometric operations, such as computing the first-order approximation of local parameterization, adaptive sampling in the parametric domain, lifting the samples to the tangent plane, and projection to the curved surface.

2. PUGeo-Net is the first upsampling network that can jointly generate coordinates and normals for the densified point clouds. The normals benefit many downstream applications, such as surface reconstruction and shape analysis.

3. We interpret PUGeo-Net using the local theory of surfaces in differential geometry. Quantitative verification confirms our interpretation.

4. We evaluate PUGeo-Net on both synthetic and real-world models and show that PUGeo-Net significantly outperforms the state-of-the-art methods in terms of accuracy and efficiency for all upsampling factors.

5. PUGeo-Net can handle noisy and non-uniformly distributed point clouds as well as the real scanned data by the LiDAR sensor very well, validating its robustness and practicality.

2 Related Work

Optimization based methods. Alexa et al. [19] interpolated points of Voronoi diagram, which is computed in the local tangent space. Lipman et al. developed a method based on locally optimal projection operator (LOP) [20]. It is a parametrization-free method for point resampling and surface reconstruction. Subsequently, the improved weighted LOP and continuous LOP were developed by Huang et al. [21] and Preiner et al. [22] respectively. These methods assume that points are sampling from smooth surfaces, which degrades upsampling quality towards sharp edges and corners. Huang et al. [23] presented an edge-aware (EAR) approach which can effectively preserve the sharp features. With given normal information, EAR algorithm first resamples points away from edges, then progressively upsamples points to approach the edge singularities. However, the performance of EAR heavily depends on the given normal information and parameter tuning. In conclusion, point cloud upsampling methods based on geometric priors either assume insufficient hypotheses or require additional attributes. Deep learning based methods. The deep learning based upsampling methods first extract point-wise feature via point clouds CNN. The lack of point order and regular structure impede the extension of powerful CNN to point clouds. Instead of converting point clouds to other data representations like volumetric grids [28], [29], [30] or graphs [31], [32], recently the point-wise 3D CNN [33], [34], [27], [35], [36] successfully achieved state-of-the-art performance for various tasks. Yu et al. pioneered

PU-Net[24], the first deep learning algorithm for point cloud upsampling. It adopts PointNet++ [27] to extract point features and expands features by multi-branch MLPs. It optimizes a joint reconstruction and repulsion loss function to generate point clouds with uniform density. PU-Net surpasses the previous optimization based approaches for point cloud upsampling. However, as it does not consider the spatial relations among the points, there is no guarantee that the generated samples are uniform. The follow-up work, EC-Net [25], adopts a joint loss of point-to-edge distance, which can effectively preserve sharp edges. EC-Net requires labelling the training data with annotated edge and surface information, which is tedious to obtain. Wang et al. [26] proposed a patch-based progressive upsampling method (MPU). Their method can successfully apply to large upsampling factor, say 16. Inspired by the image super-resolution techniques, they trained a cascade of upsampling networks to progressively upsample to the desired factor, with the subnet only deals with 2case. MPU replicates the point-wise features and separates them by appending a 1D code , which does not consider the local geometry. MPU requires a careful step-by-step training, which is not flexible and fails to gain a large upsampling factor model directly. Since each subnet upsizes the model by a factor 2, MPU only works for upsampling factor which is a power of 2. PUGeoNet distinguishes itself from the other deep learning method from its geometry-centric nature. See Sec. 4 for quantitative comparisons and detailed discussions. Recently, Li et al. [51] proposed PU-GAN which introduces an adversarial framework to train the upsampling generator. Again, PU-GAN fails to examine the geometry properties of point clouds. Their ablation studies also verify the performance improvement mainly comes from the introducing of the discriminator.

3 Proposed Method

3.1 Motivation & Overview

Given a sparse point cloud with M points and the user-specified upsampling factor R, we aim to generate a dense, uniformly distributed point cloud , which contains more geometric details and can approximate the underlying surface well. Similar to other patch-based approaches, we first partition the input sparse point cloud into patches via the farthest point sampling algorithm, each of which has N points, and PUGeo-Net processes the patches separately.

As mentioned above, the existing deep learning based methods are heavily built upon the techniques in 2D image domain, which generate new samples by replicating feature vectors in the abstract feature space, and thus the performance is limited. Moreover, due to little consideration of shape geometry, none of them can compute normals, which play a key role in surface reconstruction. In contrast, our method is motivated by parameterization-based surface resampling, consisting of 3 steps: first it parameterizes a 3D surface S to a 2D domain, then it samples in the parametric domain and finally maps the 2D samples to the 3D surface. It is known that parameterization techniques depend heavily on the topology of the surface. There are two types of parameterization, namely local parameterization and global parameterization. The former deals with a topological disk (i.e., a genus-0 surface with 1 boundary) [38]. The latter works on surfaces of arbitrary topology by computing canonical homology basis, through which the surface is cutting into a topological disk, which is then mapped to a 2D domain [39]. Global constraints are required in order to ensure the parameters are continuous across the cuts [37].

Fig. 2: Surface parameterization and local shape approximation. The local neighborhood of parameterized to a 2D rectangular domain via a differentiable map . The Jacobian matrix provides the best linear approximation of , which maps (u, v) to a point on the tangent plane of . Furthermore, using the principal curvatures of , we can reconstruct the local geometry of in the second-order accuracy.

In our paper, the input is a point cloud sampled from a 3D surface of arbitrary geometry and topology. The Fundamental Theorem of the Local Theory of Surfaces states that the local neighborhood of a point on a regular surface can be completely determined by the first and second fundamental forms, unique up to rigid motion (see [52], Chapter 4). Therefore, instead of computing and learning a global parameterization which is expensive, our key idea is to learn a local parameterization for each point.

Let us parameterize a local neighborhood of point to a 2D domain via a differential map so that (see Fig. 2). The Jacobian matrix provides the best first-order approximation of the map , where and are the tangent vectors, which define the first fundamental form. The normal of point can be computed by the cross product .

It is easy to verify that the point is on the tangent plane of , since . In our method, we use the augmented Jacobian matrix to compute the normal and the point . Matrix T is of full rank if the surface is regular at . Furthermore, the distance between x and is , where and are the principal curvatures at , which are the eigenvalues of the second fundamental form.

As shown in Fig. 3(a), given an input sparse 3D point cloud, PUGeo-Net proceeds as follows: it first generates new samples in the 2D parametric domain. Then it computes the normal . After that, it maps each generated 2D sample to the tangent plane of by . Finally, it projects to the curved 3D surface by computing a displacement along the normal direction. Fig. 3(b) illustrates the network architecture of PUGeo-Net, which consists of hierarchical feature extraction and re-calibration (Sec. 3.2), parameterization-based point expansion (Sec. 3.3) and local shape approximation (Sec. 3.4). We adopt a joint loss function to guide the prediction of vertex coordinates and normals (Sec 3.5).

3.2 Hierarchical Feature Learning and Recalibration

To handle the rotation-invariant challenge of 3D point clouds, we adopt an STN-like mini-network [40], which computes a global 3D transformation matrix

Fig. 3: Overview. The top row illustrates the stages of PUGeo-Net: learning local parameterization, point expansion and vertex coordinates and normals refinement. The middle row shows the end-to-end network structure of PUGeo-Net. The output is colored in red. The bottom row shows the details of two core modules: feature recalibration and point expansion. applied to all points. After that, we apply DGCNN [34] - the state-of-the-art method for point cloud classification and segmentation - to extract hierarchical point-wise features, which are able to encode both local and global intrinsic geometry information of an input patch. The hierarchical feature learning module extracts features from low- to high-levels. Intuitively speaking, as the receptive fields increase, skip-connection [41], [42], [43], a widely-used technique in 2D vision task for improving the feature quality and the convergence speed, can help preserve details in all levels. To this end, as illustrated in Fig. 3(c), instead of concatenating the obtained features directly, we perform feature re-calibration by a self-gating unit [44], [45] to enhance them, which is computationally efficient.

where is the feature length. We first concatenate the features of all L layers, i.e., , where and stands for the concatenation operator. The direct concatenate feature is passed to a small MLP to obtain the logits , i.e.,

which are futher fed to a softmax layer to produce the recalibration weights with

Finally, the recalibrated multi-scale features are represented as the weighted concatenation:

3.3 Parameterization-based Point Expansion

3.4 Updating Samples via Local Shape Approximation

Since the samples are on the tangent plane, we need to warp them to the curved surface and update their normals. Specifically, we move each sample along the normal with a distance . As mentioned in Sec. 3.1,

this distance provides the second-order approximation of the local geometry of . We compute the distance by regressing the point-wise features concatenated with their coarse coordinates, i.e., (8) where is for the process of an MLP. Then we compute the sample coordinates as (9) We update the normals in a similar fashion: a normal offset for pointis regressed as (10) which is further added to the corresponding coarse normal, leading to (11) where is the process of an MLP.

3.5 Joint Loss Optimization

As PUGeo-Net aims to deal with the regression of both coordinates and unoriented normals of points, we design a joint loss to train it end-to-end. Specifically, let with RM points be the groundtruth of . During training, we adopt the Chamfer distance (CD) to measure the coordinate error between the and , i.e.,

where , and is the norm of a vector.

For the normal part, denote and the ground truth of the coarse normal N and the accurate normal , respectively. During training, we consider the errors between N and and between and simultaneously, i.e.,

where measures the unoriented difference between two normals, and is used to build the unknown correspondence between and . Finally, the joint loss function is written as

where , and are three positive parameters. It is worth noting that our method does not require repulsion loss which is required by PU-Net and EC-Net, since the module for learning the parametric domain is capable of densifying point clouds with uniform distribution.

4 Experimental Results

4.1 Experiment Settings

Datasets. Following previous works, we selected 90 high-resolution 3D mesh models from Sketchfab [1] to construct the training dataset and 13 for the testing dataset. Specifically, given the 3D meshes, we employed the Poisson disk sampling [49] to generate , , and N with M = 5000 and R = 4, 8, 12 and 16. A point cloud was randomly cropped into patches each of N = 256 points. To fairly compare different methods, we adopted identical data augmentations settings, including random scaling, rotation and point perturbation. During the testing process, clean test data were used. Also notice that the normals of sparse inputs are not needed during testing.

Implementation details. We empirically set the values of the three parameters , , and in the joint loss function to 100, 1, and 1, respectively. We used the Adam algorithm with the learning rate equal to 0.001. We trained the network with the minibatch of size 8 for 800 epochs via the TensorFlow platform. The code will be publicly available later.

Evaluation metrics. To quantitatively evaluate the performance of different methods, we considered four commonly-used evaluation metrics, i.e., Chamfer distance (CD), Hausdorff distance (HD), point-to-surface distance (P2F), and Jensen-Shannon divergence (JSD). For these four metrics, the lower, the better. For all methods under comparison, we applied the metrics on the whole shape.

We also propose a new approach to quantitatively measure the quality of the generated point clouds. Instead of conducting the comparison between the generated point clouds and the corresponding groundtruth ones directly, we first performed surface reconstruction [50]. For the methods that cannot generate normals principal component analysis (PCA) was adopted to predict normals. Then we densely sampled 200, 000 points from reconstructed surface. CD, HD and JSD between the densely sampled points from reconstructed surface and the groundtruth mesh were finally computed for measuring the surface reconstruction quality. Such new measurements are denoted as CD, HDand JSD.

4.2 Comparison with State-of-the-art Methods

Table 1: Results of quantitative comparisons. The models were scaled uniformly in a unit cube, so the error metrics are unitless. Here, the values are the average of 13 testing models. See the Supplementary Material for the results of each model.

We compared PUGeo-Net with three methods, i.e., optimization based EAR [23], and two state-of-the-art deep learning based methods, i.e., PU-Net [24] and MPU [26]. For fair comparisons, we retrained PU-Net and MPU with the same dataset as ours. Notice that EAR fails to process the task with R greater than 4, due to the huge memory consumption, and MPU can work only for tasks with R in the powers of 2, due to its

natural cascaded structure. Note that the primary EAR, PU-Net and MPU cannot predict normals.

Quantitative comparisons. Table 1 shows the average result of 13 testing models, where we can observe that PUGeo-Net can achieve the best performance for all upsample factors in terms of all metrics. Moreover, the network size of PUGeo-Net is fixed and much smaller than that of MPU. Due to the deficiency of the independent multi-branch design, the network size of PU-Net grows linearly with the the upsample factor increasing, and is comparable to ours when R = 16.

Visual comparisons. The superiority of PUGeo-Net is also visually demonstrated. We compared the reconstructed surfaces from the input sparse point clouds and the generated dense point clouds by different methods. Note that the surfaces were reconstructed via the same method as [50], in which the parameters “depth” and “minimum number of samples” were set as 9 and 1, respectively. For PU-Net and MPU which fail to predict normals, we adopted PCA normal estimation with the neighbours equal to 16. Here we took the task with R = 16 as an example. Some parts highlighted in red and blue boxes are zoomed in for a close look.

Fig. 4: Visual comparisons for scanned 3D models. Each input sparse 3D point cloud has M = 5000 points and upsampled by a factor R = 16. We applied the same surface reconstruction algorithm to the sparse and densified points by different methods. For each data, the top and bottom rows correspond to the reconstructed surfaces and point clouds, respectively. As the closeup views, PUGeo-Net can handle the geometric details well. See the Supplementary file for more visual comparisons and the video demo.

From Fig. 4, it can be observed that after performing upsampling the surfaces by PUGeo-Net present more geometric details and the best geometry structures, espe-

Fig. 5: Visual comparisons on CAD models with the upsampling factor R = 16. The input point clouds have 5,000 points. We show the surfaces generated using the screened Poisson surface reconstruction (SPSR) algorithm [50]. Due to the low-resolution of the input, SPSR fails to reconstruct the geometry. After upsampling, we observe that the geometric and topological features are well preserved in our results.

cially for the highly detailed parts with complex geometry, and they are closest to the groundtruth ones. We also evaluated different methods on some man-made toy models. Compared with complex statue models, these man-made models consist of flat surfaces and sharp edges, which require high quality normals for surface reconstruction. As illustrated in Fig. 5, owing to the accurate normal estimation, PUGeo-Net can preserve the flatness and sharpness of the surfaces better than PU-Net and MPU. We further investigated how the quality of the reconstructed surface by PUGeo-Net changes with the upsample factor increasing. In Fig. 1, it can be seen that as the upsample factor increases, PUGeo-Net can generate more uniformly distributed points, and the reconstructed surface is able to recover more details gradually to approach the groundtruth surface. See the supplementary material for more visual results and the video demo.

Comparison of the distribution of generated points. In Fig. 6, we visualized a point cloud patch which was upsampled with 16 times by different methods. As PUGeoNet captures the local structure of a point cloud elegantly in a geometry-centric manner, such that the upsampled points are uniformly distributed in the form of clusters. Using PUGeo-Net, the points generated from the same source point are uniformly distributed in the local neighborhood , which justifies our parameterization-based sampling strategy. PU-Net and MPU do not have such a feature. We also observe that our generated points are more uniform than theirs both locally and globally.

Fig. 6: Visual comparison of the distribution of generated 2D points with upsampling factor R = 16. We distinguish the generated points by their source, assigned with colors. The points generated by PUGeo-Net are more uniform than those of PU-Net and MPU.

Table 2: Verification of the effectiveness of our normal prediction. Here, the upsamplin raito R is 8. PCA-* indicates the normal prediction by PCA with various numbers of neighborhoods.

4.3 Effectiveness of Normal Prediction

Moreover, we also modified PU-Net, denoted as PU-Net-M, to predict coordinates and normals joinly by changing the neuron number of the last layer to 6 from 3. PU-Net-M was trained with the same training dataset as ours.

The quantitative results are shown in Table 2, where we can see that (1) the surfaces reconstructed with the normals by PUGeo-Net produces the smallest errors for all the three metrics; (2) the number of neighborhoods in PCA based normal prediction is a heuristic parameter and influences the final surface quality seriously; and (3) the PU-Net-M achieves the worst performance, indicating that a naive design without considering the geometry characteristics does not make sense.

4.4 Robustness Analysis

We also evaluated PUGeo-Net with non-uniform, noisy and real scanned data to demonstrate its robustness.

Fig. 7: 16upsampling results on non-uniformly distributed point clouds.

Non-uniform data. As illustrated in Fig. 7, the data from ShapeNet [30] were adopted for evaluation, where 128 points of each point cloud were randomly sampled without the guarantee of the uniformity. Here we took the upsampling task R = 16 as an example. From Fig. 7, it can be observed that PUGeo-Net can successfully upsample such non-uniform data to dense point clouds which are very close to the ground truth ones, such that the robustness of PUGeo-Net against non-uniformity is validated.

Noisy data. We further added Gaussian noise to the non-uniformly distributed point clouds from ShapeNet, leading to a challenging application scene for evaluation, and various noise levels were tested. From Fig. 8, we can observe our proposed algorithm still works very on such challenging data, convincingly validating its robustness against noise.

Real scanned data. Finally, we evaluated PUGeo-Net with real scanned data by the LiDAR sensor [53]. Real scanned data contain noise, outliers, and occlusions. Moreover, the density of real scanned point clouds varies with the distance between the object and the sensor. As shown in Fig. 9, we can see our PUGeo-Net can produce dense point clouds with richer geometric details.

Fig. 8: 16upsampling results on non-uniform point clouds with various levels of Gaussian noise.

Fig. 9: 16upsampling results on real scanned data by the LiDAR sensor.

4.5 Ablation Study

We conducted an ablation study towards our model to evaluate the contribution and effectiveness of each module. Table 3 shows the quantitative results. Here we took the task with R = 8 as an example, and similar results can be observed for other upsampling factors.

Table 3: Ablation study. Feature recalibration: concatenate multiscale feature directly without the recalibration module. Normal prediction: only regress coordinates of points without normal prediction and supervision. Learned adaptive 2D sampling: use a predefined 2D regular grid as the parametric domain instead of the learned adaptive 2D smapling. Linear transformation: regress coordinates and normals by non-linear MLPs directly without prediction of the linear transformation. Coarse to fine: directly regress coordinates and normals without the intermediate coarse prediction.

From Table 3, we can conclude that (1) directly regressing the coordinates and normals of points by simply using MLPs instead of the linear transformation decreases the upsampling performance significantly, demonstrating the superiority of our geometry-centric design; (2) the joint regression of normals and coordinates are better than that of only coordinates; and (3) the other novel modules, including feature recalibration, adaptive 2D sampling, and the coarse to fine manner, all contribute to the final performance.

To demonstrate the geometric-centric nature of PUGeo-Net, we examined the accuracy of the linear matrix T and the normal displacement for a unit sphere and a unit cube, where the ground-truths are available. We use angle to measure the difference of vectors and , where (i = 1, 2, 3) is the i-th column of T. As Fig. 10 shows, the angle is small with the majority less than 3 degrees, indicating high similarity between the predicted matrix T and the analytic Jacobian matrix. For the unit sphere model, we observe that the normal displacements spread in a narrow range, since the local neighborhood of is small and the projected distance from a neighbor to the tangent plane of is small. For the unit cube model, the majority of the displacements are close to zero, since most of the points lie on the faces of the cube which coincide with their tangent planes. On the other hand, s spread in a relatively wide range due to the points on the sharp edges, which produce large normal displacement.

Fig. 10: Statistical analysis of the predicted transformation matrix normal displacement , which can be used to fully reconstruct the local geometry.

5 Conclusion and Future Work

We presented PUGeo-Net, a novel deep learning based framework for 3D point cloud upsampling. As the first deep neural network constructed in a geometry centric manner, PUGeo-Net has 3 features that distinguish itself from the other methods which are largely motivated by image super-resolution techniques. First, PUGeo-Net explicitly learns the first and second fundamental forms to fully recover the local geometry unique up to rigid motion; second, it adaptively generates new samples (also learned from data) and can preserve sharp features and geometric details well; third, as a by-product, it can compute normals of the input points and generated new samples, which make it an ideal pre-processing tool for the existing surface reconstruction algorithms. Extensive evaluation shows PUGeo-Net outperforms the state-of-the-art deep learning methods for 3D point cloud upsampling in terms of accuracy and efficiency.

PUGeo-Net not only brings new perspectives to the well-studied problem, but also links discrete differential geometry and deep learning in a more elegant way. In the near future, we will apply PUGeo-Net to more challenging application scenarios (e.g., incomplete dataset) and develop an end-to-end network for surface reconstruction. Since PUGeo-Net explicitly learns the local geometry via the first and second fundamental forms, we believe it has the potential for a wide range 3D processing tasks that require local geometry computation and analysis, including feature-preserving simplification, denoising, and compression.

References

1. Sketchfab, https://sketchfab.com 2. Orts-escolano, S., Rhemann, C., Fanello, S., Chang, W., Kowdle, A., Degtyarev, Y., Kim, D., Davidson, P., Khamis, S., Dou, M. Holoportation: Virtual 3d teleportation in real-time.

3. Lafarge, F., Mallet, C. Creating large-scale city models from 3D-point clouds: a robust ap- proach with hybrid representation. International Journal Of Computer Vision. 99, 69–85 (2012)

4. Musialski, P., Wonka, P., Aliaga, D., Wimmer, M., Vangool, L., Purgathofer, W. A survey of urban reconstruction.

5. Xu, Z., Wu, L., Shen, Y., Li, F., Wang, Q., Wang, R. Tridimensional reconstruction applied to cultural heritage with the use of camera-equipped UAV and terrestrial laser scanner. Remote Sensing. 6, 10413–10434 (2014)

6. Bolognesi, M., Furini, A., Russo, V., Pellegrinelli, A., Russo, P. TESTING THE LOW-COST RPAS POTENTIAL IN 3D CULTURAL HERITAGE RECONSTRUCTION.. International Archives Of The Photogrammetry, Remote Sensing & Spatial Information Sciences. (2015)

7. Paine, J., Caudle, T., Andrews, J. Shoreline and sand storage dynamics from annual airborne LIDAR surveys, Texas Gulf Coast. Journal Of Coastal Research. 33, 487–506 (2016)

8. Nie, S., Wang, C., Dong, P., Xi, X., Luo, S., Zhou, H. Estimating leaf area index of maize using airborne discrete-return LiDAR data. Ieee Journal Of Selected Topics In Applied Earth Observations And Remote Sensing. 9, 3259–3266 (2016)

9. Chen, X., Ma, H., Wan, J., Li, B., Xia, T. Multi-view 3d object detection network for au- tonomous driving.

10. Li, B. 3d fully convolutional network for vehicle detection in point cloud. 11. Fioraio, N., Konolige, K. Realtime visual and point cloud slam. 12. Cole, D., Newman, P. Using laser range data for 3D SLAM in outdoor environments. 13. Held, R., Gupta, A., Curless, B., Agrawala, M. 3D puppetry: a kinect-based interface for 3D animation..

14. Santana, J., Wendel, J., Trujillo, A., Su´arez, J., Simons, A., Koch, A. Multimodal location based servicessemantic 3D city data as virtual and augmented reality. (Springe,2017)

15. Hakala, T., Suomalainen, J., Kaasalainen, S., Chen, Y. Full waveform hyperspectral LiDAR for terrestrial laser scanning. Optics Express. 20, 7119–7127 (2012)

16. Kimoto, K., Asada, N., Mori, T., Hara, Y., Ohya, A. Development of small size 3D LIDAR. 17. Lai, W., Huang, J., Ahuja, N., Yang, M. Deep laplacian pyramid networks for fast and accu- rate super-resolution.

18. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y. Residual dense network for image super- resolution.

19. Alexa, M., Behr, J., Cohen-or, D., Fleishman, S., Levin, D., Silva, C. Computing and render- ing point set surfaces. Ieee Transactions On Visualization And Computer Graphics. 9, 3–15 (2003)

20. Lipman, Y., Cohen-or, D., Levin, D., Tal-ezer, H. Parameterization-free projection for geom- etry reconstruction.

21. Huang, H., Li, D., Zhang, H., Ascher, U., Cohen-or, D. Consolidation of unorganized point clouds for surface reconstruction. Acm Transactions On Graphics (tog). 28, 176 (2009)

22. Preiner, R., Mattausch, O., Arikan, M., Pajarola, R., Wimmer, M. Continuous projection for fast L1 reconstruction.. Acm Transactions On Graphics (tog). 33, 47–1 (201)

23. Huang, H., Wu, S., Gong, M., Cohen-or, D., Ascher, U., Zhang, H. Edge-aware point set resampling. Acm Transactions On Graphics (tog). 32, 9 (2013)

24. Yu, L., Li, X., Fu, C., Cohen-or, D., Heng, P. Pu-net: Point cloud upsampling network.

675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719

25. Yu, L., Li, X., Fu, C., Cohen-or, D., Heng, P. EC-Net: an Edge-aware Point set Consolidation Network.

26. Wang, Y., Wu, S., Huang, H., Cohen-or, D., Sorkine-hornung, O. Patch-based Progressive 3D Point Set Upsampling.

27. Qi, C., Yi, L., Su, H., Guibas, L. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.

28. Maturana, D., Scherer, S. Voxnet: A 3d convolutional neural network for real-time object recognition.

29. Riegler, G., Osmanulusoy, A., Geiger, A. Octnet: Learning deep 3d representations at high resolutions.

30. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J. 3d shapenets: A deep representation for volumetric shapes.

31. Rieu, L., Simonovsky, M. Large-scale point cloud semantic segmentation with superpoint graphs.

32. Te, G., Hu, W., Zheng, A., Guo, Z. Rgcnn: Regularized graph cnn for point cloud segmenta- tion.

33. Qi, C., Su, H., Mo, K., Guibas, L. Pointnet: Deep learning on point sets for 3d classification and segmentation.

34. Wang, Y., Sun, Y., Liu, Z., Sarma, S., Bronstein, M., Solomon, J. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog). 38, 146 (2019)

35. Komarichev, A., Zhong, Z., Hua, J. A-CNN: Annularly Convolutional Neural Networks on Point Clouds.

36. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B. Pointcnn: Convolution on x-transformed points.

37. Campen, M., Bommes, D., Kobbelt, L. Quantized Global Parametrization. Acm Transactions On Graphics (tog). 34, 192:1–192:12 (2015)

38. Hormann, K., Greiner, G. MIPS: An Efficient Global Parametrization Method. Curve And Surface Design: Saint-mal. 2000 pp. 10 (2012)

39. Xianfeng, G., Shingtung, Y. Global Conformal Parameterization. 40. Jaderberg, M., Simonyan, K., Zisserman, A. Spatial transformer networks. 41. Ronneberger, O., Fischer, P., Brox, T. U-net: Convolutional networks for biomedical image segmentation.

42. He, K., Zhang, X., Ren, S., Sun, J. Deep residual learning for image recognition. 43. Huang, G., Liu, Z., Vandermaaten, L., Weinberger, K. Densely connected convolutional net- works.

44. Hu, J., Shen, L., Sun, G. Squeeze-and-excitation networks. 45. Zhang, H., Goodfellow, I., Metaxas, D., Odena, A. Self-Attention Generative Adversarial Networks.

46. Yang, Y., Feng, C., Shen, Y., Tian, D. Foldingnet: Point cloud auto-encoder via deep grid deformation.

47. Deng, H., Birdal, T., Ilic, S. Ppfnet: Global context aware local features for robust 3d point matching.

48. Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M. Pcn: Point completion network.

720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764

Designed for Accessibility and to further Open Science