Proactive 3D Scanning of Inaccessible Parts

ACM Trans. on Graphics (Proceedings of SIGGRAPH 2014)

Feilong Yan1     Andrei Sharf2     Wenzhen Lin1     Hui Huang1     Baoquan Chen1,3
1Shenzhen VisuCA Key Lab / SIAT     2Ben Gurion University     3Shandong University

Figure 1: Proactive scanning of a tangled Banana tree. We freely move the scanner around to capture the scene while physically movingaside occluding leaves to scan the trunk and branches (a). In (b) we zoom


The evolution of 3D scanning technologies have revolutionized the way real-world object are digitally acquired. Nowadays, highdefinition and high-speed scanners can capture even large scale scenes with very high accuracy. Nevertheless, the acquisition of complete 3D objects remains a bottleneck, requiring to carefully sample the whole object’s surface, similar to a coverage process. Holes and undersampled regions are common in 3D scans of complex-shaped objects with self occlusions and hidden interiors. In this paper we introduce the novel paradigm of proactive scanning, in which the user actively modifies the scene while scanning it, in order to reveal and access occluded regions. We take a holistic approach and integrate the user interaction into the continuous scanning process.

Our algorithm allows for dynamic modifications of the scene as part of a global 3D scanning process. We utilize a scan registration algorithm to compute motion trajectories and separate between user modifications and other motions such as (hand-held) camera movements and small deformations. Thus, we reconstruct together the static parts into a complete unified 3D model. We evaluate our technique by scanning and reconstructing 3D objects and scenes consisting of inaccessible


Figure 2: Overview of our algorithm. We initially over-segment scanned frames into piecewise smooth patches(a). Next, we perform pairwise non-rigid registration of consecutive frames (b) and compute trajectory vectors. We cluster long trajectories (c), belonging to the user’s interactive modifications, and accurately reconstruct the complete scene (d-e).

Figure 3: Modification motion guides registration between exterior and interior parts. Given an exterior (a), an interior (c) and a modification(b) frame, we demonstrate incorrect registration of (a)+(c) in (d) vs. correctly registering (a)+(b)+(c) in (e). Note that in (b), motion of the doors is segmented out by our algorithm, leaving only static parts to guide full registration.

Figure4: Results of our proactive scanning with piecewise rigid modifications of scene parts. Left-to-right are three individual frames, the modification motion, full 3D registration and reconstruction.

Figure 10: Proactive scanning with non-rigid modifications of scene parts. Left-to-right is the modification motion, a zoom-in into motion,complete 3D registration and reconstruction.

We thank Prof. Daniel Cohen-Or for insightful discussions and reviewers for their valuable comments. This work is supported in part by the NSFC (61232011, 61379090, 61161160567), National 863 Program (2013AA01A604), CAS Young Scientists Plan (2013Y1GA0007), Shenzhen Innovation & Technology Program (CXB201104220029A, ZD201111080115A, KC2012JSJS0019A, JSGG20130624154940238, JCYJ20120617114842361), the Israel Science Foundation and the European FP7.


@ ARTICLE {yan2014proactive,

    title={Proactive 3D scanning of inaccessible parts},  author={Yan, Feilong and Sharf, Andrei and Lin, Wenzhen and Huang, Hui and Chen, Baoquan}, 

    journal={ACM Transactions on Graphics (TOG)},







Copyright © 2016-2018 Visual Computing Research Center