Offsite Aerial Path Planning for Efficient Urban Scene Reconstruction

ACM Transactions on Graphics (Proceedings of SIGGRAPH ASIA 2020)

Xiaohui Zhou1    Ke Xie1    Kai Huang1    Yilin Liu1    Yang Zhou1    Minglun Gong2   Hui Huang1*

1Shenzhen University    2University of Guelph

Fig. 1. With a 2D map and a satellite image, we can plan an aerial path offsite that successfully reconstructs a large region with only 875 images.


With rapid development in UAV technologies, it is now possible to reconstruct large-scale outdoor scenes using only images captured by low-cost drones. The problem, however, becomes how to plan the aerial path for a drone to capture images so that two conflicting goals are optimized: maximizing the reconstruction quality and minimizing mid-air image acquisition effort. Existing approaches either resort to pre-defined dense and thus inefficient view sampling strategy, or plan the path adaptively but require two onsite flight passes and intensive computation in-between. Hence, using these methods to capture and reconstruct large-scale scenes can be tedious. In this paper, we present an adaptive aerial path planning algorithm that can be done before the site visit. Using only a 2D map and a satellite image of the to-be-reconstructed area, we first compute a coarse 2.5D model for the scene based on the relationship between buildings and their shadows. A novel Max-Min optimization is then proposed to select a minimal set of viewpoints that maximizes the reconstructability under the the same number of viewpoints. Experimental results on benchmark show that our planning approach can effectively reduce the number of viewpoints needed than the previous state-of-the-art method, while maintaining comparable reconstruction quality. Since no field computation or a second visit is needed, and the view number is also minimized, our approach significantly reduces the time required in the field as well as the off-line computation cost for multi-view stereo reconstruction, making it possible to reconstruct a large-scale urban scene in a short time with moderate effort.

Fig. 2. Our pipeline for constructing a 2.5D proxy with a no-fly zone for the drone, given a map and a satellite image of the to-be-reconstructed area.

Fig. 5. A running example of our MaxiMin optimization on a synthetic scene (NY-1 from [Smith et al. 2018]). All surface samples are color coded according to the reconst ructability values and the viewpoints are coded using redundancy values, where red corresponds to a higher value and blue to a lower one. The initial view set of our method is very dense, which leads high surface reconstructability and high viewpoint redundancy (first column). The first step of redundancy minimization reduces the view number, hence both the samples’ reconstructability and viewpoints’ redundancy are decreased, and eventually become uniformly distributed; see the transition from the first to fourth column). In the second step of reconstructability maximization, although we may hardly observe changes on viewpoint redundancy, the increment on the reconstructability can be seen on quite a few sample points, i.e., dark blue turns to lighter blue in the last column.

Fig. 11. Comparison between our method (left) and [Smith et al. 2018] (right) on virtual scene NY-1. The bottom row shows zoomed-in views of the final reconstructions at labeled areas, where red boxes are our results and blue ones are from [Smith et al. 2018]. Note that the proxy, planned viewpoints as well as the trajectory of [Smith et al. 2018] are all acquired from their released file, where 433 views are used. Although our method only uses 248 images, the final reconstruction quality are quite comparable to theirs; see Table 6, row 1 vs. row 4, for the quantitative comparison.

Fig. 13. Running Plan3D under different numbers of views, where the leftmost visualizes the reconstruction result of Colmap by an initial flight, which serves as the geometric proxy for space voxelization in Plan3D. At the rightmost, we show a zoomed-in comparison on a fine detailed region between our method and the final reconstructions of Plan3D. For the complete reconstructed model and more detailed visualizations of our result on NY-1, please refer to Fig. 11.

Fig. 16. Experiments on four real scenes. The top row shows the proxies we built, as well as the planned path and viewpoints by our method. The next two rows show the reconstruction results with and without texture mapping, respectively. Specifically, the proxy of Hitech has three layers, which was reconstructed from three footprints using different extrusion heights. The two lower layers used 2D building footprints extracted from Google map, whereas the top one uses a manual drawn rectangular shape. The heights of the three layers were evaluated from shadows whose lengths were manually verified. The last two scenes, ArtSci & Gym, contain dense, low-rise buildings, some of which are also connected by walking bridges. Extracting accurate proxies for individual buildings is difficult and unnecessary. Hence, coarse proxies based on the overall shapes of the collections of buildings are used instead, which is more effective.

Fig. 17. Comparison with oblique photography conducted by DJI-Terra on scene Residency. From left to right, top row shows the 2.5D proxy we built, the final reconstruction overlaid with the path planned by our algorithm, and the reconstruction result of oblique photography overlaid with the path planned by DJI-Terra. Four comparisons on the local region details are shown at the bottom row (middle and left), where red box relates to our result, and blue shows the result of oblique photography. The plot at the bottom left shows the height distributions of viewpoints planned by both two methods, where the drone is clearly kept at a fixed high elevation by oblique photography. Our approach instead provides many tilted viewpoints at different low elevations, making it possible to capture the ground objects in more detail and observe the building facade well from the side.


We sincerely thank the reviewers for their valuable comments. This work was supported in parts by NSFC (61861130365, 61761146002), GD Talent Plan (2019JC05X328), Guangdong Science and Technol- ogy Program (2020A0505100064, 2018KZDXM058, 2018A030310441, 2015A030312015), LHTD (20170003), National Engineering Labora- tory for Big Data System Computing Technology, and Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ).



title = {Offsite Aerial Path Planning for Efficient Urban Scene Reconstruction},

author = {Xiaohui Zhou and Ke Xie and Kai Huang and Yilin Liu and Yang Zhou and Minglun Gong and Hui Huang},

journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH ASIA 2020)},

volume = {39},

number = {6},

pages = {192:1--192:16},

year = {2020},


Downloads (faster for people in China)

Downloads (faster for people in other places)