Model-Guided 3D Sketching

IEEE Transactions on Visualization and Computer Graphics

Pengfei Xu1       Hongbo Fu2      Youyi Zheng3      Karan Singh4      Hui Huang1       Chiew-Lan Tai5

1Shenzhen University     2City University of Hong Kong     3Zhejiang University     4University of Toronto     5HKUST

Fig. 1. Our model-guided 3D sketching system allows users to create interesting sketch-based redesign of existing 3D models with ease. The right in each example is the physical realization of 3D sketches around 3D-printed models.


We present a novel 3D model-guided interface for in-situ sketching on 3D planes. Our work is motivated by evolutionary design, where existing 3D objects form the basis for conceptual re-design or further design exploration. We contribute a novel workflow that exploits the geometry of an underlying 3D model to infer 3D planes on which 2D strokes drawn that are on and around the 3D model should be meaningfully projected. This provides users with the nearly modeless fluidity of a sketching interface, and is particularly useful for 3D sketching over planes that are not easily accessible or do not preexist. We also provide an additional set of tools, including sketching with explicit plane selection and model-aware canvas manipulation. Our system is evaluated with a user study, showing that our technique is easy to learn and effective for rapid sketching of product design variations around existing 3D models.

Fig. 3. Main interface of our system.

Fig. 5. Inference-enabled sketching workflow: first sketching (a) and then 3D plane confirmation (b), followed by in-plane sketching (c). These steps are repeated to add sketches plane by plane (d), leading to the final result (e).

Fig. 14. Gallery of 3D sketches generated by our system.


We would like to thank Jiyuan Zhang for the initial exploration of this project, the reviewers for their constructive comments, and the user study participants for their time. This work was partially supported by grants from NSFC(61502306, 61602310, 61522213, 61761146002, 61861130365), Guangdong Science and Technology Program (2015A030312015), Shenzhen Innovation Program (JCYJ20170302154106666, KQJSCX20170727101233642, JCYJ20151015151249564), the Research Grants Council of HKSAR (CityU11300615, CityU11204014, HKUST16201315, HKUST16210718), NSERC, and the Open Project Program of State Key Lab. of Virtual Reality Technology and Systems, Beihang University (VRLAB2018C11).


title = {Model-Guided 3D Sketching},
author = {Pengfei Xu and Hongbo Fu and Youyi Zheng and Karan Singh and Hui Huang and Chiew-Lan Tai},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {25},
number = {10},
pages = {2927-2939},  
year = {2019},

Downloads (faster for people in China)

Downloads (faster for people in other places)