Augmented Sketching as Interactive Markers: Generating Heightmaps in Mobile AR

Augmented reality (AR) has traditionally relied on predefined visual markers or spatial tracking to anchor virtual content, limiting user agency in shaping the interaction space. This constraint presents challenges for creative tasks where expressive, tangible input is desired. To address this gap, we explore the potential of hand-drawn sketches as interactive AR markers that allow users to organically embed meaning and function into physical drawings. While prior work separates the physical act of drawing from digital augmentation, few approaches unify the two into a cohesive, real-time experience, especially on mobile platforms. We introduce Augmented Sketching, a mobile AR approach that transforms freehand sketches into 3D heightmaps. By mapping contrast to elevation, the approach creates manipulable spatial structures that respond to user input. This approach reclaims analogue drawing as an active interface, enabling a more seamless blend between physical creativity and virtual feedback. Applications range from interactive art to educational tools and game mechanics, supporting new forms of spatial storytelling and tangible interaction in mobile AR applications.