Transparent AR Markers

Year Active: 2023

We explored the affordances that transparent PVC films offer for tangible interactions. Stack AR introduces a novel alternative input method for augmented reality: stacking transparent fiducial markers as a tangible, compositional interface. Unlike traditional AR systems that rely on single, opaque markers or touchscreens, Stack AR enables the computer system to recognize the combination of layers. I made a music synthesis game Stackable Music, to showcase how it can be used in AR games.

If you are interested in the technical details, please read (Chen et al., 2023).

Technical Highlight

Custom ArUco Marker Library
A bespoke ArUco dictionary was created to support marker stacking on transparent film, minimizing interference and ensuring that stacked combinations generate unique, detectable codes. Marker regions are designed in concentric “rings,” allowing selective stacking and clear layer separation.
Computer Vision Integration in Unity
The Unity application employs the OpenCV for Unity plugin to enable real-time marker recognition and pose estimation from a top-down camera. Custom logic parses the detected markers, identifies their combination, and maps them to distinct musical or visual elements in the AR environment.
Detection and Interactive Synthesis
By precomputing all viable marker combinations, Stack AR can identify both individual and stacked markers in a single frame, triggering the correct set of audio, 3D models, and particle effects. The system is resilient to partial occlusion, variable lighting, and the material challenges of transparent.

Presentation

Stack AR was selected for presentation at the MIT Reality Hack 2025 (Jan 23–27) as part of an art grant. The project can be easily set up with a desktop, camera, and printed transparent markers.

Demo Video

Presentation Video

References

2023

Stackable Music: A Marker-Based Augmented Reality Music Synthesis Game

Max Chen, Shano Liang, and Gillian Smith

In Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play, Stratford, ON, Canada, Mar 2023

Abs DOI Bib

Augmented reality (AR) allows the rendering of digital content on top of the physical space, which is a promising medium for tangible interaction. Marker-based AR is widely used thanks to its low cost and ease of integration, but the gameful aspect of manipulating the physical AR markers remains understudied. In this paper, we explored the stacking mechanics of transparent AR markers and described the creation of an AR music game called Stackable Music. Stackable Music can be developed, assembled, and set up at the home or office with a printer using several sheets of transparent film and a PC or mobile device with a camera.
@inproceedings{chen2023stackable, author = {Chen, Max and Liang, Shano and Smith, Gillian}, title = {Stackable Music: A Marker-Based Augmented Reality Music Synthesis Game}, year = {2023}, isbn = {9798400700293}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3573382.3616071}, doi = {10.1145/3573382.3616071}, booktitle = {Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play}, pages = {22–28}, numpages = {7}, keywords = {AR Game, Marker-based AR, Tangible Interaction, Transparent AR Markers}, location = {Stratford, ON, Canada}, series = {CHI PLAY Companion '23} }