top of page

Interacting with Ancient Poem

An interactive installation designed to interact with the ancient poems through people's  hands shadow.

project info


Nov.2021 (2 weeks)

My role

Concept Design

Visual Design


Video producing

Key words

Computer Vision

Display Design

Interactive Installation 

Group members

Xinyi GUO

Pin HU

Interacting with Ancient Poem

Getting inspiration from animals in many poems in ancient China, in this project, we designed an interactive installation to let people interact with the ancient poem with their hand's shadows. The interaction was created with the use of the projector. Through the projector's light, people can clearly see the shape of their hand's shadow (as shown below). 

截屏2022-01-13 20.22.47.png

👆Figure 1: Butterfly-like shape in hands shadow, photo from our project

Final Design
Interacting Process

People can make animal-like hands shadow the wall to emulate some animals in ancient Chinese poems. Then our camera would catch your hand shadow's shape and recognize the animal you emulated. The corresponding animals in the poem would swim or fly into the frames, and a visualized animation of the Chinese poem would be projected on the wall accordingly. At the same time, the poem's characters would gradually reveal on the wall.

Of course, before people make a gesture, there is a hint in the animation to tell them what kind of animal gestures can be recognized by the computer.

👆Concept Video

Anchor 1
Design process
Childhood Memory: A Hand Shadow Game

When we were young, we didn't have many toys, so our parents often played hand shadow games with us. It can create fascinating images when making simple shapes with our hands and projecting the image onto a wall under the light of a lamp. At that time, I marveled at the lifelike images that a random gesture could transform. The game became one of my favorite childhood memories.

截屏2022-01-13 21.34.20.png
Design Concept

Edutainment is a scientific and interesting method of children's enlightenment. Through shadow games and interactive combinations of ancient poems, children can learn ancient poems. 

So we designed an installation for children to interact with ancient poems: we selected several ancient poems with animals and visualized the content of these poems into several animations in ancient style. Through the shadow of their hands in the projection, children can make some hand shadows similar to the relevant animals and then trigger the corresponding poem animation containing that animal to step on the wall.

Visual Design

The homepage animation


Poem TianJingSha · Autumn

Main Animal: Bird





Poem Visiting Taoist Priest Dai Tianshan failed 

Main Animal: Dog







Poem Spring Day

Main Animal: Goose




Poem River Melody

Main Animal: Butterfly

杜甫  《曲江二首》


Technology Implementation

We used the machine learning engine provided by Ali Cloud to train the image recognition of hand shadow. The model trained by Ali Cloud platform was used to identify the shape of people's hand shadows on the scene. Finally, through the interface of Python SDK, different images recogniction results would trigger transition of  different animations of poems.

截屏2022-01-14 20.35.51.png
截屏2022-01-14 20.36.05.png

👆Ali Cloud Visual model training platform


Video switching logic through Python: Different image recognition results from Ali Cloud trigger the system to play different videos


👆Image Recognition Results

Exhibition Design
  • Sketch

Use the projector to project the animation. Use the camera behind the audience to identify hand shadow.

截屏2022-01-14 21.01.02.png
  • On-site Comissioning

The end
bottom of page