Interacting with Ancient Poem
An interactive installation designed to interact with the ancient poems through people's hands shadow.
Nov.2021 (2 weeks)
Interacting with Ancient Poem
Getting inspiration from animals in many poems in ancient China, in this project, we designed an interactive installation to let people interact with the ancient poem with their hand's shadows. The interaction was created with the use of the projector. Through the projector's light, people can clearly see the shape of their hand's shadow (as shown below).
👆Figure 1: Butterfly-like shape in hands shadow, photo from our project
People can make animal-like hands shadow the wall to emulate some animals in ancient Chinese poems. Then our camera would catch your hand shadow's shape and recognize the animal you emulated. The corresponding animals in the poem would swim or fly into the frames, and a visualized animation of the Chinese poem would be projected on the wall accordingly. At the same time, the poem's characters would gradually reveal on the wall.
Of course, before people make a gesture, there is a hint in the animation to tell them what kind of animal gestures can be recognized by the computer.
Childhood Memory: A Hand Shadow Game
When we were young, we didn't have many toys, so our parents often played hand shadow games with us. It can create fascinating images when making simple shapes with our hands and projecting the image onto a wall under the light of a lamp. At that time, I marveled at the lifelike images that a random gesture could transform. The game became one of my favorite childhood memories.
Edutainment is a scientific and interesting method of children's enlightenment. Through shadow games and interactive combinations of ancient poems, children can learn ancient poems.
So we designed an installation for children to interact with ancient poems: we selected several ancient poems with animals and visualized the content of these poems into several animations in ancient style. Through the shadow of their hands in the projection, children can make some hand shadows similar to the relevant animals and then trigger the corresponding poem animation containing that animal to step on the wall.
The homepage animation
Poem TianJingSha · Autumn
Main Animal: Bird
Poem Visiting Taoist Priest Dai Tianshan failed
Main Animal: Dog
Poem Spring Day
Main Animal: Goose
Poem River Melody
Main Animal: Butterfly
We used the machine learning engine provided by Ali Cloud to train the image recognition of hand shadow. The model trained by Ali Cloud platform was used to identify the shape of people's hand shadows on the scene. Finally, through the interface of Python SDK, different images recogniction results would trigger transition of different animations of poems.
👆Ali Cloud Visual model training platform
Video switching logic through Python: Different image recognition results from Ali Cloud trigger the system to play different videos
👆Image Recognition Results
Use the projector to project the animation. Use the camera behind the audience to identify hand shadow.