Interacting with Ancient Poem
An interactive installation designed to interact with the ancient poems through people's hands shadow.
project info
Duration
Nov.2021 (2 weeks)
My role
Concept Design
Visual Design
Programmer
Video producing
Key words
Computer Vision
Display Design
Interactive Installation
Group members
Xinyi GUO
Pin HU
Overview
Introduction
Interacting with Ancient Poem
Getting inspiration from animals in many poems in ancient China, in this project, we designed an interactive installation to let people interact with the ancient poem with their hand's shadows. The interaction was created with the use of the projector. Through the projector's light, people can clearly see the shape of their hand's shadow (as shown below).
👆Figure 1: Butterfly-like shape in hands shadow, photo from our project
Final Design
Interacting Process
People can make animal-like hands shadow the wall to emulate some animals in ancient Chinese poems. Then our camera would catch your hand shadow's shape and recognize the animal you emulated. The corresponding animals in the poem would swim or fly into the frames, and a visualized animation of the Chinese poem would be projected on the wall accordingly. At the same time, the poem's characters would gradually reveal on the wall.
Of course, before people make a gesture, there is a hint in the animation to tell them what kind of animal gestures can be recognized by the computer.
👆Concept Video
Design process
Background
Childhood Memory: A Hand Shadow Game
When we were young, we didn't have many toys, so our parents often played hand shadow games with us. It can create fascinating images when making simple shapes with our hands and projecting the image onto a wall under the light of a lamp. At that time, I marveled at the lifelike images that a random gesture could transform. The game became one of my favorite childhood memories.
Design Concept
"Edutainment"
Edutainment is a scientific and interesting method of children's enlightenment. Through shadow games and interactive combinations of ancient poems, children can learn ancient poems.
So we designed an installation for children to interact with ancient poems: we selected several ancient poems with animals and visualized the content of these poems into several animations in ancient style. Through the shadow of their hands in the projection, children can make some hand shadows similar to the relevant animals and then trigger the corresponding poem animation containing that animal to step on the wall.
Visual Design
The homepage animation
Poem TianJingSha · Autumn
Main Animal: Bird
《天净沙·秋》
孤村落日残霞，轻烟老树寒鸦，一点飞鸿影下。
青山绿水，白草红叶黄花。
Poem Visiting Taoist Priest Dai Tianshan failed
Main Animal: Dog
《访戴天山道士不遇》
犬吠水声中，桃花带露浓。
树深时见鹿，溪午不闻钟。
野竹分青霭，飞泉挂碧峰。
无人知所去，愁倚两三松。
Poem Spring Day
Main Animal: Goose
《春日》晁冲之
阴阴溪曲绿交加，小雨翻萍上浅沙。
鹅鸭不知春去尽，争随流水趁桃花。
Poem River Melody
Main Animal: Butterfly
杜甫 《曲江二首》
穿花蛱蝶深深见，点水蜻蜓款款飞。
传语风光共流转，暂时相赏莫相违。
Technology Implementation
We used the machine learning engine provided by Ali Cloud to train the image recognition of hand shadow. The model trained by Ali Cloud platform was used to identify the shape of people's hand shadows on the scene. Finally, through the interface of Python SDK, different images recogniction results would trigger transition of different animations of poems.
👆Ali Cloud Visual model training platform
Video switching logic through Python: Different image recognition results from Ali Cloud trigger the system to play different videos
👆Image Recognition Results
Exhibition Design
-
Sketch
Use the projector to project the animation. Use the camera behind the audience to identify hand shadow.
-
On-site Comissioning
