Conversation Garden
Timeline
Tools Used
November - December 2020
As schools transitions into a online space, we find classrooms experiencing a common problem of a lack of conversation. Therefore, using Google's Teachable Machine, we wanted to create a Zoom function that would motivate student to make more engagement.
HTML/CSS/JS/p5js
Teachable Machine
Collaborators
Alice Fang, Charmaine Qiu, Mimi Jiao, Sebastian Carpenter
Customize your own conversation flower
See your flower accompany your calls
& see it grow as you talk
Balanced conversations,
increase virtual engagement,
and always have fun!
The Challenge
In order to encourage more conversation in online classrooms, we wanted to some sort of engagement that would reward students for more contribution but also not penalize those who don't talk. This is so that we could incentivize students to talk rather than making people feel uncomfortable if they choose not to show their screen.
Process
Taking care of your own engagement flower
We brainstormed multiple options in visuals that we could do to achieve this such as how students can earn coins to then "purchase" Zoom accessories for their videos. But this could be distracting to the class environment, therefore, we pursuing the idea where each student is responsible for growing their own conversation flower.
Amplifying gestures in real-life conversations
As part of our system, I felt that including features that could help the professor further evaluate the tone of the classroom and amplify natural gestures that one would make in conversation would further add complexity and meaning to our project. This resulted to the team exploring how to incorporate gesture recognition with the voice recognition from Teachable Machine.
Face tracking
I tested using PoseNet for our gesture, however, this program only works well in regards to the whole body. Therefore, we used BRF (Beyond Race Reality) that only tracks the face instead. We also explored how visuals would respond such as expand in size (seen on the leftmost image) as someone talks for longer periods of time.
👎
👍
✋
🕐
🔇
🔈
🔊
Portraying key gestures
We decided on certain gestures to implement into the flower characters such as thumbs up/down, hand raise, and timeout (away). We decided on poses that were static because this would be captured by the BFL software more accurately than a pose that changed over time (such as clapping).
Connecting with OBS camera with Zoom
With our animations implemented into our code with Teachable Machine and BFR, we then used OBS camera to use with Zoom. This is a snapshot of a conversation taking place on Zoom with our program.
Expanding our system with customization & Future steps
For our futures steps, we would have wanted to create a website where users are able to create their own flower character and use Teachable Machine to match with their own faces for accurate gesture recognition.
