Emoji Scavenger Hunt
URL: https://emojiscavengerhunt.withgoogle.com/ (Works best on phones)
Google Brand Studio project, in collaboration with Google PAIR.
Emoji Scavenger Hunt is a browser-based game built with machine learning that uses your phone’s camera and a neural network to try and guess what it’s seeing. Powered by Tensorflow.js, the game is built to run efficiently on your phone’s web browser without needing to access backend servers. Importantly, no images from your camera are collected or stored.
Brief: “Humanize Google (and A.I.)”
In 2017, Google's CEO announced a shift from "Mobile first to AI first" at Google I/O, underlining a new focus on AI for all product experiences. As part of the brand team, our challenge was to change people’s perceptions of AI. With AI being a relatively new technology, people had mostly negative or uncertain impressions. Therefore, we aimed to demystify AI by showcasing the benefits and possibilities of the technology, thereby humanizing it.
Background and Design Process:
While exploring this new technology, we discovered one of the research projects, "deeplearn.js," an early version of tensorflow.js. This Javascript library enables running machine learning models on a browser. We developed a prototype for on-device, browser-based image classification using JavaScript, which helped us understand this technology's unique aspects.
Real-time ML based image classifications on Chrome with Pixel2 XL. The debug window shows the ML model updating a list of detected objects and confidence level scores about 15 times a second. Try it for yourself by accessing this URL.
Our analysis highlighted three unique experiences made possible by this technology:
Works on the phone's browser.
Exceptionally fast – it can run about 15 times per second, thanks to on-device machine learning.
Makes forgivable mistakes – the AI often gets confused, such as mistaking chili peppers for strawberries or bamboo shoots for potato fries.
Guided by these insights, we developed a fun browser game to demystify AI and demonstrate tensorflow.js. We announced the project at the 2018 TensorFlow Summit and Google I/O 2018 as one of the product demos and open-sourced it to inspire web developers to incorporate TensorFlow.js into their work.
My Role:
As the Creative Lead, my responsibilities included:
Identifying potential opportunities and setting creative direction.
Facilitating cross-functional collaboration between the Research team and Brand Studio.
Leading the UI/UX and game design, and overseeing sound design.
Spearheading go-to-market strategy and providing creative direction for marketing materials.
Crafting presentation narratives for the TensorFlow Summit.
UI Design with animations created in AfterEffects, visualizing how the final UX would look and feel like.
UI Design with animations created in AfterEffects, visualizing how the final UX would look and feel like.
Level Design: To guarantee a balanced game level and a delightful user experience, all available emojis were cross-referenced with objects that the machine learning model could identify. These were then mapped based on certain level criteria to ensure gameplay balance."
Additionally, I wrote a case study and UX article for Google Design Library, titled "ML and the Evolution of Web-Based Experiences," to help designers learn how to utilize ML for UX design. My contributions to this project earned me a VP award.
UX article for Google Design Library, titled "ML and the Evolution of Web-Based Experiences"
Result:
Without any paid promotions, the project was organically shared and gained popularity on social media. Within a few months after the launch, users hunted 3 million objects, including 300,000💡 and 280,000 pairs of👖. The GitHub repository gained 800 stars and 200 forks.
Although our target audience was primarily developers, the project has also been widely shared by school teachers and used in classrooms (as it works on Chromebooks!). We were delighted to see the project being used to introduce AI to students.
Selected Honors:
Site of the day from FWA.