duolingo snapshots

Imagining a camera learning function to bring the real world into the Duolingo app for more immersive language learning.


Timeline: 3 Weeks

Roles: Product Designer, UX Researcher

1.0: Bridging The Gap Between Duolingo & Its Heritage Speakers


“Anyone can learn a language with Duolingo”: it’s the mobile language learning app that aims to provide functional, personalized, and fun language learning to all, with over 6 million English speaking users actively learning Duolingo Chinese.

According to a special Duolingo language report on Asian languages, Chinese learners are more motivated to study for family reasons than learners of other languages, and nearly 3.5 million people in the US are speaking Chinese as a second language in the home.

Many of these Chinese learners are heritage speakers. Heritage speakers are people who have learned a certain language informally from exposure home, and are often fluent in speaking and listening — but weaker at reading and writing about subjects beyond personal experiences due to a lack of formal learning.

But how does Duolingo address Chinese heritage speakers’ desire to effectively improve their reading and writing skills, given their unique spoken fluency? And how might we how might we adapt language learning for more experienced language learners? 

2.0: Taking Advantage of "Language Touchpoints"


I dove further into the needs and pain points of Chinese heritage speakers by interviewing 5 students who learned Chinese from familial exposure, and actively speak the language at home. Participants were asked about their desired fluency in Chinese, past learning experiences, and obstacles to learning as a Chinese heritage speaker.

We can see that heritage speakers have a unique experience in the language learning world — they are often already surrounded by language “touchpoints,” or places where they encounter the language in their daily lives, due to their day-to-day immersion in the culture.

Therefore, it seems that heritage speakers would learn best by taking advantage of their “touchpoints” or existing immersion. So…

How might we bring real world touchpoints into the Duolingo app for more immersive language learning?

 3.0: Shortcomings: Limited Phrases & Revisitation


To understand how Duolingo currently helps its users practice the language and bring real-world practice into learning, I did an audit of the tips page shown on each unit, and the Practice Hub’s collections.

Tips

While the tips page is helpful for describing concepts in each section, it lacks a specific definition page for phrases or characters of interest. This leaves learners having to guess certain phrases, and definitions’ nuance is often negated. The tips page also lacks examples for specific phrases or characters, causing a constrained sense of learning. 

Collections

Next, the mistake collections appears as just a list of phrases to be reviewed. If a user wants to refer to all of their collected phrases so far, it only displays a translation and its pronunciation. It has no cues for the learner to remember where or when they made this mistake, giving users no connection to the phrases they’d previously missed.

4.0: Low-Fi Mockups


Through my ideation while keeping Duolingo's approach in mind, I opted to build a camera tool for Chinese learners to capture familiar artifacts from the world around them — if heritage speakers are having trouble with reading media that’s interesting to them, why not save and practice those phrases in particular? This significantly expands the vocabulary that Duolingo users practice when using the app, and ensures that they’re truly relevant and words of interest for the user.

  1. Upon navigating to the camera icon in the navigation bar, users are prompted by Duo to begin “capturing snapshots” with their phone camera.

  1. Users can change between languages they’re currently learning.

  1. From there, recognized phrases and words will be highlighted, where users can select and confirm the phrase of interest.

  1. They are then taken to the phrase’s definition page, where the pronunciation, definition, and example phrases are provided.

  1. Inside the Practice Hub, “Snapshots” are now an additional collection and a type of daily review.

  1. The Snapshots collection displays all the previously collected phrases alongside their respective pictures.


4.0: Low-Fi Mockups


Through my ideation while keeping Duolingo's approach in mind, I opted to build a camera tool for Chinese learners to capture familiar artifacts from the world around them — if heritage speakers are having trouble with reading media that’s interesting to them, why not save and practice those phrases in particular? This significantly expands the vocabulary that Duolingo users practice when using the app, and ensures that they’re truly relevant and words of interest for the user.

  1. Upon navigating to the camera icon in the navigation bar, users are prompted by Duo to begin “capturing snapshots” with their phone camera.

  1. Users can change between languages they’re currently learning.

  1. From there, recognized phrases and words will be highlighted, where users can select and confirm the phrase of interest.

  1. They are then taken to the phrase’s definition page, where the pronunciation, definition, and example phrases are provided.

  1. Inside the Practice Hub, “Snapshots” are now an additional collection and a type of daily review.

  1. The Snapshots collection displays all the previously collected phrases alongside their respective pictures.


5.0: Meet Duolingo Snapshots!


Introducing the interactive prototype of Duolingo Snapshots! With Snapshots, Super Duolingo users are able to capture any written media around them to save and practice, aiding their learning with relevant, real-world vocabulary. With the help of Duo and the rest of the cast, users can bring their environment right to the Duolingo app for more immersive learning.

5.0: Meet Duolingo Snapshots!


Introducing the interactive prototype of Duolingo Snapshots! With Snapshots, Super Duolingo users are able to capture any written media around them to save and practice, aiding their learning with relevant, real-world vocabulary. With the help of Duo and the rest of the cast, users can bring their environment right to the Duolingo app for more immersive learning.

In sum, there were three major redesigns:

  1. The tips page and hanyu learning page now has been redesigned for individual character definitions, with opportunities for more nuanced definitions and applicable example phrases.

  1. The individual collections pages have been redesigned to accommodate visual snapshots alongside learned phrases, allowing users to attach more significance and memory to the phrases they learn.

  1. The camera function ties these existing systems together to bring a sense of real-world connection, targeting and teaching meaningful phrases of interest.  

5.0 Implementation and Duolingo Snapshots’ Potential


Duolingo Snapshots would ideally implemented as an included Super feature within the Practice Hub, exclusive to dedicated learners who are more likely to be immersed in the language and motivated to revisit their phrases of interest that they’ve encountered.


Duolingo Snapshots has a lot of potential for expansion! Here are some nice-to-haves and additional implementations:

  1. Location feature: logging where snapshots were collected.

  2. Social function: sharing what snapshots your friends have collected recently, or what snapshots were collected near a given location. 

  3. Notifications: personalized reminders of past snapshots.

  4. Edge case: when the character is misrecognized by the camera, implementing button that says “This isn’t right!” and allows the user to manually correct the character.