[jahy-rahy]
Language learning in the context of your life.
Learn a new language through a context based and immersive mixed reality platform.
Mixed reality can be leveraged to provide an immersive learning experience to the user, creating a cognitive link between their own reality and the language they are learning. Not only can this method of language learning create the immersive experience, but also allows for contextual and relevant education to take place.
Duration
Aug 2021 – Aug 2022
Committee
Mike Strobert
Adam Smith
Design Tools
Maxon Cinema 4D
Adobe After Effects
Figma
How can a mixed reality based experience be leveraged to bring an immersive language learning experience to the user within a relevent context?
There are so many benefits to learning a new language...
Learning a new language has many cognitive benefits, as well as social benefits through connections with new people and cultures.
✚ Improved communication
✚ Enhanced problem solving skills
✚ Better memory
✚ Enhanced worldview
...but it's so hard!
Many people want to reap the benefits of learning a new language, but, your chance of becoming completely fluent in a new language plummets by the age of 10.
Immersion has been found to be a successful method, but most people don’t have the ability to move to a new country to learn the language.
One school* has found that through full immersion in the classroom, their students have gained similar proficiency to non full immersion learning in approximately 50% less time.
The Solution
With Gyri, simply learn a new language in the context of your life.
Through the use of image recognition, simply point your phone at an object you want to learn in a new language, click, and learn!
Explore.
Learn the words that are relevant to your own life. Simply pick up your phone, point and click on what you want to learn, and watch the translations immerse you through augmented reality (AR).
✚ Point your phone at the object you want to learn
✚ Click on the object to see translation cube hover above
✚ Explore the world around you
✚ Click on any object to see how to write, pronounce and read it
Collect.
Curate your own cube collection to catalogue your progress or save for later. The AR platform allows for the use of 3D cube sides to give pieces of information. This is a dual coding approach (both hearing and seeing information) that is key when learning a new language.
✚ Click on a cube to unfold to see all sides at once
✚ Click + to add cube to your collection
✚ Each side of the cube shows a different piece of information
✚ Move around the cube in 3D or click to see cube unfold
Play.
Chose your collection and play to test your knowledge. Match your chosen cubes to the item in real life. This allows for spaced repetition as well as solidifying context behind the vocabulary.
✚ Select your collection you want to play
✚ Stats for the collection are shown
✚ Green = words you have gotten correct on the first try
✚ Red = words you have gotten incorrect on the first try
✚ Click play to begin
✚ Throw cubes onto the correct object.
✚ Glow indicated if correct (green) or incorrect (red).
The Process
Topic Research
Language Learning
There are multiple tried and tested techniques for successful language learning. In particular, immersion is one of the most effective methods of language learning.
Immersion
Immersion is much like the way children learn their own native language. By fully immersing yourself on a daily basis, you can constantly be learning new words.
Dual Coding
The theory states that having both verbal (hearing) and visual (seeing) input helps the user to remember, long-term, the information being presented to them.
Spaced Repetition
A proven memory technique consisting of repetitive reviewing of words, so that eventually the interval between review can become much longer while still remembering.
Study Every Day
Immersion in the language on a daily basis is important. Practicing daily will help the user to keep their memory sharp and remember words faster.
User Research
User Interviews
Potential Gyri users were interviewed to further understand the pain points of current language learning solutions, as well as specific needs of different types of users.
The Student
"I have a hard time learning Spanish long term, after a semester at school I have forgotten all of the new vocab. I have no use for the random words taught to us in class either."
The Stay at Home Parent
"As a mom I focus most of my energy on taking care of my newborn, but I want to be able to keep my mind sharp. I'd like to learn a new language but I don't have the time to attend a class, and I find a lot of the language learning apps a little intimidating.
The Professional
"My job requires me to travel quite often, and I find myself having to learn select words in many different languages. I need a way to learn the relevent words to be able to communicate with the locals."
Key Findings
Users Don't Want:
Complex interface
Random irrelevent vocabulary
Long lessons
Work
Users want:
Intuitive Interface
Vocabulary relevent to them
Flexible time commitment
Fun to use
Market Research
Current Solutions
There are many language learning apps, websites, and even VR and AR experiences. These current solutions leave a gap that Gyri aims to fill. An immersive language learning experience that is relevant to the context of your life.
Augmented reality can be leveraged to provide an immersive learning experience to the user/s, creating a cognitive link between their own reality and the language they are learning.
DuoLingo
A free language learning application that utilizes short “bite sized” lessons, allowing users to easily learn in their downtime.
Immersion is not a part of this application, as it only exists on your device, and doesn’t interact with the physical world other than recording your voice.
Google Lens
Scan something in real life with your phone, and instantly have the ability to search on the web.
Using image recognition technology, this app can find relevant information to what you are seeing in front of you.
ImmerseMe
A VR experience that gives a more comfortable speaking experience for new learners, allowing for less pressure on the learner to get it right. The user is placed in a 360 view of an interaction.
Realistic conversations/interactions are explored through VR.
Toucan
A browser plug-in that translates random words on the webpage you are browsing.
Allows for contextual learning, in a passive way.
User Flows
Explore
The user flow is as simple as point, click, and learn!
Collections + Play
Access your collections to view saved words, and play to solidify your chosen vocab.
Design Exploration
Cube Iterations
Deciding how to present the information within the space to the viewer was an important step in the process.
I initially started with cards, but eventually settled on a cube as the 3D nature would integrate better within the environment using mixed reality, and each side allowed for different pieces of information to be displayed, without being visually overwhelming.
Final Cube Design
Building the Cube
The cube was built and animated in Maxon Cinema 4D. All animations were tracked onto video footage to show how the user can interact with Gyri within their environment.
UI Iterations
The UI had to be simple and graphic, so that it would be legible when overlayed within mixed reality.
The only item on the final screen is the collections icon, keeping the interface intuitive and simple as to not take away from the immersive aspect of the experience.
Final UI
Collections Iterations
The collections took some fine tuning to figure out what information was crucial, and what could be minimized to remove distractions.
3D Cubes that you have saved to collections are shown within each category, allowing for the user to scroll and see which they have gotten correct or incorrect. Total percentages are shown for the selected category as well.
Final Collections Screen
Visual Identity
Why Gyri?
Gyri [jahy-rahy] are the ridges and bumps you find on the brain, giving more surface area for better cognition in the small space. One of the important areas made up by Gyri is Wernicke’s area, which is vital when it comes to language learning and comprehension.
Simple & bold
The limited color palette of black and white creates an easy to read, yet elevated look, allowing for immersion without distraction.
Green and red are used minimally to indicate if an answer is correct or incorrect.
Blocky sans serif typefaces were chosen for ease of legibility, and also to allude to the cube's shape.
Montserrat Bold
Montserrat Semibold
Montserrat Regular
Thanks for Looking!