;

PRODUCT

WIREFRAME

LEARNING

RESEARCH

MORE?

RESUMÉ

Enabling Inclusive Communication on the Apple Vision Pro

AR/VR Design

Accessibility

Project Management

View the full case study here.

View the full case study here.

My Role

My Role

Project Manager
Lead UI/UX Designer

Project Manager
Lead UI/UX Designer

Area

Area

Product Strategy
User Experience

Product Strategy
User Experience

My Contribution

My Contribution

Product Concept
UX Research
Rapid Wireframes
High-fidelity Prototyping

Product Concept
UX Research
Rapid Wireframes
High-fidelity Prototyping

Tools

Tools

Figma
Swift UI

Figma
Swift UI

Teamwork

Teamwork

Software Developers*4
Business Analysts*2
UI/UX designers*2

Software Developers*4
Business Analysts*2
UI/UX designers*2

My Role

My primary focus was to build a beautiful and impact user experience (UI, Wireframes and Concept), as I got more involved with concept building I assisted my co-pm (Kunal) with cross-functional alignment and overall product strategy

My primary focus was to build a beautiful and impact user experience (UI, Wireframes and Concept), as I got more involved with concept building I assisted my co-pm (Kunal) with cross-functional alignment and overall product strategy

Project overview

Over six months, we created a new experience for an accessible-focused environment we had no experience in. Design, Engineering and Business/Marketing worked together to build a platform where users hard-of-hearing were able to communicate with others regardless of ability well.

Over six months, we created a new experience for an accessible-focused environment we had no experience in. Design, Engineering and Business/Marketing worked together to build a platform where users hard-of-hearing were able to communicate with others regardless of ability well.

Timeline

A linear timeline that was not so linear.

Exploration & research

The Challenge

Simple conversations is a daily
struggle when one person can hear
and the other cannot.

Simple conversations is a daily
struggle when one person can hear
and the other cannot.

Once we began research, the data was undeniable. The challenge wasn't just about the tech, it was about learning how to design and develop for an area that was completely new, while making sure our solutions would help people communicate better in their daily lives.

Idenitifying needs

Interviews and contextual inquiry reveal that individuals hard-of-hearing feel invisible, isolated and dependent.

Interviews and contextual inquiry reveal that individuals hard-of-hearing feel invisible, isolated and dependent.

Technology dependence can falsify perceived attention

Technology dependence can falsify perceived attention

Users need to have autonomy given any moment

All existing technology applications carry friction in experience

All existing technology applications carry friction in experience

critical thinking

Exploring

Exploring

Technology research

VisionPro Is Here. Is Mixed Reality the Right Move?

VisionPro Is Here. Is Mixed Reality the Right Move?

Seeing an opportunity in the new technology was exciting, but we had to verify to make disciplined choices and explore other solutions. I began thinking about spatial design, placement of the navigation bar. Many of my design iterations were derived from experience on the headset.

Seeing an opportunity in the new technology was exciting, but we had to verify to make disciplined choices and explore other solutions. I began thinking about spatial design, placement of the navigation bar. Many of my design iterations were derived from experience on the headset.

Experiencing the VisionPro in-person for the first time enabled the beginner's mindset

Feasability & kNOWLEDGE

Navigating intersections between Spatial Design, Accessibility, and AR/VR Technology

Navigating intersections between Spatial Design, Accessibility, and AR/VR Technology

I had to understand what it meant to converge between three non-technical and technical disciplines.

I had to understand what it meant to converge between three non-technical and technical disciplines.

Clear visuals and lighting:

Using zone and fewer occlusions so people can rely more on visual cues and lip-reading

Clear visuals and lighting:

Using zone and fewer occlusions so people can rely more on visual cues and lip-reading

Personalized settings:

Letting users adjust features to match different signal processing needs

Personalized settings:

Letting users adjust features to match different signal processing needs

Adaptive, immersive cues:

Building in environmental signals that scale with perception and spatial awareness.

Adaptive, immersive cues:

Building in environmental signals that scale with perception and spatial awareness.

ReseArch summary

User A, B and C indicated feeling excluded in conversations and lack of physical presence. Leveraging Mixed Reality (XR),we can tap into spatial awareness and UI for multimodal screen integration to facilitate more inclusive interactions- recognizing both sign language and speech- while enabling more accessible task flows

User A, B and C indicated feeling excluded in conversations and lack of physical presence. Leveraging Mixed Reality (XR),we can tap into spatial awareness and UI for multimodal screen integration to facilitate more inclusive interactions- recognizing both sign language and speech- while enabling more accessible task flows

HOW MIGHT WE…

Improve communication for people with hard-of-hearing in order to empower them with autonomy and a physical space resulting in confident conversations?

HOW MIGHT WE…

Improve communication for people with hard-of-hearing in order to empower them with autonomy and a physical space resulting in confident conversations?

cONCEPT PROPOSAL

A mixed-reality (XR ) communication tool for sign language that provides live speech-to-text, multilingual support, and customizable settings for users with hearing impairments.

cONCEPT PROPOSAL

A mixed-reality (XR ) communication tool for sign language that provides live speech-to-text, multilingual support, and customizable settings for users with hearing impairments.

Feature Implementations

Transitioning from Research to Action-oriented Insights

The results from our research gave us concrete insights on what users who are hard-of-hearing needed from an accessibility forward mindset. We understood that the environment was complex and by breaking the physical and virtual space we could offer solution to these pain points.

The results from our research gave us concrete insights on what users who are hard-of-hearing needed from an accessibility forward mindset. We understood that the environment was complex and by breaking the physical and virtual space we could offer solution to these pain points.

  1. Reducing the activation energy to start the communication process

  1. Reducing the activation energy to start the communication process

  1. Provide effective translation in flexible environments

  1. Provide effective translation in flexible environments

  1. Empower users to communicating with any individual with any ability

  1. Empower users to communicating with any individual with any ability

Scoping and narrowing down the features

The team and I ran cross-functional sprints between engineering, design and business to ask three questions for each feature: 'Does this meet our user’s needs? 'How feasible would implementation be? 'What would the timeline look like? '

The team and I ran cross-functional sprints between engineering, design and business to ask three questions for each feature: 'Does this meet our user’s needs? 'How feasible would implementation be? 'What would the timeline look like? '

Icons next to the features indicated whether it would be implemented at this stage

Low-fidelity sketch

Implementing Concept into AR/VR Spaces

The low-fidelity enabled me to quickly validate usability and feature placement early on.
We narrowed it down to three main features- Sign language to Text, Speech-to-Text and Translation

The low-fidelity enabled me to quickly validate usability and feature placement early on.
We narrowed it down to three main features- Sign language to Text, Speech-to-Text and Translation

Conceptualizing and decision making through low-fidelity wireframes Enables quickly validate usability and feature placement early on.

Mapped out how users would move through key actions, ensuring clarity and efficiency before adding complexity

Over 70+ wireframes to refine ideas

Focused on team alignment between accessibility & feasibility post-development

Focused on team alignment between accessibility & feasibility post-development

I began by creating accessibility-focused wireframes, but once the first feature was developed, I noticed major gaps. The implementation needed improvement to align with WCAG AA guidelines. We set up focused meetings, to ensure that accessibility was also carried through into development.

I began by creating accessibility-focused wireframes, but once the first feature was developed, I noticed major gaps. The implementation needed improvement to align with WCAG AA guidelines. We set up focused meetings, to ensure that accessibility was also carried through into development.

Post-developement Session A

High-fidelity wireframes

Diving into key design decisions

Diving into key design decisions

How were these decisions made?

Feature highlight website

Built a Product Feature website highlighting Features and basic marketing efforts

Apple VisionOS App

VisionTranslate

VisionTranslate is an AR/VR-based sign language tool that provides real-time speech-to-text, multilingual support, and customizable settings for users with hearing impairments.

Real-time translation of sign language to text & audio

Real-time translation of sign language to text & audio

When you sign to someone else, your signs will display as a text and can be voiced out aloud for another person to hear live.

Understanding someone else's sign language

Understanding someone else's sign language

There are 300+ sign languages, so if you don't know one- don't worry! VisionTranslate instantly converts sign-language to text

Speech to Text mode: View live captions instantly when someone else is speaking

Speech to Text mode: View live captions instantly when someone else is speaking

Be physically present while you sit in a room with multiple people, you are able to view live captions when they talk or choose not to

Customizable Settings
& Personalization

Customizable Settings
& Personalization

Change your settings based on your comfort. Whether you want a bigger font, slower transcription or higher contrast do it all here

Expanding Impact

Expanding Impact

Learnings

This was not a feasible product.

This was not a feasible product.

This project is a good example to show that the application's success metric was also defined by the success of over-heading product: VisionPro. The physical product served as the primary influencer before our application made any contact. While the VisionPro served as a fantastic medium to get our idea implemented, this was not a great example of a feasible product in the market.

This project is a good example to show that the application's success metric was also defined by the success of over-heading product: VisionPro. The physical product served as the primary influencer before our application made any contact. While the VisionPro served as a fantastic medium to get our idea implemented, this was not a great example of a feasible product in the market.

Scoping is important first

Scoping is important first

As I took a leadership role for an end-to-end process, I learnt how important alignment between teams was- it was the answer to making or breaking the product quality. There were moments of disconnect between the engineering and design team, especially after implementation. With a little more scoping and planning, we could have shipped the MVP at a higher quality of work that I would have liked it to be (a classic cross-functional team set back)- and today I remember certain ideas I can do to make the next project better.

As I took a leadership role for an end-to-end process, I learnt how important alignment between teams was- it was the answer to making or breaking the product quality. There were moments of disconnect between the engineering and design team, especially after implementation. With a little more scoping and planning, we could have shipped the MVP at a higher quality of work that I would have liked it to be (a classic cross-functional team set back)- and today I remember certain ideas I can do to make the next project better.

The purpose of a MVP is to collect data

The purpose of a MVP is to collect data

Looking back, the purpose of a product on the market is to validate certain hypothesis, assess the product and derive insights that shape the next iteration. As a team we did pretty well only until the MVP launch. I could have said, "Beta on the way" or something, but let's be real considering school timelines and shifting priorities, we did not move forward. Now, I would like to see how this would play out if given a shot to improve.

Looking back, the purpose of a product on the market is to validate certain hypothesis, assess the product and derive insights that shape the next iteration. As a team we did pretty well only until the MVP launch. I could have said, "Beta on the way" or something, but let's be real considering school timelines and shifting priorities, we did not move forward. Now, I would like to see how this would play out if given a shot to improve.

We won the Michigan Ross + Tech design Sprint 2024! 🏆

We won the Michigan Ross + Tech design Sprint 2024! 🏆

View the full case study here.

View the full case study here.

Previous

[Lucid Motors Internship] New Design System & Developer Tools

[Lucid Motors Internship] New Design System & Developer Tools

Sole UX Designer, Delivered Immediate And Long-Term Solutions

Development and design

/Design_System

/Cross-functional_experience

Next

Stellantis Smart Energy System for Electric Vehicles

Jeep Energy System for Electric Vehicles

Automotive UX Design

AI/ML and AR/VR integration with Automotive UX

Auto UX

AR/VR Design

13:32 EST

Devyani Jain

devyanij@umich.edu

Logo
13:32 EST

Devyani Jain

devyanij@umich.edu

Logo
13:32 EST

Devyani Jain

devyanij@umich.edu

Logo
13:32 EST

Devyani Jain

devyanij@umich.edu

Logo