gtag cover 4-02.png
Show (1).png

Overview

GTag aims to assist visually impaired and blind people in doing grocery shopping independently in an urban context. It consists of a mobile app and third-party NFC tags. The primary function of GTag is to record grocery-related information (e.g., expiration dates, cooking instructions, etc.) while in-store and replay them anytime post grocery shopping. 

Timeline

08/2021 - 12/2021

Tools

Figma

ProtoPie

Miro

Adobe Illustrator

Otter.ai

Team

Watson Hartsoe

Tommy Ottolin

Meichen Wei

Qiqi Yang

My Role

UX Designer

UX Researcher

 
problem.png

Problem

In the context of grocery shopping and consuming purchased food products, there are some common problems for people with vision loss.

Obtain information from food packages, especially for expiration dates

identical food-02.png

Identify foods in the same size and shape

In the grocery shopping context

How might we minimize cognitive load for visually impaired people by making information accessible?

 

Solution

solution.png

Whenever no one is around to be your "eyes", Gtag is here to help!

+

=

Write

Users can store desired information on food packaging into designated NFC tags by creating audio recordings. New NFC tags need to be scanned to log in before storing any information.

write 1-06.png
write 2-06.png

Read

To access information, users simply scan tags hanging on food packages and listen to them. Tags can be read in or out of the app.

read 1-06.png
read 2-06.png

Erase

Information stored in each tag can be deleted.

erase 1-06.png
erase 2-06.png

Edit

Edit existing Information stored in tags.

edit 1-06.png
edit 2-06.png
 

Research

Search.png

Comparative Analysis

First, we searched for commonly used mobile-based assistive applications that aid everyday tasks for visually impaired people. This provided us the knowledge on the ecosystem of assistive technologies for our users, which helped us in designing a user-friendly product.

Be My Eyes

A mobile application that utilizes sighted remote assistance operators (volunteers) to assist visually impaired users to see objects and circumstances.

Seeing AI

AI(Artificial Intelligence)-powered mobile app that serves the same purpose as Be My Eyes.

Contextual Inquiries & Interviews

Our Visually Impaired Participants:

Participant 1_edited.jpg

01

Participant 2.png

02

Participant 3.png

03

Participant 4.png

04

Participant 5.png

05

Participant 6.png

06

b92aed64d74f836efdacc7653b794ab1.gif

All discussions and observations on participants were mainly on their grocery shopping experience. We also asked for participants' perspectives on popular assistive technologies, their medical conditions, daily life, and careers. We collected notes from contextual inquiries and interviews and analyzed them which assisted later research and design.

User Scoping Spectrums

The spectrums are high-level analysis of participants' behaviors collected from contextual inquiries and interviews to find relationships between user behaviors data. Some have stronger relationships compared to others.

 

The spectrums helped us narrow down the user group to the visually impaired population living in urban contexts. Our product would lie in the current assistive technology ecosystem to promote independent shopping.

Early Adopters vs. Late Adopters in Technologies

Autonomous vs. Reliant according to lifestyles

Affinity Map

We gathered notes from contextual inquiries and interviews and arranged them into two groups, grocery shopping and beyond grocery shopping. Then, we separated notes into smaller groups by similar patterns of issues and preferences gave high-level descriptions to each group.

We
identified design ideas from some of the notes and darkened colors of some notes to signify importance. Finally, important research findings are synthesized and applied to later product design.

My First Board - Affinity Map.jpg

Hierarchical Task Analysis (HTA)

HTA helps us view the process of grocery shopping for the visually impaired population in detail and a big image. We zoomed in to the task analysis to identify common problems and contextual factors that we can improve by designing a product for low visions.

grocery shopping.png

Research Findings

According to observations in contextual inquiries and interviews, we generated research findings from patterns of issues and preferences in grocery shopping.

#1

Planning vs. Exploring Shopping Styles

#4

sensory-14.png

Use of sensory inputs in grocery shopping

#2

3 factors-12.png

3 most applied factors in selecting grocery items

#5

memorize-15.png

Visually impaired memorize the entire world

#3

urban-13.png

In-person grocery shopping for urban shopper

#6

helper-16.png

helper's knowledge and personalities affect experience

 
design.png

Design

Design Requirements

From research findings, we created functional (specific to functions of the product) and non-functional design requirements. We narrowed down to three of the most important and influential design requirements(circled) to be used as guidelines in designing the product.

Functional Requirements:

1. Human

1A. Design empowers visually impaired and blind shoppers to independently shop with helpers

1B. Design supports cognitive recall in store and at home

2. Computer

 

2A. Design is multimodal, not just visual

2B. Design supports cognitive recall in store and at home

3. Interaction

3A. Design presents information at the point of choosing the product: expiration, sale, and related items

3B. Design presents information at the point of using the product: expiration and cooking instructions

Non-Funtional Requirements:

4. Supportive of Diversity

Design should support and accommodate different shopping styles and allow users to access desired information to make purchase decisions

5. Intuitively Usable

Design should be compatible with the present ecosystem of support which requires minimal cognitive labor in learning and use 

6. Collaborative

Design should promote healthy relationships between visually impaired users, shop assistants, and other stakeholders involved in decision-making

7. Trustworthy

Design should assist users in confirming desired items to purchase with certainty and to recall information easily and accurately when they want to

Brainstorming Ideas

We brainstormed together to come up with as many ideas as possible around 4 selected topics related to the design problem. We individually chose 5 ideas from the brainstorming session and sketched them. Finally, we narrowed it down to 10 unique ideas and refined the sketches. 

 

From the 10 ideas, we found that multiple ones can effectively solve a problem or satisfy a need in the entire grocery shopping experience for the visually impaired. 

Idea #1: Phone-A-Fruit

An Internet browser plug-in can analyze the user's shopping cart on grocery retailing websites.

We presented and discussed the ideas with 2 visually impaired participants to ask for their opinions. Finally, we picked the idea that best met the design requirements. The final design is a combination of 2 design sketches and further defined and developed as one product, the GTag.

Idea #5: AI Expiration Date Scanner/Tracker with Audio Cue

expiration date and key info scanner.png

Idea #10: Food Information Locator

+

Hut's Conned.png

+

Prototyping

Sighted Helper

Haptic

Audio

Smartphone

NFC Technology

How did we decide to use NFC technology for our product?

Our User Group

Visually Impaired People 

Satisfy 2 interactions for visually impaired users with commonly use technology for the whole user group

NFC tag scanning for mobile:

Place the upper back of the mobile phone close to the NFC tag. Able to scan without touching the tag within 4cm height range.

There are 3D-printed NFC tag carriers made for attaching tags onto food packages. We have tried other methods like applying adhesive materials to stick onto the packaging, but using carriers and rubber bands is the most suitable way to easily attach and detach NFC tags onto desired items.

gtag real-10_edited.png

The 4 user flows established for the GTag app were created based on the flow of using NFC technology and customized to our users and the grocery shopping and consuming settings.

"Read" Flow

Wireframes

When designing the GTag mobile app, we were referring to WAI’s guideline on applying WCAG 2.0 and other guidelines to mobile content (Mobile Accessibility). The visual style guide shows the considerations of accessibility for the GTag app.

Colors

All color used reaches the Enhanced Color Contrast level (Level AAA) of 7:1.

Type

To ensure legibility, Sans Serif typeface Roboto was used. 

6 styles of fonts are used to establish visual hierarchy.

We researched how smartphones provide audio feedback to people's actions and applied various audio cues to the app. We also added vibration as haptic feedback in addition to audios.

Tag loaded successfully

Audio Cues (Sound Effect)

Click the button to play the audio!

Start and stop recording

New tag created

Erase tag

Voiceover

- Read content by taping the content once

- Voice to indicate actions

- 1.5 times speaking speed

Tap once:

"Create A New Tag
Double Tab to create"

Normal

The following design guidelines are created to help us meet the design requirements. There are examples to show evidence of applying the design guidelines to design GTag.

Perceivable in diverse ways

write 1-06.png

Voice- “Ready to Scan”

Sound+vibration- “Bling!” after successfully scan a tag

Click the button below to play the audio!

Support reconfirmation

Accessible interface

Color contrast

Text size

1.2-1.5 times larger

Dictation (speech to text)

Touch target’s size and spacing

Touch target bigger than 9mm*9mm

Inactive space around small size touch targets

Audio outputs

After scan

Click the button below to play the audio!

Requires minimum learning

GTag utilizes pre-existing control schema to help users with simple commands.

- Same actions to scan and load NFC tags on the app

- Tapping once to record and tapping once again to stop build on actions

 
Tick Square.png

Evaluation

We conducted a User Evaluation session with a visually impaired shopper and a sighted helper by joining them on a grocery shopping trip. We played different roles in evaluations, which were moderator, notetaker, and wizard (facilitator in performing the prototype).

Screen Shot 2022-01-23 at 11.37.49 PM.png
IMG_0323.jpeg

User Tasks

#1

Write

Helper's Task

Shopper chose the information for the helper to write.

Shopper witnessed the whole process.

#2

Read

Shopper's Task

(VoiceOver On)

Shopper scanned NFC tags written by Helper.

 

Shopper read pre-recorded information to distinguish items with similar packaging.

#3

Erase

Shopper's task

(VoiceOver On)

Shopper performed the erase function under instructions.

We conducted short interviews with the helper and shopper separately to collect feedback.

Key findings from User Evaluation session:

01

Helper could not tell how and when to start and end voice recording.

 

Shopper did not have issues holding their phone throughout the trip or handing it to the helper for creating new tags

03

Helper was able to perform the task or record a new tag faster as they use it more.

 

Shopper was able to retrieve recorded information to distinguish items with similar packaging

02

04

We also conducted an Expert Evaluation session with another project team from our class. There was a free click-around session for participants to interact with GTag and guess its purposes. Then, we asked them to role-play as shoppers and helpers to perform the above tasks. 

Prototype Improvement

I made improvements to the GTag prototype based on feedback collected from User and Expert Evaluations.

Onboarding pages for first-time users to familiarize the process of creating new tages

Wipe ---- Erase

Switch to more universally understandable names to avoid confusion

Future Improvement

- Program GTag to be compatible with VoiceOver(screen reader)

- Continue improving and testing GTag with visually impaired and sighted users

- Make GTag more customizable to users with different financial statuses, living situations, and technological needs

- Try and test new technologies for improving visually impaired people's grocery shopping experience

improvements.png
 

Prototype

Category.png

This is a video demo for GTag (VoiceOver Version). It showcases the process of using the Write, Read, Erase, and Edit features of GTag.