Culture at Hureo and everything that follows
January 25, 2022Insights for Usability Studies with Visually Impaired Participants
Guidebook and Recommendations
User research focused on accessibility can be challenging, particularly when it comes to understanding the experiences of participants who are visually impaired. Drawing on our experience at Hureo, we’ve compiled several recommendations for the technical, contextual, and methodological elements to consider when conducting usability studies with visually impaired participants.
1. Utilize Multiple Cameras and Angles
Observation plays a crucial role, and you may need to review the session recordings multiple times. To capture a comprehensive view, consider using more than one video device to record different angles. Aim to incorporate as many of the following as you can:
- Overhead Camera: To monitor hand movements on devices such as touchscreens, keyboards, touchpads, or mouse.
- Front-Facing Camera: To capture facial expressions.
- Screen Recorder: To track navigation and interactions on the screen.
Session on smartphones using a device which captures hand movements and broadcasts on the moderator laptop Source: Hureo archives copyright reserved
Session on a laptop using an over-the-shoulder camera which broadcasts on the moderator laptop screen Source: Hureo archives copyright reserved
2. Let the participant use their own device and keep a test device as a backup
Participants are most comfortable using their own devices, which allows for a more authentic testing experience. By letting them use their own devices and keeping a test device as a backup, you can simulate real-life usage and gain more accurate insights.
- Accommodate Familiarity: Allow participants to use their own devices during testing, as they are more comfortable and familiar with them.
- Simulate Real-Life Experience: Testing on their own devices provides a more realistic view of how they will interact with your product in everyday life.
- Use a Test Device as a Backup: Have a test device available as a backup in case technical issues arise with the participant’s device, ensuring the session can continue smoothly.
- Gain Authentic Insights: This approach helps you understand how your product performs in real-world scenarios, offering valuable insights into its usability for potential users.
3. Test with a Variety of Devices and Tools
To ensure your product works well for all users, it’s essential to test it across different devices, browsers, and screen readers. This helps identify any compatibility issues that might arise in real-world usage.
- Use Multiple Devices: Conduct tests on a range of devices, such as smartphones and laptops, to ensure broad compatibility.
- Incorporate Different Browsers: Test your product on various browsers, including Chrome and Internet Explorer, to identify any browser-specific issues.
- Evaluate Multiple Screen Readers: Use different screen readers like JAWS and NVDA to catch bugs and errors across accessibility tools.
- Identify Compatibility Issues: Testing with various combinations of screen readers and browsers will help identify any compatibility problems, ensuring your product works smoothly for all users.
4. Importance of Conducting Pilot Sessions
Running pilot sessions is key to refining your research methodology and reducing anxiety during accessibility studies. Diverse participant selection and multiple pilots will help you set up your test environment effectively.
- Run Multiple Pilot Sessions: Start with a pilot session and follow up with at least one more. If you are engaged in accessibility-related research, conducting multiple pilots will help solidify your methodology and reduce any anxiety associated with conducting these studies.
- Diverse Participant Selection: Ensure that your pilot sessions include participants with different types of visual impairments, such as:
- Partially sighted
- Completely blind
- Color blind
- Enhance Test Setup: Including a range of visual impairments will be beneficial in setting up any test devices you plan to use, as it ensures you incorporate all necessary software and meet other essential requirements.
5. Reevaluate Usability Scenarios
After pilot sessions, it’s crucial to review and adjust usability scenarios to ensure they are inclusive and relevant for all users. This process helps refine the scenarios to better capture diverse user experiences.
- Broaden Scenario Scope : After conducting the pilot sessions, take the time to rewrite or adjust the usability scenarios to ensure they are more inclusive and applicable to a wider audience.
- Understand User Motivations : Recognize that the reasons for using the same product can differ greatly between sighted users and those who are visually impaired. Tailor your scenarios to reflect these different perspectives.
- Use Inclusive Language : For instance, consider rephrasing a task like, “Imagine you saw an advertisement on a hoarding and were curious to know more.” Instead, you could say, “Imagine you are curious to learn more about a product you came across.” This promotes a more universal understanding of user engagement.
- Test with Diverse Users : Engage users with varying abilities in your usability tests. This will help ensure that the scenarios resonate with everyone and that you capture a range of experiences.
- Iterate and Refine : Continuously revise your scenarios based on feedback from pilot sessions and usability testing. This ongoing refinement process will help improve the overall effectiveness and inclusivity of your usability scenarios.
6. Avoid Creating Separate Tasks
It’s important to assign the same tasks to all participants, regardless of their abilities, to accurately assess how your product performs across different user groups. This approach will highlight usability challenges and provide comprehensive feedback.
- Uniform Task Assignment : Ensure that all participants, regardless of their abilities, complete the same usability task. For example, if the task involves booking a cab from point A to point B, assign this identical task to all users and observe the differences in their navigation approaches.
- Monitor Navigation Differences : Pay close attention to how both visually impaired and sighted participants navigate the task. This will help identify specific challenges faced by each group and uncover any usability hurdles.
- Maintain Task Complexity : Avoid simplifying tasks for visually impaired users. Providing the same level of complexity for all participants will yield a more accurate understanding of the product’s usability.
- Do Not Provide Additional Guidance : Refrain from guiding visually impaired participants to complete tasks more quickly. Allowing all users to navigate independently will reveal genuine interaction patterns and difficulties they encounter.
- Highlight Product Challenges : Using identical tasks across different user groups will more clearly point out the product’s usability challenges and their severity, offering valuable insights for improvements.
- Gather Comprehensive Feedback : After task completion, collect feedback from all participants about their experiences. This comprehensive data will contribute to a more holistic understanding of the product’s usability for all user types.
An excerpt from a study conducted at Hureo for a tech media product – We had recruited sighted and visually impaired participants of various types. We had consulted an expert and by advice had not changed my script.
Our first pilot session for accessibility testing was with a completely blind participant. There was one task where users had to browse through a carousel of images. When in the flow of the research, the researcher came on to that task, got nervous and hesitated to ask the user to perform this task due to their bias – ‘how could this user see these images, it is not possible’. The researcher went ahead anyway as it was difficult for them to reword on the go.
The participant was of course able to complete the task. The participant heard the captions on the images and very well understood the purpose of the feature, understood what the images are, was able to swipe to various images in the carousel, and was able to rediscover the same content.
Notably, this task – browse through image carousel – was a task which entirely failed with sighted participants as no participant was able to navigate and was not able to comprehend the feature, however all visually impaired participants completed the task without any assistance!
7. Objective Observation of Visually Impaired Participants
When observing visually impaired participants, it’s essential to recognize their unique interaction styles. Contextual note-taking and maintaining dual attention during the session will provide a more accurate understanding of their experience.
- Recognize Diverse Interaction Styles : Understand that visually impaired individuals may navigate the product differently compared to sighted participants, and their screen interfaces might be customized to suit their needs.
- Contextual Note-Taking : Take context-specific notes that reflect the unique ways participants engage with the product, particularly when they are using features like zoomed-in displays.
- Focus Beyond the Screen : Acknowledge that what appears on the device screen may not accurately depict the participant’s actions. It is crucial to observe their interactions, which may involve physical adjustments, such as bringing the phone closer for better auditory feedback.
- Dual Attention : As a researcher, maintain awareness of both the participant’s actions and your view of their phone screen to gain a fuller understanding of their experience.
8. Understanding Screen Readers and Accessibility Tools
Understanding the tools that visually impaired participants use is crucial for accurate usability testing. Familiarizing yourself with screen readers and accessibility settings will improve your ability to interpret user interactions.
- Explore Accessibility Settings: Familiarize yourself with screen readers and other accessibility tools by experimenting with the accessibility settings on your devices.
- Learn Functionality: Gain a deeper understanding of how these tools function to better interpret user interactions during testing.
- Simulate User Experience: Use these tools yourself to simulate the experience of visually impaired users, enhancing your empathy and insight into potential challenges.
9. Reflect on Your Experience
Reflecting on your experience as a researcher is essential for effective communication and understanding during usability studies with visually impaired participants. Acknowledging your own learning curve can lead to better interaction with participants.
- Encourage Slow Screen Reader Speed: Ask participants to slow down their screen reader speed to improve your understanding.
- Acknowledge Challenges: If you’re unfamiliar with screen readers, let participants know, and ask them to repeat tasks if needed.
Additional Do’s and Don’ts
When conducting usability studies with visually impaired participants, certain considerations can make the experience more productive and comfortable for both the participants and the researchers.
- Provide Adequate Incentives : Offer higher compensation since the sessions are longer, and consider covering transport to your lab and other related expenses if possible.
- Ensure Accessibility : Rearrange the office furniture to create an accessible environment.
- Use Natural Language : Communicate in a way that feels comfortable and familiar. Avoid overcompensating for any impairment; for instance, it’s perfectly fine to ask, “What do you see here?”
- Maintain a Steady Pace : Be patient with participants, allowing them to complete tasks at their own speed without rushing them or providing assistance in finishing.
