Autonomous Driving | Competitive Research Study
Benchmark usability testing to evaluate the end-to-end usability of an autonomous ride-share app and its competitors.
Senior UX Researcher | 8 Weeks | Evaluative Research | 2023
Business Needs
This evaluative user research project was constructed as the first round of rolling research for an autonomous ride share company to check for improvements in or degradation of usability over time and to inform recommendations for product changes with the service roadmap.
Process
1. Study Planning
- Collaborated with 1 Project Manager, 1 Research Director, and 1 UX Researcher to review previous research (e.g., customer success outcomes), assess technological constraints, and plan logistics for two-weeks of in-person testing in San Francisco.
2. Screening and Recruiting
- Partnered with a local recruitment agency to screen and schedule 40 existing and prospective customer participants for 30 to show. Planned for 60-minute interviews between 2 Researchers and 2 Research Assistants.
3. Client Alignment Workshop
- Utilized client’s customer success outcomes (CSO) to plan and lead a 2.5-hour alignment workshop including design activities such as CXO statement refinement, prioritization, and dot voting as a group through Mural.
3. Session Guide Creation
- Created a session guide using our project objectives and research questions as a north star. Incorporated various tasks to gauge the ease and comfortability of the autonomous driving and ride-sharing experience. Ensured consistency throughout all competitor session guides.
4. Hiring and Onboarding Research Assistants
- Screened and interviewed 5 Research Assistants with the Research Director for the 2 open positions.
- Onboarded 2 Research Assistants to the project topic, methodology, and note-taking protocol while leading the team of researchers on-site.
5. Conducting In-Field Testing
- Facilitated 16 ride-sharing and autonomous vehicle sessions in downtown San Francisco (Embarcadero) with 2 types of experiences (one autonomous vehicle and one ride-share).
- Managed full data collection days with 6 sessions run concurrently with a secondary researcher (3 per person per day) while organizing note-taking through a spreadsheet, Qualtrics survey, and task board.
Analysis and Outcomes
- Synthesized and presented high-level findings and emerging themes to the client based on my and a secondary researcher’s initial impressions and observations one day after the final interview session.
- Created an 80-slide deck with benchmark methodology, success criteria, severity ratings, participant segmentation, task scorecards, usability findings, survey responses, process documentation, video highlight clips, quotes, and recommendations.
- Presented the final deck to 5 stakeholders during a 1-hour walkthrough and discussion.
- Followed up with an internal project retrospective amongst team members to reflect on what went well and what we learned.
