User Research, Usability Testing
Second Street is a successful start-up based in St. Louis that specializes in promotion software and products.
I joined the team to help Second Street with user experience design, user research, and usability testing for their thirteen different products and 2,000 daily active users.
Although Second Street has been in business for ten years, the company did not have experience when it came to user research and usability testing. When new features and updates got released, it was rare for us to measure the performance and check for design bugs.
To help prevent Second Street from skipping usability testing, I created a guide for conducting usability tests. This guide helped others on my team understand the importance of usability testing to allow for our process to be updated so we could start validating features and benchmarking the user experience.
For unmoderated usability testing, Full Story is a must-have.
Luckily, Second Street had Delighted with plenty of user feedback for usability testing ideas.
To create a process that integrated well the existing processes, I had to do some research on what would be best with our available resources. Since there wasn't stakeholder buy-in yet, I had to use the tools Second Street already had.
After reading what felt like endless Nielson Norman Group articles, I felt confident enough to experiment with some pilot studies. Utilizing Full Story and Delighted, I was able to come up with goals for usability studies and segment user sessions for moderation.
NNG has the best resources and references when it comes to user research.
A glimpse of the testing checklist.
Now that the framework for conducting a usability test was complete, it was time to put it into action. I started by benchmarking the setup process of a specific promotion type.
The main purpose of the study was to serve as an example of how Second Street could utilize usability testing to validate design decisions, measure updates, and discover insights. To make this a part of our regular processes, it needed to show a return on the investment of additional time and resources. Otherwise, the stakeholders in the company wouldn't believe it's worth doing.
Overview of the Usability Test:
Some basic insights from the study. Nothing too surprising, but now we have an anchor to compare updates to.
Glancing at the data, it's honestly quite plain. However, one thing I started to notice was the number of users who would throw errors when it when they would use date inputs. So I started tracking the number of users who created errors in addition to the initial goals of the study. It turns out that 30% of users who created a promotion, encountered an error in the exact same place.
Following this insight, I decided to follow up with another usability study. This time, focusing specifically on the save states to figure out if I could find the reason so many users were having the same error.
Overview of the Usability Test:
On this usability test, I chose to do 40 sessions so I could have a sample size of 20 for each type of saving. Turns out we could reduce the majority of user errors simply by implementing manual save for steps with dates.
Once I was able to demonstrate the value of remote unmoderated usability testing, the company was accepting of adding it to our processes. In fact, we are discussing the possibility of adding more advanced research methods for the product team. Going forward, I hope to help Second Street add A/B testing and moderated user testing to their UX toolbelt.
I learned a lot from developing this process within my organization. It was challenging to raise awareness of the importance of usability testing in the beginning, but once there were data and insights the team really started to come around. It felt rewarding to advance the process of the product team, as well as, create more enjoyable experiences for users.