August 08, 2017
Are you seeking validation for this really cool product idea you have? Or are you rather trying to understand why your users do not engage with your website as much as you’d hoped? Whatever the reason, chances are high that you are about to conduct some sort of qualitative study and are now looking for ways on how to recruit participants.
One of the major impediments to conducting user testing when it is needed is the time it may take to find appropriate users if one is not prepared. — Nielsen Norman
Finding the right people in a timely manner is crucial for the success of a product, but where and how to start?
This article aims at providing a guide on how we at WATTx recruit participants and includes our insights and learnings. In our role as a venture builder we conduct focused research, triggering informed solutions that we take from concept to prototype, and from pilot to company formation in under six months. This fast-paced environment requires rapid action which is especially crucial when it comes to validating our own ideas. Here’s what we’ve learned so far.
We have come to learn that the best time for the recruitment of participants is shortly after we’ve started working on a new topic: Within our UX team, we kick off the project by brainstorming our assumptions in order to identify areas of research and the stakeholder landscape. What follows is an extensive but short, 2-week research phase where we consult secondary literature and research papers from domain experts. After having built a basic understanding of our topic, we move on to recruiting participants.
Five users are enough to gain a first grasp of the bigger picture.
Every research team that recruits participants will see themselves confronted with this question: How many and which participants do we need to talk to in order to yield representative insights?
We find that for qualitative studies, be it interviews, usability tests, heuristic evaluations or the like, five users are enough to gain a first grasp of the bigger picture or to find most usability problems a system suffers. More do not necessarily mean better results or additional insights. However, make sure that you are talking to five people with the same characteristics. If you have two or more different stakeholders, you should recruit at least five of each group.
And how do we decide now on who exactly to talk to?
Let’s use the learnings from our shipping project as an example: During the first extensive research phase, we identified one major problem area that stakeholders operating within a port environment seemed to struggle with, namely port congestion. We found out that truck drivers often deal with waiting times of up to six hours before they can pick up their containers. So we created a concept that aims at optimizing container pick up times by providing real time data on ports to truck drivers. To validate our product idea, we selected truck drivers and port terminal operators as the people we wanted to talk to.
Note: There can be a lot of variation in a user group such as truck drivers. What we did, was narrowing our focus down to those that have experience in regularly delivering and picking up cargo in ports.
Example questions for a screener
Once you know who to target, a short recruiting screener (used to determine if a potential participant matches user characteristics) comes in handy and might also help in further figuring out characteristics that you want participants to have as well as characteristics that are reasons for exclusion.
When recruiting participants, you are naturally left with two main choices: You either consult a recruiting agency who does the work for you, or you do your own recruiting.
The big downside of recruiting participants through agencies (even though it’s a quick and easy solution) is that you are usually not allowed to recontact people they provide to you. In our case, it is critical to build lasting relationships with the people we talk to, therefore we made the decision to build our own participant database and are continuously extending it with every new project.
So far, what has proven to work extraordinarily well for us is leveraging our personal and professional network as well as asking for further referrals from our study participants.
More strategies that have proven to be fruitful, were: - Contacting experts on LinkedIn (once you know what professions people you want to target have, it’s easy to go from there) - Going to exhibitions in the target field and talking to experts in person - Participating in hackathons (if you have the technical abilities, hackathons offer the perfect playground to experiment, validate ideas, get inspired by ideas from other hackers, and talk to experts from the field) - Looking for articles written by experts in the target sphere (writers feel flattered if you reach out to them to talk about this awesome paper you read) - Doing cold outreach via email. There is many ways to find email addresses of relevant people, however, be aware of privacy concerns!
Something else we usually do is making use of our social media channels: leveraging our website, spreading the word on Facebook, Linkedin and Twitter as well as sending out newsletters. We also tried and succeeded in recruiting participants in forums, groups and communities or even by approaching them directly in the streets.
For us, next experimental steps will be trying out new strategies, such as: giving talks in relevant settings and letting participants find us; creating a recruiting brochure that can be given to study participants to share; putting up well-designed recruiting posters or setting up booths in strategic locations; and placing ads in professional journals or publications.
Each month, we organize a two-day internal hackathon during which WATTx-ers dedicate their time to...
We investigated the overall consumer sentiment towards smart homes and the potential of smart home...
Each month, we, WATTx, organize a two-day internal hackathon.