Done correctly, user testing is a quick and less expensive way to validate your concept before you invest hours or money into developing your mobile or web app. Testing with users often highlights usability issues with how your information is structured or where users have concerns regarding their security/privacy/ease of use.
Why You Don’t Need Everyone’s Feedback
Now that you have a prototype, you might be going around asking your friends and family what they think. It’s likely that most of them will look at the prototype once and tell you how cool it is that you’re trying to be an entrepreneur leaving you with fluff for feedback. A few of them may be critical of the whole concept of your prototype, leaving you doubtful or defensive.
The trick is to NOT look to everyone for feedback. You need to screen out the people (and the noise) in order to efficiently narrow in on the applicable insights that will improve your design. You need to write a screener!
What is a screener?
A screener is typically a survey you send out to potential users that screens for the people with the right qualities that fit your target user personas (most often your customers). Focus on people who would actually use your product because it solves some pains of theirs.
If while testing you find that actual users have different behaviors from the target user personas you created then update them to better inform your product and design decisions. User personas should always be living and changing documents.
How to write a screener
Review your own user personas & requirements – so you know who you want & don’t want. In a scenario for a music discovery app, we want users who are:
- 24 – 35 years old, male
- browses on Youtube/Spotify/Google Music/Itunes/Pandora for new music to listen to
- spends about 1-2 hours a day finding new music
- plays 1-2 musical instruments (guitar, drums, bass)
- has 2-3 bands that he follows closely and wants to listen to something similar
- Be broad and narrow down
One thing to note is that user testers are usually compensated for their time. Thus, some testers are incentivized to take as many tests as possible, by selecting answers they think you want. In order to best screen out people who do not fit your requirements, we will have to start broad.
For example, which question is easier to guess? [brackets denote whether to accept or terminate the user from the test and is not shown]
- Are you interested in music?
- Yes [continue]
- No [terminate]
- What are some activities that you are interested in doing often (select all that apply)?
- Watching Movies / entertainment
- Exercising
- Cooking
- Playing Sports
- Arts and Craft
- Playing/Browsing Music [must select]
- Playing Video games
- Reading the News
- Outdoor recreational activities (ie. camping, etc)
- None of the above [terminate]
Clearly, screeners need to stay away from Yes/No questions, since it’s a 50/50 chance that the user may continue along the screener.
After users select questions, you can narrow it down even further by asking clarifying questions:
You indicated that Playing/Browsing Music was one of your interests. How often do you browse for new music?
- Not at all [terminate, all else can continue]
- 0 – 2 hours per week
- 3 – 5 hours per week
- 6 – 8 hours per week
- 9+ hours per week
Of the hours you spend browsing for new music, what websites do you use (separate each by comma)?
And so on…
Filter out by behaviors instead of collecting demographic information
As you can read in the line of questioning, the screener is used to filter out participants instead of collecting data. If narrowing down by demographics is important, use questions with multiple options so that the testers cannot guess through your screener.
Be quick to follow up
You don’t want to miss out on scheduling the user tests with those that do fit your requirements. The quicker you’re able to collect test data, the more you’ll know about how your product would perform with actual customers/users.
Why it could be beneficial to have non-users test/review your product
While testing with actual users of your product will result in valuable insights, testing or reviewing your product with some of “everyone” could potentially be useful. Some examples:
- It they’re industry insiders/veterans
- If they have specialized skillset for specialized feedback (ie. back-end data architect to review if your prototype is possible)
Running user tests can be a great way to make sure you’re developing a quality product that meets the needs of real users. However, user testing can be a time consuming and costly process, so it’s always best to make sure you’re testing with the right kind of users, not just any users.
