For anyone about to embark on User-Testing, firstly, good-luck, it’s daunting. There’s plenty of niches and learning that you really wouldn’t have expected for something that on the face of it seems like a very simple task, to get information, from people. So here are 10 pieces of advice i’ve put together from my experience which hopefully you can take away and use to avoid any undue pain.
- Be confident yet humble
This isn’t contradictory. You can be confident in the work that you’re doing and humble enough to listen and value people’s views at the same time. I believe it’s very important to give off a confident persona even if you’re not exactly exploding with self-belief. What’s important is that they (the participants) believe you and have confidence in you and for that you need to be confident in yourself. So whatever the user-test, however sophisticated (or lack thereof) your workstation and equipment may look, just be honest with yourself about the aims of the session and make sure you put your best foot forward. If you’re not, this may set alarm bells ringing in the participant, set yourself off track and potentially have a knock on effect on the quality of the session, so if you can, deep breath and think positive. If you are confident, make sure this isn’t too overbearing on the participant either. Being a good-listener is an undervalued quality in many aspects of life, but hugely important to User Research. Make them feel comfortable, promote a chatty environment and let them waffle on, remembering bring the conversation back on track when needed. Structuring the conversation will help you to not take up too much time ‘off-topic’, but if they should stray, you can always pick out the key bits of what they’ve said in the analysis after.
2. Don’t be leading and don’t assume
‘Leading the witness’ as people like to say generally evokes the response you want to hear, not the one you need to hear if you’re going to improve your product. Have a few key phrases at hand to help push the participant along if they need it, but don’t ask a leading question as to guide their train of thought. For example “When was the last time you ….X? What do you think of ….X? You said X can you explain why?”. Try to prompt the user, then let them do the talking. Imagine it as a conversation with a friend who’s just come back off holiday, let them get it of their chest, it’s your job to listen.
3. Remember that it’s their first time every time
If it’s your last participant of the day you’d be forgiven for being a little bit lethargic, a little bit fed-up possibly and generally just wanting to go home. However, people are giving up their time and it’s important to respect that, especially if you want them to come back in the future, which you might. Don’t make assumptions (as mentioned) because of what you’ve heard from participants earlier in the day and treat each person like a new customer coming to your business. It’s their first time, so make them feel comfortable and valued.
4. Allow time in-between for participants
There’s no point having a rushed, uncomfortable researcher and similarly, there’s no point having panicked, rushed participants, it’ll only end in a worse quality of results. Put plenty of time between participants, people will frequently be late, no matter how much faith you might have and sessions may naturally run over. So just try to err on the side of caution and give yourself and them a little buffer to stay on track.
5. Have a back-up (and then maybe a backup of that)
Things go wrong, things get lost, issues occur (see Murphy’s Law). Just make sure you have a back-up. Make sure your laptop and/or phone is fully charged, that you’ve got the charger as well just in case. Make sure you’ve access to WIFI, know where the toilets are etc. and be sure to print off or have to hand more than is necessary of any items you’re planning on giving to users in case any might get ruined or lost and need replacing.
6. Get participant’s contact information (before the test)
Sound simple, i’ve made the mistake plenty of times. Email, phone number, whatever its is, make sure you get it off them when they sign-up and give them a gentle reminder perhaps the morning of the user-testing session. If one person is late and you’ve set an ambitious number of people to trial during the day, then any set-back can have a undesirable effect on your next sessions for that day. People forget things, so just try to mitigate it by being on top of who’s next and give them a little nudge if needed.
7. Expect Drop-outs
If you’ve never had a drop-out to your session, well done you. Unfortunately i’ve always found this to be par for the course, but you can help to mitigate this by giving people the gentle reminders (with the contact info). With this in mind you may want to try recruit more than is necessary (e.g. + 5 to 10% of your target) depending on your total. This is especially important for those conducting quantitative research and require a threshold level to get any statistical significance as dropouts may mean you don’t have enough to ascertain statistical significance even with strong results.
8. Try to be focused in the research
Trying to get as much information as possible back from the participant can often make it hard for you as the researcher to capture and also difficult for the user to articulate their feedback if there’s lots to consider. If possible, try to be focussed in an area you want to know about or test and break it down in to more easily tested chunks. You’ll be able to get really riched informed information on the aspect you do chose to test. However, there is certainly a different benefit in testing something as the sum of its parts rather than individually. If this is the case, think thoroughly about what it is you’re wanting to test and ask and ensure that these key issues items are tested and not missed amongst the general patter of the user-testing conversation.
9. Don’t do user-testing with a working prototype which only has low functionality
This might sound unusual, but i’ve learnt from experience on this one. I’ve found it’s always better to test early in terms of fidelity for a few reasons. The first two being cost and time, it’s obviously easier to knock-up a sketch, wireframe or interactive prototype then one which has been fully coded by the developer, besides any changes needed you’ll be able to make yourself when iterating. Secondly and probably more importantly is people’s expectation levels. It’s genuinely astonishing how much this matters when you give someone a working live environment, which has been developed, but is missing a lot of future functionality which will be created in future sprints (for example). You can preface it to the participant, however there is every chance you’ll end up with them pointing out issues, which you believe you are awaiting development, or criticisms which are fair, but are born from issues which would otherwise be not existent in a fully-working model. You may spend too much time on these issues, not where you want to focus and probably already have one-eye (as a team) on fixing the issue anyway. It can put the whole session out of sync and probably isn’t presenting the user with a fair representation of the product anyway, even when looking at a small area as they could well be off put by what they’ve seen. Low-fidelity helps to shape their expectation. Sketches and wireframes remain malebale in their minds and their more likely to understanding that the product is ‘in development’.
10. There’s always qualitative data to be gathered (or almost always)
In essence, if you’re speaking to someone and getting them to go through a user-test to collect quantitative data, then it might be a missed opportunity for you to get their thoughts and opinions (qualitative data). If they’re rating, or completing a task which is being measured, chances are they’ll be talking to you about it, so don’t miss the opportunity to note this down. Qualitative data ultimately helps to tell us the ‘Why is X happening?’ to the Quantitatives ‘What is happening?’. Even if it means a dictaphone to record what’s being said, just to be sure to obviously get their permission. If speaking or note taking from the participant may actually be a confound, either obstructing the user or biasing the results, then you will obviously want to avoid this, but it’s worth asking the question when you are creating your user-test.
11. Always run a pilot test!
…and finally, this will help to iron out any inevitable bumps or kinks in the test, remind you of anything you forgot and potentially make you rethink the approach completely if you find it doesn’t actually work. Plan it in as an essential part of the user-testing so that a pilot test doesn’t go uncompleted. Completing one beforehand is invaluable, as will be known by anyone who has conducted a user-test before.
Why Confidence is Key and 10 other useful tips for User-Testing was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.