Why unbiased questions matter

Imagine investing time and resources into developing a new feature for your product and gathering extensive user feedback, only to discover post-launch that the feedback was biased. Users become lost within the navigation structure, forcing them to guess the next step. Customers lose connection and feel misunderstood, so they leave and you can’t figure out why.

This is the true hidden cost of biased research questions.

For example, asking a user "How easy was it to use our new application process?" may seem harmless, but it presumes the process was easy to some degree. This leading question prevents users from expressing if they found the process difficult, resulting in an incomplete picture of the user experience.

Multiply this effect across an entire research study, and you can see how biased questions can lead to flawed data, misguided design decisions, and poor business outcomes. The consequences are simply too high to leave the quality of your research questions to chance with biased questions.

Crafting unbiased questions

Got it? Alright, great. Now what does it take to ask unbiased questions, which will uncover meaningful insights? Let me explain. Here are some key principles we follow at Craftzing during user research which we used on numerous of partnerships like the Federal government of Belgium.

1. Use neutral language

• Avoid words that have a positive or negative connotation which could influence responses from users. This is what we call “leading questions”.

  • For example: Instead of asking "How simple and usable was the feature?" try "What do you think of the interaction with this feature?".

2. Ask open-ended questions

• Yes/no questions can limit the depth of user feedback. Ask questions that encourage users to share their thoughts and experiences in more detail.

  • For example: Instead of asking "Did you find the navigation easy to use?" (yes/no) try "What was your experience using the navigation?".

3. Avoid assumptions

• Do not presume any problems or solutions within your questions. This goes hand in hand with neutral language.

  • For example: Instead of asking "Why was the navigation confusing?" try "What was your experience using the navigation?".

4. Avoid “double-barreled” questions

• Make sure that each question only asks one thing at a time to avoid confusion and ensure clear responses.

  • For example: Instead of asking “What did you think of the product and its usability?” try to split them up in “What did you think of the product?” and “what do you think of the usability of the product?”.

5. Balance positive and negative questions

• Only asking what users found positive will influence the bias of the interview. To counter this ask them what they disliked.

  • For example: Ask both “What did you like about this feature” as well as “What did you dislike of this feature?” Here as well: avoid “double-barreled” questions.

Note that a bias also can occur during the observation of an interview, so you can’t fully prepare unbiased questions. It still takes another skillset to be unbiased during the interview when users don’t do as expected. The goal is to discover the deeper motivations, figuring out the “why” after each move a user makes.

Conclusion

By following these quick guidelines, you can already improve the quality of the user research. If you’re ready and kept the bias out the interview, it’s time to analyze those results.

If you have any questions or want to know more about our approach to UX research, feel free to contact us.

By Tommy Buelens