Skip to content
User research

Recruiting testers: why and when to recruit inside your product or outside it

By Raphael Alapini 7 min read

Recruit your testers in the app or outside it? The pros and cons of each approach, the role of recruitment agencies, and the right moments to launch your user tests.

In user research, everyone wants the best possible testing process. It's a central step in designing your product, and you don't want to miss it.

Yet many questions come up about the right timing and best practices for recruiting your testers. Why recruit my existing users when I launch a new feature? Is it better to recruit testers from outside the product? When is the best time to do it?

Screeb has supported user research for major brands like Doctolib and Pennylane for more than 2 years. We've gathered the lessons you need to give you our best advice and help you make the best decisions. This article was written in collaboration with Tandemz, the self-service tester recruitment platform.

Recruiting testers in the app: pros and cons

Let's keep it simple: here are the pros and cons of recruiting testers in the app.

Pros

  1. Authentic feedback. Your existing users already know your tool and have experienced its features, which makes their feedback more authentic and more relevant. By recruiting from your user base, you gather opinions from real users who use your tool every day. This feedback helps you identify problems or friction points your development team may not see, and make improvement decisions based on data.
  2. More efficient and less costly. Recruiting from your existing users can be more efficient than external recruitment methods. They're already engaged with your product: asking them to join user tests is faster and simpler than recruiting new testers from scratch. In addition, in-app recruitment reduces the need for external incentives or compensation, which keeps costs down.
  3. Better retention and greater loyalty. Involving your existing users in testing can improve their retention and loyalty. When they feel their opinion is valued, they stay more engaged with your app and keep using it. By involving them in development, you create a sense of belonging and loyalty, which supports retention and word of mouth.

Cons

  1. Biased feedback. Testers from your user base may be biased in favor of your app: they already know it and may have an interest in giving positive feedback.
  2. Limited diversity. In-app recruitment can limit the diversity of your testers: you only test with users who already use your app, and you risk missing other demographic profiles or user segments.
  3. A small sample. Depending on the size of your user base, in-app recruitment can produce a limited sample, which doesn't let you properly understand your app's usability and features.
  4. User fatigue. Your existing users are already engaged with your app and may not have the time or desire to take part in tests, which can lower participation rates.

In short, recruiting users inside your app can be very useful, but you need to segment your target audience carefully to collect insightful feedback. Also account for the biases and limits of this technique when you analyze the feedback you collect.

Recruiting outside your app: a better option?

To choose the best option, you need all the cards in hand. Let's take a close look at the pros and cons of this second method.

Pros

  1. A fresh perspective. Testers who don't know your product can give unbiased feedback and a fresh perspective, which reveals problems or improvements the internal team or existing users may not see.
  2. Varied insights. Recruiting testers from different demographic profiles, user segments, or locations brings diverse perspectives and helps you spot usability problems or preferences specific to certain groups. Testers from outside your product aren't influenced by past experience or bias, which gives more objective, impartial feedback on your app's usability, features, and user experience.
  3. A broader testing scope. Testing outside your product lets you assess your app's compatibility with different devices, operating systems, browsers, or networks, and discover problems specific to different environments.

Cons

  1. Cost and effort. Recruiting external testers can require additional costs and effort, to find them and pay them for their time and feedback.
  2. Time-consuming. Finding and onboarding external testers takes time, which can delay testing and affect your app's development schedule.
  3. Lack of product knowledge. External testers don't necessarily know your product or its features, and their feedback may not match your target or your users' goals. Of course, some parts of the product (onboarding, for example) benefit from external feedback.
  4. Potentially lower engagement. External testers don't necessarily have the same level of engagement or motivation as your internal team or existing users, which can affect the quality and quantity of feedback.

In practice, tools make it easier to combine in-app and outside recruitment. Tandemz, for example, handles recruitment against all your criteria, while Screeb helps you segment your users by their habits.

Now let's look at the best time to launch your user tests.

Going through a tester recruitment agency

Working with a tester recruitment agency can simplify finding the right profiles for your product. Players like Userlytics, Respondent, or TestingTime specialize in finding and selecting testers who match your target audience, which saves you time and resources. They give you access to varied panels, for more complete insights. Whether you need in-app testers or external users, these agencies tailor their services to your needs and make your user tests more effective.

When should you launch your user tests?

When you need them!

As a general rule, a few key stages are ideal moments to launch user tests:

Initial development phase. It's never too early to start. From the earliest phases of development, usability tests with a small group of users let you identify problems, gather feedback on early designs, and fix things early, before investing more resources. That avoids costly redesigns or rework later in development.

Beta or pre-launch phase. When your product is nearly finished but not yet open to the public, the beta or pre-launch phase is an ideal time to run user tests. You get feedback from real users in a more realistic environment, which lets you uncover last-minute problems or adjust final details before the public launch.

After launch. User tests aren't limited to pre-launch or beta phases. Continuing to run them after launch gives you valuable insights into how real users interact with your product in real-world conditions. You identify opportunities for improvement, optimize the user experience, and gather feedback for upcoming updates.

Major updates or redesigns. Every time you make a significant change or redesign to your product, it's the right time to test. You validate that the changes work, gather feedback on new features, and make sure your users welcome the updates.

Now you know everything: it's time to launch your first user tests, in the app or outside it.

Get in touch with our friends at Tandemz or the Screeb team to learn more. And to learn more about the process to follow for effective user research, read Tandemz's article: how to recruit inside and outside your product.

Written by Raphael Alapini All articles

Did you like this article?

In 30 minutes, see what Screeb shows on your own journeys.

Shall we look at your journeys together?