Location, Location, Location: On-site, Remote, or Lab Research

When it comes to qualitative research, your location choice matters. Unlike quantitative research, where your only decision is whether to use a snail-mail or online system, and hardly anyone does snail-mail these days, when comes to qualitative you have lots of options and choosing the right one is dependent on the research objectives.

Here is a simplified list of location options...

Option 1: On-site

On-site has more than a few drawbacks. It biases the research because participants come in knowing who the research is for, this can affect their ability to be honest or give negative feedback. They're not dumb. They know you're watching and walking into someone else's office space can be intimidating.

You're also only drawing people from the local area to participate. Unless you're only serving your local community, that's a really limited sample. Plus, if you're conducting the research during normal work hours you are limiting yourself to people who either work odd hours, are in a position to be away from work during your session, or have no job at all.

So why do it? A really good reason is that you're testing a prototype or some kind of media or product that hasn't been released to the public and you want to keep it under wraps. Having your company secrets stay at the office is a great way to make sure they stay secret.

Option 2: Remote Research

Honestly, this is my favorite kind of research. It's relatively cheap, there are no associated travel expenses, and it allows for a lot more flexibility for respondents, researchers, and clients. With remote research, the research is being conducted online using either a web-conferencing system or an online research platform.

I recently worked on a study with another researcher where, between the two of us, we interviewed and did user testing on a voice interface with thirty CTOs across the United States. My research partner and I each had the device that was being tested near our computers and we asked the CTOs to interact with the devices using their webcams. CTOs, and other high level professionals, are very strapped for time and our conducting the research via webcam allowed us to be available at their convenience. If we had done these interviews in-person, it would have taken much longer and required a lot of money in travel expenses and extra money to make it worthwhile for the participants (the CTOs).

Webcam research is great for in-depth-intervews (IDIs or one-on-ones), but what about focus groups? When speed and convenience is key, such as a design sprint, I recommend online bulletin board platforms. Invite research participants to answer questions using an online bulletin board system over the course of two days. Anywhere form fifteen to thirty participants can come and go as they please as long as they contribute two hours of their time over the two days the platform is open. Once participants answer the predetermined questions, they are then allowed to comment on other participants' answers, and the moderator can probe as needed. These bulletin boards can be incredibly helpful for requirements discovery and concept testing.

Option 3: Lab Research

A lab is usually a focus group facility that has been set up to do user testing. The room should be set up with all of the devices you need for your test, picture-in-picture video capturing the interface being tested, a way to pipe that video into the back room for observers to see, and technical support.

This is expensive. You are paying for everyone's travel expenses (researcher and the observers). You are paying more for incentives because you are inconveniencing the participants. You are also paying for someone who knows what they are doing to set everything up, have backups for your videos, and be able to handle the problems which arise without fail. I once had an agency try to save money by doing the lab setup themselves. They sent the facility a laptop PC and a cheap webcam to use and I went cold with terror. I, the moderator with limited technical skill, would be responsible for making this work. Everything that could go wrong with that project went wrong. Videos were lost, devices didn't pair, fingers were pointed. It wasn't fun and it distracted from the work.

Lesson: If you're going to shell out for a lab test, shell out for technical support too. Peace of mind during a research study is priceless.

Lab tests are also limiting. Instead of talking to people from a dispersed geographic area, you're only talking to people from that metropolitan region who are able to spend a few hours away from work or home to be available during your window.

So what are the benefits? First person experience for one. Nothing is more convincing to a stakeholder than seeing a user actually have a problem in the moment or say something about the experience that hits home. If your team feels stuck or disconnected from your customers, getting outside the office and hearing from users directly is invaluable.

***

Regardless of your budget, you have options when it comes to doing good research. Just because your budget is small and you need a fast turn around, you can still get quality insights. Ask a research professional about methodology options before you assume it can't be done.

Big Design 2017 Presentation on Survey Design

Magic Numbers