In life, there is a very natural instinct to look for the “right answer” or a “formula for success.” To seek advice that's guaranteed or proven to work.
But this approach will lead you nowhere. Just look at any professional. You won't find one formula of success repeated over and over, but many. Some unconventional. Some even unwise.
I had this realization many years ago when I was in graduate school. I decided that my goal wasn't to only learn how to conduct good research, but to learn how to conduct research the way I wanted to.
In this way, I didn't rely on my supervisor to teach me everything I needed to know. I visited other research labs - some for weeks, others for months. I consulted with other scientists on methods and study design. I even co-founded a podcast, where I had the privilege of speaking with and learning from all types of researchers.
I left graduate school with my own set of rules for what is good research, or a good collaborator, or a good leader.
These were my rules to live by, change as necessary, to fail at completely, and to learn from, always.
So find your formula. And like anyone who makes that choice, have faith. Faith in your actions. Faith that you'll have the power and strength to keep going in the face of duress. Faith that your actions will eventually lead to confidence.
I recently finished a study designed to help our team decide what to build next for one of our product’s core features. A secondary objective was to determine how we could make that feature more engaging to our users.
To design this future state, I asked educators to describe their perception of student engagement with our current feature and provide specific examples supporting their views.
What happened next was unexpected. Educators struggled to provide recent examples that corroborated their perception of students' level of engagement with the feature content.
In hindsight, I now understand why. The feature was used sporadically throughout the year. Most educators were trying to recall something that happened 2-3 months ago - of course, they’d struggle to describe the specifics!
If I were to do this all over again, here’s what I would have done differently.
Lesson #1. Understand how frequently the feature is used over the course of a week. The less it’s used, the less meaningful it is to rely on recall.
Lesson # 2: Understand the factors that affect engagement and preserve them in your study design. For instance, since lessons are done in the classroom and not for homework, it’s better to study engagement in the context of a classroom vs home. Likewise, since educators facilitate the delivery of the lessons, then it’s important to see how that might impact student engagement.
With that in mind, I would’ve designed that portion of the study around contextual interviews instead of remote 1:1 interviews with educators. This would allow me to observe how the lessons are being delivered, what student engagement actually looks like, and probe into the reasons that lead to high or low engagement.
Ultimately, studies can fail for the same reasons products do. We haven’t designed them to understand the problem or solution space well enough. We rely on what’s comfortable vs leaning into what will actually work. We’re satisfied with good enough, yet expect great results.
The next time you’re designing a study, be honest if the methods you’ve chosen will actually meet the research objectives.
The difference between market and user research lies in the first word: “market” and “user”.
A market consists of all the people who might benefit from using or purchasing your product. They’re not customers or users yet, but they have the potential to be.
The goal with market research is to help the business convert non-users to users. So you study trends, you study competitors, you study different segments of people, you study various problem spaces, etc.
In the end, you establish a set of key value propositions for a target market, then build those things to acquire them.
A user is someone who has already committed to purchasing and using your product. Whether it’s their first time using it or they use it all the time, they have skin in the game.
The goal with user research is to help the business engage and retain existing users. So you study the problems and needs they have, how to solve/address them, and how we can make the things we create even better.
In the end, you establish a set of core experiences and solutions, then iterate on them as your users’ needs and expectations evolve.
What is it about demonstrating and articulating research impact that scares researchers?
What makes us comfortable completing studies, but not following through on how they’ve been used, informed decision-making or influenced team/organizational processes?
When did it become okay to focus on output and not impact? Why don’t researchers treat their own work and practice as individual studies?
I believe there are several reasons for this. For one, our definition of success may not always align with our stakeholders. For instance, in academia, impact is equated to the number of publications and citations. That’s often enough in that world. That would be equivalent to us completing studies and having them cited in presentations.
But we must go beyond that. Our studies must inform decisions that otherwise wouldn’t have been made, UX/CX improvements that otherwise wouldn’t have been realized, process improvements that otherwise wouldn’t have been implemented, business value that otherwise wouldn’t have been created. That’s what success - or impactful research - looks like to our stakeholders.
Secondly, impact is highly dependent on the quality of your relationships with the decision-makers and doers. The more they trust you and see the value you can create for them, the better they can articulate their needs to you, and the better you’ll be able to support them and achieve your goals.
Last but not least, grounding your work in impact requires you to be more thoughtful in your own decision-making and processes. How you scope, design, deliver, and report on a study. You need to elevate your attention to detail and craft based on the context you’re in. Relying on best practices isn’t good enough.
People often seek out research because they want more or less of something - more understanding, more value, more confidence, less confusion, less uncertainty, less stress, etc.
I know it's tempting to want to learn all things, but the most meaningful research is when we stay focused on finding out what the team needs to know to push their work forward.
But the hardest part is separating curiosity from necessity. That's where scoping comes in.
I like to think of "curiosities" as questions that, while interesting, aren't tied to immediate action or current priorities.
The "necessities" are questions that can inform immediate decisions, strategies, and investments. Whether they be product, design, or business-related.
The "necessities" are questions that can inform immediate decisions, strategies, and investments. Whether they be product, design, or business-related.
The true value of research - in the business context - lies in prioritizing the necessities.
To define necessities, ask what your key stakeholders are working on now and in the near future. Have them walk you through those artifacts in detail, and identify areas where they're stuck or unsure. Also, take note of points that must be right or where there's capacity to change.
It is not enough to simply ask, "what decisions will be made with this research", and run off with what's expressed. You have to study the progress that has already been made and closely dissect what's in the works. Your research should aim to validate the former and inform the latter.
Some might say I have a reputation for asking good questions. I say it's in the spirit of discovery, not interrogation. I'm respectfully curious.
Here's how my curiosity presents itself during interviews. I always start off by asking if I've sufficiently answered all their questions or if there's anything I said that they'd like me to explain further, clarify, or give more examples.
I prepare more questions than I have time to ask, but with a hiring manager, I stick to these three when we first meet...
- Where do you feel your research team is really excelling at and where do you see gaps between what your team can do and what your partners need?
With this question, I'd listen out for any strengths that align with yours and any skill, knowledge, or experience gaps that you can fill. If you haven't shared these parts of yourself, now is the perfect moment to briefly introduce them. Reinforce them in future interview rounds.
- What are the team's biggest priorities right now? How do you see the new hire fitting in?
This is a natural follow-up to the first. It'll give you more insight into what's happening in the business, what's expected of the team, and the impetus behind the opening. Ideally, you're energized by these priorities, and they align with your interests and career goals.
- 4 or 5 months from now, how would you know if you hired the right person?
I always end with this question. Their answers tells me if they've thought about what they want and expect in the new hire. If you get hired, come back to this. Bring it up in a 1:1 after the first 90 days and ask how you're doing. Hopefully, you've lived up to those expectations, and if you haven't, you have 1-2 months to make up for it.
I've shared these questions with my peers interviewing for roles in engineering, product management, operations, and research. It's been helpful to them, so I hope they're helpful to you.
In a future post, I'll share the questions I ask potential teammates and partners - i.e. other researchers, designers and PMs.
Just a friendly reminder that it's OK to prioritize yourself and your needs in the workplace.
Be generous in moderation when it doesn't serve your goals, beliefs, and vision. Be generous in abundance when it does.
Where your attention and care goes, grows. Focus that attention and care inwards and only direct it outwards if/when it energizes you.
Cultivate that spark and sense of urgency in yourself, before doing it for someone else.
Choose you. You're the prize.
