Just as with any product, you need to understand the current state of your users before you try to change anything. That’s why it’s so important to get a full picture of your product operations function before you try to make any big changes. While it takes time to understand the entire picture around how the product team is functioning, there are a few questions I prioritize when establishing a baseline.
Each of these questions maps back to some of the key pillars of product operations: using data; understanding users; team ownership; and cross-departmental communication. The faster you can understand the way your team is working in each of these areas, the better you can diagnose the best course of action.
The goal is not to just ask these of the product managers, but of the entire product development organization and key stakeholders. The more you understand alignment around these key product concepts, the more you’ll be able to make suggestions that move the needle.
Five of my favorite questions to ask to quickly understand how well the product operations function is running
Do you know what is on the roadmap for the products you’re working on or work with?
This question focuses on communication, both on a team and across departments. If the engineers and designers don’t have roadmap clarity, they’re going to struggle to make great decisions about the product as invested owners. If stakeholders across the org lack understanding of what’s coming up, they’re going to struggle to make sure the product is promoted and supported appropriately.
Do you know how to find product and business performance data? When is the last time you accessed it?
I recently worked with a client where everyone had a strong sense of the business data. Everyone knew whether purchases and overall utilization were trending up or down. But they didn’t have a great sense of product performance, so were unsure about the leading indicators to that business performance data. This slowed down their velocity overall because it meant that to see the effects of changes they were shipping, they had to wait for that business performance data to shift.
I ask this of everyone because even if every person at the company doesn’t use this data frequently, everyone should be able to access it when needed.
Do you know which products are performing well and which are not?
This may seem similar to accessing the data, but it’s got some nuance. Just because someone can access the data doesn’t mean they are looking at it. And sometimes reports are sent out about product performance that don’t require you to actually access the data.
In addition, this helps indicate if everyone agrees on what good product performance looks like. Sometimes different people have different opinions on what the target is, which can indicate a lack of alignment or shared understanding across teams or departments.
How many times have you talked to a customer/user in the past 4 weeks?
For the product team, this question helps you understand the sophistication of their product discovery process. If everyone in the product trio isn’t talking to customers with some frequency, you’re leaving value on the table.
For stakeholders, this helps you understand company culture. Is this an organization that sees the value of talking to customers across the board? It can also indicate how much product is bringing stakeholders in early on work to build alignment.
How frequently do you hold retros? What was a change that came out of a recent one?
When a team has frequent, well-run retros and follows up on action items after, half of product operations work will be taken care of. Good retros identify areas that need improvement across data and communication, help the entire team feel ownership and responsibility over their work, and get everyone invested in team processes.
This is just the beginning
As with all user research, make sure to listen actively and ask good follow-up questions. Don’t just take the question verbatim and move on. Listen for what’s being left out of answers as much as you listen for what’s being included.
When I run a product operations assessment, I ask these questions and quite a few others. I then pair it with a survey and generate a pretty extensive report. But if you’re looking to just get started and target some critical areas to focus on at the beginning, these five questions will get you quite a bit of data and insight to work with.