CPO Series: Aligning Around the Right Success Metrics

Read the Full Transcript

The following transcript has been altered for readability. 

Becky Flint: So, without further ado, I will hand this over to Lydia. Please go ahead. Share with us what’s your observation and experience on a good and a great.

Lydia Varmazis: Well, point of view is different for every single one but a good Chief Product Officer focuses on the how and the what, but a great Chief Product Officer focuses on the why, to frame the problem, align success metrics with business and to scale and grow the business. I think we’ll talk about that a lot more in the presentation. We may even have better answers than that. So, a huge thanks to all the attendees who took the time out of their busy day or their busy morning to come join us. Here’s this morning’s agenda. I’m going to speak for about 15 minutes on the first four topics. And then we’ll leave lots of time for questions and hopefully some engaging discussion at the second half. So, write down your questions.

Let’s start by base lining how we describe the role of VP of product versus CPO. I recognize that many of you come from vastly different types of companies and depending on the size and the stage of your company, this role can be very different. I also know that many companies use these titles interchangeably. They may call it the VP of product, the CPO or the head of product. For example, when I joined Checker, there were no C-suite titles. Everyone who reported to the CEO was a VP. So, let’s separate scope and responsibility from title. I also know that in some companies, a CPO owns the technology and the marketing functions. It just depends. For today’s discussion, let’s assume that you’re a product-centric CPO and you probably own the design function as well. As you can see, the CPO is considerably more expansive than the VP. For example, rather than talking about the roadmap, a CPO communicates the roadmap and the business impact of that roadmap.

It’s kind of a given that you’re going to execute on the roadmap. That’s why you’re there and that’s why you have the title. When I work on strategic and product roadmap planning, I often get this question from my customers. Is it about top down alignment and aligning the product features with a company OKR or is it about bottoms up innovation from the product managers who’ve got these really great ideas where they’re getting from the customer feedback. The answer is it’s both. As a Chief Product Officer, you need to identify, manage and prioritize this multidimensional matrix of priorities. It’s never easy and it’s never perfect. You must use customer insights and feedback to inform the roadmap and build stakeholder alignment using success metrics. What do I mean by that? Well, let’s talk about that more in the next slide.

Many of you may have read Stephen Covey’s, The 7 Habits of Highly Effective People. If you haven’t, it’s a good and quick read. It talks about two things. Well, it talks about many things but it does talk about your circle of influence versus your circle of concern. Your circle of influence is things that you can actually control. Your circle of concern are things that are happening to you that you can’t control like the economy or the weather. As a Chief Product Officer, your circle of influence you need to have expands considerably. When you start up as a VP of product, you’re likely to be working with the CEO, obviously, and then the marketing and the technology and the customer success team. Your external stakeholders are probably your customers and your partners. When you move to the CPO product, you’ll find that your circle of influence expands considerably. Suddenly you’re working with the board and with investors. You may be asked to present the roadmap and the strategy to the board at least once a year and report in multiple times throughout the quarter. They may even reach out to you and ask you questions about what’s going on within the product or make recommendations for people that you might want to consider for hiring.

In addition to that, one of your external stakeholders is the communities where you conduct business. Even though your focus is predominantly on your customers, you have to think about the impact of your product in the business, where you do business. Are you actually harming the communities? I often use this very geeky but informative way of looking at it. It’s the Star Trek prime directive. If the product you are doing is causing harm to the communities where you do business, that’s not the way you work. You have to think about that. It’s not just about growth. In addition to that, your internal stakeholders become much larger. You have to think about your finance team, your revenue team, the people’s team and operations. It’s not that you don’t work with them when you’re a VP of product. It’s just as an executive in the C-suite, you’re probably working with them a lot more because your scope and responsibility has excelled. Let’s talk about how that relationship will work in later slides.

As your circle of influence expands, you have many more internal and external stakeholders each with their own point of view on the organization’s opportunities and challenges. One way to build trust with this multifaceted group is to articulate success using the same frame. Use metrics. For example, your revenue officer says to you, “We need to increase revenue.” Okay, what kind of revenue? New revenue? Expansion revenue? With what products? With what customers? Your goal, as a Chief Product Officer, is not only to advocate for the customer and communicate the product vision but namely what you are building, but more importantly, why. As your circle of influence is critical, you begin to recognize that less and less of what you do is about building new products and features and more and more about building teams, your product portfolio and your relationship with others.

You need to leverage this to scale and grow the business. It’s your relationship with your teams, with your customers and with your stakeholders that will define your success in addition, of course, to the execution of the roadmap in alignment with the strategy of the company. And for the next part of this section, we’re going to go through some CPO scorecards and I’m going to provide you some examples of how I use these scorecards to drive value back to the company. As a product example, executive and consultant, the most frequent concern I hear is that we need to increase top line revenue. So, I’ve put that down as my example objective. Why? Because it drives growth and because it’s easily understood. Of course, as a product person, we immediately believe that the number one way to increase revenue is to drive and drive more value back to the customer is by offering new products and new features.

What if you don’t have this? What if this isn’t available? You have other options. I’m using the orange arrows on these scorecards to identify where you can use product and technology resources in the product portfolio to improve this success metric, in this case growth. But let me give you examples where I didn’t have to use features but I was still able to drive growth. I recently had a client engagement with a consumer product company where the technology resources were already committed. So, when they said to me, “We really need to increase average order value and top line revenue growth,” I suggested like any smart consultant, “Oh, that’s easy. Let’s just increase prices.” And they said they couldn’t because they were a mission-driven company and they had to ensure that their product was within reach of everybody in their community. So, here’s what I suggested.

They had a three-tier pricing plan. What if we kept one plan at the same price? We never changed the price, even in the next 10 years. And we identify it as the measuring offering. And then we increased the prices of the other two plans. We didn’t change any new features. Instead, we realigned the other plans, redesigned the pricing page and clarify the language we used to describe the features. To validate our assumptions, we conducted a value study and a willingness to pay study with new customers, understood their willingness to pay and aligned it with our ideal customer profile. The study informed the future roadmap. So, features that once the team was available we could build, we rebranded the existing features using the customer’s words. They immediately saw a lift in average order value and top line revenue. The study took two and a half months and we saw that impact within 30 days. Moving on.

Retention. This is one of the most challenging and interesting metrics. It requires a ton of effort from the product and the development teams as you can see here with the many orange arrows. When you want to improve retention and you don’t have product resources, what do you do? You need to go deep on the data and truly understand what the problem is. And let me forewarn, this is the number one metric after revenue that your board is paying attention to. In case you don’t know what net revenue retention is, let me explain. It is the percentage of recurring revenue for an existing cohort of customers over a specific period. So, if this many customers were buying the product last month, are the same number of customers in that cohort still buying the product or have they churned? It’s also a great way to figure out if you’ve got product market fit.

The best SaaS companies have at least 120% net revenue retention each year. If it’s less than a hundred percent, it’s problematic and people are going to start asking questions. So, let’s talk about why we can actually look at the data and understand that. One of my customers had an SMB segment that was churning over a hundred percent. Why was this happening? Because they probably didn’t have product market fit in the product anymore. So, our solution wasn’t to actually change the product to meet SMBs because we had moved up market towards the enterprise. The solution was to actually sustain the offering. What do I mean by that? What I mean is, no new sales, simply end of life, the product itself by taking it off the product roadmap but continuing to support the customers that are in that plan. We continue to fix critical bugs and we told the existing 65 customers what we were doing.

We said to them, “We’re going to sustain these features but if you want to move to the more expensive plans, we’re going to provide you a considerable discount for the next two years and you’re going to be able to move up the food chain.” If you didn’t, they probably would have turned anyways. I took some time to get over… Now, this was hard for the company because it was their initial pricing plan and it was how they built the company. But once they got over that and they recognized that the data said that, they were able to sustain the product and over 40% of their customers took advantage of this pricing discount within the first 60 days. This is how, again, I used net revenue retention to align the entire executive suite on what we could do to address this problem with churn.

Sales and marketing efficiency. Many of you are probably wondering why, as a Chief Product Officer, why did I put sales and marketing efficiency as part of the CPO scorecard. Because the CPO is responsible for helping revenue teams understand the product and when it’s necessary to close big deals. You may have a sales operation team in your company already but if you’re in growth stage that may not be available. So, it’s a shared responsibility with your team and the product marketing teams to keep the revenue teams up to date on new products, pricing, positioning, promotion and how that works with the marketing team. So, rather than just focusing on, “Here are the new features salespeople,” what can we do to help them understand the pricing and packaging, understand the value proposition of the new features and optimize the journey?

Go ahead, Becky.

Becky Flint: That’s a really, really good question and timing for this topic, which is if there are questions around, how do you empower your team when there’s heavy emphasis on growth? I think it’s really tied to this one. We would love to hear your answer.

Lydia Varmazis: That’s a good one. That’s a very good one and I absolutely hear it because I hear it all the time and because I hear growth at all costs. And that’s a really great question. So, yes, everyone cares about growth, especially the revenue team but there are ways to optimize growth that’s a little more sophisticated than up into the right. Especially if growth is only activating churn, which of course is increasing your CAC, which is actually decreasing your sales and marketing efficiency. So, understanding that relationship between growth and CAC is the number one way to mature your sales efficiency. If all you’re doing is getting customers acquired, at maybe a dollar per user or $300 per user and they’re churning within the first month or the first six months, it’s probably not worth the acquisition of the customer. And we’re going to talk about that when we talk about lifetime customer value.

Thank you, though. Nice setup. We’ve got two more and then we can wrap this up.

So, margins. This is where it gets interesting. We started this conversation by starting with Stephen Covey’s, 7 Habits, and we talked about the things you can’t control. Well, one of the things that we all have in our circle of concern right now is the economy. It’s pretty tough out there. So, as a CPO, it is well within your scope and especially in product, if you have product and loss responsibility known as P&L, to talk about how you deploy capital, whether it relates to CapEx or OPEX and specifically reviewing cogs and cost of ownership. I love building products and I love delivering value to customers that drive revenue, exactly to your point, Becky. But do you also know what I love more? I love creating products that also deliver value to my investors and my shareholders.

Becky Flint: It’s such a big, big differentiating factor of a VP product versus Chief Product Officer. And I hear you touched upon so many things around business and metrics and around drive business. So, awesome. There are lots of questions coming through, so I want to continue and then I’ll bring them in around metrics for scaling and so on.

Lydia Varmazis: Great. I want to be respectful of time, so I’ll go through this and then we’ll get to the questions.

If you want to scale a company and survive in the good times and the bad, it can’t be all about exponential growth at all costs. “To the moon!” There needs to be a balance. You need to be thoughtful when you’re deploying your resources, including your customer acquisition, including your sales efficiency and you have to work with your partners in the revenue team, finance and operations, to make sure that you’re being thoughtful. What can you do to optimize cogs? Can you optimize your APIs? Is total cost of ownership what’s impeding your metrics there? Can you improve your ML, machine learning, to make your AI even more efficient? As a CPO in tough times, you need to make EBITDA, earnings before interest tax deductions and adjustments, lifetime customer value and GM, gross margins.

Final one, engagement. As much as we like to talk about our customer’s engagement, monthly active users, daily active users and session, time important metrics that prove product market fit and validate your job as a Chief Product Officer. There’s so much more. You aren’t just building products or scaling a business. You are also building people and your leadership is in service of them. Therefore take the time to talk about engagement of your employees. Employee retention rate is a critical metric that, as a CPO, you need to have your eye on. You need to be looking at, are you compensating people competitively? Are you making sure that there are internal promotions and are you creating a healthy culture? You are accountable and responsible for that and that should be apart of your ownership in your metrics portfolio.

Wrapping it all up. I think I’ve beat this over and over again but it’s defining success by creating alignment, framing the problem with your internal and external stakeholders. Always understand the, “Why,” focus on your customer, deploy and scale and grow the business thoughtfully. Clearly communicate proactively and create a culture of highly engaged and innovative top performing teams. And finally, recognize that the most important asset in any scorecard is your relationship with your people and how you work with them. I’ll just add some initial reading in case you ever want to learn some more about this and if you want to read some really great books about leadership. Here’s some examples and also follow David Sacks on Twitter. He talks about SaaS metrics a lot. And Reid Hoffman has a great podcast on the master of skills where you can learn from really great people and who has done this well.

From there, I’ll move to the Q&A.

Becky Flint: Hey, thanks so much, Lydia. I saw two of your resources, David Sacks, and Reid Hoffman. All coming from our hometown, right?

Lydia Varmazis: I didn’t even think about that. Yeah, they’re both PayPal.

Becky Flint: Right. And so far, some of you may not be familiar Lydia and we actually both have a pretty extensive experience at PayPal. So, really excited having her to come in. So, we’re going to start some audience Q&A. Before I do that, I want to just touch upon some of the things you covered, Lydia, in your CPO scorecard around VP of product and CPO. They’re definitely a very, very different focus. So, you already touched upon that. I want to just cover one thing you mentioned around the business metric where I think there are most of the product audience here, some of them are relatively new to being a product executive and many of them have pretty good experience in leading product driving customer. So, what would it be your top suggestion as someone from a product background to go into the C-suite, directly working with the CEOs and interface with the other executives?

Lydia Varmazis: That’s a great question. I think the number one thing you need to do is build bridges and have shared metrics. So, very often in C-suite executives where I’ve worked, I’ve seen like, “Oh, here are the product OKRs and here are the marketing OKRs and here are the revenue OKRs.” And it actually, unfortunately, at times creates fiefdoms between the organizations and it’s like, “Well, we shipped the product. If they can’t sell it, that’s their problem.” Actually, if you create the OKRs where they’re shared OKRs, where both C-suite executives actually own them holistically, of course they each own their segment of that, then you create alignment between the teams and you build bridges. The second thing is I would spend time having one-on-ones with your C-suite executives every week and understanding their pain points. Understanding what pressures they have and how you can help alleviate that.

Becky Flint: Wow, it’s so cool to hear. That’s something where we hear, time and again, about how Chief Product Officers coming really is first to focus on business teams. Their stakeholders versus first going into their teams and looking at the product strategy and so on. But eventually we’ll get there but thinking about how we start with your partner, think about your business teams and how we create shared goals. It’s so refreshing. One of the questions our audience has is regarding the success metrics. What is something that didn’t make the chart in terms of scaling but really it’s important. Meaning they may not be the number one or two. What are other things you should consider?

Lydia Varmazis: Well, that’s a great question. So, obviously my list is very generic and it does depend on the stage of growth of your company. So, one thing I didn’t include, which I kind of think is obvious but it’s implicit, is your Say:Do ratio. So, your Say:Do ratio, for those of you who aren’t familiar with that, is, “Hey, we said we were going to do this at the start of the quarter. And we did that.” That’s your Say:Do ratio. So, hopefully it’s something above 80%. Obviously when you do roadmap planning, you can’t predict every single thing that’s going to happen. There are escalations and there’s new customer features. If you’re in a compliant business, sometimes the regulators tell you things two weeks in advance and you have to go figure things out and obviously move the roadmap.

Ideally, you shouldn’t be radically changing the roadmap within the quarter. Unless, again, there is an escalation. So, your Say:Do ratio is something that you hold between you and your development team and your program team that says, “We said we’re going to do this at the start of the quarter. What did we do?” Obviously you track and communicate. And so, hopefully that’s above 80% to 85%. And you’ve built that credibility with your sales team so they say, “Oh, when we said we were going to do this, we actually do this.” And the marketing team knows they can launch products, et cetera, et cetera.

Becky Flint: Right. So, there’s another question coming in saying, “How do you see the product operations role in the product team? It seems, does it really matter? It seems to be a buzzword these days. What kind of value do they provide?” What’s your experience?”

Lydia Varmazis: Oh, I work with operations teams all the time and it’s really, really important, especially if you have a product that has heavy cogs. So, if you have cogs… Good analysis in case people were wondering what that is. It’s the components that make up the product. If it is pure software and pure SaaS, the only cogs are really the resources and the human people but then you also have Amazon Web Services, Snowflake and whoever else you’re working with. That can actually grow considerably and if you’re buying data from third party sources to build your product, that also can actually add a fair bit. So, you should be working with your operations teams to understand the inputs and how you can negotiate better deals. Maybe the deal you negotiated with Amazon when you were a small startup for Web Services was X. Well, now you’re X plus, whatever.

Can you go back and negotiate a better deal? Or maybe the marketing team is using Asana and the development team is using JIRA. Som everybody has three seats of everything. That actually adds up over time as your company grows. So, can you align on those things? And then the second thing is pricing. I work a lot on pricing as a product leader and working with your operations team and your sales team and your marketing team… Usually I create what’s called a pricing council to think about how we optimize our prices based on our analysis and looking at margins. Your operations partner is critical in that.

Becky Flint: What is your top one or two suggestions on product leaders to navigate today’s environment. Is it going to be any different or similar moving us forward?

Lydia Varmazis: What I would do is I would make your top three priorities. Write them down and communicate the, “Why,” as to what you’re doing. “This is why we’re doing it.” And then write an email to yourself and say, “If I wrote this email and I read it back in three years in the future,” not in the past, “What would my three plus year old self say about this decision? And can I live with that decision given the information I have right now?”

Becky Flint: All right. I know that there are a lot more questions coming through. So, what I’ll do is we’re kind of running out of time today. What I’ll do is I would gather the question and I’ll send it over to Lydia. And then we’ll try to answer that. It’s great to hear a lot of insight from you, Lydia, and great question. A Chief Product Officer’s scorecard is something that we haven’t seen very much talk about. So, this is a really great start. I’ve seen lots of questions coming through. So, we’ll follow up with you and hopefully we’ll send it out to those. Continue to send questions to us. We have a short session and join us again. We have a more to come. And finally, we have more to come every Thursday at the 8:00 AM Pacific time. And if you are interested to know how to run your portfolio, how to tie your product roadmaps to outcomes and how you get more insight and tips and tricks around leading your product organization, go to dragonboat.io/CPO to learn more.