Christina Lau (Host): Next up, we will hear from our panel of experts on how they’ve worked through the challenges in moving their business forward. So we will also have a Q&A at the end of this session. So leave your questions as we go. And I’m excited to officially introduce Scott Williamson, a product advisor and former CPO of GitLab. We have Cory Gaines, the CPO of Blackhawk Network. We have Ryan Polk, the CPO of Stack Overflow, Carmen Palmer, the CPO of Woman In Product, and our panel moderator, Becky Flint, Dragonboat’s CEO and founder. Welcome to all of our panelists and thanks for being here with us today. I am now going to let you take it over, Becky.
Becky Flint: Thank you, Christina. Hello everyone. I’m Becky Flint, founder and CEO of Dragonboat. At Dragonboat, we partner with amazing CPOs and product leaders to deliver products and accelerate outcomes. And today I’m honored to be able to join with quite a few world-class CPOs to share with our audience on how great CPOs move the unstuck organization and accelerate business forward.
So what I’ll do is I’m going to start with the first question and I’ll pass to one of you and then please do provide a couple of lines of overview of your background and in context here, answer the first question and then you pass it to… Popcorn to the next person. And then we go around a little bit like that. Without further ado, I’ll start.
The first question, I’ll come to you first, to Carmen, which is, let’s talk a little bit about a few things around what are some of the accelerators and the roadblocks to achieving more outcomes faster? Carmen, it’s up to you.
Carmen Palmer: Sure. Hi, I’m Carmen Palmer. I’m the chief product officer of Women In Product and we’re a nonprofit organization looking to bring more women into leadership positions and really get women and non-binary folks to thrive in the product management space. I’m relatively new to the role, but prior to that I was a chief product officer at MarketCast, a data and analytics and technology-enabled organization. So thank you for having me here.
As far as accelerators, I think when we’re talking about how do you achieve outcomes faster, I think some of the things that some of the things Osama spoke about are really critical, which is around having very much clarity on, first of all, the company mission and goals, and then how does that relate to your product vision? I think you need to be really clear and communicate within your organization about what that is and focusing on outcomes. Because once you can get that alignment just within your organization and across, then hopefully you’ll be able to reduce friction as you move through developing your strategy and your execution plan.
So I can’t reiterate how much I think communication, so clear documentation and how the vision and strategy are going to roll out, and get that out to all of your stakeholders and teams. Because I think when you really focus on the outcomes and be really customer focused and you put that analysis into that portion of your planning, then you can spend more time on the things that are really going to roll out. I would say even though it sounds like a slowdown to speed up, I do think that those really critical planning and communication steps are important for accelerating.
And so let me pause there. All right, I’ll popcorn over to Scott.
Scott Williamson: All right. Hey everybody. Thanks for listening in. I’m Scott Williamson, product advisor. Prior to my product advisory pivot, I had a long career in product leadership, most recently as chief product officer at GitLab. Prior to that was VP of product at Sendgrid and Twilio.
One, this is kind of a tactical answer about how to drive clarity and focus. GitLab became a very large R&D organization. We had about 40 different teams, we called them groups, that were working on various parts of the GitLab product experience, so a lot of breadth to what we were doing. To try to communicate a detailed roadmap for each team would’ve been mission impossible. Nobody can pack that into their head.
I chose to use three annual themes as sort of a galvanizing, rallying cry to try to simplify what we were doing, try to communicate what mattered most, try to make them pithy and memorable so that anybody in the company could at least rattle off of those three things that we were trying to get done as a product organization.
For example, at one point our SaaS service started growing much faster than our on-premise delivery model. And so we had an initiative called SAAS First, which was like think about SAAS First, think about the user experience on SAAS First because that’s where this is all headed. And by using those, I think it got people more focused. When trade-offs were necessary, they could tie back to the top three things we were trying to do and make sure what mattered most got done. I will pass it to Cory.
Corey Gaines: Thanks Scott. Hey everybody, Cory Gaines, the chief Product Officer for Blackhawk Network, which is a large stored value payments provider. Been here for about three years. Prior to this I was in the healthcare business. I was the chief product officer for a large healthcare administration company that’s now part of Optum. Prior to that I was at PayPal for about eight years as a product leader in the merchant group. So thanks for having me. Great to be here.
I would just leverage off of what Carmen talked about, starting with the clear understanding of the outcomes, the objectives that all the stakeholders are trying to accomplish. I tend to leverage a design thinking framework, which is start with what the outcomes your stakeholders are trying to achieve and then try to come to some loose arrangement on a framework for evaluating new opportunities because you’re going to get bombarded almost daily with new requests and new ideas and concepts. And if you get alignment early on in what your outcomes are and have some loose framework around how you go about evaluating new things that come about that get you to those outcomes, then you can minimize the amount of churn and debate you have because everybody agrees in the guardrails of how you evaluate new opportunities.
And then I would say if you can do that well, plus communication and frequent check-ins, which can be a big administrative overhead for the product team, but it’s really essential for maintaining alignment, you’ve got to find a way to have those check-ins be transparent, be clear, plain-spoken. And I think sometimes… I’m from the east coast, so my directness in the west coast doesn’t always fit, but it does get you to alignment much quicker without a lot of back and forth and a lot of traffic. So that’s what I try to do to keep momentum or build momentum and then keep it throughout the year. I will pass it on to Ryan.
Ryan Polk: Hey, I’m Ryan. I am newly the chief product officer for Stack Overflow. Previous to that was an operating partner at Insight Partners building their R&D community of practice basically, we helped scale and grow our portfolio companies. We had over 750 portfolio companies. We helped to scale and grow their R&D organizations as a whole, including product. Prior to that, chief product and technology officer for Carbon Black, cybersecurity firm, and head of product for Rally Software in that chain.
I love what everyone said so far. I think I’m a plus one for just about everything. I think that there’s a couple of core elements when we talk about outcomes. Defining outcomes by customer outcomes versus business outcomes is important to me in driving towards what problems are we solving for our customers. That includes a lot of really boring stuff though, like having a really solid concept of buyer and user personas along with ICPs, ideal customer profiles, and having that well aligned across the leadership team so that we understand who we’re talking to, why we’re talking to them, and what we’re selling to them, and how we’re going to solve their problems.
Those are just key strategy elements that are important, but not… Even more boring than that, but even more important to me is a well understood work breakdown hierarchy. Whatever that may be, whether or not using any OKRs to then align work to higher level objectives, or if you’re using an initiative style system, where you’re aligning initiatives to higher level objectives and then aligning that to features to teams and so forth. Being able to show the connection top to bottom across the organization is absolutely key.
I’m going through a lot of this right now in building out some of the framework for Stack. We’ve got great teams. They’re incredibly responsive to the customers. They’re doing an amazing job and we need to create visibility of that work and connect it back to the strategy of the business. And so that’s a large portion of what I’m doing coming in the door, is rebuilding that system and connecting the amazing work the teams are doing back to the leadership team’s priorities. Work breakdown hierarchy. Really, really boring principle, but it should be number one on everybody’s list.
Becky Flint: Right. This is super cool. Thank you for all of the insights. Interesting, you can see almost the flow of the breakdown of want to align to goals to actually technically how to align goals and to some of the challenges to go through.
Now, I hear from a lot of product leaders, which I think two or three themes you mentioned. One is having goals, but then also there are multiple goals. So which goal is more important? And the other part is you have a team that’s working and sometimes you achieve the goals there, sometimes it’s not there. So we take this similar concept. Let’s talk a little bit of some of the common challenges for the things that you think is the right thing to do. Accelerators, so what are some of the common challenges that you run into? And then how does your team organize to achieve that, to solve that? How do you organize these goals and then what’s the big hurdle to overcome? Probably start from the beginning, Carmen, on this question and would’ve love to hear your insight. What are some of the challenges that you have to go through?
Carmen Palmer: Yeah, certainly I think that the challenges in creating that vision, that map and that work breakdown piece, again, I have to reiterate it, it goes back to making sure that all of the stakeholders are really aligned. I think in some of my past roles where I’ve had to come in and really build a product-led organization where you have to put that design thinking in and you have to think about repeatability of what you’re launching and be super customer focused, is that each of the different stakeholders in the organization, whether it be sales or maybe even the marketing team or the engineering teams, we need to be aligned on where we’re going because it’s very easy, for example, that we’re trying to build repeatable, scalable, very customer-focused products that solve for an outcome. But oftentimes if goals are not aligned, you might start to get lots of custom requests. And then how do you prioritize those requests against what you’ve established as part of your roadmap?
So I think in terms of the prioritization process, it all still ladders back into that strategy. And then as you go through breaking that down, you have to look at what is the value that each… If you get down to the feature function level, what is the value that that’s actually bringing and how is that going to help you meet that outcome?
So in terms of prioritization, it’s looking at, yes, customer focus and what are the actual problems we’re trying to solve and how does that… Even now on the feature level, how does that roll into that? But then you also have to look at how can you roll that out also in the context of what is going to be healthy for the business as well. And you need to find that balance between what are the right things to push out there, but what are also the things that you have other stakeholder goals, whether it’s your board or your CEO? What are those other things that you need to be solving for?
So in terms of that prioritization model, you really have to be able to look at the product itself, but also how does it ladder into what your organizational goals are as well. So I’ll pass that over. I don’t know who was next. I think Scott.
Corey Gaines: I think it was me. We’ll stay in the same order to make it simple. Yeah, absolutely. I totally agree. I think that the biggest challenge, the biggest hurdle is just constant distraction. I think about the life of a product manager every day. It’s a constant game of what do I focus on right now at this very moment? And you’re constantly flip-flopping between protecting an existing relationship, dealing with security and compliance, which is a pain in my ass at all times. Figuring out ways to grow the business. And making space in room to do experimentation and innovation. It’s a constant battle to figure out which of those things takes precedence any of you every time.
So I go back to the framework that I spoke about earlier, which is having some alignment around what are the key output metrics you’re focused on, the outcomes you’re trying to achieve, and having some loose agreement within the organization on the ways to evaluate anything that comes at you. And I’ll give you an example. We sell gift cards,, and we constantly looking for places to distribute gift cards, both physically in stores and online. And every day I’m asked if a new destination would be a great next place to sell gift cards. You would just come up with something very simple that everybody can grab onto. How much traffic does it drive? Does the content that we have aligned with the property that we’re trying to go after? If it’s a homemaker’s website, do we have content that’s relevant to that? What’s the compliance overhang?
And if you just do a couple of simple little checks and balances, then hopefully people can get better at evaluating what could be really important versus what’s not as important. I think going into the year with a good understanding of what your output is and what your key evaluation criteria are, and then just… Sometimes it’s as much art as it is science, just making sure you’re balancing correctly between growing the business, protecting your existing business, finding a little space for innovation and experimentation, which really is the foundation of innovation. And then dealing with compliance and security. That’s just something you have to do every day really well.
Becky Flint: This is really cool. So I think one of the things talking about strategy and the goals, adding the success to that, it’s definitely a key part for bringing down to more closer to the product managers. I think Scott is the next one, so I want to make sure… You guys had a little swap here, but all good.
Corey Gaines: No, you’re right, I did. Sorry about that. I was a little aggressive. Sorry, Scott.
Scott Williamson: It’s all right. We should switch up the order so Ryan doesn’t have to go last every time. There’s a lot of hard aspects to being a product leader. The first is setting the direction, and both Cory and Carmen have talked about that, so I’ll focus on operationalizing the direction. Once you know where you’re headed and what’s most important, then getting the org to execute against that is non-trivial. There’s a lot of organizational inertia that can come into play. There’s distraction that Cory mentioned. You get pulled in way too many directions. So how do you stay true to the direction you’re pulling? How do you make sure that the org is aligned behind it?
I mentioned GitLab having 40 teams. We were in 80 different market categories, 10 different DevOps segments. Prioritization of our efforts across the giant product surface area was a constant challenge. I created this thing called an Investment Allocation Framework, which looked at monthly active usage, contribution to revenue and the addressable market size in the three-year timeframe, and scored each area on that and use that as a slightly more objective way to evaluate and communicate our investment decisions.
As a concrete example, we had 10, what we call DevOps stages, from planning to source code management to CI to CD, and we decided to put 80% of our people on four out of 10. So we had relatively heavy investment in four stages and relatively light in six, whereas before it was fairly peanut buttered across. So it was a more decisive product investment strategy, but we felt like that was necessary to make sure that we could win in the core areas we had to win. And then you have to get everybody aligned to it. We probably did 20 different small reorgs along the way where we would align people to where we really needed the investment. And all that takes time and energy and good communication. Ryan?
Ryan Polk: Yeah. So I’m drawing back to a couple of key things. Being the last person, I think everybody’s got some great talking points that I love. I would say that a couple of key things that I like to pull back to here on the strategic side, and then I’ll go to a little bit more tactical. From my experience on the other side, on the dark side of the VC world, I learned a lot about how shareholders understand the value of the business and I like to tie our objectives back to shareholder value, actually taking these things back to core shareholder value and how we calculate that as an organization.
Not just talking about revenue growth, but actually talking about quality of revenue and multiples produced by that revenue and then giving us an idea. At Stack, we have multiple different avenues right now. We have our consumer product and we have our enterprise products, and so we have to balance the opportunities for revenue across both sets of product.
So that’s an interesting framework that I like to apply and align with the organizations. The other thing I would say is what are we not going to do? Because lots of times as the leader of product, we end up being the person who says no. And so creating a systemic way of saying no, that is not no, but what is the priorities? How do we prioritize the top-level items on our roadmap, and how do we as a leadership team align on those so that your head of sales, your head of marketing, your head of finance, are all aligned with you when their teams come in and say, “Why aren’t we doing this other thing?” And so having a specific perspective on no and how you present that to the leadership team is incredibly important.
Becky Flint: Thanks, Ryan. I’m actually going to go reverse order so you can go first this time, but the question is I think a lot of… Actually, this whole summit really stems from how product leaders has already… Like Melissa said earlier, product leaders are no longer just product leaders. Product leaders are business leaders, especially when you are CPO, you’re more C than P. So the driving business forward from an investor and shareholder and perspective makes a ton of sense. And I heard the conversation around the allocation, around… The allocation is a way to say not do some of the other stuff. And this is really, really good insight across some of the traditional product management thinking, as users and stories and journeys and experiences. They’re definitely foundational, but it’s additional to that.
So want to take the lens of a PM. If I’m a PM, then I’m started working on something that was important. I spent a lot of blood and tears in building and then all of a sudden come out and say, “Look, this is a priority and this is 60% of investment here and then this is no longer a priority.” Especially in today’s world, how does your team navigate those changes? Tell us some specific examples of how you, as the business’ priority change, as the landscape change, how does it make from a potential roadblock to a accelerator?
Ryan Polk: So I get to go first? All right. First things first, base principles. This is a silly little one, but I don’t like my teams to be named after functionality or after areas of the product or anything like that. I like them to actually have their own names. And it seems silly, but we have to build a mindset of I am not my product and that I can actually throw away the things that I’m working on at any point in time and have that experimental mindset of, “This isn’t working, let’s move to the next thing.” So that’s a base level activity that I look to encourage in my product managers is that we’re not our product, we are experimenters that are trying to build something that is bigger than what we are.
We have to build that mindset in. I’ve seen so many teams that when we change their focus, because we have to, we have to make a shift in the roadmap to drive a strategic direction. We have organizational like white blood cell reaction to these types of things and it’s like, “No, that’s…” We’ve obviously missed something in how we’ve actually designed and matured our teams when that happens. So base foundation is ‘I’m not my product, my team is named something fun.’ So that team Kraken or team whatever is focused on being a team and producing the best thing they possibly can for the customer.
Now, beyond that, then being able to connect back and having perspectives, I love the book Team Topologies, but having a perspective on the types of teams you’re building around component versus feature versus X, Y, Z teams, product teams and so forth. And having a good framework for understanding how you’ve laid out your organization gives you flexibility. The more you can move work between teams and flex the work, the better off you are. But that flexibility comes at a cost of specific experience in certain domains and other things. And so you have to balance that as you build your organization.
Reversed the question here a little bit because you asked for examples of where we’ve worked through change. What I tend to do is attack the organizational structure first to make it so that we’re more flexible to change in the future. But I’ve seen plenty of examples where people are stuck in an area or stuck on an area of the product because they think this is their job and they must continue to deliver on something that we’ve all known has been long dead and is not providing customer value. And we need to build organizations that are better at flexing. And I’ll go… We’ll go… Let’s see. Let’s really mix it. We’ll go Cory next.
Corey Gaines: I love that. I am not my product. I need to have that tattooed somewhere. It is tough. I would say that I try to make sure that everybody thinks like an owner. I had an old boss at PayPal that told me, “Run it like you own it.” And that stayed with me because when you think like an owner, you’re constantly forcing yourself to uplevel your thinking away from maybe the feature you’re responsible for or the initiative you’re working on. And it gives you at least the mental agility to change your perspective and be open to new concepts and new ideas. Part of it is always checking your ego at the door to some extent, which great ideas come from everywhere. So yeah, think like an owner is what I would start with.
I remember early in my career at PayPal, the product management organization owned the P&L and there was a proposal to remove the P&L ownership from the product team and move it into, at that time we called it the regional teams, the business teams. That was a struggle for many of the product leaders because they were so invested in their product and the revenue they generated was attached to themselves. It was almost like taking a piece of your body apart and leaving it somewhere else and you ask yourself, “What am I going to do now?”
And I think part of the thing that helped get us through that was just taking more of a short-term versus long-term view. And we leaned into the P&L very important on a quarterly basis, particularly for public companies. But taking the long-term review, you’re not as focused necessarily on the short-term outcomes, but how do you generate value and accelerate value over the course of years? And that gave them some ownership that they still have control over the P&L, but just maybe not short term, not the short-term focused.
It’s a tough one though. Your ego is caught up in everything you work on. It’s very difficult to separate those things, but trying to keep yourself as uplevel and at the ownership thought process I think is very helpful. Carmen, you want to take a stab at this one?
Carmen Palmer: You bet. I really will play off of that because I think that the empowered product teams is really critical where they’re really looking at those outcomes in terms of what they deliver and not necessarily so specifically tied to this feature functions that where on the roadmap. Because a lot of times we’re going to have to pivot. So I’m going to uplevel it a little bit into ensuring that, as leaders, we’re driving this culture of design thinking and iteration, and really looking to not just the customer but the market in general and what we have to react to. So if there’s that visibility into, “Hey, this is what you’re working on and here’s the roadmap, but we have to be adaptable.” I think that especially now, the ability to adapt and change and how do we pivot with things moving so quickly in our tech space? You want to build that culture, but then you also want to build that ability of being able to surface things and then be able to act on them.
So there’s the high level of building that culture, of it’s okay to pivot, we’re going to consider all those scoring points that I think Ryan talked about on how we make those decisions. But then you pivot. If we take it to a tactical perspective, how do you do that? Because you also want to make sure that your teams aren’t feeling that they’re churning and that we’re constantly changing and moving, and then you don’t actually deliver on what you set out to do because you’re all over the place. And I think that in roles that I’ve had before, it’s about establishing governance or establishing some frameworks around how do you move these new things through the pipeline, having the team have a clear understanding of how do you make those decisions and then the ability to move forward. Because then that takes that adaptability and that ability to fail forward, to take that into what is the next thing you’re going to do. And I think that reduces a lot of churn.
Again, just to reiterate, I think that there is that long game of building that culture of iteration, test, evaluate and pivot, and then putting in those structures in place to actually operationalize that. So I think we’re on Scott?
Scott Williamson: All right. Change can be hard. People get invested in what they’re doing. Others have said that in different ways. So when you’re a product leader and you need to drive change, I think… Thinking a lot about a nuanced communication plan, in particular focusing on the why is really important, especially with knowledge workers that have strong opinions.
As an example, there was a moment at GitLab when our revenue growth rate dropped fairly materially around COVID. Prior to that we basically had a blank check for hiring and then all of a sudden we had to pivot into this mode where we were much more concerned about the cost of R&D as a percentage of revenue. We had to start managing to that. We were a year and a half, two years out from an IPO. Growth rate couldn’t sustain unlimited R&D spending. So we had to really start asking teams to do more with less. We had to reallocate, like I mentioned before.
So we put together a really detailed communication plan. We did live calls. We did Q&As. We tried to explain the business reason why and not just bark orders. And I think that helped us make that change with near–zero attrition and people jumped on board much more quickly than I even expected.
Becky Flint: Super helpful. I think since we’re on the topics about team and communication, it’s incredibly nuanced, like Cory mentioned, of how we all know product managers is are incredibly… Lots of ideas and opinionated, and it can be quite a bit of work to… Especially managing changes when people are so invested in things.
I want to just, using this opportunity, to move on our second part of the conversation, which is around people, process, organization. We touched upon some of that already today earlier. The part that when you look at the teams or when changes happened, some of the teams struggle more, some teams are relatively less. What are some of the tips? You called out a few of them, threw out a little bit, but if we were to say what are some common patterns on teams that are struggled more, and some of the teams are actually, for whatever reason, just accelerate better. Even in your own organization you do the same thing. You talk to them, you give them the goals, you go through the strategy. Some of teams just for some reason do better than the other.
Do you see any patterns, any tips in terms of how you see the potential challenges of teams that are struggling versus the other? And how do you identify them and help them to get along with the rest of the… Unstuck them, let’s say. I’ll start with you Scott, since we moved things around a little bit. Let’s do it.
Scott Williamson: I think when teams are stuck or struggling, oftentimes it has to do with a lack of autonomy. I think teams that have more autonomy typically move faster. And a lack of autonomy could be for a lot of reasons. It could be because management just tells them what to do. It could be because they have tons of technical dependencies with other teams and they just can’t move on their own. It could be that they don’t have a clear strategy or clear KPIs to drive towards and therefore everything looks like a good idea. I think the key elements of autonomous teams are, A, they have a clear strategy with a KPI that they’re managing to. They have a clear charter that’s separate, distinct from other charters of other teams. And they have a relative lack of technical dependencies.
So if those three things are true, and you have good people in the roles, that’s another glaring fault in many organizations as teams just don’t have skills to run in an empowered way. You have to be able to interview customers, you have to be able to make priority calls, you have to be able to communicate well, all this… And you need engineering PM and design skills that are all clearing the bar. If those things are all true, then teams can move really fast. So ask yourself what’s the biggest slowdown here and try to address that first. I will pass to Cory. You’re muted.
Corey Gaines: This is why I never go off mute. I never go on mute. I think, Scott, you nailed it. When you said autonomy, it rung a bell. It reminded me of the four agile transformations I’ve been through. I consider myself a pretty good agile practitioner now, but the very first one, I was not having it. I was not changing the way that I worked. I was the biggest pain in the agile coaches side because every time they said I had to do something, I asked them why 15 times. I refused to change. And it wasn’t until they actually left me and my teams alone that we began to embrace the methodology for what it was good for, which is operating and being effective in very ambiguous situations. But until you give people the ability to think on their own, the autonomy to learn and grow on their own, I think it’s a struggle.
I think many businesses force change on teams without helping them understand the why. And I think where I see struggles with a lot of these teams, either they have a bias of not embracing change, or they haven’t been explained the why very clearly. And I’m sorry, I think we’re all adults, and I look for product managers that are inherently difficult to deal with because those are the most skeptical ones and the ones that tend to build the best solutions in my opinion. But they also were sometimes difficult to control. And I think that management and leadership, it’s really upon them to make sure those teams understand the why and give them the room to embrace it and ingest it at their own pace. Yeah, I think ownership and autonomy is at the heart of that. Should I pass it to Ryan?
Ryan Polk: All right. I have three basic principles and I’ve actually covered I think two of them already. Well, I’ll cover two out of three on this. One is persistent and high functioning teams, which is core of this conversation, which actually means having a very standard template for what a team looks like inside of our organization, having all of the roles staffed. Unfortunately, when you get to 50 plus teams, as you scale organizations pretty massively, that is actually almost a full-time job just being able to sit there and look at the overall battle map of all your teams and understand how they’re staffed even, much less what work is going to them. And so having a consistent definition of team and driving that.
I find that a strong pattern for success is a highly engaged product manager at the team level who is a facilitative leader, someone who understands that they need to take on… We used to say Scrum Masters was this core role from an agile perspective. Quite honestly, I like to see highly facilitative product managers who pull the team through a lot of the process, own and manage the backlog, but really they think of themselves as somebody who is bringing the team together to come up with ideas and drive towards customer outcomes. And so it’s a mix between an ownership mentality and driving the roadmap and a facilitator mentality of driving the team.
On the other side of that, principle number two. Principal number one was well-formatted and functioning teams. Principal number two was a work breakdown hierarchy. Well understood work breakdown hierarchy. Number three is build your organization around one and two. There are supporting roles outside of the teams that are needed to make sure that we actually can function as an organization. We need to get to 10 teams. You need to have the basic concepts of a value stream or whatever you want to call that thing. You get to 20 teams… the basic concepts of multiple value streams. And as you grow, you need to have these supporting roles. Product operations is one of them. I saw somebody mentioned in the chat. Having the ability to actually run a process and be able to have a consistent process and manage across those teams gives a ton of value at the team delivery level and pulls away a lot of the BS that they have to deal with. Letting them focus on customer value. Giving strong, consistent ties back to the strategy, the business with a simple hierarchy that does it.
There’s other supporting roles. I am a fan, although there’s a… It’s a great beer debate over. I’m a fan of having multiple layers of product managers, having team level product managers and having strategic product managers that own higher level initiatives or OKR objectives and that their job is to work across the teams to make sure that we’re coordinating work. Because I believe you have to have both evolutionary change at the team level and revolutionary change at the multi-team level. And you have to have mechanisms in people supporting both of those. So there’s multiple org charts you could build there, but I do believe there’s a layout of people and roles that help support that as well. Oh, and Carmen, I call on you.
Carmen Palmer: Awesome. I think Scott, Cory and Ryan really covered a good chunk from the top level as well as the tactical. I think the other things I would add in from some of the experience where my teams have gotten caught up sometimes is around decisioning. So getting stuck in that churn of what is the right thing to do and maybe not getting that right direction. So when you’re talking about at whatever level of teams that you’re talking about, whether it’s at the feature level or above that that Ryan was talking about, is what is the right way to be able to make the decisions and move through? So again, a framework for how do you surface things when you are the person who is… As a leader, you’re the person who’s empowered to make those decisions and making sure that there’s a way to communicate those decisions out, and ensure that we’re going in the right direction.
And then I would say from an overall leadership perspective is providing the right level of support across your team. I think the one thing we haven’t touched upon in terms of change is that I think the thing that’s really difficult about change in priorities is that unknown and that uncertainty and how do you lead your teams through that so they can focus. So I think we’ve talked a lot about how do you get that focus through the structure and the frameworks and the communication and that transparency, but I think there’s also that, as leaders, we also have to look to is there anything else that is really blocking the teams? And some of that might just be your leadership and culture to create that space so that they feel comfortable to move forward and move ahead.
Becky Flint: This is super, super insightful. So we’re going to have one last question that’s planned, and then we’re going to take a more audience question. So this is an interesting question, which is around you. Obviously, we have lots of CPOs in the session. We also have quite a few audiences, not quite a few, hundreds of audience, but probably not a CPO today, but they want to get into their role at some point of her life. So the question may not be as discussed often, is how about you? How is the CPO evaluated? How does your CEO look to you? How do you look at yourself in terms of do a good job or something that I need to improve? And then how does your board looking at you in terms of what is a good CPO and what kind of KPIs do you use? Now, some of you… Who hasn’t gone start first once? Let’s do that. Scott, maybe you can start. You haven’t done as the first starter once, have you? I lost track, but let’s just do that.
Scott Williamson: Yeah, let’s go. At GitLab, the two primary metrics for product were monthly active users and we called it stages per user. GitLab is a broad product. It did lots of things and we were trying to drive breadth of usage across the product. So get more unique people on it, get them to use it more broadly. I also owned growth and so growth had a bunch of different metrics like around conversion rate, uptiering rate, that kind of thing, that were specifically because I had growth. I’ll pass it to, Ryan.
Ryan Polk: I think I mentioned a little bit, but I’ve been on the other side of this for the last few years, and so I got a good view of what the investors are looking for that are sitting on your board, which is different than board members, to be specific. Not all board members are investors, and so there’s different variables here for sure. What I would say though is we all need to be able to speak better in shareholder value and quality of revenue and retention and how we understand the multiples that drive the ownership structure so that we can impact that and start to grow shareholder value. And we need to put things in those terms for our investor board members because what you see as the sales organization, our revenue teams are really good at this. They really focus on those numbers. And when it comes to our investors, you have to speak in that same language.
I learned a ton of this being in the VC side. We would literally sit there and watch our board members thumb through the board decks, see the roadmap section go and then move on to the… I used to think that rating myself on consistency of delivery of the roadmap and customer outcomes and growth and revenue and retention and those core business metrics was key. What I was missing was the step of tying those back to shareholder value and actually putting that in terms that the investors can understand. So something to think about when you’re in your next board deck is coming back to that as a key variable.
What do I think that I’m rated on? I think in the end it’s building a business that is healthy, growing, and solves our customer’s problems. And I think our CPOs need to take more responsibility for being that business leader and somebody who is really driving the strategy of the full business, not just the product, and really being focused on facilitating and driving that across the company. Some companies were given that opportunity, sometimes we’re not. So you have to determine what is your opportunity in your next CPO role to be that person driving the strategy for the business, not just the product. That’s a key variable I look at when taking a role. I’ll pass it to Carmen.
Carmen Palmer: To play on that. Thanks. Thank you, Ryan. I think that it also depends on what kind of organization that you’re in. So we’re talking about if you’re a large public company versus a mid-size privately-held company, or a nonprofit that I’m at right now. I would say that in a lot of my experience too, there is that need to look at that sustainable and repeatable revenue. So if you’re an organization that’s looking to transform from something that’s more project-led to being product–focused, there is that measurement of what is that sustainable and long-term repeatable revenue. So there are those types of elements that you have to… Again, playing on what Ryan said is what are your stakeholder needs and what are they looking for?
From a nonprofit perspective, we are mission-driven every day. I think it’s a privilege to be mission-driven every day and to look at how we’re pushing that mission forward and ensure that it is aligned with how we need to run the business. But the first thing is is our community growing? What is the actual outcome? So in a way, there’s a lot of the things that we really try to look at from an idealistic perspective as product leaders, this is something I get to do every day. So when we look at measurement, a lot of the measurement, for example, might be more subjective, but we also have qualitative metrics, and how we look at our impact reports and things like that. So again, I think that as a CPO and a leader, you have to look at what is the stakeholder value, what do you need to do because it’s going to be different across different organizations.
Corey Gaines: Yeah, I agree with all the points that were made. I think there’s the obvious functional responsibilities, which I think Ryan already mentioned. Do we have products that get adopted? Are they contributing? Do they generate value? Those are more obvious. But because the product role, the modern product role I think is becoming a lot more known and well understood and more valued, I think two things Ryan said that I’ll lean into. One is you’re part of the leadership team, the executive team, which means you are a foundational part of the business strategy, not just delivering the roadmap, the business strategy.
That one I’ve grown over time into, but the last role that Ryan talked about was you are also a functional part of what the board sees as an asset to move the company forward. And that’s been a learning for me because I’m in a job now that I’ve never had so much interaction with the board. And I think the learning for me is looking at it through the investor’s eyes, and I agree, Ryan, the investors are different than the board members, but for me it’s all the same. They look at your business very different than you may look at your own business or your own product portfolio. And so I think those three levels are something that people that are aspiring to get to the CPO role today and its more modern form, I think you should keep those string things in mind. It’s not just about delivering the roadmap, it’s about business strategy and it’s about how you generate value for the organization overall.
Becky Flint: Super helpful and very, very insightful. I can’t believe… This session, I don’t know how many are tweetable already or X, or whatever that is. So now let’s go to some audience question. I’m not sure we’re going to have time to go through all of them. They’re pretty robust discussion here now. So for some of you, if we haven’t got a chance to finish up, we would probably follow up with you in emails and other forms. The first question actually to you, Cory, you mentioned a framework for evaluating opportunities. Can you distinguish big expansion disruption case from an improvement type of decisions? How do you resourcing and make that kind of decision?
Corey Gaines: I have to make sure I… It was a lot in there. So it was the framework for making decisions on intake and then how I might think about investment or resourcing to them?
Becky Flint: Yeah, it’s more like conflicting goals in terms of big expansion versus improvement. How do you make that kind of decision?
Corey Gaines: Yeah, my partner in crime, my CTO Nick Sathe, he uses a term called fuel mix. We use that all the time because we’re constantly bombarded by accessibility, compliance, and security vulnerabilities that are impeding my ability to generate revenue, generating products and features. So the question is how do you balance between those two things? It’s not a broad brush view. I think you have to look at the product maturity, where it is in the maturity curve. I think you need to think about what your business is really trying to accomplish and what the important factors are there. But at the end of the day, I think you’re trying to figure out what the right fuel [inaudible 00:51:07] is between these different activities that you have to align to.
The framework I talked about, figuring out what are the key input variables that you can move to get to your outcome metrics. The same kind of concept would play against how much do I do in tech refactoring? How much do I do in covering security vulnerabilities? How much do I do in innovation and experimentation? How much do I do in growth-related feature work? I think that is something that every team, depending on how you’re structured, every team that’s managing a certain amount of capacity and is responsible for a certain amount of value creation has to come up with that fuel mix. And the fuel mix may change over time. And again, it’s probably based on where you are maturity-wise and also competitively where you sit in the market. I wish I can just give you a snappy formula that will work in all cases. I don’t think that’s the case. But I think if you think about your investments as where I put my chits on any given day, that’s how I would approach it.
Becky Flint: That’s awesome, thank you. So next question, I’ll probably send it over to you, Carmen. You mentioned a little bit of in the not-for-profit organization it’s a very different way to do prioritization and decisions. How do you define product strategy and how is it different from, let’s say, others? So I’m just paraphrase some of the questions around product strategy definition.
Carmen Palmer: For sure. Great question and thank you for that. I think that coming from the for-profit world, I actually bring a lot of the same sort of thinking into the frameworks and the structures, but the way that you score and the way that you prioritize is different. Some of the advice that I’m also learning is every day the KPIs that you focus on really need to be about our community and the mission. I think someone mentioned in here, what is the engagement? So if our goal, for example, is to improve or provide development opportunities and get more women into product management in all levels, then we need to find measures to do that. Now, at the same time, we still look to sponsorship and donation and things like that to fuel what we do. So that’s still a very important part of what we do. And we need to look at the needs across our different stakeholders, our stakeholders sometimes being our sponsors as well as our community, and then our board and where we want to go, and us as a business.
I would say that I wouldn’t really change necessarily what we’ve talked about here, it’s just where the… Our main driver is really that mission piece. And then the belief is that if we can really address those customer needs or our member needs the best, then the other parts come along, which is definitely different from the for-profit world where you have different stakeholder KPIs that you have to look to. I’m not sure that answers the questions, I’m happy to chat about that more. And a lot of the things that we do within our community, if you guys have more questions about best practices for CPO, we just did a CPO podcast, Path to CPO at Women In Product. So please check that out.
Becky Flint: That’s awesome. Cool. Thank you. And I think, to your point, other than the missions is slightly different, that there’s a lot of a similarity in terms of defining strategy, have the different stakeholders. Anyone want to add more in terms of how do you draw product strategy in your space? Scott or Ryan?
Scott Williamson: I think a good product strategy is probably two years in length. You want it to be longer than your normal planning cycle, but not super far out. It has to make choices. You can’t do it all. So you want to really draw, okay, here’s the playing field we’re going to be on for the next two years, here’s what we’re not doing. So it should be ambitious enough to cover two, three years, but make enough choices that you can make trade-offs and prioritizations. I like the framework of what’s our situation, what challenges do we face and how are we going to fix those problems through our product activities? It’s a very simple layout, but I think it connects what we’re doing. Going back to the why, it connects what we’re doing to the situation we’re in and the business problem we have. And if you can get your team largely in that mode, you’re going to make fewer suboptimal decisions that might be about what’s cool or technology or what the last sales rep said. Definitely that’s how I generally think about product strategy.
Becky Flint: This is awesome. I love the highlight. Really important to say what we will not do, I think be not afraid to say and very decisive on that. This is so critical for the organization. Ryan, you want to add anything before we move on?
Ryan Polk: Very fast version of that, I would say. I like to focus on combining product strategy and business strategy as a facilitated process. I like to use Playing to Win as a framework. So it’s a great book out there as well. I like to use that as a framework for an iterative process throughout the year with the executive team on how we co-create the strategy for the business and for the product. And we use that as… Each quarter we’re doing a different activity. So first quarter we’re talking about our competitive environment in our current market. Second quarter we’re talking about our SWOT analysis and how we can win in that market. Third quarter is what do we need to do to change the roadmap in the future to do that and solidify our overall strategies and goals. And fourth quarter is operationalize that by changing the company, changing the dynamics of the organization, growing, shrinking, scaling, whatever.
And we think of those as activities that are built into a yearly calendar that drives our strategy process, which then drives our product strategy as well. And so I think of that as a part of my role is to facilitate that and drive that inside the organization.
Becky Flint: That’s great. So we have a quick round, so everyone maybe get one minute to go through a question I think if we don’t talk enough, which is oftentimes you’re CPO a larger organization, but when you’re a startup, a lot of time you’re the only PM, you the CPO, is it a PM, then everything. So, how do you decide what are the relevant metric for a company? Much smaller, it doesn’t have daily active values. How do you know you’re moving towards the right direction? So I’m going to go… Since Ryan started last, let’s go from you and then we’ll go around close at Cory.
Ryan Polk: Relevant metrics are we’re talking for-
Becky Flint: Small startup.
Ryan Polk: Small startup, all right. All right. So I would tell you that the key element at the very beginning is of course product market fit and getting to a degree where you can actually gauge and measure that. So you have to tailor your own set of metrics around product market fit and how do you know you’re getting the right engagement from your customers? It’s not revenue yet. It’s whether or not they’re engaged inside of the product, they’re finding value, and they’re driving towards some form of outcome that they will find valuable in the future, even if it’s an incremental outcome on a path to something bigger. And so I’m looking at as much instrumentation as I can inside the product so I understand what they’re doing and how they’re interacting so that I can get a larger swath of behavioral data to then drive my decisioning to get to that product market fit and really drive that to be really precise at that stage.
Becky Flint: Thank you, Ryan. Scott?
Scott Williamson: Early stage, I agree with Ryan, it’s all about product market fit. I think the underpinning to product market fit is having a tight ideal customer profile that you can sell to repeatably. And until you get there, that should be the number one priority. So as you’re working with customers, think a lot about the segments within all the customers that you’re doing business with. Try to find the segment that has the most repeatability, the best unit economics, the best path to getting to whatever your next stage of revenue is. Let’s say it’s 10 million, which segment can get you there? Segment your customers, instrument the user journey, and then you’re going to want to of course keep track of financial data and that kind of thing. One key metric that maybe may precede revenue is user retention. You want to see that flatten out. If you start to see user retention flatten out, that means that some segment in there is seeing repeatable value in your product and hopefully they’ll end up paying for it. Cory?
Corey Gaines: Yeah, I can’t add much to what Scott and Ryan said. I would just make sure, in that target customer, you have in that profile. I would have a diverse set of folks that represent that profile so that you are interpreting their behavioral patterns and their interaction correctly. It’s like a science experiment, you could easily misinterpret your results. And so good instrumentation, diversity of users that represent the target market you’re going after, I think that’s the best thing you can do to measure early on.
Carmen Palmer: Yeah. Cory, that’s spot on. I was going to say, what are the experiences that you’re actually driving? Are they addressing the problems you’re trying to solve? So a lot of that stuff is sometimes qualitative. You can look at your metrics and say, “People are coming and engaging,” but are they actually solving the problem that they need and are they going to come back? So that’s your sustainability. And then also as you look to build out the rest of your roadmap, are they behaving in your product the way that you’re expecting them to? So yeah, plus one to what everyone else said.
Becky Flint: Wow, what insightful session today. We never going to have enough time to go through all the questions. Thank you so much, Carmen, Scott, Ryan, and Cory for sharing your insight.
