How to Run an Elite Product Organization

Read the Full Transcript

The following transcription has been altered for readability.

Connie Kwan: Welcome to the session on ‘How to Run an Elite Product Organization in 2021.’ With me here today is Becky Flint, a technology executive based in Silicon Valley. She has built and scaled product and engineering organizations globally for both startups and fortune 500 companies. I’m very excited to learn more from her because she’s taken her understanding of program management at all these companies and created a product called Dragonboat. Dragonboat is a strategic product portfolio tool for leaders to align strategies, execution, and deliver near-term and long-term results. Her experience includes executive roles at feedzai, big commerce, Shutterfly, and PayPal, and she’s managed more than 10 million hours of products.

I’d like to start with this question- what is an elite product organization?

Think about product organization like a research team or basketball team, where a product manager can be a runner – a key member of the team, but not the only member or role. An elite product organization exists to fulfill one key thing, which is building products that sell well. And it sounds kind of obvious, but the difference is that a company does two things: builds something and sells something. For product organizations, it is in the building part where all the elements of the pieces come together.

When building a product, you need a great product manager, and you also need the team that builds the product. These two groups work together in the ecosystem to bring pieces together. They work to build a product that not only achieves what your customer needs and wants, but it also achieves the business value you need and want. That is the key differentiator of being an elite product team.

Connie Kwan: What have you seen in terms of building blocks of a team like that? What are all the people involved in creating this elite team?

Becky Flint: Going back to the analogy I had earlier about sports teams. Let’s say you’re a racecar team. You have drivers, the pit crews, mechanics, and you can have technicians. If you think about a leader of product teams, first of all, you will have to have really talented people with the various skills needed. You have product managers, designers, and the technologist, so the people’s talents are really important.

The second part is product craft, I think all of you as product managers are very familiar with the product craft, the user-research, market-research experience, AB testing, product analytics, all the things in terms of building a really good product. There are two more things that don’t seem very obvious. But in my experience they have really differentiated, a product organization, versus an elite product organization. So these two ingredients, I called product portfolio craft, or ppm craft, and then the other part of that is the cadence.

So if you can imagine, there are four building blocks. We will have to have “People Talent” – the people in various functions needed to build the product. The “Product Craft” the skills in problem discovery, product design and customer journey etc.. The product portfolio craft, which is holding the pieces together.And finally the Cadence which is how the operating model and rhythms within product organization and between product teams and other functions.

Those are the key ingredients of an Elite product organization.f we have one piece not there. It’s hard to keep everything together.

Connie Kwan: At Product Maestro, we are storytellers so I want to ask you what an elite team feels like. And then afterward share with us a story about a team that wasn’t so elite and what that felt like.

Becky Flint: Professionally I “grew up” at PayPal. I joined PayPal a long time ago. And it was a pretty young company. At that time we were also part of eBay. So there’s something very interesting about how these two companies operate at the bigger scale building product.

PayPal has an amazing product manager group where almost every single PMs I interacted with was super smart and great. And the same thing goes for eBay. What is uniquely difference is that at eBay, PMs are very metrics-driven, very customer-driven, and very data-driven. And eBay kept expanding and was doing really well.

At PayPal, PMs are also very metrics driven, they will look at expansion and conversion and managing transaction costs. But PayPal PM’s also constantly evaluated new market opportunities. eBay wanted PayPal team to optimize payment penetration on eBay, as it’s a huge market. PayPal felt they should go after merchants not on eBay.

Whereas eBay saw it as not the best ROI because investing / optimizing payments on eBay yields 10x or higher return than investing in unproven product space. Plus enabling merchants to conduct ecommerce (using PayPal check out), it reduces these merchants reliance on eBay and hence reduces eBay’s revenue.

PayPal and eBay product team disagreed on how to move forward. So the result of the negotiation is to allocate a small budget (aka engineering resources) to the new Merchant team (akak off eBay). A unique set of success metrics was set, that’s not ROI, to evaluate whether the off eBay “experiment” was achieving its goals. And the rest is history/

Fast forward many years later, things became very different. Back in the days, eBay accounted for maybe 90% of PayPal’s revenue. And PayPal was 3% of the market share of SMB online payment. Today (in 2020), eBay accounts for around 5% of the total PayPal revenue.  Could you imagine what would have been had the PayPal PM teams didn’t insisted on the “off eBay” product?

So the story is that as an elite product management, group, or organization, if you look at today, and you look at the future it won’t coincide. If you look at the customer you have today, and you look at customers you don’t have today, if you really focus on metrics, that is great for now but it may not lead to future expansion.

Connie Kwan: So what I’m getting at here is there’s this dreamer DNA that needs to be in the team but at a reasonable level. Otherwise, if you have too much of that, then nothing gets done.

It’s a very tricky line. Usually the CEO comes in and drops the ball asking for this thing they’ve wanted since starting the company. But there’s other priorities that need to get handled today like a stack of customer service tickets or a bunch of tech backlog. And then of course, we have these incremental iterative features that everybody’s asking for and beating down doors for. So I totally get that point. But I think the craft of doing the dreamer part is actually really challenging. So I’d love to hear more, you talked about product portfolio management earlier, is that the secret sauce for how you take dreams and make them real?

Becky Flint: It’s one of the things every organization needs. When I started in tech everyone did waterfall. Someone determined a three-year vision, a one-year plan, and everyone followed.

When agile happened, there are scrum teams with its own product managers with their own backlog and customer focus. Although there are many “product” internally, from a customer’s perspective. it is just one product. For example, you think about Google Search. Nobody thinks Google Search is 3000 products. But how many product managers do you have inside Google Search? The point is that no matter how big or small you are, as a product organization, even if you think you only have one product, you have a product portfolio.

It’s similar to how in the finance world, you have your retirement portfolio. You slice and dice your retirement portfolio in different ways, essentially, it is a portfolio of opportunities you can go after. The portfolio of opportunity is a secret sauce to build an elite product team because we cannot do everything. I just do one thing, we can all say new customer acquisition is more important than tactic. Or retention is more important than new customer acquisition. So how do we decide all of these pieces? Does that mean you cannot use the peanut butter strategy?

Connie Kwan: I’m curious. What is the peanut butter strategy?

Becky Flint: The peanut butter strategy is that if you cannot decide who gets something, you just spread it across everyone.

Connie Kwan: I love the analogy of the financial portfolio, because everybody understands you want to diversify your risk. So you have a number of different investments. But one thing you said earlier was interesting, the peanut butter strategy is still a diversified strategy. But it’s a flat diversification.

It’s like, I have 10 buckets, everybody gets 1 over 10 dollars of the money, right? But what you’re saying is it needs to look more like a wave, where over time, it actually shifts. So each bucket gets a different allocation. How often do you update that? That allocation, and what happens on the ground when teams get their funding pooled? It creates a lot of anxiety and churn and so how often do you make the shift? And then how do you manage your way in and out of that shift?

Becky Flint: That’s a very good question. There are actually three processes.

The first question is, is a peanut butter strategy, a good strategy or not? I would say it’s better than not having diversification, but you also have to think about every portfolio and the success of the portfolio. The outcome of the portfolio is not that important. How you allocate is more important.

If you talk to a financial advisor, that’s sort of the key point. It’s just like how you individually adjust your financial portfolio, based on where you are in life. It is the same thing for products, and we call that responsive. So it is responsive, meaning, we don’t really know how this is set up with the product features, and exactly how it will work out. We have a hypothesis, we think it’s going to drive revenue, we think it’s gonna drive sign up. However, we don’t have the metrics there.

Connie Kwan: How often are you checking?

Becky Flint: It depends on the business. Back in the day, there was a 3-year-strategy, 1-year-execution. Today, it’s something more like how frequently you can measure the output. If you can measure the outcome very quickly, it takes longer to see the effect so you may want to look over a longer period. Most of the companies I’ve seen so far have sort of an annual strategy or annual vision. You have either quarterly or maybe 2 month check-in, and then you have your typical spreads. That way, each step of the way, you can see if you achieve what you wanted to achieve and then you can make adjustments to that. So that’s the frequency.

Connie Kwan: So every 2 months, you check your numbers, there are some numbers leading indicators and you can get those numbers pretty quickly. But there’s some lagging indicators, right? Sometimes you need that lagging indicator to be your metric. How do you manage that?

Becky Flint: I think that most people today if you talk about participation, are always some sort of scoring. It’s actually very challenging to say whether data would achieve the outcomes or not. So think about whether your outcome and impact is probably more important in terms of metrics.

So if we do something like NPS, it would take a while, and it’s not inclusive. Therefore, we cannot just keep investing in their seat, and we could be over investing. So the idea is that we have a hypothesis, then we have to keep track of the hypothesis. If you can see there’s movement, that’s great, if not, you work on that.

It’s not immediate, it’s not like your financial portfolio for something. Even your financial portfolio, even though you buy something, it goes up or down in value, you still want to see the six month cadence. It’s almost like a business has somehow figured out a way to operate in a quarterly season, a quarterly report, etc. So that quarter from company to company, seems to be in a good cadence for at least a good sized company. Smaller companies can go faster, you can go monthly, for example.

Connie Kwan: So it seems like you plant the seed for the ones where you can’t check the result yet, those trees just grow slower. So you plant the seed, and then you don’t keep doing anything else, or come back and check on the tree when it’s ripe. And then you invest again from that point. What do you do with the team though? Because the team is working on a project and you’re only investing one month in the project and then they’re supposed to wait for the tree to actually grow and see if it comes out?

What happens after one month, do you put them on a different project?

Becky Flint: What happens is that we can look at two schools of thought. And if some of us were more familiar with the Spotify model, the team is actually more, more fluid, right? If you especially look at the newer ones, they actually go from mission to mission. You work on something, and when it is at least somewhat wrapped up, then you can move on to the next mission that’s mostly driving the outcome. So the team actually moves more without tearing the team apart. But mostly, the team is following where the most important outcomes are commonly kept.

The other one is very interesting to us. What happens is that when you work as a team together for a while, you actually create a new way of thinking, and then you bring energy to other teams. Not everyone has control on how your team potentially can work, but mixing things up and even having people work in different initiatives that move over time seems to be more helpful. People get to do different things, they have new perspectives, they still can bounce ideas off each other. It creates a very good balance.

Connie Kwan: It’s interesting you say that, because innovation is happening at the intersection of worlds.

For example, food innovation comes when you have culture A and culture B, and they visit each other. And now you know, the West is using soy sauce, and the East is using cheese. Then you have these new cuisines, that’s how you create this innovation. So what you’re saying about having this built-in kind of mix and match with the teams can really spark that new thinking.

In one of the teams I worked in, we did that too, but we hit a point where it became exhausting for the engineers. So I think there’s a balance. They were experiencing too much change, I’m learning new stacks all the time, their efficiency went down. There is a happy path of maintaining efficiency with new experiences. But then at some point, if you shake things up too much it does slow things down. What is the balance, you’ve seen that works?

Becky Flint: First of all, the frequency is very important, you don’t want to do it faster than a quarter, usually two or three quarters. In most cases, people settle in and do something, then they learn the area for a while. So that’s one thing.

The other one was, and I am borrowing a term that one of the VP of engineering I used to work with used to say, and the way he described this is, that you can bring the team work or you can bring the work to the team.

This means that you can either break up teams and instead of working together, you can divy up the work. You can have two pairs of people working together and take one pair to another area that needs more work or resources to achieve certain goals. That’s called bringing the team to work.

The other ones bringing the work to the team. You have the same team, but you’re going to work on different things so that you also create something new and exciting. Sometimes you have a team that’s been working together for a while, and they have really good synergy so they don’t want to be broken up, but they are ready for something new. That’s a situation where you bring work to them.

In contrast, sometimes the team has worked together for a while, and they want to work with others to figure out what’s going on. Then you can try breaking them up and bringing them to the other team. And so you can balance teams and work depending on the needs.

Connie Kwan: We talked a lot about larger companies. But I understand that portfolio product management also applies, even in startups, even if you’re one product manager, right? This is a mindset that can help, so can you talk through with an example of portfolio product management even at smaller companies?

Becky Flint: There are a couple of things, and the portfolio is the concept of those choices. How you categorize the product, and sculpt it into different categories is important.

One of the ways to look at that is whether you want to focus on new customers, because they’re struggling with your product. The onboarding might be complicated, so you decide to move your investment to your resources, your product features, and investment, to make onboarding work great.

The second part is that your existing customers could be seeing issues with your product. Maybe there are feature gaps, the widget not lined up correctly, and so on and so forth.

Additionally, a lot of times even with the same product, you have different use cases. So instead of selling to individuals, I can sell in a different way. I can sell to partners, I can sell and go to a different market expansion. That’s the extension of your product. That’s from a pure product perspective.

The other part of that is goals.  We’re responsible for more than one goal. The company needs to grow, but we have a lot of other goals. So the question becomes where do you find your time and support one objective versus the other, and when does your time support your own objective versus someone else’s objectives?

Obviously OKRs become hugely popular today, and everyone’s talking about it. There are two dangers and you have to be aware, so you can build the product around it. One of the first dangers is it’s important to make sure when we look at our OKRs, that it covers both tangible short-term and 6 month objectives. Also be sure that it allows us to innovate and stay ahead of the curve for our product.

The second part of OKRs is to think about how much your OKR is shared with your objectives versus being lined up to individual teams. We have a great blog on dragonboat.io/blog called “Is your OKR, hurting your innovation?” What that means is sometimes a company implements OKR, and then you have  a VP OKR, and then you have a product OKR? Are they really broken down? So when you break down your objectives into different areas, there are two problems that happen.

One is every team is going to have a smaller set of goals that limit you to think big. If I will go directly to the top goals, maybe I have a different approach versus my goal. That’s why when you say our goal is to grow, our goal is to grow revenue. And then we have to line up our strategies across the board.

Connie Kwan: Right, and one of the first things I did at my last company was to bring everyone on the same page. Everyone has different opinions and all of them require product work but it won’t work out if everyone is working towards different goals.

And with so many products shifting to subscription and cloud, how does customer success and adoption fit into the product organization?

Becky Flint: This is something a lot of companies are still trying to figure out. Customer Success has always been an integral part of the product, especially around inputs to existing customers. And some of them also drive you to new customers. There are almost three parts of it.

One part of that is to have a cadence of how we collect customer feedback and needs into its standalone category. And then we can categorize standalone and repository, so that they can be aligned to some of the product themes that a product organization would have. Then you can build that into your roadmap process, just to see how much of that could have fit into the category. This is similar to having existing customers, new customers, and maybe different personas as a segment of customers. And if you tag your customer feedback and an insight into this category, then you can bake them into your product planning process. This means Customer Success is in a close partnership with the product organization. It’s almost an extension in terms of getting the insight and first-hand experience into product.

Customer success is a very integral part of the product input process.