Drawing the Line Between PO and BA

The Scrum Business Analyst

I have heard more than once “There is no BA in Scrum.” Imagine how your BA’s feel when a transformation starts!  At best, they are uncertain what their role ought to be. At worst, it is made clear by everyone else in the process that the BA is no longer needed or wanted.

The irony, for an agile coach viewing this as an outsider, is that numerous individuals throughout the value stream who are also struggling to cope with the shifting sands of transformation, frequently report that mistakes, lack of prioritization, failure to clear dependencies, and miscommunication are due to “being too busy.”

Obviously, just from this “too busy” problem, there are two important things the BA ought to do as an active member of a Scrum Team in a scaled environment:

  1. Act in a WIP-clearing capacity to the extent their t-shaped skills allow.  To whatever extent they do not have T-shaped skills, the moment they are not clear on how to utilize their time is the perfect opportunity to develop these skills.
  2.  Capture the very broad “reminders of a conversation” about a story that, in a large enterprise, occur across a larger number of individuals, over a longer time period, and in more geographically distributed locations than “core scrum” implies.

Roles and Accountability

Now we can draw the line between the Product Owner and the Business Analyst.

The Product Owner is accountable for decomposing an Epic or expressing a single enhancement as User Stories.  The Product Owner creates a Story card in JIRA for this initial Story list that includes a JIRA Summary and the User Story in classic format:

As a {user persona} I want {action} so that {expected value to the user}.

This is an expression of “Commanders Intent” and represents why the story is being developed and who cares whether or not it is developed.  Thus, the User Story is an expression of product strategy, and represents trade-off choices and prioritization.  The decision to expend finite on and expiring resources – time, energy, money, and talent – on one product change versus another is the most critical accountability of the Product Owner.

Although the what and how is negotiable, the intention of the Product Owner serves as a litmus test for all subsequent decisions.  The what and how are the realm of operational effectiveness rather than strategy.  It includes the framework of economic decision making and the processes, practices, and tools that streamline communication and align strategic direction of a distributed control system.

The Business Analyst uses the Description to succinctly express the what and how that has already been determined so that no context is lost in subsequent decisions.  The what and how remain negotiable to the extent these better serve the “Commanders Intent” of the User Story.

In an analog Scrum board, there is typically an agreement on “front of the card” and “back of the card” content that serves as the “reminder of a conversation” for the team.  In a scaled environment relying on a digital board like JIRA, the Summary and Description fields serve a similar purpose.  As the number of individuals contributing to the value stream increase, the need to detail the conversations that have already occurred increases as well.

In the process of detailing each Story Description, it will often be apparent – due to test data or testing scenario coverage – that a Story ought to be split into two or more stories.  The Business Analyst completes this activity and is accountable for communicating the split to the Product Owner.

 Stories may also be further split during Backlog Refinement or Sprint Planning based on additional insights from the team. Attendees should collaboratively decide who will capture this decomposition within the tool, but the Product Owner is accountable for prioritization decisions (if the split impacts this).  

Purpose of the Story Description

So, to meaningfully define the role of the Business Analyst, we need an understanding of what value is created if one individual “owns” capturing the elements of a Story Description as the number of these predetermined elements continue to grow. To the extent at scale that the team is unable to economically interact with every other value add activity in the value stream, the purpose of the Description is a succinct expression all value-add activities and decisions that have influenced the User Story prior to development. While we want to express these in the fewest words possible, and work toward distributed control of decisions, we do not want previous insights “hidden” unnecessarily from the Scrum Team.

Several important activities have likely occurred prior to our Sprint:

  1. Business decisions fundamental to the economics of our interaction with the customer.
  2. Funding based on an overarching strategic initiative.
  3. Customer research and analysis of product metrics.
  4. User Persona definition and Empathy Mapping.
  5. UX Proofs of Concept and/or A/B Testing.
  6. Stakeholder meetings.
  7. Success Metrics defined.
  8. Technical dependencies fulfilled (such as a new or updated web service API).
  9. User Story decomposition.
  10. Other Stories already developed related to the feature.

Thus, many details needed “downstream” should be easily expressed in advance of the Sprint:

  1. Why are we building this story?
  2. Who is the User?
  3. How is this User unique in our Product (i.e. relate persona to an account type)?
  4. What Test Data will need to be requested to test the story?
  5. What steps does the User follow to obtain the value of the story?
  6. What will the User see when they finish the story?

DevOps Athleticism

Let’s face it – being small makes it look easy to be nimble.  Look at your average kindergartener: they may not always be graceful, but their capacity for unexpected action or a rapid spontaneous change in direction at full speed is frequently mind-blowing (for each of mine, it was usually a sudden jump into the risk of oncoming traffic).

So it’s understandable for the established enterprise to look at the youth (and occasional hyperactivity) of startups – and small companies who never grew up – and feel a little fat, a little old, a little bit brittle.

That metaphor doesn’t need to end there, though, because there are also large companies maintaining a portfolio that balances finding new opportunities on the one hand and exploiting new opportunities on the other.  These are two very different operations, though, and companies find balance difficult.  Just like hyperactivity is internalized in the executive function of adults who had physically hyperactive childhoods, that rebellious startup creativity can survive unscathed within the mature organization. By doing so, you can simultaneously continue category-killing through innovation despite staying the course and reaping decades of fruits on already-mature markets and products.

Likewise, to extend the agility metaphor, life is full of athletes in the top quartile of height and body mass index. They top the BMI charts compared to average scrawny-but-chubby adults despite doing it with rather lean body composition.  They are practically outliers, and definitely don’t fit the “standards” set by statistical BMI.  More importantly these individuals put “nimbleness” to shame; and just look at those kindergarteners revere them. Look at the heroes of the NFL: agility doesn’t begin to describe their mastery of movement.  Look at star hockey players in the NHL: they move with the power of an elephant, the stamina of a gazelle, and the grace of a ballerina.  The stereotypical Hollywood karate master black belt may have a very thick, potbelly body-type, but the master needs very little movement, in just the right places at just the right time, to send an opponent flying.

So it’s simplistic at best to “think like the startups” or “be more agile”.  You cannot transplant a culture.  Size alone is not their advantage, strength alone is not their advantage, tenacity alone – as much we love a good underdog story – is not their advantage.  To emulate any ONE  attribute of the lean-agile startup on the rise is foolish.

Stop talking about enterprise at-scale agility like you’re trying to be that kindergartener veering into the street unexpectedly. 

It’s more than being lean, or gaining experience or agility. At scale you need to build repertoire of enterprise-grade DevOps Athleticism!  It’s one thing to have an impressive vertical jump but quite another to jump over a fence, hurdle over a tackling safety, or parkour up a building,

Training for Obstacles

For the support engineer team, kanban looks a lot like an Olympic marathon team.  You constrain WIP (focus/movement pattern), create a sustainable pace, and fuel as you go – you train for the long haul by (basically always) running for distance.  It may be fragile spaghetti code built over the decades, but you your crack team knows it inside out.  But that’s not the majority of your at-scale enterprise.  As you get further from a continuous flow of relatively similar requests and move toward innovation and greenfield disruption efforts, kanban and even scrum are going to fail unless you include the assumption of uncertainty and churn in your overarching process.

This is like the difference between a New York marathon versus Tough Mudder or another obstacle-rich competition.  If you don’t build your capacity for speed-strength and coordination against unexpected obstacles, you’re likely fall short. The long and short of it is, if you can count on a marathon, kanban your way to glory.  If unpredictable obstacles and risk-taking for glory are fundamental, stop whining and start training. Look for extra opportunities to beat down tough challenges instead insisting on a slow and steady pace.  Speed and sustainability need to be loosely coupled in a strong DevOps process.

Jumping the Fence

I can tell you from experience that maximizing raw strength in the barbell squat does not correlate to jumping higher.  If you need to jump higher to make a layup, you do things like box jumps.  If your really daring, leverage your box jump into jumping over a 3-4′ fence (or hurdle) from a standing position.  Raw strength on the one hand and sprint speed on the other don’t give you the actual agility, coordination, and explosive power you’d need.

Similarly, coordination of multiple teams is more complex than strengthening, quickening, or improving communication with each team individually.  Establishing a cadence of synchronization and opportunities for cross-pollination of ideas – even as you increase the independence of each team  

This extended metaphor has a great caveat – go find a 3’ tall picket fence, stand in front of it, and try to jump over.  Assuming you haven’t been practicing, I bet your body stops you.  The same is true of a healthy DevOps system – when you try to launch into a painful bout of stupidity, the developers stop you.  If you’re smart, you don’t force yourself into a giant failure.  Instead, you practice a bit and ramp up your agility to get your entire body confident you won’t end up in the hospital.

Throwing a Punch

You don’t throw a punch with your arm.  You throw it from the ground up, leveraging the perfect twisting launch of one square inch of fist powered by your entire body.  If you’re the square inch that gets to land the winning knock-out blow, don’t get cocky.  You’d be nothing without the support and power of the entire body.

5 Simple Rules for Real Agility 

There is a very wide spectrum from the mythical evil-waterfall-monolith to a perfectly lean and utterly agile continuous delivery of value. In fact, scratch that, it’s not really a spectrum. It’s not even a consultant’s two-by-two matrix. It’s a complex multivariate systems-view that is needed, and fighting religiously for absolutes doesn’t help improve any of the real trials and tribulations on the front lines.
Come on – Don’t try to enforce one-size-fits-all agile or lean or kanban. Instead, create your own adaptive center of excellence from any approaches & tactics, new or old that can provide insights into the variety of techniques that can be adopted for each situation.  

For example, a really well-formed support team could move to kanban and get rid of most of the scrum ceremonies. Likewise, not every multi-scrum-team product needs a cadence of Program Increment planning – but (be honest) some do.

The bigger the organization, the more important it is to be practical in your approach to change you’re passionate about. Self-check: Are you asking for a change that makes the organization easier to work in, better at serving the customer, or more sustainable as it grows? Great! Don’t let fundamentalism get in the way of the spirit of agility and the purpose of your organization.

….Especially if you are an enterprise that is “mid-transformation”.

In fact, rather than worrying about drastically changing to a new and elaborate process with all the answers (which will definitely FAIL), there are just a few underlying principles that will help evolve an enterprise toward lean agility:

1 – Each time you make a business decision, make one that makes it faster to validate your assumption than if you hadn’t made a decision at all.

Anytime you make a decision, make it easier to recover from it as an organization if you are wrong than if you had done nothing at all.

2 – Challenge the biggest assumption first, working to disprove one hypothesis at a time

3 – Each time you touch code, leave it simpler to understand and easier to change than if you hadn’t coded at all

4 – Create an environment of psychological safety for your people

5 – Do no harm to the user

The DevOps Secret that IT Won’t Tell You

DevOps. SAFe. Scrum. Kanban. Did you shiver? The experts have you scared you’re doing it wrong (you know, IT – all of IT). The certificate-teachers need you to need them so they can get paid. The coaches and consultants are taxing your #FOMO (Fear Of Missing Out) on ALL THE AGILE.

Get. Real.

It’s an impossible dream: The perfectly lean and agile software process would be just one person – a masterful engineer with great people skills and business acumen who can hang around a group of people to see what troubles them, create an intuitive software tool that relieves the pain, and collect cash from them while obtaining ultra-constructive feedback. In other words “dev” and “ops” as one person.

Ever seen that happen?

That said, every additional person involved in the flow of DevOps information is either a value-addition or a source of systemic waste. More often, everyone involved is a bit of both. As more bodies are thrown at a problem, it is likely any one of them can only spend 10-20% of their time actually adding value. The other 80% – and all overtime – is typically waste due to waiting on information from others, or over-documenting our value-add so others can consume it.

So it isn’t that IT won’t tell you the secret, if they know at all. It’s easy to crack jokes about how little other people accomplish despite working long hours, but less likely that anyone raises their hand and volunteers the truth about how much time is wasted.

– Waiting for clarification

– Waiting for updates to install

– Waiting for SVN code check-in

– Waiting for someone else to check their code, setup a VM, or answer what their code means

– Waiting for the next meeting

– Waiting for the inspiration to work on something meaningless to you

The great big DevOps secret is that no methodology will fix all that waiting. Over-specialization creates a world in which teams throw big piles of shit over the fence, each group speaking it’s own language that no other group understands. New philosophies won’t fix bad relationships, a toxic culture, and shit tools. It takes time, work, and fearless leadership.

Otherwise, Agile, Scrum, Kanban, and DevOps – or any other recipe for “when to meet” and “what to document” – are like 30-day fad diets; it’s a false hope and a gimmick and it doesn’t change the sedentary fast-food lifestyle that keeps you fat in the first place.

The Virtue of Collaborativeness

This is part of a series on the key virtues of excellent Product Ownership.
The Virtue: Collaborativeness

Collaboration is, by definition, the ability to work with someone to produce or create something. This excellence, as I like to describe it to people, is the ability to “delight and inspire” a mix of engineers and artists to empathize with the end user and work together to address their pain. This also requires trusting others to do their job well to the extent of their expertise.  

If you slip into a mode where you have too little active participation in building a common understanding with your team, the outcome of their work and the pace at which it is produced will suffer. At the other end of the spectrum, micro-managing and looking over shoulders sends a clear signal of distrust. You might be able to have a nice relationship-building moment sitting with a developer and showing you care what the code does that she is staring at – just don’t overstay your welcome. You have a backlog to manage.  

Without getting overly disruptive of other people’s work, be sure to get feedback. Don’t get in people’s faces, but when you are getting coffee and one of your developers comes into the kitchen, it doesn’t hurt to genuinely care about things like: “Do you think I should be sizing any of the stories differently?” Or “Is there anything I can do to make your life easier today?” Somewhere between ignoring people or annoying them lies caring about what you produce together.

What this really requires of you is genuinely caring about the customer, the end user, and the people building the product for you. Don’t care about your product. Genuinely care about whether your developer knows who’s pain is being alleviated and why. If you don’t care, it shows. Why would anyone follow you or stay engaged in their work if you don’t believe in it? Sort that out, then find a balance that builds a great dynamic with the team.

Scrum Backlog Zombies!

Product Owners – Do you really want an algorithm to take all the thought out of backlog prioritization? You do?


You’re fired.

Pack up your things and request an immediate transfer to a different department. It’s over. You’re a zombie worker. Purposeful prioritization is your one wildly important purpose in the role!

You MUST prioritize the inter-party flow of information between stakeholders, customers, the development team, and your ever-storied users.

You MUST prioritize the flow of disciplined experimentation as the only scientist who can connect “innovative solution” supply with “ever-evolving pain-driven-need” demand.

You MUST prioritize the flow of economic value-add with enough vision and discipline and multi-partisan politics that the agile train doesn’t come off the rails.

Oh by the way, you’re definitely not fired.

What I meant to say before was, you’ll need to light a fire under your ass because you’re insanely important. Product Ownership may not matter to SOME people, but it’s VERY important to the REST of US.

You know, if you are a Product Owner in a large company, I get it if you didn’t ask for all this power that no one fully explained to you. In fact, I’m sorry. It is serious malarky that you’ve been empowered SO MUCH but nobody will tell you until after you fail. That’s because you’re the single point of failure in a system they didn’t want, and they definitely don’t understand what to do with. They don’t know your great power or your great responsibility, how can they possibly help you?

Your boss probably hates agile.

Maybe you hate agile.

But hey, here you are, a bridge between two broken systems, prepared to be blamed as the “single neck to choke” because a ScrumMaster said so.

I’m here for you, and I’m not giving up on you. Whatever tips, tricks, or advice you may need, just reach out.

(Image credit Walking Dead)

Entrepreneurial Product Ownership

As a Product Owner, is “entrepreneurial” or “innovative” in your job description?  Yeah, I thought so. But what does that even mean?

As I’ve mentioned before, the ability to ensure work is meaningful to the user-in-pain and the customer paying to resolve that pain (occasionally but not always the same person) is super important to the health and motivation of the engineers. The shorter the time between empathy and resolution, the more meaningful the work, the more likely the direct correlation with revenue, and the safer the jobs and projects of the engineers become.

The big “evil” business “side” in a large company doesn’t need any of that in order to maintain risk aversion and product agility. Cancelling a three year project before a customer saw it because a better project was two years in WAS their agility.

Congratulations on your agile transformation, but when you said “agile” you really meant lean concepts like “small batching” and “level-loading” and “systems thinking”. As a Product Owner, you must obtain a clear understanding around lean startup principles. That may shock you. Alas, the moment your team became an agile team, you became a new venture.

At scale, as a Product Owner overseeing delivery of your backlog, it is of tantamount necessity that you strengthen your relationship with your Product Manager. To the extent the Product Manager handles budget approval from the C-Suite on your behalf and builds up buy-in from your top customers, treat them like a venture capitalist with many connections in the VC/Angel community – and this is your first company. The expected return is 10X due to the extra risk. Many of them are having to let go against their will and trust that you know what you’re doing.

Within this context that you bear on your shoulders, the burden-of-proof that the budget can be trusted to the agile engines of production, you now have to proactively validate all learning about three things:

Product risk – The right thing was built to solve the pain of a large (enough) but narrowly (enough) focused group.

Market risk – The ability to build and continuously improve the viability of your product’s business model.

Customer risk – The ability to get the solution into the hands of the customer.

You are the single point of failure for information flow about your product backlog.

Project risk, as you once knew it, is gone. In fact, waterfall was the good ol’ days for the free riders and career passive observers among us. That was part of a bubble in the economy. SO long-lived that it caused no pain to investors, so you never heard it called a bubble. But the era of software-in-itself as a competitive advantage is gone. The fat shall be trimmed.

So no, you don’t have a deadline to track against but you do have the angel investments from your Product Manager(s). Also, you don’t really have “technical risk” anymore right? Come on, there isn’t much that you can’t build because most of us aren’t so imaginative that our requirements are science fiction. The risk that your estimates were wrong is so tertiary that some groups do away with it entirely.

No no no, YOU, my Product Owner friend, risk the loss of intrapreneurial “investor” confidence… the confidence that you know what you’re doing, mostly. Can good forecasting and transparency help? Sometimes. Unfortunately, some of us get promoted from T-Ball to the Majors with no notice that it even happened. We need a pocket-MBA and a time machine to make the new demands reasonable.

What can you do?

OWN your own destiny. The path to autonomy, mastery, and purpose is a purposeful pursuit of excellence in your craft.  If you don’t love agile, you may start to. If your boss hates agile, you’ll win her over. But it’s on you to make it happen. 

There are three “steps” to excellence in Product Ownership. Repeating these steps will take your abilities to the next level.

Step One: Establish Communication and Discovery Ceremonies

Agile transformations seem to fizzle at your level (the Product Owner). That’s because the Scrum ceremonies of the builders “under” you are well-established and grow from the ground up. They have become so commonplace that refusing the development team’s desire for a Sprint Retro seems just silly. The ceremonies that keep the team moving are established – the ceremonies you hold with your stakeholders, not so much. Without knowing the stakeholders and the meetings you already have with them, giving specifics here is hard. But – for whatever problem you’re having, looking outside your company. There is a vendor in the services industry that overcomes your challenge as a full-time business model. Prioritizing a longer term roadmap, creative brainstorming sessions, Lean UX workshops, Lean StartUp coaching – large companies have had the problems you’re facing for a long time. Look for a consultant, google what they offer, read blog posts, attend MeetUps. They are excited to talk about what they do. They know that they have to “out teach the competition” to be seen as the expert. Reach out to me for specifics. At the end of the day, borrow those workshop tactics and meeting agendas, get buy-in from each stakeholder, then establish a cadence for those activities.

Step Two: Pursue Cross-Functional Team(membership)

Never stop learning. Create a plan. Get a mentor or friend to share it with and if possible include it in your check-ins with your manager. As a Product Owner, you are the conduit between nearly every other function. Because the “development side” has begun fighting toward cross-functional teams, it’s critical you develop cross-functional communication skills. A simple way to get started is to ask people what they think works best about their jobs. An even more robust strategy is to list out all the job titles of the people you must successfully communicate with, check online for a job description, pick a skill or requirement and prioritize a backlog skills to learn. Should you master photoshop? No. But a few YouTube tutorials as you play with GimpShop (a free alternative) will be a real eye-opener. Remember this isn’t about becoming a jack-of-all-trades, this about mastery and excellence in your product innovation and delivery ownership role. Sharpening your ability to translate between specialists and clarify your intentions about prioritized effort IS your specialty.

Step Three: Define The Art and Science of Prioritization

I’m writing about this quite a bit else where. Check my blog posts, follow any links I gave and come up with your approach. If you need more specific answers, reach out to me.

Defect Prioritization

Defect Prioritization: Everything you ever wanted to know but were too afraid to admit that you needed to ask.

One of the biggest agile religious debates that seems to get people up in arms is backlog prioritization when planning has to balance known defects (especially in production) against new feature work.  Let’s dig in and find a sensible approach here.

First of all, realize that it isn’t a fun situation for developers: fixing a defect older than two weeks, even if you wrote the code yourself, is like looking at someone else’s work.  Especially if it is the cause of a problem, it hurts inside to look at it.  You wish you could re-write the whole thing because you’ve grown and learned and you’re a better developer than you were then!  I get it.  Naturally, it sucks that you can’t do that because you’d end up down a refactoring rabbit hole due to all the other code that depends on your code.  So you end up feeling like you’re adding duct tape to a hole in the hoover dam.  If you’re fixing a bug in code you’ve never seen before, its like your Product Owner told you to fix a hole in the Hoover dam, but said it in a language you don’t speak, while handing you a box of duct tape and shoving you in the opposite direction of where you need to go.  Support engineers – if they actually like what they do – are a very special breed and should be your best friend.  Get them a Snickers bar and a thank you card sometime.

Second, the “triage” work for identifying the importance of bugs (if that happens at all) in which a manual QA tester writes up a ticket and picks a “Criticality” level is a joke.  Even if you had an elaborate definition for each one, who cares?  A crash that impacts 70% of users is “critical” why?  The defect is critical to… what?  To who?  How many people?  How much money?

Now, the lazy moral high ground of newly trained agilists is to insist you should’ve never let the bugs out at all.  Six Sigma Quality baby!!!  That’s a nice thought, and a very valuable standard if you are starting a completely new project on the latest and greatest stacks.  You know, an iOS 9+ iPhone 6s or later mobile application from scratch.  Then, I do advise you to build less than you think you should, think harder about whether or not each feature is actually important, and ensure that no defect gets into the App Store. 

That isn’t most software and that isn’t the problem established software companies are grappling with while in the middle of an agile-at-scale transformation.  If are a Product Owner for 10% of an application older than three years, you definitely have defects and you definitely need a rule of thumb for what to do about them.  It isn’t your fault, but it is your responsibility.  Operating systems evolved underneath you.  Hardware was replaced.  Vendors changed.  SDKs stopped getting updated.  People changed.  Deprecations occurred.  Now you have a list 1,000 decisions to make.  In that scenario, the moral high ground “you shouldn’t have made any defects!” is lazy and unhelpful.  That’s not the reality and it provides no answer for what to do once you already have defects in production.  There’s really four approaches to consider.

1 – All About the Money:

On the one hand, calculating the ROI of every User Story then attempting to apply the same methods to your production or other leftover defects will require a pretty rigorous approach to finance, accounting, and statistics.  A simple example – if a LinkedIn share crashes every 100th time I cross-post to Twitter, what is the ROI of fixing it?  I’m not a paying customer nor is Twitter.  Should I just leave the crash and hope people don’t complain too much?  No, I don’t think anyone would suggest that.  That said, there definitely is a statistical algorithm for whether or not that crash is likely to impact my decision to become a Premium Member in the future.  But if the crash takes 11hrs to fix, test, and deploy while the data mining, analysis, etc takes 60hrs to gain a certainty of 75% – why on earth would anyone not fix the defect and move on?  Eric Reis popularized the saying “Metrics are people too” while Ash Maurya adapts this to say “Metrics are people first.”  That is to say, if you have crash count of 13, not a very actionable metric.  If you have a percentage of engaged users experiencing a crash in version 1.9.3 – you have an actionable metric, but that metric represents real people who are annoyed in real life about that crash!  Quantitative data needs to drive qualitative insight, not ever-more-complicated quantitative analysis.

On the other hand, there are important occasions when the money makes a difference.  If you have customers with a Service License Agreement or paid SAAS users threatening to leave or your actionable product metrics are moving in a scary direction on account of a defect or the perception of poor performance, the money should be the incentive you need to prioritize fixes over any new feature.  Once a customer is gone they are incredibly unlikely to come back.

2 – Actionable Product Metric: Oops

Oops!  We stopped talking about money and started talking about product metrics!  There is a good reason for that – if you prioritize the development effort that improves Acquisition, Activation, Retention, Referral & Revenue (AARRR!) then you are by default increasing the money.  ROI is not even the money question to solve, is it?  If you have a fixed team contributing to the revenue of a product, ROI variability or Gross Margin variability is what you actually want to track – as long as the costs per month to maintain and improve my product is outpaced by the growth of revenue from paying customers, the ROI is there and the Margins are there and everyone is happy!  The problem is that revenue, ROI, and Gross Margin are extremely lagging indicators of success.  They are a good indication of the stability of the performance of a company over time, which gives investors the confidence they need to keep the money there, but multi-year lagging indicators are very poor metrics for the decision-making of the teams that managing, maintaining, and improving the product.  Growth of total users or active users can be a great indication of possible network effects long-term.  Both of these long-term performance indicators are symptoms of competitive advantage.  NOT the cause.

Now we have the cart before the horse though.  If you have legacy production defects in your backlog and want to move to cohort-based split-testing, your defects are the NOISE in your SOUND.  In the example crash above, if you knew cross-posting to Twitter was an important proxy metric for Referral, the possibility of a crash is also the possibility that I don’t try to engage a second time.  If that defect existed before you began using cohort analysis and split-testing, your viral coefficient is already distorted.  So if your agile release train is making an exciting and important stop at the actionable metrics station, make sure to prioritize any defects that could distort the reliability of your pirate metrics and future experimentation.

3 – Fuzzy-Weighted Economic Value Algorithm: 

A core concept for how the backlog should be prioritize in a lean-agile environment is maximizing the flow of value-add and minimizing waste.  If you are continuously deploying, the scope of feature releases can take a back seat to actually committing to the long-term awesomeness of the product.  Of course, as Ash Maurya says, the product isn’t the product, the product is the sustainable business model – and that’s not a fixed-duration project, that’s a commitment to continuous improvement of your unique value proposition.  So in good lean-agile, at any given Program Increment planning session, you do not need to be certain that your ROI calculations are perfect or your developers will be fully allocated or your that your schedule is on track.  You have a fixed release cadence, only scope may vary.  You have a large backlog, selecting the best possible thing to build matters.  As we said above, if your revenue growth outpaces your cost growth, the ROI takes care of itself.

The Weighted Shortest Job First approach is very useful for exactly this.  Because everything else is a lagging indication of good decisions, the Relative Cost of Delay and the Relative Job Size are the most important factors in job sequencing.  Let me reiterate.  Sequence = prioritization.  Under the assumption that small legacy defects require small individual effort while large value-add features take multiple sprints to roll out, you’ll likely always fix your bugs FIRST.  Which is good.  No one likes a crappy product, no matter how much “SOCIAL!!” you add to it.  Burn your customers long enough and they will abandon you.  Every software product is replaceable if you make the pain of use greater than the pain of switching to an alternative.

Over time, three things will happen.  You’ll get to a point where the outstanding defects will require large-scale refactoring effort while your stakeholder (hopefully) get wise to the fact that a smaller improvement with more certain economic value add is more likely to get prioritized.  At that point, you have flow and hopefully rational planning and discussion will rule the day on deciding what to do next.  On that note….

4 – Politically-Intelligent Fuzzy-Weighted Economic Value Algorithm: 

If you aren’t entirely lean-agile, aka you are still mid-transformation, aka “the top” still works using their old plan-driven paradigm while somewhere down the line an agile-savvy person tries to smooth the flow of that work, you need to add something for portfolio-level politics that impact your program-level prioritization.  In this case, while you may not share it too publicly, adding more scores for which stakeholder you are pleasing should be considered.  You can then weight each of the relative scores, like proxy-voting for your stakeholders.

Yes, that means the old-school problems will be continued because you are giving the important people at the top some blank-check preferred stock when it comes to your backlog prioritization.   The unfortunate alternative is that you live in silly denial that their perception matters or that backlog prioritization is not a political question as much as an economic value you question and the people “at the top” or “in the business” continue to hate agile and second-guess every decision you make.  Concessions to a powerful VP today help earn you the trust you need in order to move prioritization to a more rational approach later.  Admit where you are, challenge it bit-by-bit, and work to improve it.

What does that look like in practice?  Hopefully there is an IT leader at the top too in your large-scale silo-heavy organization.  Hopefully that IT person or a Quality person up “at the top” can be one of the political variables in your weighting approach.  Don’t pit the CEO against the CTO as a Product Owner, that just makes you look like a chump who can’t make difficult decisions yourself.  Gain buy-in and provide enough visibility before planning sessions that no one gets blindsided by your decision to prioritize refactoring over that VP’s screams for “SOCIAL!” 

To reiterate – THE BACKLOG IS A POLITICAL ARTIFACT. 

Your solution is only partially economic or financial or social.  This is not a democracy.  Don’t go asking for votes.  It really isn’t a democratic republic, either.  And it is definitely isn’t just user story meritocracy.  Slapping a relative business value tag and sorting is begging for failure and distrust at your methods.  It looks lazy and it is.  You have to influence the right people to make the right decisions and that takes work.  If you’re lucky, its close to a full-time job and you have job security now.  Congratulations.  But seriously go fix those bugs.  They’re lame.

The Virtue of Scientific Discipline

This is part of a series on the key virtues of excellent Product Ownership.

The Virtue: Scientific Discipline 

I admit, this one combines a few qualities that make someone’s approach to problem solving incredible. The excellence of scientific product innovation requires the ability to identify and challenge assumptions in isolation so that purposeful experimentation and learning can occur. To succeed at disrupting a market, there is a permanent restlessness and curiosity, a willingness to challenge the status quo. The discipline part requires an understanding of what to measure, how to measure it, and the patience to consistently track and communicate your findings.

The vice of deficiency here is a total lack of experimentation. Say “no” to 99% of requests that don’t fit your vision or serve the longer term goal, absolutely; but if you say “no” to 100% of ideas that come your way, focus on locking things down and playing it safe, you’ll destroy any chance of innovation and your customers will abandon you. As Product Owner, it’s assumed that you are controlling a continuous stream of changes. Passively following an original plan or passively fulfilling stakeholder requests is a lack of experimentation of your own. If you don’t insist on tracking the results of their ideas, you’re also just plain unscientific. Empirical process control (Scrum) implies learning through purposeful experimentation. As the Product Owner, your laboratory is the product-market fit and your target result is the sustainable viability of your (Product) business model.

Of course, the vice of excess here is uncontrolled chaos. Drastic changes without rhyme or reason do not lead to adaptation. Evolution takes the discipline to kill tiny bad ideas as they fail and build on the successes. In Scrum this is an unfortunate use of “super-epics” like “LETS DO SOCIAL!!” Adding 7 social networks at once to a product is the undisciplined throwing-spaghetti-at-the-wall chaos I’m talking about here, and it is inevitable when you separate market planning from development planning. As soon as the developer don’t seem overwhelmed by the size of the backlog, stakeholder start feature-stuffing so they don’t feel like agile means “paying people to sit around”.

So the virtue in the middle that is absolutely essential as a Product Owner – whether you are an entrepreneur with a 5-person Scrum team or you are in the middle of a 5,000-strong software-at-scale enterprise – is scientific discipline feast or famine, insist on controlled experimentation not chaos. Insist on purposeful validation of each hypothesis, not hand-wavey assumptions. Insist on actionable metrics and clear articulation and confirmation on whether the product is progressing correctly.

Because you are the single point of failure for information flow, only you can courageously take a stand before things are out of hand. Stay disciplined and rational, but enjoy creative experimentation.

The Virtue of Sincere Honesty

This is part of a series on the virtues of excellent Product Ownership.

The Virtue: Sincere Honesty

Sincerity – the quality of being free from pretense, deceit, or hypocrisy.

As a Product Owner, you are the conduit of an immense amount of information. You are the tip of the spear for what gets built next, which requires you to establish – with your stakeholders, peers, customers, users, and cross-functional development team members – that you can be trusted. In most organizations that I have spoken with, the Product Owner is the single point of failure for information symmetry. Does the Product Manager know what the ScrumMaster knows? Does the UX designer know what the Social Listening Analyst knows? Does the development team know what the customer knows? The duty to prioritize the backlog is derivative. The core purpose of Product Ownership is to prioritize the flow of information.

This isn’t about keeping stories straight or “transparency” and it definitely has (almost) nothing to do with deadlines. Your Product Manager or equivalent stakeholder must trust that you prioritized the right things to build or you’ll lose their “investor confidence.” The customer must trust that you are giving them their money’s worth. The developer must trust that you know the user and their needs and have told them the right thing to build.

So this is about the visibility of information, and your role in promoting the most important insights for each party.

The vice of deficiency is not “lack of documentation” any more than bad product performance is due to lack of velocity. There are no documentation or utilization problems, only communication and engagement problems. So the real deficiency is lack of effective communication of ideas. When you are apathetic but your words scream “ASAP!” that’s hypocrisy and people know it.

The vice of excessive “honesty” is what most people do in the name of transparency: Over-supply of data with no context or prioritization. Just like the overwhelming number of things that you could build make prioritization the cornerstone of managing the backlog, the overwhelming amount of information that could flow through you requires prioritization of what you communicate.

The virtue, in between data-forwarding without context and insincere platitudes of precision and urgency, is sincere honesty – providing the information that you believe is most important to the success of everyone involved. Whatever tool or process you need, be authentic, be genuine, and do your best.