The DevOps Secret that IT Won’t Tell You

DevOps. SAFe. Scrum. Kanban. Did you shiver? The experts have you scared you’re doing it wrong (you know, IT – all of IT). The certificate-teachers need you to need them so they can get paid. The coaches and consultants are taxing your #FOMO (Fear Of Missing Out) on ALL THE AGILE.

Get. Real.

It’s an impossible dream: The perfectly lean and agile software process would be just one person – a masterful engineer with great people skills and business acumen who can hang around a group of people to see what troubles them, create an intuitive software tool that relieves the pain, and collect cash from them while obtaining ultra-constructive feedback. In other words “dev” and “ops” as one person.

Ever seen that happen?

That said, every additional person involved in the flow of DevOps information is either a value-addition or a source of systemic waste. More often, everyone involved is a bit of both. As more bodies are thrown at a problem, it is likely any one of them can only spend 10-20% of their time actually adding value. The other 80% – and all overtime – is typically waste due to waiting on information from others, or over-documenting our value-add so others can consume it.

So it isn’t that IT won’t tell you the secret, if they know at all. It’s easy to crack jokes about how little other people accomplish despite working long hours, but less likely that anyone raises their hand and volunteers the truth about how much time is wasted.

– Waiting for clarification

– Waiting for updates to install

– Waiting for SVN code check-in

– Waiting for someone else to check their code, setup a VM, or answer what their code means

– Waiting for the next meeting

– Waiting for the inspiration to work on something meaningless to you

The great big DevOps secret is that no methodology will fix all that waiting. Over-specialization creates a world in which teams throw big piles of shit over the fence, each group speaking it’s own language that no other group understands. New philosophies won’t fix bad relationships, a toxic culture, and shit tools. It takes time, work, and fearless leadership.

Otherwise, Agile, Scrum, Kanban, and DevOps – or any other recipe for “when to meet” and “what to document” – are like 30-day fad diets; it’s a false hope and a gimmick and it doesn’t change the sedentary fast-food lifestyle that keeps you fat in the first place.

Agile Priorities: Keeping Documentation Lean

Agile Transformation

If you want your agile transformation to fail, allow the development team and all supporting functional teams to focus on the waterfall deliverables and dependencies to which they are accustomed.

In “Fixed Budget, Fixed Scope? Be Agile Anyway.”  I argued that agilists, especially newcomers, must understand that the Agile Manifesto is a statement of priorities – at times, tools and interactions, comprehensive documentation, contract negotiation, and following a plan are all part of the product that is created by the process that keeps your teams in business.

In software development, this is the exception rather than the rule.  

For “core” Scrum, the proverbial “one-team-in-a-garage”, this is pretty simple.  Everyone you need is in one room.  When scaled, a delivery organization is often “compound-complex” – it is “compound” when two or more independent functional knowledge areas must cooperate and “complex” when at least one knowledge area has dependency on another (these aren’t fancy concepts, just basic grammar).  The moment a delivery process moves beyond one-team-in-a-garage Scrum, there is compound-complex value stream:

The web team and mobile team will build a user-facing experience, while the middle tier team provides RESTful services, only after sales executes the contract and the Product Owner builds the backlog; meanwhile, UX, BA’s, and QA’s will provide documentation.

In this value stream, the “Wildly Important Thing” is the software that a user will consume in order to drive revenue.

Because Communities of Practice have their own traditions and methods of proving extrinsic value, shifting their priorities is much more difficult at scale.  Plan to change the physical distribution in order to overcome the functional “clumping” that Waterfall encouraged.  Moreover, agile transformation will require finding and eliminating the agency dilemma of managers who built their kingdoms around waterfall values.

Resolving Wasted Effort

If agile at scale has not improved company-wide quality and velocity, it is likely due to waste in the overarching process – you must remove any activity that does not drive value creation that a user consumes and the business will capture.  This is the central rule of the User Story – if it does not make a difference to the user (and occasionally the owner) it is wasted energy, a distraction.  You find this clearly stated, often, in Jeff Sutherland‘s various descriptions of “Bad Scrum” – without delivering working software that matters to the user an organization will be stuck in a self-reinforcing cycle of inefficiency and ineffectiveness.  No one is making a difference in the world, selfishness increases, velocity never improves, no software is delivered.

For a compound-complex process to stay agile, the Lean principles of Just-in-Time and Just-Enough to must be applied to all forms of documentation – business rules, persona descriptions, wireframes, UI designs, sliced assets,  value mapping, technical constraints, market conditions – all are a means to pleasing the user, not the end goal (software that a user values).  The User Story is the cornerstone of efficient, effective product delivery because it is a “reminder of a conversation” about a result from which a user will derive value.  For some teams, a set of wireframes can all but replace “cards”.  An image of a login screen is sufficient to drive the conversations a team needs to have about client-side validation, security requirements, and authentication method.

The Product is Everyone’s Deliverable

To move to next-level agile, at scale, an organization will need to blur its understanding of “business planning”, “requirements documentation”, and “UX Design”.  While there may be a need for full, distinct teams of experts at each of these disciplines, these approaches to defining what the software will be should converge on a single goal – building great software.  A popular article on Lean for UX, Lean UX: Getting Out Of The Deliverables Business leads with this statement:

Wireframes, site maps, flow diagrams, content inventories, taxonomies, mockups […] crystallized the value that the UX discipline brought to an organization. Over time, though, this deliverables-heavy process has put UX designers in the deliverables business — measured and compensated for the depth and breadth of their deliverables instead of the quality and success of the experiences they design. Designers have become documentation subject matter experts, known for the quality of the documents they create instead of the end-state experiences being designed and developed.

Having started my career as a Business Analyst, this is painfully resonant.  It is unfortunately easy for management move from training “the tools of the trade” – spreadsheets, slide decks, requirements documentation, technical specifications – to judging the tools independent of the software to be delivered.

Here are a few rules to follow to get UX, UI, and BA’s out of the deliverable business and into the well-formed team:

  1. Never say “final sign-off” regarding artifacts – whether working for internal stakeholders or external clients, if software is the deliverable, it is the only item requiring sign-off.  Every other artifact is is a tool for getting working software in the hands of users.  Make sure you are “responding to change” at the request of real users of working software – no amount of planning or documentation will survive contact with the user.
  2. UX/UI Refinement should mirror Backlog Decomposition – Prioritizing working software does not mean there is no product roadmap, it does mean that time and resources should only be invested in the software most likely to be built in the near future.  This starts as a process diagram of all possible Epics that could be built, and how the user gets to each of these features.  The upcoming 2 to 3 features should have relatively actionable wireframes.  The team should feel certain about the UI design of the current and next sprint.
  3. Create paradigms, not screens – the most important expectation to break is the annotated UI composition per screen.  Some screens absolutely benefit, many do not.  A well-formed team should have enough shared understanding of technical constraints, design paradigm, and business context that a conversation is sufficient to drive the whiteboard-to-working software process.  Wherever possible, establish a paradigm for a product is intuitive to build.  If it isn’t intuitive to the developer without extensive documentation, how likely is is it the user will find it intuitive?
  4. Document releases proactively, but in accordance with your level of certainty – If there is a product that will require knowledge hand-off, a support guide, or other formal documentation, maintain the tension between what makes sense to proactively document (so you don’t spend a full week post-release) while wary of what will change (likely anything more than a sprint in the future).

Following these rules, the software development process in an at-scale, compound-complex organization can focus on the user enjoying a working product.  Aligning vision and expectations along the agile priorities will remove fear of failure or criticism, increase team velocity, and result in the best possible products.  Then, your team, business, and enterprise can pursue kaizen – and the real magic can happen.

Fixed Budget, Fixed Scope? Be Agile Anyway.

Innovation is Sexy:

Scrum is typically taught with its most innovation-focused form as the example, born from fast-paced software companies who disrupt markets and kill categories.  This is the sexy version of Scrum: a highly elastic process for delivering complex products with emergent design.  In the real world this is not the only context in which Agile, Scrum, Lean, and XP principles can and are utilized to increase quality and velocity.

If you are a newly-trained Scrum professional, bright-eyed and full of hope, do not get discouraged by convergent, fixed-budget, fixed-scope, fixed-timeline projects – be agile anyway!  This is how to do it.


The Agile Manifesto:

It is important to remember that the Agile Manifesto is a set of priorities, not a set of laws. To any extent we can, we strive to prioritize in the direction of team collaboration, continuous improvement, and emergent value creation.  However, there are some occasions when you will have to put additional energy in the secondary priority.  Strategy is the art of trade-offs, so make sure your organization is aligned and the strategic vision is deliberate:

  1. Individuals and interactions over processes and tools – Never lose sight of prioritizing the people who huddle around your solution, never stop collaborating.  Encourage face time and green time.  However, there are highly regulated industries with static expectations, accounting control benchmarks, FDA guidelines, or stable dividends paid to investors for decades.  A well-documented, repeatable process for how software, hardware, or other product will be delivered over the course of a long-term project will assist tremendously with stakeholder confidence.  Make priorities clear on all sides of the contract, set the expectation that communication and availability are mission-critical, but understand that there will be little risk tolerance for out-of-control processes.  Maintain your agility by keeping your document alive and lean, maintain elasticity, but make sure the tools used and the processes adopted, and the pace of process change, gives your fixed-scope, fixed-budget project a stable context for predicting delivery and ROI.
  2. Working software over comprehensive documentation – Delivering builds early and often is the best way to be truly certain of product status.  In emergent product design with incremental feature addition this feels natural.  There is an MVP planned so an opportunity to pivot is assumed, the feedback loop on the progress of high-quality incremental delivery is the best way for stakeholders and users to see and feel the value that is emerging.  When working with waterfall-minded stakeholders – especially if they are an external client – fixed scope, fixed budget, and fixed release dates cannot always be avoided.  However, convergent delivery does not preclude the XP and Scrum best practices of well-formed agile teams and an openness to documentation-heavy products will drive velocity and success waterfall organizations have never seen.  Instead of negative reactions to the expectation of comprehensive documentation, approach documentation as a secondary product and maintain the same expectations of prioritization, collaboration, and responsiveness.  Moreover, a convergent, finite project likely came to you with extensive documentation.  This documentation will require some additional work, but it should not be ignored.  Even though this is a source of waste that is removed during agile transformation and lean consulting in “sexy” emergent product delivery, handling waterfall requirements as documented stakeholder demands can be incredibly helpful in convergent product delivery.  The Product Owner still must aggregate, consolidate, and complete backlog decomposition with the team.  The XP user story and a prioritized backlog should still set the delivery cadence.  Be open to documentation and transparent to stakeholders about its actual ROI after it is delivered.
  3. Customer collaboration over contract negotiation – Good fences make good neighbors.  Some large companies simply cannot risk an open-ended and loosely-defined relationship, especially with vendors or contractors.  If you’re competitive strategy relies on these types of contracts, inspect and adapt.  Find how negotiation can be expedited, build in the process of collaborative relationship change, and build well-formed teams that are as adept at ramp-up as they are at innovation.  The capacity of a Scrum team to quickly assimilate a knew business and architectural context will drive success in products and relationships of all types.  Answering the internal needs of various business functions or developing solutions in new industries requires a team aptitude for understanding new contexts effectively and efficiently.
  4. Responding to change over following a plan – Change is inevitable in product delivery – the demands of the market, the technological tools available, and the state of the product being developed are a state of continuous evolution.  New requirements previously unnoticed will surface in the course of a project.  Even in a mostly-waterfall project, a great Product Owner will apply XP and Lean principals to minimize waste and maximize up-front value.  Great Scrum teams will perpetually improve, increasing velocity.  We often talk about “waterfall projects” versus “scrum projects” when we really mean emergent versus convergent delivery – fixed budget, scope, and deadlines do not preclude scrum, lean, and XP practices, they constrain them to a known outcome.  The difference between delivery by a helpless group of Project Managers and Business Analysts disengaged from team performance versus delivery by a powerful Product Owner and kaizen-crazed ScrumMaster and team is not dependent on emergent product delivery!  Collaborating around the highest value backlog items up front, swarming impediments, and tracking the variance of forecasted-to-actual product-level burn down will allow fixed dates to be met appropriately.  Responsiveness to change needs to be supported by a strong relationship of trust, which must earned.  If you are in a custom software shop, responding to RFP’s and entering into fixed projects is often the inevitable first step to earning the trust that will lead to a more agile relationship, so make efforts to meet the comfort level of the customer’s stakeholders while including a healthy process for addressing requirements changes.

Conclusion:

The values, practices, and behaviors of agile, scrum, lean, and XP have wider applicability than open-ended emergent product design.  Staying open to the unique lessons that can be learned in seemingly less-than-ideal projects will be a terrific opportunity for the kaizen-driven team to grow.  If a rigid process and convergent projects have caused the shine on your Scrum mindset to dampen, step up or step out: don’t let the context defeat you, stay true to your agile values.  On the other hand, if you find yourself in a long-term no-win situation in which you cannot drive change for your team(s) and users, maybe a sexier career path is out there for you to consider.

Mike Cohn’s Opening Keynote: “Let go of Knowing”

Special thanks to Mike Cohn, who did a great job kicking off the Global Scrum Gathering in Phoenix, AZ with his opening keynote, Let go of Knowing: How Holding onto Views May Be Holding You Back.  It set the tone for the week and opened up my mind for gathering new perspectives!

Overcoming biases, challenging the assumptions we bring into conversations and new opportunities, and admitting when our beliefs are proven wrong – this is the Intellectual Humility at the core of being agile rather than doing agile.  Scrum is not a best practice in itself, it is an empirical method for finding better “next” practices.  Cohn quoted one of my favorite philosophers on the virtue of intellectual humility:

“In the case of any person whose judgment is really deserving of confidence, how has it become so? Because he has kept his mind open to criticism of his opinions and conduct. Because it has been his practice to listen to all that could be said against him; to profit by as much of it as was just, and expound to himself, and upon occasion to others, the fallacy of what was fallacious. Because he has felt, that the only way in which a human being can make some approach to knowing the whole of a subject, is by hearing what can be said about it by persons of every variety of opinion, and studying all modes in which it can be looked at by every character of mind.”

– John Stuart Mill

The rapid feedback loops we strive for in Scrum are meaningless if we are unwilling to revisit our assumptions and beliefs.  The agile frameworks and philosophies we explore and promote – with the goal of building a complex product more efficiently and effectively – are intended to create a safe-to-fail environment, not a fail-safe system.  Not only do we need to encourage this mindset in our organizations so that our teams are free to experiment, we must be the example in this and admit when our assumptions were incorrect.  Process stability allows us to effectively measure data, but process rigidity is likely to ignore metadata.  Cohn gave some great examples of ideas held by agile thought leaders that did not stand up to to the test of time:

  • Ron Jeffries has abandoned estimating completely after doing it for many years as part 

  • Ken Rueben once argued the ScrumMaster should NOT be a team member, but now believes it can work

  • Robert Martin was a complete skeptic of TDD when he first heard of it until he tried pair programming

While we joke about “Scrum, but” companies (e.g. “We do Scrum – but we only test every third sprint”).  Mike put things nicely into perspective, setting a great tone for the week:  Scrum was born through companies that were essentially “waterfall, but…” companies.  As in Art and Literature, it took a new era to name the previous era, but Agile, Scrum, XP, and Lean all evolved over time.

“In science it often happens that scientists say, ‘You know that’s a really good argument; my position is mistaken,’ and then they would actually change their minds and you never hear that old view from them again. They really do it. It doesn’t happen as often as it should, because scientists are human and change is sometimes painful. But it happens every day. I cannot recall the last time something like that happened in politics or religion.”

It was fantastic to start my week with this mindset, looking for new perspectives and challenging my beliefs along the way.  I picked up a few non-core-Scrum tactics along the way.  The only way to stay innovative and keep improving is to risk the possibility of proving yourself wrong along the way.

User Stories 101

Background:

The concept of the “User Story” originated in Extreme Programming in the late 90’s and has become the fundamental building block of product planning and delivery across various forms Agile, Scrum, and XP adopters.  Learning to write effective User Stories is an art that takes time and must adapt to the team and product for which the stories are written.


Basics:

The basic structure of the user story defines the user’s role, necessary action, and expected outcome:

“As a [role] I want to [interaction] so that [outcome].”

Here is an easy example of the traditional “post-it note” User Story:

“As an online shopper, I want to share my purchase on Facebook so that my friends see it on Timeline.”

Each of these are important to the developer:

  • Outcome – the result expected by the user is the most important element of the user story.  The focus on user outcome was the major drive in Scrum and XP to adopt the User Stories – software developers wanted to know what value for the end user was being created.  If an outcome is not testable by new user (“hallway” testing) with minimal ambiguity of expected outcome, we have failed the user!  By aligning the User Story around a valuable outcome expected by the target user, the development team (including testers) and the Product Owner can easily align around what is being built, why it is being built, and whether or not it creates value.  As highest criteria for business value creation, it is the outcome that drives the Product Owner’s prioritization of the Product Backlog.
  • Interaction – the outcome lets everyone align around the goal of what is developed, while the interaction necessarily defines how the user does it.  In new product development, while the Product Owner works with the stakeholders to define the most valuable outcomes for the target users, the Well-Formed Team works cross-functionally to define how the user creates the valued outcome.  This is where the “ideal state” of 3-minds-and-a-whiteboard ought to be able to define user experience, technical implementation, and testing approach with a few conversations.
  • Role – role can define two things quickly and easily.  1) The relationship the user has to the product (admin, owner, end user).  2)  The business context that helps define the Interaction design.  In the above example the “online shopper” in the story is loaded with important information that can be quickly assimilated:  the user has made a purchase, so they are on the last screen in the check-out workflow, they are not an admin or owner, they have likely registered in the past and are now authenticated due to the necessity of payment system security.  A wealth of business context is alluded to in a few words.  In a narrowly-focused product, this might be left off completely because the role does not change across stories.

I.N.V.E.S.T. Criteria

As I described in Incremental AND Iterative Product Delivery the effectiveness of a User Story is driven by its “nimbleness” and user focus.  Once an architectural runway and design paradigm are established, new vertical slices should be easy to add with minimal refactoring.  Vertical product slices follow the I.N.V.E.S.T. criteria:

  • Independent, end-to-end, “shippable” increments of the emergent whole.  The slice could be delivered fully tested without any other product increment and create value for the user or owner.
  • Negotiable, in planning and expectations with users and stakeholders, allowing delayed incorporation of enhancements or, if a major pivot becomes necessary, easily removed from the product roadmap (without previous over-engineering, over-planning, or over-documentation).
  • Valuable to the user or owner, likely in a way that is sufficiently noticeable that it is monetizeable.
  • Estimable by the delivery team because the User Story does not generalize or hide uncertainty.  An inestimable story is often an Epic.  It is either complex enough to warrant a technical spike or compounds enough feature work that it should be broken into independent, smaller, user stories.
  • Small enough to be estimated by a team, with certainty, and deliverable fully-tested within a single sprint.
  • Testable by virtually any team member because the expected outcome is qualitatively or quantitatively noticeable (as part of its being valueable) to the target user.

The I.N.V.E.S.T. method for user stories ensures that planning and development occur just-in-time so that the emergent product can regularly evolve in response to changing stakeholder demands.  More importantly, in the fast-paced markets in which digital product programs compete today, incremental delivery maximizes the freedom to release the Minimum Viable Product and pivot in a new direction based on real-time feedback from users.



Conversation Starter, Conversation Reminder

In Scrum and XP circles a User Story is succinctly defined as “A reminder of a conversation.”  It should be simple in focus but rich in context as a reminder for conversations that have been had and conversations that need to take place.  Note that this definition contains nothing I wrote above!  This was done intentionally as it aligns with the tenets of the agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

via agilemanifesto.org

Lengthy documentation about “business requirements” and “product specifications” created several unhealthy habits in Waterfall organizations – the “requirements” often had nothing to do with making the life of the user better, the specifications often were no longer relevant by the time an engineer needed to build the product, and the multiple decision-makers made conversations impossible!  A user story is conversation starter and a conversation reminder because it is intended to encourage interaction, collaboration, responsiveness, and tangible progress.

If you are going old-school and using actual cards or post-its, the User Story (role, action, outcome) should easily fit on the front.  This is enough for a Product Owner to prioritize the stories and remind him and the team that a conversation needs to take place.  During Sprint Planning, if interaction or technical implementation decisions are made, these can easily fit bullet-point-style on the back a reminder of what was already discussed.

This method works great at company like Spotify, where static feature teams can become subject-matter experts and the business context is understood by the cross-functional, self-contained team members.  If you are writing stories for Rapid Application Development, especially as a contractor to another firm – more details captured in a tool like JIRA may be appropriate, and “just-enough” additional documentation can be useful until the new product achieves the normal team velocity.

Looking for more insights?  Feel free to ask questions in the comments, attend a Scrum User Group Event or connect with me directly.


Connect With Me:

LinkedIn       Twitter

FREEDOM: How using Scrum un-impedes our full potential!

Scrum frees teams from the tyranny of the urgent:

A key goal of the Scrum transformation is breaking the parent-child relationship between “business” and “development”.  Technical planning, implementation, commitment, and delivery is in the hands of the team while the Product Owner provides a single priority-order backlog of consolidated items from stakeholders, users, and the developers, for the team to plan around.  Once a well-formed-team finds the right mix of XP and Lean practices (especially continuous integration and frequent releases) the pressure of “we have to do this now” is easy to manage.  If major items frequently pop up mid-week on a “release train” that deploys weekly:

  • The team leaves room in their commitment for these items
  • The Product Owner sets realistic expectations with stakeholders on which release will include the request

This is part of the power of Lean planning for making strategic product and release decisions – once the Minimum Viable Product is in the open market, data from the users can determine urgency of backlog items.  Frequently, this is a point where a pivot is desirable.  Through frequent releases, the users, stakeholders, and the team can relax and look at the big picture, free from the fear arbitrary deadlines and urgent scope “creep”.

Scrum frees teams from the fear of making mistakes:

While the extreme co-dependency that drives the tyranny of the urgent is overcome by putting an end to arbitrary deadlines and poor scope decisions, the fear of making mistakes is overcome through the key roles defined by Scrum.  Note, these are roles and this label is intentional – these are NOT by necessity positions or titles.  As I describe in Why Product Owners? a role is the confluence of empowerment, responsibility, and accountability for One Big Thing that rests on the shoulders of a single person.  Teams are free to voice ideas because the Scrum framework encourages collaborative implementation planning, cross-functional brainstorming, and constant discovery of best practices.  With XP Pair Programming, this becomes even easier as a single screen and keyboard gets two minds and four eyes that can converse on best practices, mitigate against knowledge silos, and build team relationships.  In the end, all of these adults acting brilliant and mature, huddled around a problem still have the Product Owner as the proxy to the users and stakeholders.  The team is free to determine one small solution at a time, just-in-time, and just-enough – then iterate or pivot.  When tough decisions need to be made, the Product Owner makes the call, relying on the expertise of the team, sets expectations, and maintains accountability.  Moreover, the user story – as the building block of incremental product delivery – isolates risk of mistakes to the smallest valuable user interaction and continuous integration ensures that when a new increment is not “potentially shippable” it can be easily taken out until additional iteration is completed.

Scrum frees organizations from business as usual:

When things go wrong in a large Waterfall project, multiple helpless Project Managers ask for updates but are unable to check boxes, development teams are powerless to drive the direction of the product, and stakeholders become progressively frustrated by the unfinished pit into which money is being thrown.  Then they cancel the project!  What is missing here?  JOY!  Menlo Innovation’s Richard Sheridan and author Joy, Inc. describes the importance of team joy in an interview with InfoQ:

There is in fact tangible business value to joy. But understand this: our focus is external to the organization. What we want more than anything else is that the work of our hearts, our hands, and our minds gets out into the world and delights people. That’s our definition of joy. We want somebody to stop us on the sidewalk and say, “That thing you built, I love it. Thank you. You made my life better.”

When I build a new Scrum team, guiding them with the help of my Scrum Master through the process of forming, storming, norming, this is the number one fear I must guide them through:  I say “Yes, the requirements will change.  Yes, what you build today will be modified tomorrow.  What matters is that you are a creator, people will actually use what emerges, and it will make a difference to them.”

The move from cancelled projects to meaningful user feedback has an incredible impact on an organization.  The drive to constantly improve, tighten feedback loops, and evolve through empirical process control will make disruption the norm and remove the fear of inevitable change and new ideas.

Scrum frees organizations from process rigidity:

There are three overarching goals driven by the roles, ceremonies, and artifacts in Scrum:

  1. Notice ineffective processes, workflows, or behaviors quickly and cease them.
  2. Notice effective processes, workflows, or behaviors communicate this knowledge
  3. Recommending new processes or practices that can be tested for the length of a sprint

At the organization level, the same process should occur.  While the empirical methodology monitoring “the process” – a mix of lean, XP, and other function-specific practices – is held constant, the practices that are monitored within “the process” are under constant review at the team and organization levels.

At this year’s ScrumAlliance Global Scrum Gathering in Phoenix, Arizona, I attended a terrific session  “Scrum at Scale – Free yourself from the myth of one-size-fits-all scaling led by Scrum Inc.’s Alexander Brown.  Look for an upcoming post on his argument that Scrum is by definition modular and object-oriented, such that organizations scaling Agile, Scrum, Lean, and XP can utilize empirical process control and a mix-and-match approach to the various frameworks most appropriate to the unique needs of each team.

Incremental AND Iterative Product Delivery

Highly-effective agile teams deliver user-valuable software incrementally and iteratively. What’s the difference?

Incremental Delivery:

An increment is “something added as part of a series of regular additions.” Scrum relies on incremental by employing the user story (originally promoted by Extreme Programming and written about extensively by agile champion Mike Cohn).  The user story is the fundamental building block of how sprint commitments in Scrum are managed.  Incremental delivery relies on intra-sprint-based “vertical slice” delivery, in which new features are divided up like sashimi – as thin as the skill of the chef can accomplish.  Effective vertical slicing is a best practice in Scrum and XP because incremental delivery minimizes uncertainty in any given sprint and release, throughout the development of a complex product.  Vertical slices follow the INVEST rules for user stories:

  • Independent, end-to-end, “shippable” increments of the emergent whole.  The slice could be delivered fully tested without any other product increment and create value for the user or owner.
  • Negotiable, in planning and expectations with users and stakeholders, allowing delayed incorporation of enhancements or, if a major pivot becomes necessary, easily removed from the product roadmap (without previous over-engineering, over-planning, or over-documentation).
  • Valuable to the user or owner, likely in a way that is sufficiently noticeable that it is monetizeable.
  • Estimable by the delivery team because the User Story does not generalize or hide uncertainty.  An inestimable story is often an Epic.  It is either complex enough to warrant a technical spike or compounds enough feature work that it should be broken into independent, smaller, user stories.
  • Small enough to be estimated by a team, with certainty, and deliverable fully-tested within a single sprint.
  • Testable by virtually any team member because the expected outcome is qualitatively or quantitatively noticeable (as part of its being value) to the target user.

The INVEST method for user stories ensures that planning and development occur just-in-time so that the emergent product can regularly evolve in response to changing stakeholder demands.  More importantly, in the fast-paced markets in which agile/scrum/lean/xp programs compete today, incremental delivery maximizes the freedom to release the Minimum Viable Product and pivot in a new direction based on real-time feedback from users.

Iterative Delivery:

An iteration is “a repetition of successive approximations in which progressive versions evolve to a desired level of accuracy.”  Iterative delivery relies on an openness to prioritizing value that is good-enough rather than perfect.  The perfection of a single feature or a single screen, as defined by one Stakeholder, Architect, UI Designer, or Engineer and taken out of the context of the overall emergent product leads to unnecessary delays, lack of decisiveness, and delayed time-to-market and ROI.  The Scrum team avoids working in a silo through collaboration, entrusting the Scrum Product Owner with accountability for decisions.  In contradistinction, over-analysis and churn prior to meaningful user feedback is utterly inefficient and ineffective.  Thus, while Just-in-Time vertical slices maximize the freedom to bring Lean MVP’s to market and pivot as appropriate, iterative and “just-enough” Lean delivery is equally critical to ensure that increments can evolve based on user feedback instead of non-plans made due to (unnecessary) fear of scrutiny.


Connect With Me:

LinkedIn       Twitter

Frameworks for Scaling Agile and Scrum

3Back:

I had the pleasure of attending 3Back consulting’s “Scaling Scrum with Scrum” training last week with Dan Rawsthorne.  It was an EPIC day due to the dense amount of information provided, philosophical analysis completed, and a very worthwhile way to spend eight training hours.  I also completed my Certified Scrum Product Owner training with 3Back’s Doug Shimp because, as a repeat customer, I know Doug and Dan present more than one perspective, offer their personal experiences to the mix, and drive collaborative discussion.  There is no one way to Scrum, and 3Back gives a broad yet practical introduction to the debate.

Frameworks:

The fundamental element of Scrum is the Well-Formed Team.  Philosophical Scrum, your Scrum 101, might make it easy to gloss over this building-block of Scrum – there is an immense amount of information to cover and the importance of the Well-Formed Team is quickly lost on the beginner.  However, after three years of driving Agile, Scrum, XP, and Lean in various organizations, I must reiterate:  the need to build on the Well-Formed Team as a paradigm cannot be overstated.

What does a Well-Formed Team – a “Scrum” in the truest sense, huddling around a problem to solve – look like?

  • It is Self-Organized:  No one on the team directs the work of anyone else on the team.  Everyone is self-started, self-directed, and accountable for their commitments to the Product Owner
  • It is Self-Contained:  We often call it “cross-functional” but this is insufficient.  A truly well-formed team has every resource it needs in one room, ready to solve an issue on a whiteboard together.  The members value working together as a team, work constantly to improve, take pride in their “chores” (effort not directly involved in completion of committed backlog items), take seriously the due-diligence and appropriate standard of care for the product increments that are built for the stakeholders and users, and practice this excellence with complete integrity as professionals.

This Well-Formed Team is completely self-contained – every discipline necessary for working software is in a room, sharing a proverbial pizza.  Pragmatically speaking, “classic” Scrum would have a team that looks like this:

  • 4 iOS developers
  • 1 Middle-tier/ DBA
  • 2 QA testers
  • 1 UX/UI designer

How they grow to become cross-functional is part of their self-organization:  The iOS developers will assist the Middle-Tier and QA in System Integration Testing, collaborate on Human Interface Guidelines with the UX/UI designer.  The middle tier looks to UX/UI and the iOS developers for what information needs to be delivered by a web service.  The QA testers are perfect for “hallway” testing of the UX/UI designer’s rapid prototypes.

framework is a collection of patterns– the Scrum framework is built from the pattern of the Well-Formed Team.

The second pattern consists of the Scrum Ceremonies – the feedback loop of Daily Stand Up, Planning, Review, Retrospective.  The last pattern is the role of the Product Owner as the single point of accountability to the users, stakeholders, business owner, and team for the value created by the prioritize Product Backlog (for more on this, see my post Why Product Owners?).  In traditional Scrum, the Product Owner is outside this Well-Formed Team.  50% of the Product Owner’s time is spent collaborating with the team about the backlog, 50% is spent with the Stakeholders.  Adding a Business Owner that handles most of the sales and business “chores”, this could be an effective Lean Startup!

Scaling the Patterns:

The 3Back course reviews three approaches to scaling the patterns set up by Scrum:

In the interest of time, and because I recommend learning this from 3Back, LeSS, or SAFe yourself, I will skip to my take-aways.  The number one thing to recognize one a framework scales is where accountability is shifted.  This is the biggest with the concept of the “Product” Owner in traditional Scrum – once several teams are delivering a single large product (imagine Spotify or Microsoft Office for desktop), or a single team becomes responsible for a long series of very different products (imagine a team in a typical custom software development company, delivering 4 to 7 products per year), the “ownership” becomes an expectation that must be made clear.

SAFe – Scaled Agile Framework:

SAFe takes the large organization and introduces agile in a scaled manner (in juxtaposition to scaling from small to large as a company grows).

SAFe

It divides the delivery of a large organization into the product, program, and portfolio levels.  The Agile Release Train and (presumably) the ceremonies are on a single schedule for the entire portfolio.  At each level, backlog development is done at the program and portfolio level, leaving the teams to deliver groomed and prioritized backlogs to deliver.  While the Agile Release Train teams, whether delivering a distinct Product or features as part of a large system, have an accountability owner, the key issue with SAFe is how accountability is scaled.  The pattern indicates scaled scrum teams, but those teams do not have Accountable Owners for the decisions or the teams!  You can see how, despite sprint-based delivery, this will eventually result in the Waterfall Witch-Hunt”.

LeSS – Large Scale Scrum:

Large Scale Scrum, is the methodology currently supported by ScrumAlliance.org for its new Added Qualification: Scaling Scrum Fundamentals.  The Body of Knowledge presented on the LeSS site is thorough and full of terrific information on the Well-Formed Team, organizing in the Large, and bringing together the  – I am still revisiting the content and will likely write more on the topic in the future.

As Dan Rawsthorne quickly pointed when tying together the patterns in SAFe and LeSS, the owner of accountability, regardless of what (s)he is called, emerges in three ways from:

  1. Traditional Scrum: The Product Owner is outside the team and gives them a prioritized but not fully groomed backlog
  2. Modern and Scaled Scrum: The Product Owner is part of the team that is handed a prioritized and groomed backlog, some level of interaction with stakeholders is sacrificed
  3. Dysfunctional Scrum: The Product Owner is outside multiple teams serving multiple competing stakeholders for several products (typically, a proxy owner ends up with responsibility but no accountability

In SAFe, the key problem breach-of-pattern is that the diffusion of responsibility overcome through a Product Owner does not scale to the Program and Portfolio levels.  The key issue with LeSS, in contradistinction, is that the the pattern forgets to give each Scrum team their product owner!  It may be be implied that a Business Analyst or Technical Lead is meant to take on the responsibility of ensuring the backlog is fully groomed, but without recommending a solution for this I predict Dysfunctional Scrum – the person accountable, the person responsible, and the person empowered are not housed in a single role.

Scaling Scrum with Scrum:

The Scaling Scrum with Scrum framework recommended by Rawsthorne and Shimp at 3Back attempts to maintain the patterns that make Scrum successful and repeat them across several teams working on cohesive product offering.  Partly drawing on the success of Spotify there is a clear effort to maintain a clear chain of accountability ownership at any level of the organization.  This is not by necessity a command-and-control hierarchy – the reporting structure can rely on Communities of Practice so that skill-specific positions can report to an expert in that field despite working on separate teams.  The goal best driven by SSwS is organizational learning.  The clarity of the One Big Role at each level, where the is a facilitator and aggregator of feedback (scaling the ScrumMaster) and a clear point of accountability for decisions related to the feedback (scaling the Product Owner) stayed true to the philosophical line of thought set up in the first half hour.

Having spent several hours with a pattern-based critique, I quickly realized the main limitation of SSwS – the pattern “scales” 50 to 100 developers in scrum teams up to a cohesive Product Offering.  As the other half of my career was focused on Management Strategy Consulting, I could see that scaling accountability beyond SAFe’s portfolio level in a many-product firm competing with unique strategies in disparate markets quickly becomes management problem, not Scrum problem.  Dividing up strategic divisions, defining positioning in the market, and set the focus of a complex portfolio of investment is strategy solution, not a process solution.  Rawsthorne alluded to the longer discussion of how to Scale SSwS with SSwS and I plan to reach out to him regarding it – Chief Operating Officer as ScrumMaster and Chief Executive Officer as Accountability Owner?  “Clark, I can picture it in my mind and its breathtaking.”

Conclusion:

Scaling Scrum with Scrum is a terrific crash course for someone looking for a philosophical comparison of multiple approaches to scaling agile or taking a large organization and introducing scaled agile.  I would primarily recommend it to trainers, coaches, and change agents.


Connect With Me:

LinkedIn       Twitter

RSVP Now: ScrumAlliance Chicago Group Meeting May 12th

Scrum Chicago

FREEDOM – Using Scrum to Un-Impede Your FULL Potential

In parallel with this year’s Global Scrum Gathering in Phoenix, at our May User Group Meeting we will discuss “Freedom” and creating an environment that maximizes potential for yourself and others.

Next-iteration Scrum drives freedom from:
– The tyranny of the urgent
– Fear of making mistakes
– Business as usual
– Process rigidity

With an eye to the past, we will look at how to optimize current environments and adopt a forward-thinking approach to Scrum.

Eventbrite - FREEDOM - Using Scrum to Un-Impede Your FULL Potential

The one outcome you MUST achieve in Sprint Zero

Scrum, Lean UX, & Extreme Programming Sprint Zero:

The length of sprint “zero” for a new product and the (debatable) necessity of common sprint zero expected outcomes may vary, but there is one task that must be completed, or the full benefit of your Scrum and XP practices will suffer:

Complete setup of the repository and automated build server – and send a build.

This is a critical step in establishing team transparency and Product Owner accountability – frequently delivered builds, even with mid-sprint potentially un-shippable product increments – are the exclusive insight of the PO into the progress of the product.  It is the Product Owner’s responsibility to determine the worthiness of the stories delivered and whether additional iteration is necessary; delaying this feedback loop robs the PO of crucial insights and undermines the team’s ability to gain fast and honest feedback.

The longer the team waits to send the first build, whether due to a still-emerging architecture or lack of finalized copy and assets, the more perfection becomes the enemy of the good (or in this case, the deliverable).  Scrum requires transparency and accountability:  remove dysfunctional Hawthorne Effect as early as possible by setting the precedent – from Sprint Zero – that the build is delivered to QA and the Product Owner early and often.