Agile Documentation

Agile values working software over comprehensive documentation – it is 1/4th of the original manifesto.  That doesn’t mean don’t document!  It means don’t document more than you need to document.  Documentation does have value, but the practice of documenting got excessive – that’s why a reaction to the bad stuff earned a spot as one of the pillars of agile.  How do you avoid over-reacting when changing a culture of over-documentation?

The Need for Documentation

We need documentation to help us communicate.  You can define anagile team as “the people creating the product.”  You can interpret“creating” as the people we normally think of as executing to accomplish the goals, or you can be more inclusive, and think of the people who have the goals as being part of the team too.

Philosophically, agile stresses collaboration.  Usually it is couched in terms of (1) people who are executing collaborate to devise the best solutions and (2) people who are part of the team collaboratewith the people for whom they are creating the product.  Personally, myexperience has been that projects with committed sponsors succeed, andprojects without them fail – so I always think of the sponsors as part of the team.

Regardless of your definition – collaboration requires communication, and communication benefits from documentation.

Temporal Communication

Communication among groups of people happens over time.

Some collaboration is transient – communication happens right now, and is only important right now.  Other communications are persistent – the collaboration happens right now, but we need to remember later what we agreed upon and why.  There are variations of “right now” and varying degrees of “later”, but slightly oversimplifying,

• There is communication for now.
• There is communication for later.

Both are important.

Documentation, in the vision from your nightmares – giantrequirements documents, architectural analyses, and detailedpsychographic profiles – is very inefficient when communicating for now.

These massive documents, while getting in the way of a conversation,do provide the opportunity to remember (in the future) why we make thedecisions we make (today).

Documentation, as emphasized in most agile discussions – user storiesand acceptance criteria, on an index card, taped on the wall – is not arobust solution for communication that happens later.

This low-overhead approach to capturing today’s ideas actuallyfacilitates and improves today’s conversation – but does not give us arecord we can use in the future for why we made our decisions.

As an agile team, there are ways we can approach both thepersistent and transient documentation tactics to make them morevaluable.  We can tweak our transient documents to make them more usefulfor persistence.  We can take an agile approach to development ofpersistent documents, so that we only do just enough documentation – trading off the minimum amount of short term efficiency to avoid long-term blunders.

Transient Documents for the Future

The image above is from a workshop defining the product backlog foran initial sprint for a new project.  The team identified the mostimportant stories for the first sprint, their acceptance criteria, andthe size, in story points, for each story.  As part of scrum, thestories are written in user-story format, with acceptance criteria foreach story identified and documented on additional index cards (taped tothe index cards for the associated stories).  Great transient way tocapture the must do and makes it better stories that the product owner is thinking about.

As a team, we made a couple minor tweaks to make these story cardsmore useful.  First, all of the stories were organized into themes thathelp establish the context (for the team), and combine the stories into“meaningful areas of focus” for communication with stakeholders.

Each story got a blob of color in the top left hand corner,identifying which of the five themes were being supported.  You can see afew of them circled in the image above.

We also developed a set of 8 proto-personas (prototype, early draft,very rough personas – almost stereotypes) that the overall product /project was designed to support.  Each persona got a different colorstar.

Each user story got a corresponding colored-star sticker to indicatefor which persona the story has been written.  Specifically, whenmultiple personas could use a story, we identified for which persona wewere implementing the story in this sprint.

In the “right now” world, these tweaks helped us understand,question, and confirm the themes and personas being supported in thefirst story.  Those conversations allowed us to reach pragmaticcompromises about which stakeholders were going to benefit and when. They also allowed us to craft the outbound message that is step one ofstakeholder expectation management.

The list of themes changed while we were doing this – moving from 4to 5 themes, as we realized that there was value in splitting one of thethemes.  The group of proto-personas grew from an initial set of 3 to aset of 8, as we quickly realized that the team was trying to “buildsomething for everyone” and we wanted to avoid the elastic user problem, and instead, roll out capabilities sequentially that satisfied valuable groups of users with similar goals and perspectives.

Culture Change for Stakeholders

As you would expect, having a bunch of cards stuck on the wall provides a great, visceral, now communication for the team – both in implementing and communicating.  However, cards on the wallis very transient.  It also doesn’t provide a good way to communicatewith old-school stakeholders (imagine the execs who have their admins print out their emails for them to read and annotate – those folks still exist!).

This can make it hard for the “not part of the implementation” part of the team to collaborate and communicate.

In User Stories Applied,Mike Cohn stresses that the brevity of user stories is intentionallydesigned to facilitate (and I would add “dependent upon”) conversation. I find it to be both a strength and a weakness of user stories.  It isstrong because you can cover a lot of ground quickly (breadth) andcapture a number of stories, much like the list above.  It is weakbecause the requirements documentation does not stand on its own – itrequires conversation to fill in the details.  I’ve had some successusing the “Verify that…” user acceptance tests as the method ofdocumenting those details (depth), in conjunction with the brief, easilyconsumable stories.

Stakeholders in a Barrel

Those conversations happened as part of getting the stories defined,getting them on the board in the right order, and defining theacceptance criteria.  Stakeholders often have a short memory, especiallywhen they were the ones making a compromise.  You need some way to retain the context of the conversation that led to the particular organization and content of the stories on the wall.

Persistent Documents in the Present

Agile proponents complain about the massive get-in-your-way documentsthat slow down the start of projects, prevent the teams from adapting to changing market conditions,and otherwise make it harder to deliver great products.  I agree. Agile proponents also rail against the “build it all, then release it,”waterfall, process for creating software – and propose an alternative –iterative development of the software.  I agree again.

Why not apply the same principle of iterative development to persistent documents?

Consider the example above of using colored-star stickers to identifythe people for whom each story is being implemented in any givensprint.  The majority of the time spent on this analysis is aroundunderstanding which people are important (to the success of theproduct), and for which people the ability to perform this user story isimportant (to the person).  The combination of these two importances isan input to prioritization.  A minimal amount of time is spent puttingstickers on cards, and creating multiple cards for the same story (eachwith a different sticker for a different persona and potentially different acceptance criteria).

Add a tiny bit of overhead, and capture the information in a spreadsheet (or whatever).

The image above shows a mapping of use cases to actors.  It could just as easily show a mapping of user stories to personas.  The transition from thinking about actors to thinking about personas is a small one.  This chart comes from an earlier article on communicating a delivery schedule with use cases – it helps stakeholders get a big picture view of who benefits when – a user-centric, yet still top-down perspective.

Note: Another nuance to incremental delivery is that you canintroduce the “bare bones” version of a story now, and the “better”version of it later.  You can also communicate that stories get better over time, for some users, with the same type of chart.

A very similar, and also simple to create table can show how eachtheme is getting “representation” within each sprint.  That viewfacilitates great discussions about trying to emphasize one theme andthen another sequentially, versus doing the most important items in eachtheme first – gradually making progress in each theme.  Creating thatview, and using it to communicate, has helped me address concerns that“external” stakeholders have shared about how a team appeared to be“chaotic and random” in their approach to prioritization andimplementation.

Many Models

There are many types of documentation – and many types of models foremphasizing and discussing particular aspects of a complex system (likeusers and goals and solutions).  Creating a simple diagram like thefollowing whiteboard sketch is almost required for effective transientcommunication in some projects.

[from Simple Agile Model Example]

When most folks are complaining about documentation, they are notcomplaining about the sketch above.  They would consider that sketch tobe augmenting a conversation.  Fine.  That’s what I did, at thetime.  Then I took a picture of it.  Suddenly, it was documentation.  Astakeholder came by later, confirmed that the system needed to workthis way, and signed the whiteboard – and the photo was updated.

Every heavyweight document can start out this way.  And it can evolve.  It should evolve.  If you’re going to succeed, it must evolve as your team gets smarter, your competitors react, and your market changes.

The trick is that you don’t have to write the final version before you get started.  Capture at a high level what you know right now – just like a roadmap.  Capture in just enough detail what you need right now – just like a story backlog.  And change it as you go.

Conclusion

Documentation is not bad.  Documenting stuff you will never use isbad.  Documenting stuff you don’t need for a long time is risky, becauseit will probably change before you refer back to your document.

Documenting why you made decisions right now, as part of transient collaboration and communication is important, so that you collectively remember.  That documentation persists and your team can build upon knowledge, incrementally – just as your team builds upon the evolving code base, incrementally.