Retrospectives: Best Practices, Common Problems and New Ideas

Here are the slides for my presentation at the May 2012 Phoenix Scrum User’s Group meeting. If you attended the meeting and have any further questions, please contact me, clayton@integrumtech.com.

Agile Principles: Tear Down These Cubes

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
– Agile Principles

Manager, Tear Down These Walls

If you were tasked with designing a system that made spontaneous collaboration difficult, you couldn’t go wrong with the modern cube farm. Members of a “team” dispersed over an area of the office, isolated in their drab tchotchke ridden cubicles. Instant message, e-mail and chat might be great for passive communication, but when it comes to real collaboration, being physically present is essential.

But We’re Distributed!

Co-located teams are better than distributed teams. However, sometimes we cannot be co-located. That does not mean we resort to passive forms of communication and digital tools to convey our progress and ask questions. Skype is better than a phone call which is better than e-mail. Don’t underestimate your brain’s ability to gather important insight from the facial and body language of the person on the other end of that video chat.

Agile Principles: Self-Organizing Teams are Motivated Teams

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
– Agile Principles

Self-Organizing

Allow the people doing the work to self-organize and discover the best implementation they can given what they know right now. Protect them from organizational politics and outside distractions, enable them to focus on the work at hand.

Self-Directing

While we encourage self-organization, we remember the need for direction. While the team does not self-direct, they need a clear vision and an understanding of success in order to attain the autonomy and purpose required to feel motivated.

Agile Principles: Collaborate Everyday

Business people and developers must work together daily throughout the project. – Agile Principles

Email is not Enough

Working together involves more than exchanging e-mails about the software. It’s more than passive instant messenger conversations. It’s not reviewing a digital agile tool in isolation. Working together means collaborating, negotiating and exploring the software that is being built.

Too Busy for Success?

If you’re too busy to interact, fairly frequently, on a daily basis with the rest of the team, you’re too busy to enable product success. High performance teams don’t allow their schedules to be overrun by meetings. They don’t let anything get in the way of daily collaboration, negotiating and exploration.

Agile Principles: Frequently Deliver Working Software

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
– Agile Principles

Quick Feedback

A short feedback cycle is incredibly valuable. A short feedback cycle helps ensure you’re building the correct thing. A short feedback cycle helps you decide if you should keep going. A short feedback cycle helps you know if you should stop.

Product vs. Project

Shift your mindset from working on a project to working on a product. Products don’t have end dates, they continue to exist and the organization realizes their benefits over and with time. As time goes on, incrementally release working features for the product so that you can harness the power of short feedback cycles.

Probability, Velocity and Re-Estimating User Stories

When working with user stories, story points and SCRUM, it is not uncommon to come to the end of a sprint and realize that a smaller story took longer to complete than a larger story. While this is an excellent reminder that story points reflect complexity, not effort, it is often misinterpreted as an indication that remaining stories need to be re-estimated. Basic probability distribution shows us that over a large enough sample, all three point stories in the product backlog will take about the same time to complete. A team’s desire to re-estimate stories in the product backlog based on a few stories deviating from the norm is a mistake.

Not All Stories are Created Equal

If you were to roll a pair of dice a million times, you would see the majority rolls sum up to seven. If you graphed the data, you would see a bell curve shape. User stories are no different.

Over a large enough sample, you will start to see the same bell curve shape in the time it took to complete those stories. Some three point stories might take two hours to complete while others take twelve; but most take seven. At the same time, some of your five point stories will take ten hours to complete and some twenty four; but most take eighteen. Consider the overlap in the following diagram:

The overlap between the time to complete some of the three point stories and some of the five point stories should not be mistaken for a need to re-estimate any stories. Sometimes a three point story will take as long to complete as a five point story.

Commitment Driven Planning

One technique from which all SCRUM teams could benefit is that of Commitment Driven Planning. From Cohn’s Agile Estimating and Planning

A commitment-driven approach is an alternative way to plan an iteration. Commitment-driven iteration planning involves many of the same steps as velocity-driven iteration planning. However, rather than creating an iteration plan that uses the yesterday’s weather idea to determine how many story points or ideal days should be planned into the current iteration, the team is asked to add stories to the iteration one by one until they can commit to completing no more.

So what happens when a three point story is tasked out and comes to ten hours, and a five point story is tasked out and comes to nine? Often times the first inclination of the team is to inflate the three point story. Especially interesting is that almost no one suggests to shrink the estimate of the five point story. Again, we should expect that some three point stories will take the same amount of time to complete as some five point stories, but in the end this will even out.

Velocity is The Great Equalizer

Again from Mike Cohn’s Agile Estimating and Planning

What’s happened here is that velocity is the great equalizer. Because the estimate for each feature is made relative to the estimates for other features, it does not matter if our estimates are correct, a little incorrect, or a lot incorrect. What matters is that they are consistent. We cannot simply roll a die and assign that number as the estimate to a feature. However, as long as we are consistent with our estimates, measuring velocity over the first few iterations will allow us to hone in on a reliable schedule.

As the team complete more and more sprints, developers will come across three point stories that are a little larger than average and five point stories that are a little smaller than average. Between our bell curve distributions and velocity acting as the great equalizer, over time, the bumps in the estimations will smooth out and the desire to re-estimate stories will fade.

Agile Principles: Deliver Early and Often, but Always Deliver Value

The Agile Manifesto describes twelve principles of agile software. In a series of twelve blog posts, I will explore each principle.

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

This principle can be broken down into two separate thoughts. Continuously delivering early, and always delivering value.

Delivering Early and Often

Delivering early is more than just delivering something before the competition. Delivering early is more than just delivering before you would have had you not been being Agile™. Delivering early means specifically delivering less than you could have otherwise. Continuous delivery and iterative development are synonymous, it is not enough to just deliver early, you need to frequently do so.

Always Deliver Value

Delivering value is difficult. At a high level, it is difficult to know what is and is not valuable. Following the process and being disciplined can get you to a point where you are delivering early and often, but without a unified vision of what you are creating, it is very difficult to deliver real value.

Delivery early and often, but always deliver something of value.

Let Conversations Write Your Tests

I consider myself an early adopter of Cucumber and have spent a lot of time using, thinking about, writing about and discussing cucumber and, at a higher level, behavioral driven development. Lately however, I’ve really been rethinking how I use Cucumber and why. Am I getting enough benefit from the overhead and abstraction of plain text features? Could I do the same thing with RSpec if I just approached RSpec differently? Am I cuking it wrong?

This shift in thinking is due in part to Liz Keogh’s Step Away from the Tools and Dan North’s Who’s domain is it anyway?. Both of these got me thinking about how I’m using the term BDD (rather loosely) and how much of an investment I’m making in a specific tool (Cucumber).

More Meaningful Planning

Having meaningful planning meetings with their customer/product owner is one thing with which many teams struggle. Too often, we go too fast, don’t uncover enough detail, use the wrong language, don’t understand “done” and leave too many loose ends.

To combat this we draw screens, discuss workflows, ask leading questions and a variety of other techniques. I felt that while I was doing these things, I was still frustrated with the other part of the planning process. I’ve never liked traditional tasking and I’ve never like the idea of having to translate all of the data gathered with the customer into some other form only to express it later in a test in yet another form.

What if I wrote my tests during the planning meeting?

I decided that changing the way I gathered conditions of satisfaction, defined “done” and discussed workflows with my product owner would give me the biggest boost in value delivered, so one day I did just that.

For the following examples, assume that we’re adding a feature to an e-commerce website.

As a user I should be able to manage addresses so that I don’t have to enter my information more than once

As I’m discussing the feature with my product owner, I will discuss with them the possible scenarios, including workflows, required to use the feature. Some scenarios for this feature might include:

  • When I am on my account page
  • When I am creating a new address
  • When I am deleting an address

These scenarios might have scenarios of their own:

  • When I am editing an address
    • When I successfully edit my address
    • When editing my address fails

So far I’ve been able to ask the product owner something like “So when I’m editing my address, and I miss some required fields, what happens? What do I see? Where do I go? What are the required fields?”. I can also draw pictures to explain the workflows and ask more questions “What’s on this page? Where is the error message displayed? Do I see error messages for the fields that are missing?”.

For each scenario, I can capture assertions that come from answers to the questions I’m asking:

  • When I am on my account page
    • I should see a link to “My Addresses”
  • When I click on the “My Addresses” link from my account page
    • I should be on my addresses page
    • I should see “My Addresses” in the page heading
  • When I am on my addresses page
    • When I have existing addresses
      • I should see each address listed in format xyz
      • I should see an “edit” link for each address listed
    • When I don’t have existing addresses
      • I should see help text explaining how to add an address

If you’re an RSpec user, you might be thinking, “Hey this looks like RSpec!” and it does. During the planning meeting I can capture these scenarios and outcomes and then use them nearly verbatim for my RSpec acceptance tests. Even better, when I run my tests, I can use the “documentation” format that RSpec gives you to output that’s almost identical to the scenarios above.

The conversation required to really define these scenarios and outcomes is challenging, but at the same time, very rewarding. I have also found that it’s pretty powerful to be able to sit with the product owner and view two nearly identical documents side-by-side knowing that one is automated test output.

Do You Make This Common Mistake When Estimating?

There’s a common mistake that many software developers make when estimating projects. Here’s how you can avoid falling into this trap.

When estimating a project using the Planning Poker method, many developers like to use a baseline estimate for a given task. For example, many developers use CRUD, the creating, displaying, editing and deleting of a Model as their baseline estimate. Once they’ve got their baseline in mind, it makes it easier to estimate other stories that are more domain specific, or so it seems.

Baseline Estimates are Broken

When you’re using a task like CRUD as a baseline for your estimations, you can easily skew the estimations of the other stories in the project. Let’s say we’re using a 3 point baseline for CRUD stories.

  1. As a user I should be able to upload a profile photo – 2
  2. As a user I should be able to CRUD movies I’ve seen – 3
  3. As a user I should be able to send a private message to another user – 5
  4. As a user I should be able to create a trivia quiz for a movie that I’ve seen – 8

In this first example, the stories are probably estimated fairly well and compared to each other, the complexity is quite relative. What happens if we add a few more easy stories or a few more difficult stories?

  1. As a user I should be able to see a contact e-mail on the home page – 1
  2. As a user I should be able download the menu as a PDF - 1
  3. As a user I should be able to read a privacy policy – 2
  4. As a user I should be able to CRUD restaurant reviews – 3

With easier stories added, the CRUD story is definitely the most complex, but compared to the others it is significantly more complex than seeing a link on a page or viewing some text.

  1. As a user I should be able to CRUD portfolio photos – 3
  2. As a user I should be able to signup for a paid recurring account – 8
  3. As a user I should be able to make connections with other users via Facebook – 13
  4. As a user I should be able to send a rocket to the moon – !

When the other stories become much more complex, the CRUD task is again shown to be significantly different in complexity, this time in the other direction. In this group of stories its hard to imagine that managing portfolio photos is only two orders of magnitude away from a recurring payment e-commerce system.

Relative Complexity Works

Its important to estimate a group of stories so that the complexity of each story is relative to the next. In Mike Cohn’s Agile Estimating and Planning, he describes a method of estimating stories called “Analogy”.

When estimating by analogy, the estimator compares the story being estimated with one or more other stories. If the story is twice the size, it is given an estimate twice as large.

When you apply this technique to your estimation process, you will have a more coherent set of estimates. From this you will be more likely to determine an accurate estimated velocity and you will have a better overall sense for the scope of the project.

Hurdles to Estimating by Analogy

  • Estimating all of your stories one by one makes it difficult to estimate a relative complexity as you only have the previously estimated stories with which to compare your current story. A group of especially simple or complex stories could be waiting towards the end of the sessions.
  • Estimators who are new to the agile estimation process might have difficultly estimating a story without a point of reference.
  • Estimating with a baseline can be a difficult habit to break since it provides a convenience and familiarity to the estimator.
  • Two seemingly similar projects may have a greater than expected difference in total number of story points, even though their velocities are relatively the same.

The Secret to Awesome Agile Development

With a little hard work and my secret development ingredient, you can be a better Agile Developer

Recently my fellow developers at Integrum and I took a survey that helped us assess our team with regard to our Agile practices. When taking the survey, and now reviewing it later on, I was struck by how many of the questions were related to a single concept. Many of the problem areas that can be uncovered by the survey, along with examples of one’s successes, come back to this one theme.

Are programmers nearly always confident that the code they’ve written recently does what it’s intended to do?
Consider the following questions:
  • Is there more than one bug per month in the business logic of completed stories?
  • Can any programmer on the team currently build and test the software, and get unambiguous success / fail result, using a single command?
  • When a programmer gets the latest code, is he nearly always confident that it will build successfully and pass all tests?
  • Are fewer than five bugs per month discovered in the teamʼs finished work?
  • After a line item is marked “complete” do team members later perform unexpected additional work, such as bug fixes or release polish, to finish it?
  • Are programmers nearly always confident that the code they’ve written recently does what it’s intended to do?
  • Are all programmers comfortable making changes to the code?
  • Do programmers have more than one debug session per week that exceeds 10 minutes?
  • Do unexpected design changes require difficult or costly changes to existing code?
  • Do any programmers optimize code without conducting performance tests first?
  • Is there more than one bug per month in the business logic of completed stories?
  • Are any team members unsure about the quality of the software the team is producing?

What’s the common theme among these stories, and the secret to better agile development? Testing, testing and more testing.

The negative outcomes implied by some of these questions can be solved by testing. Spending time fixing “completed” stories? Probably something you could have tested. Conversely, the positive benefits implied by other questions can be had via testing. Want to make your code more inviting and easier to deal with for new team members or people unfamiliar with the project? Give them robust and well-written tests.