Agile Evolved

Someone said to me “surely you would expect the team’s velocity to continue to rise?”.

I have heard this many times and it really misses the point of velocity.  It’s not to say that a well performing team’s velocity will not rise but, as a product owner, you should be looking for a predictable velocity first and foremost.

Remember that the team’s velocity represents the rate at which it can complete new work
(implement change) and does not take into account technical debt.  That is to say that it is unfair to link a team’s performance with velocity.  For example, they may have inherited some technical debt that they are working on which has now taken up a lot of their capacity (velocity).

So, the goal must always be to achieve a predictable, realistic velocity.

In response to this I often get product owners saying that they fear that teams will deliberately underachieve to ensure a predictable velocity.  Sadly, that is a team morale and/or attitude problem.

So remember, the goal is to achieve a predictable velocity not necessarily an ever increasing one.

In my early days of experiencing SCRUM we used to be in a dilemma about whether or not to size bugs.  It was an old piece of software with a lot of history and therefore a lot of technical debt.

The backlog contained a mixture of new user stories and bugs.  In the beginning we were assessing the bugs and assigning story points to them.  This felt really “agile” and so we continued like that for some time.

However, the unpredictable nature of bug fixing really became a problem. During the sprint, a new high priority bug would come in and we didn’t know quite how to handle it.  Cancel the sprint? Break the principle of respecting the team’s commitment for that sprint?  How “big” is the bug? etc.

A year later and the team is very adept at working according to Agile principles and this is how we handle bugs:-

Remember that a team’s velocity represents it’s capacity to implement change (new features).  Bugs are NOT sized.  The team works in a mixture of SCRUM and Kanban depending on the business priority.  If the business needs 100% focus on bug fixing then the team falls into a Kanban mode – picking bugs off the queue as they come in (we still maintain a 2 week sprint heartbeat).  If the focus switches to new requirements then it’s in SCRUM mode – user story sizing and velocity based planning.  Most of the time it’s a blend of new features and bug fixing.  And the team’s velocity, over time, naturally reflects that.

At the end of the day, at least you have a velocity that reflects what the team is capable of in terms of implementing new features.  Which is what the management need for road-mapping and planning.

Let me pose a question to you.

Let’s imagine we have a developer who estimates that a task will take 10 days and delivers in 9 days. We also have another developer who estimates 4 days for the same task and  delivers in 6 days.

If we assume that they deliver to the same level of quality and the task is exactly the same  of course this makes this somewhat a contrived example), who is best? Which developer do you want on the project: developer A, who estimates 10 and delivers in 9; or developer B, who estimates 4 days and delivers in 6.

I’m going to say that I have a different answer depending on the type of project. If I am managing a waterfall project I am going to want Developer A – I want the confidence that we can deliver in the estimated time. However, if I am on an agile project I hope I can have developer B.

I’ll come back to why I say ‘hope’ a bit later. First of all let’s consider a few realities of estimation.

What would you think if a developer estimated 10 days and delivered in 1 day? I wouldn’t mind betting that the majority view would be that this developer was a poor estimator; not that he was a great developer who delivered good quality working software in 10% of the estimated time. (Throughout this entry I am always making the assumption that the developer delivers the right level of quality – a fundamental principle for Agile.)

A smart developer knows this. They often won’t say that they are finished in 1 day. Instead they will ‘improve’ the code until it is closer to the estimate so that they simply look like a good developer who delivers on or just ahead of their estimate.

What about developer B in the above example. You could say that this developer is 50% over estimate. I am sure that this won’t go down well with the powers that be. This doesn’t look like the profile of a ‘good’ developer. Again, smart developers know this, which is why estimates are often ‘adjusted’ by the developer to make sure that they aren’t going to exceed them – better safe than sorry.

To top all of this, an experienced project manager will know this happens and will ‘counter-adjust’ their view of developers’ estimates. All in all, you could say that estimates sometimes aren’t worth the paper that they are written on.

There is one final spanner in the works that is thrown in on an agile project. On an Agile project we expect change, we embrace change and we even encourage it (yes, we do!) to make sure we deliver the most value to the business, all in the shortest possible timeframe. In this ever-changing environment, how can we expect a developer to come up with an accurate estimate when we are potentially moving the goalposts on a daily basis?

Sounds like estimating on an agile project is an impossible task? In my opinion, that’s not the case. We can produce good estimates but we estimate to a different level of accuracy depending on where we are in the project.

Most importantly, we need honesty when it comes to estimating on an agile project. We know things are likely to change so it is totally impractical to play the blame game.

What do we want on an agile project? We want to deliver as much working software as quickly as possible in an order prioritised by the business – we want to deliver real value.

This is where I can come back to my original question and where I say I hope I can have developer B on the project. This developer clearly delivered working software in a shorter period, thus maximising the associated business value. We need to take away all the politics on estimating to ensure that we encourage developers to work at pace –we don’t want to fall into the trap of identifying as a ‘good’ developer someone who simply delivers
to their own (possibly over-cautious) estimate.

I am going to cover a lot more ground on estimating over the next few entries but I hope that this gives some food for thought until then.

Take a look at my ‘The Ten Golden Rules for Agile Projects’ presentation which was delivered at the Agile and Lean in Finance conference on 22nd September 2011.

The Golden Rules for Agile Projects

Colin Weaver

First of all, this isn’t a lesson in anything to do with money. We can all read about countries going broke, and many of us are facing some challenges with our own budgets, but what I want to talk about here relates to an agile term: the so called ‘technical debt’. As you will see, there are lots of similarities between technical debt on an agile project and real-life debt.

Is debt bad? That’s an interesting question. Could you live your life without going into any form of debt – how would you get a house, a new car? But, what happens if you over extend yourself. Your own personal debt mountain increases, interest is charged on interest; we can all imaging a vicious spiral that leads to real problems.

Now, let’s turn to technical debt. First of all, what is technical debt? I would describe at as anything that you need to do on your project that requires some work to be performed but which does not directly make the application ‘better’ for a user. Of course, this mostly means fixing bugs, defects and the like, but I like to treat the re-factoring that we often need to do as part of my project’s technical debt.

Some people say that you shouldn’t have any technical debt – they say fixing technical debt is more important than adding new functionality, so as soon as you find something wrong you should fix it straight away. This is obviously an easy way to keeping technical debt under control, but is it the most efficient way? We could compare this with real life – it would be easy to have no debt problems if you set a rule never to borrow a penny but would you really want this (remember your house)? If you tried to fix every single, unimportant ‘bug’ before delivering new functionality (what I call ‘move this button 1 pixel to the right’ syndrome), you’d never get anywhere.

Other people ignore technical debt. They have really bad projects that run awry and never get back under control (like financial debts getting out of control). Not the way to go!

The reality with technical debt is that it can be treated the same as real-life debt. It’s natural to have some at some stages in your project. Just make sure you actively monitor and manage it and deal with it before it becomes a problem.

How do I spot technical debt that has run out of control? The most common way is when I see an iteration that is purely a ‘stabilisation’ or ‘bug-fixing’ release. That’s way bad for the users and is likely to break many of the promise that we would have made about going agile.

What would I suggest? Always make sure that you monitor your technical debt, and reserve some time in each iteration to keep it under control. This way, you can spend the vast majority of time moving forwards without running the risk of the project collapsing (Greece and the Euro, anyone?)

I often see organisations who have decided to ‘go agile’ and have picked Scrum as their methodology. If you look around the Agile community, Scrum is one of the most widely mentioned and popular agile methodologies. Scrum is pretty simple to understand (most things that are good are also simple) and really embodies a lot of the agile values. However, after a few sprints, I often see Scrum projects running into difficulties.

Why does this happen? First of all, let’s look at the decision makers who are usually involved in going agile. More often than not, the decision makers are ‘leaders’ in the IT or Business departments. By definition, these ‘leaders’ are individuals in senior positions, and are usually no longer involved in the day-to-day creation of software. Their focus is often on governance, project management, processes and the like – the bigger picture if you will.

Scrum is great for the business (and indeed project managers – although some project managers will debate this long and hard). It’s clear, easy to understand and offers the business the ability to change its mind, re-prioritise and re-focus. And we all know that’s one of the biggest benefits in going agile.

However, perhaps we should spare a thought for the guys that actually create the software – the development team. Developers often start off loving the thought of agile. They are promised that they will have the chance to be much more involved in understanding the requirements; and they are told they won’t be ‘committed’ to dates that are impossible to meet, dates that other people have already promised. This all sounds good – until change starts to occur.

So much of what is good in agile is based around accommodating change. An inexperienced agile team will often promise the business that it can ‘more easily accommodate change’ – note my italics – and that sets an expectation that the business remembers.

The reality is somewhat different, of course. Change can only be ‘more easily’ accommodated when we have a set of good development practices in place. We all know the ‘change costs’. Agile should reduce the cost of change, but cannot completely alleviate it. The real effort in change comes not from making the change itself but from checking what else you have impacted – i.e. the testing.

If you remove testing from any development cycle you would always have a quicker project. However, the end product is bound to have too many bugs and won’t be fit for purpose. We have all learned this the hard way but, on agile projects, this lesson seems to have gone out of the window. On agile projects we encourage change, but sometimes we forget that the developers have to re-code and re-test their work.

This is where Scrum is poor. Whilst it covers the ‘process’ side of Agile, it doesn’t do enough to enforce good development practices. We need more, from a developer’s perspective, to help accommodate functional changes – we need some mechanism to constantly build and test our application components to make sure we are not breaking things and introducing technical debt (I will cover this in a separate posting). The things we need sounds awfully like Test Driven Development and Continuous Integration. Don’t these things originate from methodologies such as Extreme Programming (XP)?