Tag Archives: ideas

PDD, The Other SDLC

As I’ve said before, I’m no stranger to public speaking. That said, NorDevCon was my first conference and, by all accounts, I did OK.

My talk, “PDD1, The Other SDLC2” focused on how external stimuli, such as live production errors and impending deadlines, can cause development practices to break down. Ultimately this breakdown boils down to communication, or lack thereof. This talk was spawned from my talk on “Agile In The Real World” last year, and both of these talks focus on the personal experiences I have had when real life gets in the way of theory.

Personally I like this type of talk as it’s something that tends to resonate with the audience. With Impostor Syndrome and fears about “Am I doing it right?” being quite common among developers it can be quite useful to stand up and highlight where it commonly goes wrong so people know they are not alone.

Of course the irony is my own Impostor Syndrome and fears about “Am I doing it right?” have me worry the audience will respond with “No, we don’t relate to that, it’s just you doing it wrong“. Having the original author of some of the work I was building on attending the talk also added to the pressure a tad.

Having proved to myself that not only can I do it, but that I thoroughly enjoy it, I’ll be looking to see if I can do talks at other conferences3. I’m also hoping to line up so local talks too during the year (including my PDD talk if you missed it at NorDevCon). I suspect this year is going to see a lot of me using Keynote 🙂


1 PDD: Panic Driven Development

2 SDLC: Software [or Systems] Development Lifecycle

3 Ones where I don’t personally know the organiser, and run the group backing the conference 🙂

Costing and Commitment

One of the hardest aspects of Scrum seems to be the accurate costing of stories. We all know the theory: you break your work into chunks, cost those chunks, and commit to a certain amount of work each week based on costing and velocity. Pure Scrum™ then states that if you do not complete your committed work then Your Sprint Has Failed. All well and good, apart from one small problem: it’s all bollocks.

I’ve long had an issue with the traditional costing/commitment/sprint cycle insofar as it doesn’t translate from theory into the real world. In her recent talk at NorDev, Liz Keogh pointed out that Scrum practitioners are distancing themselves from the word “commitment” and from Scrum Failures as they are causing inherent problems in the process. Allan Kelly recently blogged about how he considers commitment to be harmful. Clearly I’m not alone in my concerns.

As always, when it comes to things like this, I’ve simply borrowed ideas for other people and customised the process to suit me and my team. If this means I’m doing it “wrong” then I make absolutely no apologies for it, it works for us and it works well.

Costing

Developers suck at costing. I mean really suck. It’s like some mental block and I have yet to find a quick and effective way to clear this. Story points are supposed to make developers better at costing because you’re removing the time element and trying to group issues by complexity. That’s lovely on paper, but I’ve found it just confuses developers – oh, we all get the “a Chihuahua is 1 point so a Labrador must be 5 or 8 points” thing, but that’s dogs, visualising the relative size and complexity of code is not quite as simple.

What’s worse is that story points only relate to time once you have a velocity. You can’t have velocity without points and developers generally find it really hard to guesstimate1 without times. Chicken and egg. In the end I just loosely correlated story points and time, making everyone happy. I’ve also made story points really short because once you go past a few days estimates start becoming pure guesses. What we end up with is:

Points Meaning
0 Quicker to do it than cost it.
0.5 One line change.
1 Easily done in an hour.
2 Will take a couple of hours.
3 A morning/afternoons work.
5 Will take most of the day.
8 It’s a days work.
13 Not confident I can do it in a day.
20 Couple of days work.
40 Going to take most of the week.
100 Going to take a couple of weeks.

Notice this is a very loose correlation to time, and it gets looser the larger the story point count. Given these vagaries I will only allow 40 and 100 point costings to be given to bugs. Stories should be broken up into chunks of two days or less so you’ve got a good understanding of what you’re doing and how long it’s going to take2.

With that in mind 40 points really becomes “this is going to be a bitch to fix” and 100 points is saved for when the entire team looks at you blankly when diagnosing the problem: “Err… let me go look at it for a few days and I’ll get back to you“.

Stopping inflation

Story point inflation is a big problem with scrum. Developers naturally want to buy some contingency time and are tempted to pad estimates. Story point deflation is even worse with developers being hopelessly optimistic and then failing to deliver. Throw in the The Business trying to game the system and it’s quickly become a mess. I combat this in a few ways.

Firstly, points are loosely correlated to time. In ideal conditions a developer wants to be completing about 8 points a day. This is probably less once you take meetings, walkups and other distractions into account. While an 8 point story should be costed such as the developer can complete it in a normal day with distractions accounted for, the same doesn’t hold true for a series of 1 point stories. If they’re all about an hour long and there’s an hours worth of distractions in the day then the developer is only getting 7 points done in that day.

Minor fluctuations in average per developer throughput are fine, but when your velocity starts going further out of whack it’s time to speak to everyone and get them to think about how they’re estimating things.

Secondly, points are loosely correlated to time. A developer can track how long it takes them to complete an issue and if they’re consistently under or over estimating it becomes very apparent as the story points bear no correlation to the actual expended effort. A 5 pointer taking 7 hours isn’t a problem, but any more than that and it probably wanted to be an 8 pointer. Make a note, adjust future estimates accordingly. I encourage all my developers to track how long an issue really takes them and to see how that relates to the initial estimate.

Thirdly, costing is done as a group exercise (we play planning poker) and we work on the premise of an “average developer”. Obviously if we take someone who is unfamiliar with the code it’s going to take them longer. You’ll generally find there’s some outlying estimates with someone very familiar with that part of the code giving low estimates and people unfamiliar with it padding the value. We usually come to a consensus fairly quickly and, if we can’t I just take an average.

I am aware that this goes against what Traditional Scrum™ teaches us, but then I’m not practicing that, I’m practicing some mongrel Scrumban process that I made up as I went along.

Velocity and commitment

I use an the average velocity of the past 7 sprints3 adjusted to take into account holiday when planning a sprint. We then pile a load of work into the sprint based on this figure and get to work. Traditionally we’ve said that we’ve committed to that number of story points and issues but only because that’s the terminology that I learned with Scrum. Like everything else, it’s a guestimate. It’s what we hope to do, a line in the sand. There are no sprint failures. What there is is a discussion of why things were not completed and why actual velocity didn’t match expected velocity. Most of the time the reasons are benign and we can move on. If it turns out there are problems or impediments then we can address them. It’s a public discussion and people don’t get to hide.

Epics and Epic Points

The problem with having story points covering such a small time period is that larger pieces of work start costing huge numbers of points. A month is something like 200 points and a year becomes 2500 points. With only 2000 hours in a year we start getting a big disconnect between points and time which The Business will be all over. They’ll start arguing that if a 1 year project is 2500 points then why can’t we have 2500 1 point issues in that time?

To get round this issue we use epic points which are used to roughly cost epics because they’re broken down into stories and properly costed. While story points are educated guesstimates epic points are real finger in the air jobs. They follow the same sequence as story points, but they go up to 1000 (1, 2, 3, 5, 8, 13, 20, 40, 100, 250, 500, 1000). We provide a handy table that lets the business know that if you have an epic with x many points and you lob y developers at the problem then it will take roughly z days/weeks/months. The figures are deliberately wooly and are used for prioritisation of epics, not delivery dates. We’re also very clear on the fact that if 1 developer can do it in 4 weeks, 2 developers can’t do it in 2. That’s more likely to be 3 weeks.

Epic points are malleable and get revisited regularly during the life of an epic. They can go up, down or remain unchanged based on how the team feel about the work that’s left. It’s only as the project nears completion that the epic points and remaining story points start bearing a relationship to each other. Prior to that epic points allow The Business to know if it’s a long way off, or getting closer.


1 What, you think this is based on some scientific method or something? Lovely idea, we’re making educated guesses.

2 I’ve had developers tell me they can’t cost an issue because they didn’t know what’s involved. If you don’t know what’s involved then how can you work on it? Calling it 20 points and hoping for the best isn’t going to work. Instead you need to create a costing task, and spend some time working out what’s involved. Then, when you understand the issue, you can then cost it properly.

3 A figure based purely on the fact that JIRA shows me the past 7 sprints in the velocity view.

PDD

Development Strategies

The Development Strategy triangle.


Most [all?] discussions on Agile (or lean, or XP, or whatever the strategy de jour is currently) seem to use a sliding scale of “Agileness” with pure a Waterfall process on the left, a pure Agile process on the right. You then place teams somewhere along this axis with very few teams being truly pure Waterfall or pure Agile. I don’t buy this. I think it’s a triangle with Waterfall at one point, Agile at the second, and Panic Driven Development at the third. Teams live somewhere within this triangle.

So what is Panic Driven Development? Panic Driven Development, or PDD is the knee jerk reactions from the business to various external stimuli. There’s no planning process and detailed spec as per Waterfall, there’s no discreet chunks and costing as per Agile, there is just “Do It Now!” because “The sky is falling!“; or “All our competitors are doing it!“; or “It’ll make the company £1,000,0001; or purely “Because I’m the boss and I said so“. Teams high up the PDD axis will often lurch from disaster to disaster never really finishing anything as the Next Big Thing trumps everything else, but even the most Agile team will have some PDD in their lives, it happens every time there is a major production outage.

If you’re familiar with the Cynefin framework you’ll recognise PDD as living firmly in the chaotic space. As such, a certain amount of PDD in any organisation is absolutely fine – you could even argue it’s vital to handle unexpected emergencies – but beyond this PDD is very harmful to productivity, morale and code quality. Over a certain point it doesn’t matter if you’re Agile or Waterfall, the high levels of PDD mean you are probably going to fail.

Sadly, systemic PDD is often something that comes from The Business and it can be hard for the development team to push back and gain some order. If you find yourself in this situation you need to track all the unplanned incoming work and its affect on the work you should be doing and feed this data back to the business. Only when they see the harm that this sort of indecision is causing, and the effect on the bottom line, will they be able to change.


1 I have worked on quite a few “million pound” projects or deals. The common denominator is that all of them failed to produce the promised million, often by many orders of magnitude.


“PDD” originally appeared as part of Agile In The Real World and appears here in a revised and expanded form.


Revisiting Git

I first discovered Vincent Driessen’s branching model when I was investigating Git a couple of years ago. At the time I was desperate to get away from the merge issues that we were having with Subversion and was looking into both Git, and different branching models using it. Given the steep learning curve with Git we decided not to complicate things further and to stick with our old branching model; albeit with a slight change in naming convention borrowed from Vincent.

Now I’m a little more comfortable with Git it’s time to revisit our branching strategy, and use of Git overall. I’ve looked at Vincent’s branching model again, and looked at git-flow in great depth and have concluded it’s still not for us, however, the idea of codifying the workflow, and providing git extensions to aid using the workflow appealed to me immensely.

Currently we’re doing little more than branching off master for issues, with each issue (be it a new feature, or bug fix) living in its own branch. Issues are rebased into master when they’re done. Releases are also cut from master and any patches (either to the release version in QA, or to the release version in live) are cut from the release branch, rebased into that and then rebased into master.

In order to simplify our JIRA and Fisheye/Crucible setup we also push issue branches to origin making them visible from a central point, as well as providing a backup. These branches are considered private and not used by others.

Since we have tight control over all the branches a rebase only strategy works fine for our team, although we have had some problems caused by accidental merges. The next step is to improve the workflow and the merge/rebase strategies to survive all situations, codify these steps and then script them – something I’ve already started doing.

I’m also looking at this as a potential opportunity to possibly move away from Fisheye and Crucible to Stash, potentially saving some money whilst keeping the tight JIRA integration.

BDD inline questions

I’ve started using BDD scenarios for most things now as I find the Given/When/Then syntax great for focusing on the problem and generating questions. What I have found though is that often the questions need to be written down somewhere to ask the relevant person. Correlating the questions and the scenarios can be a pain so I’ve started putting them inline into my scenario text using a ?? notation. The format is simple:

  • ?? on it’s own: denotes an outstanding question about the preceding text.
  • word??: denotes a question about the preceding word.
  • ?option/option?: denotes uncertainty about which option should be used.
  • ?question?: denotes a question being asked.

This syntax can actually be used to write the original scenario:

Given ??
When ??
Then ?? should ??

An example of this usage might be:

Given a user on a web?? form
When the user ?enters/selects? ?should we be thinking about how the provides the data? their D.O.B.
Then the form should ??

A discussion, which could be via email or some other means than face to face, can than be had to resolve the questions using just the scenario files and the annotations to the scenarios. You could even go one step further and use something like Gollum to track the changes to the scenarios.

An Evening of Game A.I.

On Thursday I’m off to An Evening of Game AI. Now, what I know about writing modern games can be written on the back of a postage stamp. What I am is a consumer of games. And a fussy one at that. So why an evening of Game AI? Well first off its a subject that I think I’ll find interesting. Even if I never out the knowledge gained into practice I’m liable to walk away thinking “that was cool”. I’m a geek; shoot me. More importantly is the concept of Gameification [hideous word] and how I might apply game AI techniques into a business context. The ideas I have a very embryonic at the moment and it may be that they’re pie in the sky, but hopefully Thursday will prove useful and give me an idea of what directions I should be looking in.