One of my current employer’s clients (a large public sector organisation) have recently been going through a process of adopting a more agile method of working – previously they had been very waterfall focused, but having had a SCRUM team from Amido embedded in their office for a while, and watching the way that an agile methodology was helping that team deliver consistent, high quality releases of software, the organisation was coming round to the idea of Agile.
As such I was asked to create and run a training workshop for a team of their Business Analysts, most of whom were either unaccustomed to agile methodologies or relatively new to the role of Business Analyst.
The training session covered all of the basics that I imagine you’d expect to see (iterative development practices, backlog management, tips for quickly gathering and confirming requirements etc) but one of the core concepts that I wanted to get across to the team was that of managing upwards and setting expectations using the Cone of Uncertainty as a tool.
As an agile business analyst, managing stakeholder expectations is undoubtedly an important part of the role. This is especially true within an organisation which is only just beginning to embrace agile practices, and as such, is more likely than not to still have a culture (particularly at management level) which is orientated towards project that follow a more traditional waterfall format.
This can cause a bit of a conundrum. Stakeholders like the benefits of agile (higher quality outputs, early visibility of project deliverables etc) but they are also accustomed to some of the perceived benefits of waterfall projects (end-to-end estimates/timelines for a whole project, well-documented solutions for all components). Often, people don’t seem to realise that adopting an agile approach is a compromise. Benefits such as the ability to pivot and re prioritise the project direction during the course of a project are logically dependent upon there being a less rigid view of the project over the long term. That isn’t always a message that people want to hear though – especially if it is being delivered in the context of a justification for why a project is overrunning.
Enter the Cone of Uncertainty (and the need to manage upwards by discussing it with stakeholders as early as possible in a project). For those unfamiliar with the the concept, it was initially conceived for use in the engineering and construction industries around 1958 but was eventually adopted in the software industry in the nineties and is now a fairly well known (albeit under-utilised) tool. This is what the cone looks like, in its original format (based on a waterfall approach).
And here’s a version I’ve slightly revised to show how the cone also applies to an agile project (I’ve used the comparative example of defining an Epic)
As can be seen from the diagrams, what the Cone of Uncertainty conveys is that the accuracy of project estimates increases relative to how near to the deployment of the deliverable we are. Or in other word – estimates created at the start of a project/component definition are highly likely to be inaccurate, by a significant variance. The comments I’ve added to the second image are just as relevant on the first – making promises early is a bad idea; estimates only really become sufficiently reliable once coding has begun.
So clearly, as a concept, the Cone of Uncertainty is relevant to both agile and traditional project delivery models. The perceived issue with an agile approach is that the variance as measured at outset is likely to be higher than that of a waterfall project, because the agile approach means that less effort will have gone into estimating work which is further along in a project plan – in agile frameworks, detailed estimates are usually only generated just ahead of the work being conducted on each sprint of work. In fact, a truly agile organisation wouldn’t even try to estimate work which is very far down the project path, precisely because the likelihood that the estimates made at outset for such components would be of low quality, due to the Cone of Uncertainty. Nevertheless, stakeholders will still want estimates, and to an extent this is understandable – no-one wants to write a blank cheque for work, and most organisations need to be able to plan for the delivery of a project against some kind of timeline.
This means that many stakeholders will push for a full set of estimates for a project even though logically, there is more danger in accepting the up front estimates for a full waterfall project than there would be for an agile project (as there is more up front work to estimate, the capacity for those estimates to overspill is increased). And herein lies the conundrum – the push to create detailed estimates for a project immediately negates the agile approach, which is about flexibility and the ability to pivot during the course of a project.
So what’s the answer? As I laid out in my workshop, I believe that the answer is honesty, and frank up-front discussions about the inherent fallibility of software development estimations. By highlighting the Cone of Uncertainty to your senior stakeholders at the start of a project, you are reminding them that even though a waterfall project may give them a more detailed plan and estimates produced at the start, the variance of those estimates is still likely to be very high. The trade off they make by adopting an agile approach is that while the overall project estimates may be less accurate at outset, the estimates produced for the iterative deliveries of work will have smaller variances (as they relate to smaller scopes) and they enable the project to gain from all of the other benefits of the agile approach – higher quality software, more closely aligned with business requirements. As an agile Business Analyst, I see it as part of my role to advocate for the latter, and evangelise on the benefits of the agile mindset.
Speaking from my own domain, as a consultant who works on projects for other companies, all of this can be a difficult ask of clients, especially from a commercial perspective. We need to remember that the attachment of senior stakeholders to plans which have detailed estimates is chiefly a product of the commercial landscape we live in, where company reporting and investment generation are so often linked to project reporting and forecasting. To adopt an agile approach where longer term estimates are less visible or have a higher stated level of potential variance is a matter of trust between the stakeholders and the project delivery team (which is often a separate entity or consultancy). Sadly, there is no shortcut to trust, so this is where it pays for consultancies to partner with companies who are open to new ways of thinking and for companies to select consultancies with strong track records for project delivery & quality.