One of the primary examples used to describe the Scrum methodology is the failure of the FBI modernization project, cited as one of the biggest software failures of all time. With the first attempt costing $170 million. The second attempt, this time lead by Lockheed Martin cost $451 million and was again an unmitigated disaster. So why had it failed the second time? Lockheed Martin is an intelligent, capable company. However, before the software development started, the engineers had planned out every detail with waterfall and Gantt charts. In reality, these Gantt charts were complete fabrication of the actual software development process, and don’t account for mistakes or overruns. The effort in this type of project management in trying to
control and restrict change and to
know the unknowable before a project starts are a hinderance to actual development. The project management with Gantt charts created a layer of abstraction from management to the developers, and some employees entire job was to manage the charts, when the project wasn’t going to plan, the Gantt charts were being used to lie about the state of development, creating conflict.
Scrum embraces this uncertainty and involves human creativity, and surprisingly the third attempt at modernizing the FBI system was a success after they implemented Scrum. The project was brought in-house, the number of contract employees dropped from 220 to only 40, while the number of FBI employees dropped from 32 to only 12. The project would also be completed in 12 months using only the remaining $20 million in the budget, and after some initial delays the database software was deployed in 18 months, and another two months later it was rolled out to the entire FBI.
This example from Scrum shows the costly mistake of trying to abstract away too much from the human-centred process. Scrum places structure around the learning process, in such a way that you can fail quickly and learn from mistakes. This approach allows real time feedback, regarding failure this allows you to check in after each sprint see what’s working, instead of the costly mistakes made by the FBI. (Sutherland, 2015, p. 1-23)
This iterative approach in Scrum is called an Action Loop, which is useful for testing ideas and learning from mistakes. In the US military this is called After Action Reviews (AAR) which emerged in the Gulf War and used before and after missions to continuously review and identify possible improvements, this allows tighter integration between thinking and action (Sastry & Penn, 2014, p. 114).
In Scrum this is called the Observe – Orient – Decide – Act cycle. Lean methodology also follows a similar feedback loop, called the Build – Measure – Learn cycle (Reis, 2011).
Failing early seems valuable because if a project’s success relies on assumptions or unknowns, then validating these hypotheses allows for more transparency and less abstraction away from the project. In software development, A/B testing has become a method of validation which splits users between two different versions of the product or website and gathers metrics in order to determine which version is performing best.
Regarding our own teams project management methodology: after a successful meeting with an industry contact the team was invited to participant in an event at The Eden Project, with a deadline set we didn’t have much time and decided to work with Lean and Agile methodologies to create a minimum viable product with only one feature, which would act as our entry into the market where we could start building a community and gathering feedback on what the community wanted as a product.
We felt this was best as it would be easy to pivot later if we already had an established community and user-base, we would allow ourselves to be flexible. In the previous journal I commented on why flexibility is important: “In contrast to normal scientists who are risk adverse, start-ups are well designed to handle the risk of new ventures and innovations. Due to the small size it makes inter-team communication easier, as the team is bound together by a shared vision they have a large amount of trust, all of which provides a large amount of flexibility—to adapt to the situation and problems as they arise. (Bessant & Tidd, 2015, p. 24)” The team knew our time at Launchpad was a valuable opportunity to experiment and test ideas, so we jumped ahead and started building our first feature – and our idea was validated by others working in this industry at similar companies. We wanted to work together and partner with them instead of competing, which would again help build our community.
However, a potential trap exists in this thinking because as Herny Ford had famously stated, customers would have wanted a faster horse if he asked them... it's not always the best advice to work off customer feedback, but as we knew getting initial users and creating a community would be important so we thought it would be valuable to launch this product at the Eden event, and quickly get user feedback and validate our idea with real users.
One of the main issues we encounter during this minimum viable product sprint, was the transparency of the programming work being done. The team has previously been using Trello as our Scrum / Kanban board and had been using high level tasks to describe our sprint goals, however this didn't translate well to software architecture, as most tasks like setting up a database or a user authentication system are hard to validate until they are functioning and the website is available on the internet for the team to view. This aspect of transparency relates back well to the Action Loop and the need for clear observation of the situation when you are trying to orient yourself, the clearer the territory is the better you can judge your relative position.
We decided to continue using Trello for our business goals, and switch to Asana for our programming goals, and our 5 cards on Trello translated to over 100 cards on Asana. This allowed the team to see that progress was being made even if no product was visible yet. With advice from Launchpad facilitators, we also switched to user-focused thin verticals, for example:
Before: Users can follow each other
After: As a user when I visit someone's profile, I can see a button under the profile picture where I can start following the user.
This allowed the team to understand what each task did, while abstract away the complexity of the software development process – and Asana allows sub-tasks where I can keep track of the software complexity.
Agile methodologies is very adaptive and I'm sure we will learn a lot more from applying it to our projects and learning from how the team works together.