Thursday, 30 January 2014

Cost of manual VS automated Releasing in Scrum (Agile)

Due to the nature of Scrum, which is an iterative and incremental process, it is required to complete the user stories by the end of the so-called iteration (also known as Sprint). In theory, at the end of a sprint, a shippable product is made available. It can be released to production. This means the software has passed successfully all the quality gates in place at that time. 

However, in order to complete and effectively close the User stories, the features (or improvements or bug fixes) need to pass all the tests. On top of this, the team must know how to release that software to Production (or to the client).

In other words, and the end of the Sprint, the software can only be:
  • Potentially deployed
  • Potentially packaged
  • Potentially released
  • Fully tested against the acceptance criteria (whether it’s manual or automated testing, the latest being the recommended option).
In order to achieve releasable and working software, we can consider two options:
  1. Finish the sprint and test/release after the Sprint (deviation from the Scrum process)
  2. Test and release during the Sprint with the goal to complete those tasks by the end of the Sprint (typical Scrum process)


The first option implicates that the features won’t be tested before the end of the sprint but rather after that. The team will not be able to close the user stories immediately. This is an alternative if the deployment and testing process is not automated and consumes some resources from the Scrum team.
During this transition phase, between two Sprints, the developers and testers will work together to discovering bugs, fixing them, deploying a new version and so forth (until acceptance criteria are met). This process is represented in the diagram below:

Note that the subsequent releases might take longer as the amount of non-regression tests increases (the system comprises of more and more features). Although the manual release process is well known, the testing might take more and more time: 
We can identify some issues with this process:
  • The sprint cannot be closed, as the user stories have not been tested. Therefore the software is not shippable at the end of the work iteration.
  • Issues and bug will be discovered late after the code was written, increasing the time needed to fix it
  • The team is busy doing manual deployment and fixing bug rather than working on the next Sprint
  • The release process takes longer as many more features are added and need to be tested again over and over again.
A better solution is to automate both the automation of testing and the deployment process. This is a required practice of Continuous Delivery.


Once the tests and deployment process have been fully automated, it is possible to continuously deploy and test as soon as the code gets changed. Developers can therefore focus on designing and implementing features. Stakeholders, product owners and testers will have access to the latest version in a Testing environment, soon after the code has been written.
That way, the deployment and testing are embedded in the sprint. See below:
The benefits of this solution are:
  • Bugs are discovered much earlier and easier to fix
  • People are busy working on value added work (designing, coding) rather than manually deploying
  • Repetitive tasks are automated, reducing the risk of errors
  • A shippable product is available at the end of sprint. It might or might not include all the new features, depending whether the team had done a good job at estimating and scoping the Sprint backlog.
The drawbacks are:
  • The automation of the testing and deployment process requires an investment in time and energy. As a result, team will have to acquire new skills for that purpose.
  • The amount of work achievable during each sprint will be less, taking into account the effort to automate the tasks
  • There is still no guarantee that all the features will be completed, tested and shippable by the end of the Sprint. We can only ensure we have the most efficient process to achieve the same.


Due to the nature of the Iterative and Incremental approach of most Agile process (including Scrum), it is necessary to automate the testing and deployment tasks, in order to deliver shippable product at the end of each iteration (Sprint).
Then only the business is able to predict what will be delivered at the end of the next iteration, while keeping the cost of releasing and testing low, even in the long run.

No comments:

Post a Comment