Software Development Process Improved with BPM
Our world is driven by software. It resides just under the surface of so much of what we touch. We take it for granted. And so we often take its development for granted too. We fail to realize the simplest software supports the broadest applications such that any error’s impact is greatly magnified. Yet, while the impact can be wide and magnified, the opportunity to avoid the error often lies in the simplest of concepts. We experienced the magnified effect of simple software development process errors in an everyday software application on a recent project related to ATM machines.
During a routine cost analysis we realized that once a particular defect reached the customer on our ATM, it cost about $750,000 to fix. Further, we averaged 20 new defects a month of which one third were repeat offenses coming through for the second and even the third time. The cost was $15 million per month. In fact, at one of our major clients, which had 25,000 ATMs running our software, the Cost of Poor Quality was nearly $200 million per year not counting the repeated defects. We immediately set out to reduce the number of defects by 50% with zero tolerance for repeat defects.
As in any good project, we began by characterizing the “As Is” state. When mapping the software development process we found the software was being developed in three different geographical areas by three different groups. The device drivers were being developed by one group, the base software was being developed by other group and the customized software package was being developed and integrated by a third group. Each group was responsible for their own unit testing and quality assurance and, on paper, this worked efficiently.
However, we found each group used a different defect tracker and project management methodologies. In fact, one group did not use any project management methodology. In addition, each group had different metrics and different quality assurance. We also discovered there was no solution testing being done until the final stage. Even testing was flawed as the device drivers and base software were just being unit tested and not solution tested.
Since metrics, processes, methodologies and testing were different and flawed, the processes weren’t stable and therefore success was not predictable or repeatable. Some projects worked well while others did not. And none of the groups could improve by learning from mistakes.
Download “Business Process Management: A Structured Approach…”
After a Kaizen event it was determined to do three things:
- Change the process to have each area base solution test the software before release.
- Have all three groups use the same defect tracker.
- Have all three groups use the same project methodology.
These changes ensured repeatable success for all three groups. Each group was able to show project management metrics and be accountable to each other for repeated success.
After several months we began measuring the defects per month. The mean defects per month declined from 20 to just under 8, or nearly a 60% decrease, which was worth over $100 million in annual cost savings. And thanks to the single defect tracker we adopted, the repeatable defects went from about 7 a month to 3 for the year.
We also setup up a PMO for all three groups utilizing project managers. This not only added continuity to the projects but we were able to show repeated success and achieve a CMMI Level 2 in a few months.
As I stated at the outset of this piece, software supports an incredible portion of our daily lives. It is invisible and unnoticeable – until something doesn’t work. Its development is sometimes thought of more as a craft done by very talented and valuable personnel than an overall system or process. But when the science is eliminated from what is an overall system, errors occur and due to the highly repeatable aspects of their usage, the costs can be very high. We now use the most basic process tools to make our software development repeatable processes that are regularly measured and improved. If you’d like to discuss this post, please contact me