Take a minute to contemplate experiences you have had navigating the traditional, strictly phase based approach to software development. Can you relate to the scenarios described below?
IT Project Manager: “We missed our sign-off deadline, because the Business did not fully complete the detailed requirements document for Technology sign-off.”
Product Manager: “The only part missing was exact verbiage for online help, which we will get from the Help Desk next week.Does this really need to impact sign off and starting design?”
IT Project Manager: “We have to cut this requirement from scope, because the Business did not complete the detailed requirement document by the milestone date for final IT review.”
Product Manager: “We met three other times with IT over the past month to review the requirement, and there were no questions. Then on the final day of review and sign-off, we received 37 questions. We can’t possibly track down all of these answers in one day.”
IT Project Manager: “If we build this page the way you requested in the requirement, we found that performance of the page loading will be very poor.I think we need to write a change request against the original requirement.”
Product Manager: “Both Business and IT signed off on the requirement.Why didn’t you bring this up sooner?”
IT Project Manager:“There was no way we could have predicted this before we got into high level design.”
While these examples may be extreme, most people involved in software development would agree that achieving sign off on detailed business requirements that Product Management believes will meet business goals, and that IT believes can actually be built given a multitude of constraints (i.e. budget, timeline, technical limitations) can be the most challenging activity on any project plan. In this project manager’s opinion, following a strict Waterfall methodology makes an already challenging activity even more difficult. In contrast, every program or project could benefit by incorporating some key Agile software development concepts into their software development processes, especially related to requirements definition and design.
Waterfall vs. Agile? Agile vs. Waterfall? Let’s start with definitions to make sure that we are all on the same page. Waterfall models refer to a sequential software development process, in which progress is seen as moving steadily downward through the phases from planning, analysis, design, build, test, etc. Waterfall models date back to the earliest days of software development and were adapted from operational processes of more traditional, highly structured physical environments, such as auto manufacturing. In these environments, changes later in the process were very expensive if not impossible. This is the basis for Waterfall’s value proposition, which is that one should only move to a new phase when its preceding phase is 100% completed and set in stone. Agile software development refers to methodologies based on iteration and overlapping phases where requirements evolve through collaboration between business and technology groups. Agile is a newer model adapted from modern operations management concepts, such as Lean Manufacturing and Six Sigma.
While there is certainly value in spending time early in the lifecycle to save even more time and money later, I believe that strict Waterfall methods should only be used in situations where mature, stable applications are undergoing maintenance releases. In “green field” situations where a new application is being built to achieve specific business objectives, strict adoption of Waterfall methods are less efficient, more expensive and take longer to implement than necessary for several reasons. First and foremost, there is no such thing as a perfect requirement. It is not possible to proceed in a purely sequential manner from completed requirements to design without first considering conceptual design and getting feedback and recommendations from a Technical Architect, Database Architect, UI Designer, etc. Does it make sense to complete detailed business requirements only to find out that your technology team is not able to build what you want? The time and money seemingly saved on the back end of the project by striving for perfect requirements often ends up lost in a quagmire of change requests to meet technical, budgetary or supportability constraints. Even worse, the requirements in question could make it all the way into the final product resulting in production issues. Recommendations regarding performance, supportability, scalability, etc. are not typically common knowledge amongst business people. My experience is that overlapping requirements and design phases actually leads to quicker, better designs, and a working application sooner, which is the ultimate goal, right? Another problem with strict Waterfall is that it lends itself better to larger, longer duration projects, which are becoming less and less ideal given the rapidly changing business environment in which we now work. Clients will no longer wait six to nine months for enhancements, when they can easily take their business to a competitor. Business opportunities that once could be capitalized on in a year now come and go in four to six months. Customers are more demanding every day, and Waterfall methodologies are simply too inflexible to adapt quickly and cheaply to changing customer realities. It does not do much good to successfully predict the outcome of a nine month project when the end result is no longer valued by the customer.
I have helped to deliver both quantitative and qualitative successes as a product of introducing some key concepts of Agile into traditional Waterfall development processes. A misconception of Agile is that rigor and documentation are thrown by the wayside, which I do not promote at all. What I do recommend is a hybrid model that blends overlapping phases, increased collaboration and earlier feedback from IT, and increased focus on working prototypes into the discipline of traditional Waterfall methods. In practice, one will find that these concepts can be incorporated into software development processes that still seem very Waterfall, and final documentation should not suffer from the perspective of requirements traceability or configuration management. As outlined in the previous paragraph, quantitative results in the overall project timeline and budget will be enjoyed, even if the milestone for final requirements sign-off is extended, because you achieve a completed design and working program more quickly with less time spent arguing over change requests and technical recommendations during design and build.
A more qualitative benefit of implementing a mixed software development methodology can be to improve the often strained working relationship between Business and IT. The nature of Waterfall causes us to focus too much on completed documentation, as opposed to a completed application. This can be seen any time a requirements document is referred to as a “Business deliverable,” or a detailed design is referred to as an “IT deliverable.” In reality, these are the “team’s deliverables,” but one particular group owns a higher percentage of the documentation responsibility. In most cases, a good requirement cannot be written without IT involvement, and a good design cannot be completed without Business involvement. To this end, Agile concepts help to promote increased teamwork and mutual accountability over all work products in the software development lifecycle.
The next time you find yourself embarking on a new project, take some time to consider adding some Agile concepts into your standard Waterfall development processes. My bet is that you will find positive results in both quality and time to delivery.
Scenario 3 (in a hybrid Waterfall / Agile software development project)
IT Project Manager: “After reviewing your first iteration of high level business requirements, we have some recommendations to make sure the data on the page renders as quickly as possible.”
Product Manager: “Sounds good.How will it impact my UI?”
IT Project Manager:“Should be minimal.Let’s complete the page prototype, and we can review it together early next week.”
For a methodology assessment, please contact Kenway Consulting at firstname.lastname@example.org.