From start to end: methodology

Showing posts with label methodology. Show all posts

Requirements in an agile world

In my last post, I wrote about differences between projects that use a traditional project methodology versus ones that employ a more agile one. In the post, Choose a methodology that suits your project, I spoke of when you should use one methodology over another one. Is there an impact on requirements from different methodologies?

Recall the main characteristics of good requirements:

These factors are essential to all requirements. However, when using different methodologies there are slight variances a business analyst should expect. The two pictures below show my thoughts on the key differences.

All business analysts should be aware of these differences and the impact to their deliverables.

Choose a methodology that suits your project

In some of my previous posts (The project and the business analyst & System development lifecycles & requirements) I wrote about different development methodologies that have been used for projects. These methodologies range from very process-oriented ones (e.g., Waterfall) to people-oriented ones (e.g., Agile.) One thought I would like to leave you with is that you should determine which methodology works best for your initiative.

On his blog, Harry Nieboer posted a write-up called On predicting the success of a project from the characteristics of its team members. The just of the post was that a team could make most projects work with any methodology and fail on any project with any methodology. He quotes Alistair Cockburn who concluded that, "...people's characteristics are a first-order success driver of projects!" I agree with this statement, for the most part. There are circumstances where I would favor an Agile approach versus a Waterfall model. And, there are times when I would favor the Waterfall (or other more process heavy) approaches.

Would you use an Agile approach to build an oil rig?

Let's suppose you are working on a software development project. You find that there are many different features that your client wants and that your client's needs are evolving rapidly. First off, you'll probably want to institute a change management process so you can bundle up work packages for your developers; but in terms of approaches, an Agile approach with small (e.g., 2-4 week) iterative cycles will allow you to show your client progress and give them a good look-and-feel. This approach encourages lots of feedback that will allow you to incorporate good ideas into the next iteration. After a few iterations you will be closer to meeting your client's needs. Note that, over time it may be necessary to revisit a component to modify it to work with another component that was developed later.

Alright, so what are the characteristics of an Agile approach:

Short iterative cycles.
Frequent client feedback.
Changes or evolution between iterations.
Modification of previous components so they will work with newer ones.

Under what conditions would you not want to use an Agile approach:

Suppose you are manufacturing a product (or in construction) and the different parts are made in different geographic locations but assembled at one place. I was watching the Discovery channel and they were showing a program documenting the construction of the Hibernia Oil Platform. They mentioned that some of the major compartments of the platform were constructed on 3 different continents and then shipped via boat. If the parts did not work together there would have been long delays, devastating cost overruns and (probably) lots of screaming & yelling.
You will not run into clients who will be constantly twiddling with different features and functions. There are relatively stable and predictable requirements that must be met.
The risk is too great if you don't know it upfront. Suppose you were an astronaut, would you want them to use a people-oriented process (e.g., Agile methodology) to build your moon rocket? Perhaps, but I'm sure you would express some concerns about your safety. Now if you were the NASA administrator signing the checks, would like this?

In concluding, I believe that some care should be taken when choosing an appropriate methodology for an initiative. Consider the costs associated with rework and the anticipated amount of evolution to requirements from clients as factors into your decision.

Make sure it makes sense

Believe nothing, no matter where you read it, or who said it, no matter if I have said it, unless it agrees with your own reason and your own common sense. (Prince Gautama Siddharta)

I've always been an advocate of applying common sense to one's life. It is usually the simple and eloquent solutions that work best. One thing I believe you should consider is the best way to invest your time. As with any investment, you want to get the best bang for your buck. Does the benefit associated with an activity outweigh its costs? What is the risk you are willing to bear?

For a large-scale complex project, complete traceability may be warranted. For a small budget, low risk project; is the same level of traceability required? I would suggest not.

Having a well defined methodology with structured processes and procedures is excellent. But you must be willing to be flexible when the situation warrants it. What works well under one set of circumstances will not necessarily work well in another. Use your common sense to guide you to make the optimal choice.

The project and the business analyst

A typical project's life cycle has the following stages: Discovery, Planning, Design, Build, Implement & Warranty. As a Business Analyst, depending on the stage of the development life cycle a project is in, you will have to support different types of activities.

Let's make some basic assumptions for the sake of this discussion:

The Business Analyst works on the project from inception to completion.
Ignore the different testing stages to keep everything simple.

Activities by phase

The chart below shows different activities that a Business Analyst may perform as part of a project. In general, there is more involvement in the first few stages of a project.

Let's investigate what each of these activities (in green) means.

Understand big picture: Understand the overall objectives and how the initiative correlates with business goals and strategies.
Work on the business case: Assist the project's sponsors to justify the need for the project. This may include high-level estimates of benefits.
Determine performance metrics: Projects are intended to accomplish something. This step is intended to provide you with something that you can use to perform a before-and-after examination. For example, did performance improve by the stated amount?
Gather high-level components: Understand the big moving pieces that compose the different pieces of the project. These pieces and sub-pieces will be given priorities in the next activity.
Prioritize components: Now that you understand the major components, estimate how much benefit is associated to each. Also, factor in natural dependencies between components to end up with a prioritization scheme that depicts the order in which items will be examined.
Gather low-level requirements: For the major components with the highest priority, delve deeper to identify the low-level requirements (e.g., functional requirements, specifications, process flows etc...)
Train end-users: Prior to deployment, train the end-users on the new solution.
Troubleshoot issues: After the project has gone live, be available to troubleshoot and provide support.
Hand-off to support: Prep the production support teams so they can eventually take over support for the new application.
Measure against metrics: After a sufficient amount of time has transpired, with the solution in production (e.g., end-users have ramped up appropriately), measure against the performance targets or metrics from activity 3.

Observations

In the beginning stages, a Business Analyst is trying to grasp the basic concepts of the project. Understand the what's and the why's and then delve down into the high priority items.
During the middle stages, a Business Analyst provides guidance and clarity to the design and development teams, ensuring that the team understands what the client needs and wants. Note that this does not imply providing design solutions to the design team!
In the final stages, the Business Analysts is performing a knowledge transfer to end-users and support staff. This is a precursor to the Business Analyst ending the engagement with the project.

Implications from Agile methodologies

The table below takes steps 4, 5 and 6 (Gather high-level components, Prioritize components & Gather low-level requirements) respectively and provides an example of what occurs in the real world. Before we get into the nitty-gritty, let's make the following assumptions to simply this discussion:

Each component can be broken down into groups of functionality called Requirement Sets.
Requirement sets are prioritized using their benefit as well as any natural dependencies they have.

Observations

A requirement set can be in a different stage than another requirement set. It all depends on its priority.
A Business Analyst will need to do different things based on the requirements set being worked on as it may be in a different phase than another set.

Summary

When using an Agile methodology, understand that groups of requirements will need different supporting activities because they will be in different stages of the system development life cycle.

System development lifecycles & requirements

One topic that I have not seen examined, is how using a different System Development Life Cycle (SDLC) methodology affects a Business Analyst's requirement gathering approach. There are many different methodologies that are used on projects. Some of the different SDLC models include:

If you view SDLC methodologies as a continuum, at one end of the spectrum you will find the waterfall model and at the other you will find the Agile approaches (the picture is based on some of the work done by a close friend.) As a Business Analyst it is very important to understand how your approach may change depending on the SDLC used. Of course, you must first speak with the project manager to clearly define and set expectations (as discussed in my previous posts: Set expectations & Time estimation.)

For a waterfall model you will want to get all of your requirements before you move to the next stage of the development life cycle (let's exclude change requests from this discussion.) Whereas when you are working on a project using an Agile methodology, you are providing discrete requirement sets based on their priority.

As a Business Analyst you must be flexible in your approach to conform to the SDLC that your project will use. When an Agile model is being employed, you must be willing and able to filter out everything that does not focus on the specific requirement set that has been chosen. For individuals that are used to a waterfall approach, letting go can be a difficult mental hurdle to cross.

Defining requirements by priority

Or how do you know where to start?

For a large number of my posts I have written guidelines to help you write better requirements. One thing that I have not covered is how you can tell which requirements you should concentrate on.

If you refer back to my post How do you know you are done? you will find a subtle hint. Recall that I suggested that you get a handle on the basic overview of what a project is trying to accomplish.

When using iterative or Agile development methodologies you group together smaller pieces of functionality into work packages or releases. The order in which items get into work packages is usually based on a combination of anticipated benefits (ROI with NPV) and dependencies.

Why not use this same methodology to prioritize the order in which different areas of the overall requirements' set will be examined? This is why I say you need to understand the 'big moving parts' (as mentioned in a previous post.) Each of these 'big moving parts' (e.g., capabilities) has a defined benefit to it. Each capability can then be decomposed into more discrete sections.

The picture below outlines the basic principles of this exercise.

Once again, each of these sections can be assigned a priority (based on natural dependencies and the section's contribution to the benefit of the capability) After this activity has been done, a business analyst (e.g., you) can then start exploring the sections based on their priority. This yields a few advantages:

You focus on the most important pieces (e.g., highest priority.)
You reduce the amount of time spent on unimportant pieces. I know this sounds like the first item but I want to emphasize something. You can use this reasoning to control your requirement gathering sessions. When your clients, because of their attention to detail and natural excitement, venture into areas outside of your area of focus you will have the ammunition to refocus them. You can politely state that you want to park their current ideas and refocus back on the pertinent requirements' section.
You will have an idea about how much is left to do.

You may think, you could implement this methodology to help prioritize everything as you delve deeper and deeper into the most atomic requirements. Would I suggest doing it?

My answer is a polite, "No." Understanding how much benefit an atomic low-level requirement contributes to the overall project may be useful information, but I feel the amount of effort needed to maintain this data far outweighs any benefits that can be gained. You need to find a happy medium otherwise eventually your productivity will hit the law of diminishing returns. Furthermore, for low-level requirements, it may be difficult to say how much benefit a specific feature is providing.

Other related thoughts

Think about how the previous picture would change if a waterfall-type of approach was used rather than an iterative one.

How do you know you are done?

A while back I wrote a series of posts on requirement do's and don'ts. I would like to revisit two of the themes from that series (Bad Practices - Part III - Escapes, Rambling & Mixing and Good Requirements - Part VI - Traceability & Modularity) to provide more of an end-to-end perspective on how you know when have enough to move to the next step.

If you look back at those two posts you will notice that I mentioned that traceability is very important as you move from a higher level requirement to a lower, more specific one. Furthermore, I cautioned against mixing high-level requirements with low-level requirements. To show you my reasoning, I would like to step back and examine the rationale behind the different parts of a project.

A project's steps

Please note that this is an abstraction and it makes a few assumptions such as the use of an iterative (e.g., Agile) methodology and the existence of a project with a good return.

Identify the business problem or opportunity.
Make sure it aligns with your business' goals and objectives.
Quantify the key performance metrics.
Measure how you are doing and have done historically (if you are enhancing a pre-existing behavior.)
Determine the major business capabilities that would address the problem (or capitalize on the opportunity.) This provides a basic overview of the big moving parts.
Determine the success criteria. What will happen if you do it right? This should tie back to the key performance metrics.
Estimate the value of each business capability (e.g., how much benefit would you get from it.) Most people equate this to ROI, however you should make sure that you account for the risk of the venture and the timing at which benefits would be realized. A project with a higher amount of risk should use a larger factor to discount the expected benefits. A project whose benefits will be realized far into the future should have those benefits discounted more than a project whose benefits will be realized earlier. After all, a million dollars today is worth more than a million dollars five years from today. This concept is called net present value or NPV. Tyner Blain has a great post on calculating ROI using NPV.
Prioritize the major business capabilities using a combination of their business value, dependencies and other prioritization criteria (e.g., legislative deadlines.)
Examine the highest value business capability and break it apart into reasonable pieces that can be investigated.
For each piece, define the key requirements keeping in mind design independence.
Determine which aspects require process changes, software solutions etc... Basically, do some solution architecture / design work. What can you reuse? What can you buy? What do you need to build?
Estimate a cost for realizing the design of the piece.
Assign a proportion of the high-level capability's benefit valuation to each piece.
Prioritize the pieces based on cost-benefit and dependencies.
Work with end-users to define the layouts, workflows and characteristics associated with the piece with the highest priority.
Start an iterative development cycle for this piece. You can also concurrently examine the next high priority capability.
Interact with your end-users to show progress, development and to keep them engaged with the project.

Observations

Cost-benefit valuations are used to assist the prioritization of components to analyze and develop.
Break things up into digestible chunks but know how the chunks fit into the grand scheme of things.
More detailed requirements were prepared, but the order in which they were gathered matched the priority associated with the component. Do not waste time on things that have low value.
Pieces that have low or negative cost-benefit may not be developed at all! The interactions with business users were deliberately setup to focus on the level of abstraction that was desired. Initially, only the big picture things were desired. Then the next level for the most important component. And finally the detailed requirements for the highest priority piece of that component.
Engagement between business and IT users promotes joint accountability for success and failure.
I have purposely avoided using names such as functional specifications, business requirements, user requirements, functional requirements and the like because I do not feel the name is what is important. A name may imply something different to another person. The important thing is the activity which transpires.

Other resources

If you are interested in this topic here are some other blog postings that you may want to check out.

Requirements vs design - which is which and why? from Tyner Blain
MRDs, PRDs, FRSes, and SRSes from Roger Cauvin
Requirements document proliferation from Tyner Blain

What are requirements? SDLC?

Functional Requirements

In order to understand what requirements are, it is necessary to know why one needs them. Basically, requirements are statements that indicate what a system needs to do in order to provide a capability (i.e. utility or benefit.) Requirements are generally prepared during the early stages of a project's system development lifecycle (SDLC.)

There are many different SDLC methodologies that are used in practice. They range from heavy-weight ones, such as the Rational Unified Process (RUP), Waterfall and Spiral methodologies to more agile methodologies such as Extreme Programming (XP), SCRUM and Crystal. Regardless of the methodology employed there are generally six basic phases (note the descriptions oversimplify the activities that occur during each phase):

(1) Discovery - "What needs to be build?" Where the bulk of the analysis (requirement gathering) occurs.

(2) Planning - Perform planning to determine the tools, procedures, budget and resourcing needs.

(3) Design - "How are we going to get a solution?"

(4) Build - Develop the solution.

(5) Implement - Deploy the solution into production.

(6) Warrantee - Monitor the solution to ensure that it functions properly, is reliable and predictable.

Requirements describe what users want a system to be able to do. Thus, well defined requirements are critical to the success of a project.

A large majority of all software defects and failures are directly attributed to bad requirements!

In practice, the percentage of errors during a project's SDLC are higher during the analysis phases than the later stages. Furthermore, the cost to fix an error increases the later the error is discovered. Put another way, the cost to fix an error during the implementation stage is several magnitudes larger than the cost to fix the same error during the analysis stage.

I have seen many different names used for requirements documentation including: business requirements, user requirements, functional requirements etc... Individual organizations will use these terms differently. Personally, I view the disctintion between different types of functional requirements to be based on the level of detail provided.

For example, a high-level (business) requirement may be, "The ability to maintain a product catalog." A lower-level requirement (functional specification), would be, "The ability to specify a date range dictating when a product is available." A good practice is to avoid mixing different types of requirements together in the same document.

Non-Functional Requirements

Most of this post deals with functional requirements. There is another type known fittingly as non-functional requirements. Non-functional requirements describe system characteristics such as:

(1) Productivity - How much a system aids users? For example, "We used to be able to handle 10 complaints in an hour and now we can handle 12."

(2) Performance - How fast should the system be?

(3) Capacity - How much traffic must the system be able to handle?

(4) Scalability - How easily can the system be expanded to handle increased throughput?

(5) Availabilty - Does the system need to be 24/7, weekdays only etc...?

(6) Recoverability - How quickly should the system be made operational after an outage?

(7) Integrity - How predictable and reliable should the system be?

(8) Exception handling - How will processing exceptions be handled?

(9) Logging - Do we need to have audit trails of activities?

(10) Security - What are the security characteristics?

(11) Manageability - How easy is it to manage the system?

(12) Useability - What interfacing is required for end-users?

(13) Interoperability - Does the system need to be able to interface with other systems? How?

(14) Extensibility - Can the system be expanded to support new functionality?

(15) Maintainability - How readily can the system be understood? How extensive is the documentation.

(16) Upgradeability - How easy is it to upgrade the system?