Apr 1, 2009 / TargetProcess Announced v.2.14 Release With Full Waterfall Support

0 comments

I am really glad that finally we released v.2.14. Check the press release: TargetProcess Announced v.2.14 Release With Full Waterfall Support.

Labels: agile tool, waterfall

posted by Michael Dubakov @ 4:30 AM

Sep 19, 2008 / Waterfall got you down with "multitasking"? It's time to take an "Agile" medicine!

1 comments

Everybody knows that multitasking is bad in general. Time is wasted due to context switching and more errors due to insufficient attention. There are many opinions and articles about multitasking problems and general advice is to eliminate it when possible. However, it is required to define what the task is in multitasking before making any conclusions. For example, is it possible to consider waterfall a multitasking process?

For simplicity purposes, let’s assume that our development process has 4 stages: specification, design, implementation, testing. We have a bunch of requirements that may be implemented in the project. In the design stage we focus on the system design only. We think about objects, patterns, communication, API, etc. Looks like quite a focused activity, doesn’t? However, at the same time we are designing many features. We are trying to consider them all and create a design that supports all of the features at once.

There are two facets to this problem:

Activity-based:
From this point of view we have a single focused task - system design.

Feature-based:
Here, we have many tasks (design feature 1, design feature 2, etc.).

What is a better view? I think it is required to focus on delivery. In the end our ultimate goal is to ship working software with a set of features. We do not ship system design, requirements documents or acceptance tests. We ship functionality (features). Feature that solves a real problem is the most important thing in software development. That is what we produce. We should focus on features (that solve real problems in a most efficient way). We should not focus on activities; they are just a part of development process. If you can release working bug-free software without testing stage - that is perfect. If you can release it without design stage - fantastic. Following common/defined/approved/considered-best process is not a rule of thumb. If you can release working, usable, high quality, efficient software without any formal process - that is fine. I am not trying to diminish the importance of processes in the software engineering, but too many people developing software using waterfall, loose focus on their ultimate goal – to ship a product that user wants!

It is better to consider feature as a task (in a multitasking term, it means that multitasking and “multi-feature” are interchangeable). Having this in hands we can say that waterfall is a kind of multitasking. It leads to many features in production cycle, thus eliminating focus. Development team tries to follow defined stages with the best possible output, but the goal is often blurred. People focus on specification, design, implementation, testing, but often they loose focus on features. Why? Well, two reasons: context switching and insufficient attention.

For example, in the design stage you have many features to design and can't prove all design concepts for all of them. During implementation stage you face many problems with wrong initial design decisions and have to change and adapt design quickly. That is natural, since design can be proved only during the implementation, there is no way to get it right the first time. However, such quick decisions may be uncontrolled and uncoordinated. They ruin the original fancy design. Insufficient attention? Yes, since it is impossible to design and prove ALL the features during the design stage.

The same is true for the testing phase where we have many features to test. We can't focus on one feature, we feel that there are many more features to test and most likely will not complete full testing of one feature. We feel that it is better to test all features briefly than test some features deeply. We briefly test feature A, then jump and briefly test feature B hoping to get back to it later and complete the testing. We have many features to test causing insufficient attention. Often under time pressure you can't get back to briefly tested features, and that leads to bugs in post-production period. Large testing stage blurs focus on features testing. It may sound strange, since it is the testing stage after all - but think about it once again.

Iterative development (especially using short iterations) solves these problems in the roots. You have few features in the production and may focus on them. It is easy to fully test several features. It is quite easy to design and prove the design right away (in waterfall you may have several months gap between feature design and implementation). You have a clear focus by not having to switch to other features outside of the iteration scope.

Iterative development is a vertical slicing. You implement "by Feature". Waterfall development is a horizontal slicing. You implement "by Activity". What do you prefer? ;)

Labels: agile, agile medicine, iterative, multitasking, waterfall

posted by Michael Dubakov @ 3:59 AM

Jan 31, 2006 / Why MS Project Sucks for Iterative Development? Part II

3 comments

Hammer and Warm Wax

See Part I

.

Let's take an average software project with full plan created up-front. It appeast that it is hard to support project plan in actual state using MS Project. It is hard to reflect real progress. But why? MS Project is great software, it should be easy. And it is easy in many cases, but not in software development. MS Project is useful when:

1. Project is very small (a month or two). But in this case maybe it is not required. Use Excel, make simple features list with estimates and that is it.
2. Project is very large and requires high-level plan. You may create very good project plan for Bridge building, Airplane or Space Shuttle construction. PERT is great in this case and project plan reflects phases, not tasks.

But most software projects have specifics:

• Average project duration is several months
• They change a lot and require fine-grained plans ("This feature will be implemented by two people. Joe will create GUI screen within 1d and Matt will program controller within 2d").
• As a result, it is very hard to keep fine-grained plan in actual state for medium and large projects for WHOLE project life-cycle (the keyword is WHOLE of course).

No matter what tool you are trying to use. Most managers and executives improperly think that Waterfall requires such detailed plan somewhere in the beginning. Waterfall is not the best way for software development, but incorrectly understood and implemented it becomes a real pain in the neck.

You may say that MS Project is just a tool and all I wrote above do not directly depend on MS Project, but on methodology and process. I partially agree, but, once again, MS Project was not designed to support anything but Waterfall or its modifications. It is used for Waterfall and not so many people tried to use it for something else. MS Project + Waterfall = Hammer, but software development in most cases is not a nail, it is a warm wax. You can't make anything but a flat from wax using a hammer.

As you see, there are plenty more reasons to find a better way. Let's step back and look at another process.

Iterative Development

If we can't have fine-grained plan for whole medium/large software project, let's split it on chunks and create fine-grained plan for each chunk when it will be necessary. This idea is not new and called Iterative Development. Whole project split on iterations (exact iteration duration depends on many factors, but usually it varies from 1 week to 3 months) and iteration planned just before start.

This may sound astonishing for some executives. "So we don't know project end date??? We don't have a plan??? Our customers will not deal with our company! What the hell you are talking me?" How can anyone answer these questions? There are two ways to go.

On the first road there is no exact project end date and this road called Customer Satisfaction. You respect them, trust them and produce software they want. Feature by feature, iteration by iteration. Exact date of final release is not required since customer knows that you'll implement as many features as possible based on defined priorities. It is real trust and it is quite rare.

On another road there are many signs with a big red "Deadline" in the end. While everyone wants to be on Customer Satisfaction road, most of us are on Deadline road in fact. This is a real world and, while we changing it, we have to drive some projects on this road. Can we apply iterative development? Yes, but with some restrictions, since there is less trust between developer and customer. Customer expects a fancy project plan, so we will create one, but low-grained. MS Project is good for low-grained plans. The plan will contain project phases and major milestones and, of course, it will contain end date. If customer asks why there are no tasks, you answer that you don't like micro-management and fine-grained plan will be developed later. In fact, customer doesn't care about exact plan, it does care about delivery dates and you've already provided them. Then you may break down each release on iterations and follow first road.

Some off topic. The other trick is to leave project/release scope flexible and this is really hard to achieve. It is harder than make a sculpture from warm wax with a hammer. If customer insists that he should get these features till that date without any deviations - you are in trouble. All you may vary in this case are people and process. And often you can't vary people as well. So the process becomes a single life-saver for you and your team. While it is powerful life-saver, it may not help. Once again, do your best to leave scope or date flexible and focus on Business Value whenever it possible. If not, use iterative development anyway and try to release something useful as early as possible. Maybe customer will appreciate your effort, trust you and you slowly turn to Customer Satisfaction road.

"OK, we may use MS Project for low-grained plans. But what should we use for fine-grained plans?"

I can't point Best-of-the-Best solution, but some important things that your tool of choice should do:

• Fine-grained plan for iteration should be visible and understandable for all developers without any restrictions
• Person who performs a task should track time spent on the task. PM shouldn't ask everyone how many times he/she spent on this and that.
• Plan should be easy-to-change (no dependencies, easy re-allocation)
• Iteration status should be visible for all (and preferably tracked automatically)
• Iterative planning should be supported by the tool

In short, the tool should be opposite to MS Project, it should be strong in those areas where MS Project becomes frustrating.

Stop! I Can Plan Iterations in MS Project!

Yes, go ahead and try. You'll quickly discover how it is useful for iterative development and how it is designed for iterative development.

I've planed iterations in MS Project for several projects and it somewhat worked, but this method lacks visibility. Developers don't see plan and don't see progress. You should update actual effort manually and assign tasks using Outlook.

How many features in MS Project you will use for Iterative Development? Gantt Chart, Resources and Tasks. That's it. Tasks will be independent, so critical path is useless and Gantt chart is not very helpful. Leveling… OK, sometimes it helps, but in most cases conflicts may be resolved very easily for one iteration. Reports? Useless in most cases, but you may need something for executives. It appears, that MS Project is too complex and don't bring enough value to justify its usage in iterative planning.

On the other hand, there are many features that you'd better have for iterative planning:

• Product/Release backlog. The main issue. If you use MS Project, you will have to store backlog in Excel or somewhere else and copy/paste features when required.
• Integrated time tracking. May be resolved programmatically and with help of MS Project Server, but this is an additional effort and MSP Server used not so often.
• Work remaining tracking. This practice is very useful to know actual iteration state. Can be resolved the same way as previous item.
• Iteration velocity concept. If you create an iteration in MS Project, Effort will be Iteration Velocity in fact. But you don't have clear velocity progress picture

So you really may plan iterations in MS Project, but it will be not so simple and natural. MS Project just will not help you much.

Labels: agile, ms project, tool, waterfall

posted by Michael Dubakov @ 1:41 PM