Sep 25, 2008 / Agile Software Tool will not help you to understand Agile

0 comments

As you maybe know we at TargetProcess develop agile project management software. We have a free community edition that is becoming popular and have about 1300 downloads already. People who request the free edition or trial provide comments. Sometimes these comments really disappoint me. Especially if I see comment like "I hope this community release help me dive into scrum. Thanks."

What's wrong with it? Well, I don't know any software tool that may help to understand agile process better. If a company starts SCRUM implementation or any other agile process implementation we do not recommend to start with a tool. Hire agile coaches, read books, try simplest tools like whiteboards and sticky notes, visit conferences, try best practices, and learn! Do not invest into any software tool till you feel the need.

The team may be distributed, or large, or needs real-time progress reports desperately, or anything else. But the team should try to operate with simplest tools first. And focus on live communication. It is the only way to "feel" agile spirit. It is the only way to dive into agile.

Do not rely on a tool. It may help to resolve some problems. It may even help a lot if you have distributed team. But it may dictate or recommend some best practices, project structure, process structure and the way how you are developing software. That is something to avoid in the beginning of agile adoption. The team should invent own best practices, own project structure, own process. With this in hands the team may decide to use a tool to resolve real discovered problems.

Agile PM tool is not a silver bullet after all. Do not rely on it blindly. It should adopt to your development process, not vise versa.

Labels: agile, agile tool, tool

posted by Michael Dubakov @ 4:32 AM

Sep 19, 2008 / Waterfall got you down with "multitasking"? It's time to take an "Agile" medicine!

1 comments

Everybody knows that multitasking is bad in general. Time is wasted due to context switching and more errors due to insufficient attention. There are many opinions and articles about multitasking problems and general advice is to eliminate it when possible. However, it is required to define what the task is in multitasking before making any conclusions. For example, is it possible to consider waterfall a multitasking process?

For simplicity purposes, let’s assume that our development process has 4 stages: specification, design, implementation, testing. We have a bunch of requirements that may be implemented in the project. In the design stage we focus on the system design only. We think about objects, patterns, communication, API, etc. Looks like quite a focused activity, doesn’t? However, at the same time we are designing many features. We are trying to consider them all and create a design that supports all of the features at once.

There are two facets to this problem:

Activity-based:
From this point of view we have a single focused task - system design.

Feature-based:
Here, we have many tasks (design feature 1, design feature 2, etc.).

What is a better view? I think it is required to focus on delivery. In the end our ultimate goal is to ship working software with a set of features. We do not ship system design, requirements documents or acceptance tests. We ship functionality (features). Feature that solves a real problem is the most important thing in software development. That is what we produce. We should focus on features (that solve real problems in a most efficient way). We should not focus on activities; they are just a part of development process. If you can release working bug-free software without testing stage - that is perfect. If you can release it without design stage - fantastic. Following common/defined/approved/considered-best process is not a rule of thumb. If you can release working, usable, high quality, efficient software without any formal process - that is fine. I am not trying to diminish the importance of processes in the software engineering, but too many people developing software using waterfall, loose focus on their ultimate goal – to ship a product that user wants!

It is better to consider feature as a task (in a multitasking term, it means that multitasking and “multi-feature” are interchangeable). Having this in hands we can say that waterfall is a kind of multitasking. It leads to many features in production cycle, thus eliminating focus. Development team tries to follow defined stages with the best possible output, but the goal is often blurred. People focus on specification, design, implementation, testing, but often they loose focus on features. Why? Well, two reasons: context switching and insufficient attention.

For example, in the design stage you have many features to design and can't prove all design concepts for all of them. During implementation stage you face many problems with wrong initial design decisions and have to change and adapt design quickly. That is natural, since design can be proved only during the implementation, there is no way to get it right the first time. However, such quick decisions may be uncontrolled and uncoordinated. They ruin the original fancy design. Insufficient attention? Yes, since it is impossible to design and prove ALL the features during the design stage.

The same is true for the testing phase where we have many features to test. We can't focus on one feature, we feel that there are many more features to test and most likely will not complete full testing of one feature. We feel that it is better to test all features briefly than test some features deeply. We briefly test feature A, then jump and briefly test feature B hoping to get back to it later and complete the testing. We have many features to test causing insufficient attention. Often under time pressure you can't get back to briefly tested features, and that leads to bugs in post-production period. Large testing stage blurs focus on features testing. It may sound strange, since it is the testing stage after all - but think about it once again.

Iterative development (especially using short iterations) solves these problems in the roots. You have few features in the production and may focus on them. It is easy to fully test several features. It is quite easy to design and prove the design right away (in waterfall you may have several months gap between feature design and implementation). You have a clear focus by not having to switch to other features outside of the iteration scope.

Iterative development is a vertical slicing. You implement "by Feature". Waterfall development is a horizontal slicing. You implement "by Activity". What do you prefer? ;)

Labels: agile, agile medicine, iterative, multitasking, waterfall

posted by Michael Dubakov @ 3:59 AM

Sep 11, 2008 / Software Development Diseases and Agile Medicine

3 comments

It is really interesting to check some established diseases in software development and investigate how they can be resolved using agile processes. If agile development is so good, it should provide solutions to common problems. Here we go.

Student Syndrome

Many people will start to fully apply themselves to a task just at the last possible moment before a deadline.

Maybe the most famous disease in software development teams. You have 40 hrs task and thoughts like "Huh, one week! I am pretty sure I will complete this task in 2 days!" You are doing something fun like refactoring or research and start working on the task Wednesday's afternoon. Suddenly you encounter a serious problem. You spent all day hunting for resolution with no luck. In a weekly status meeting you say that task was not done and 80% ready, but there is completely unexpected nasty problem that popped up and broke the plans.

Looks familiar? Yes, it is. Agile development solves the problem with several simple practices:

1. Daily Meetings. You have to say something on a daily meeting. If you committed to the task on Monday and spent all Monday researching something completely unrelated you will have bad feeling during Stand-Up. It pushes you to actually start working on task. Also serious problems will not be under the carpet for days, they will be revealed and expressed.
2. Short Iterations. In short iterations every single task is important. If you have 1 week iteration you are feeling that deadline is close enough and you have no spare time. If you have 6 months project deadline in waterfall process you have an eternity. You can do wrong things for weeks till the reality hits you in the neck.
3. Short Tasks. Two days is a good maximum for a single task. It is quite hard to convince yourself that you can start coding tomorrow if you have only two days to complete the task.

Parkinson's Law

Work expands so as to fill the time available for its completion.

Indeed it is a matter of perfection and laziness. Some people may complete the task earlier and have spare time for other things. Some people will complete the task earlier and polish the solution till it shines. Some people will digress from the task and complete it on time. All attitudes may be dangerous. The result is that a whole project delayed.

Agile approach has some medicine for this disease as well:

1. Definition of Done. The task is done when it passes all assertions in DoD. The goal is to have clear, unambiguous definition. People should know when it is ready. The phrase "it will be done when it's done" should have clear meaning behind.
2. Iteration Goals. Clear iteration goals help people to focus on getting things done.
3. Short Tasks. Again short tasks may help here. In short task it harder (but still possible) to add long safety buffer.

Conway's Law

Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.

Maybe it looks quite weird, but this law has great application in software development. If you have two modules that interact via API, you hardly create good API unless developers of the both modules communicate together. It leads to some conclusions:

• Communication between developers is very important for good design.
• Distributed teams may hurt software design in case of poor communication (and it is harder in distributed teams).
• Software can't be produced in isolation. You can't have several people working on several modules with no or little interactions.

In agile development there is a clear statement that communication is very important. It does not have explicit practices for software design communication (well, except pair programming), but it doesn't take a rocket scientist to create and apply them. Company should adapt and change to have great communication flows.

Brooks' Law

Adding manpower to a late software project makes it later.

There are two simple reasons behind this law:

1. It takes some time for the people added to a project to become productive.
2. Communication overheads increase as the number of people increase.

From the first sight agile development can't to anything with these problems. However that is not true.

1. Pair programming. Likely it is the most efficient way to familiarize new people with the project.
2. Iterative Development. New iteration and new release is a new mini-project in fact. It is easier for people to start with something new. If you join the team in the middle of one year release you feel less comfortable.
3. Small teams. Agile approach works best for small teams. When it scales, it does not bring much communication overhead. For example, a single scrum team works autonomously, while scrum master participates in scrum-of-scrum meetings. However it does not increase communication time in a single scrum team.

Feel free to continue the list of diseases ;)

Labels: agile, software development

posted by Michael Dubakov @ 2:32 AM

Sep 9, 2008 / Reveal Agile Development Process Problems with Kanban Board

0 comments

In previous post I described Kanban and its application in agile software development. Kanban board is not just a way to communicate story state, it is a tool that can reveal some problems in the development process.

There are some simple patterns that are very intuitive and can be applied to any development process. You should look for gaps on the kanban board. If you have many stories in a column before the gap, you definitely have a problem in development process.

For example, we have several user stories in Implemented column, but none in In Testing column. It means that testers are unable to process user stories. They are overburdened, or unexperienced, or lazy, or travel to Bahamas, or anything else. Kanban board can't tell what the problem is, but it shows that it is exist.

The gap may unite several columns. For example, we have some user stories in In Progress state and some in Done state. It means developers are unable to feed testers with implemented user stories. Maybe we have too many testers or not enough developers. Again it is not clear, but the problem is there.

Last case is when we have user stories in all states except Done. It is more complex case and it depends on context. In the beginning of the iteration it is definitely OK to have Done column empty, but in the halfway you expect to have something done. Otherwise testers will be overburdened in the end, we'll drop some stories, or we'll complete them in a hurry with poor quality.

As you see it is a very simple but effective way to identify problems in agile software development process. Nice side effect of the pull system!

Labels: kanban, problems, pull system, software development

posted by Michael Dubakov @ 4:32 AM

Sep 8, 2008 / Kanban in Software Development

3 comments

Lean software development is becoming a new buzzword. More and more articles, books and discussions around lean and software development popped up last year. People are using new terms like Kaizen, Kanban, Muda, Pull System, etc. Well, this is great. New thinking about software development stimulates new ideas and new solutions to old problems.

Push, Pull, Kanban

Kanban is a very popular buzzword in agile community and in this post I will try to combine various applications of Kanban for software development. In general, Kanban is a part of Pull System. Pull system determines whole supply chain. How many screwdrivers we should produce next month? Push system predicts the quantity based on history data and market parameters. For example, we predict that we need 10000 screwdrivers next year and will produce 10000 screwdrivers. Who cares if one half of the screwdrivers will be unsold till the end of the year? The plans for next year will be corrected, but unsold screwdrivers take store space and money. Pull system produces just required amount of screwdrivers. In ideal life we find a customer, produces what he needs and sell it. Anyway, Pull system reduces inventory.

Kanban is a sign or a signal in the development/supply chain. It activates some process. For example, you have small shop that sells screwdrivers, small warehouse and small plant. Imagine that each screwdriver has small kanban card on it. If a lucky houseowner purchased one screwdriver, the card returned to warehouse. It shows that one screwdriver should be delivered to the shop. The screwdriver delivered, but free kanban card returned to the plant. Plant management knows that one more screwdriver should be produced. They produce it and deliver to the warehouse. Thus customers' demands rule the whole cycle.

Product Kanban

How it can be applied to software development? I think there are several applications. One-year-long project plan is a push system. We expect to sell all these features in the next year delivery, but it may happen that customers will need (use) only one half of them. "Why the hell you add Blue Ray support? This is plain dead thing!" There situations where one-year-long plans works. If customers demand is defined and persistent, you may do that. It means that customers do not want to change anything in product scope next year. I don't know industries where it is true, but I asume they may exist.

Iterative development and product backlog is a pull system. If the item is in the backlog, it represents customer's demand. But customer is free to remove it anytime, thus changing the demand. This is a high level kanban (product kanban). If item exist in the backlog it is a signal that it should be implemented.

Release Kanban

During iteration development kanban rules Implementation/Testing activities. For example, testing team may take implemented user stories and start testing. The most popular way for this kanban is a simple whiteboard with several columns (Not Started, In Progress, Implemented, Tested). Each user story is a small sticky note. Initially all user stories are in Not Started column. When user story is done by developer, he put the sticky note to Implemented column. And that is the sign for testers.

You may think out different ways though. For example, when user story is implemented, loudspeaker in the testers room shouts "User Story As an Admin I want to moderate people comments implemented" each 5 minutes until somebody takes this story for testing. Another way is to take story card, crouch to testers room and put it into implemented user stories stack behind tester's back. You got the idea. Testers should be able to check what is available for testing anytime.

Kanban board may work for complex process without problems. Developers see what to take for implementation, testers see what to take for testing, technical writers see what to take for user guide update, customer see what is ready for check or release.

Software development differs from manufacturing significantly. The plant produces looks-alike screwdrivers. It can have robots and sharp process to produce stuff. We can't replace developers with robots. Each feature is unique, each user story is unique. We can't create detailed process, software development needs innovations, researches and a-ha! moments. Software development is a complex adaptive system that works on the edge of chaos and can't be ruled by complex procedures and instructions. Complex rules lead to dumb simplistic behavior, we can't apply such processes to software development.

It seems I went to far. OK, back to kanban. Signals in software kanban should be different. It is not enough to meet a tester and put black label (or card) into his hand when user story is ready. Tester should know what exactly user story is ready (only screwdrivers are the same!). The kanban card should contain at least user story name or ID. That is why in kanban board status coded by column/story combination.

Alternative is to use labels with different colors for each user story. For example, yellow label for implemented, orange label for testing. But each label should have user story ID (or name) on it. You may give labels to developers and testers and they will stick them to user story when required.

Maybe this method worse than usual board, but I just want to show you that there are different ways to kanban in software development and you may be creative to generate better ideas.

If you have distributed teams you will have to use web based kanban. I am not aware about online tools that provide kanban board. If you need just kanban board most likely you will have to create it in-house. There are tools with Task Board (including TargetProcess), but it is not the same. Task board shows task statuses and from tasks it is not clear when the user story is ready for testing. It can be used as a signaling system, but with less success.

A nice side effect is that Kanban board helps to identify problems in agile development process.

Labels: kanban, lean, pull system, push system, story board

posted by Michael Dubakov @ 6:17 AM

Sep 3, 2008 / Software Development IS Hard

0 comments

Yes, it is.

posted by Michael Dubakov @ 6:19 AM