Mar 23, 2009 / Simple Rules, Complex Systems and Software Development

3 comments

Many complex systems are based on simple rules. A set of several simple rules leads to complex, intelligent behavior. While a set of complex rules often leads to a dumb and primitive behavior. There are many examples.

Ants Colony

How ants search for food? They do not have cell phones, cars and mini-markets near the nest. They should have something simpler to communicate.

Here is how ants work:

1. Travel randomly  in search for food.
2. Take a piece of food and head straight back to the nest. On the way back to the nest lay down an odor trail.
3. Notify nestmates of the discovered food encouraging them to leave the nest. These newly recruited ants will follow the odor trail directly to the food source. In their turn, each ant will reinforce the odor trail until the food is gone.

Sounds simple? Take a look at this very nice ants colony model. Drop some food and enjoy the action.

Birds Flocks

Birds flocks are beautiful. You may think that the movement gets orchestrated by one savvy bird. But this is not the case. A bird glock is guided by three simple principles (every decent bird knows them):

1. Separation: steer to avoid stumbling upon local flockmates.
2. Alignment: steer towards the average heading of local flockmates.
3. Cohesion: steer to move towards the average position of local flockmates.

Simple? Yes, it is. Look at the picture on the right. It's just amazing!

Game of Life

Game of Life was invented in 1970 by John Conway. It is a cellular automaton and simulates the birth, death, etc., of organisms based on certain rules:

1. Each cell with one or no neighbors dies, as if of loneliness.
2. Each cell with four or more neighbors dies, as if of overpopulation.
3. Each cell with two or three neighbors survives.
4. Each empty cell with three neighbors becomes populated.

Simple rules. But these rules lead to fantastic diversity of the forms. Different types of the forms have been discovered e.g.  still objects, oscillators, gliders, spaceships, etc. It is impossible to predict the state of a system after several thousands steps. Cellular automaton has properties of a complex system such as emergency and butterfly effect. Small changes in the stable structure (for example, adding one more living cell) may cause death of the whole cells population.

Moreover, it is possible to build a computer based on Life! It is possible to implement logic based on stable structures and execute simple calculations. The Game of Life is a Turing machine! How can someone suggest that such a simple system based on four rules has so much power?

Take a break and have fun with Life game simulation.

Simple Rules and Software Development

Is there any relation between simple rules and software development? Sure, there is. Software development process should be simple. Process complexity leads to mechanistic and dumb behavior.

1. Information exchange and collaboration vs. standard status reporting meetings.
2. Learning vs. stagnation.
3. Emergent behavior and creativity vs. Following rules, standards, instructions.

Agile methodologies are simple. Scrum is a very simple thing. It has just several rules, roles and artifacts. Well, it is a lot harder to really implement Scrum. It is hard because you need to break stereotypes and habits. Many people are resistant and don't want to try new things. However, Scrum works. It works because of its simplicity, it lives in accordance with complex systems.

Scrum stimulates learning, feedback, communication and cooperation. Emergency is possible in Scrum.

Here is one sample of unnecessary complexity — too many hierarchy levels:

A potential problem, unlikely in small and medium organisations, is deep organisational structures. According to Peters and Waterman (1982)18, both Toyota and Roman Catholic Church have only five layers of management in comparison to Ford's seventeen.

Do you by any chance happen to have a software development process description in a huge 100+ pages document? Are you still in business?

• Part 1. Agile Software Development and Complex Adaptive Systems: Intro
• Part 2. Software Development is Complex Adaptive System. No Doubt
• Part 3. Edge of Chaos and Hyper Productive Software Development Teams
• Part 4. Simple Rules, Complex Systems and Software Development

Labels: agile, complex adaptive systems, scrum

posted by Michael Dubakov @ 4:09 PM

Mar 19, 2009 / Zero Defects? Are You Kidding Me?

11 comments

Are you familiar with zero defects mentality? It looks very good from the first sight. Zero defects... Let me think... Well, cool! I like it! I’d like to have zero defects in my projects.

So, what is zero defects mentality? Here is the quote from Lean Software Development an Agile Toolkit book:

“One of the fastest ways to kill motivation is what is called in the US Army a zero defects mentality. A zero defects mentality is an atmosphere that tolerates absolutely no mistakes; perfection is required down to the smallest detail. The army considers a zero defects mentality to be a serious leadership problem, because it kills the initiative necessary for success on a battlefield” — Mary & Tom Poppendieck

Obviously, zero defects mentality is not something that HOUSE M.D. likes ;) Moreover, I think Dr. House hates zero defects mentality. It causes several unpleasant effects:

• Not enough courage to refactor complex, messy, buggy, but important piece of code.
• Can’t make important decision, instead make less risky, but wrong decision.
• Do everything to avoid responsibility, that leads to coward and stupid behavior.

"Zero defects" may sound good. But in reality you still have errors after production. Even in predictable and (quite) easily testable industries (hardware, automobile, etc.) there are problems with 100,000 power adapters that should be replaced (or hard drive problems, or engine problems, I bet you can continue the list).

How can we expect zero defects in software development? It is harder to test, harder to define in details, harder to predict. Software development is a complex adaptive system, we can't predict all effects. Bugs in production is a normal thing, and by “normal” I mean we can't bring them to zero. We can (and should) minimize them using all possible ways, but The Last Bug is a mirage.

There are several obvious strategies that may help:

• Test Driven Development. Nice side effect of TDD is a unit tests suite. You have tests for new code, and you have unit tests for old code. More tests — less bugs.
• Continuous integration. Instant feedback helps to identify problems early and fix them early. It saves time (and money) and reduces bugs in production.
• Automated regression functional tests suite. Unit tests are good, but you need something else. Functional tests emulates user behavior and find user interface errors, integration errors, etc. Needles to say you should have continuous integration in place to succeed with automated functional tests.
• Root cause analysis. There are several ways to fix a bug. You may just hack the code and apply a patch (please, don’t do it at home!). You may think deeper and fix the bug nicely. Also you may look into the root of the problem and fix it. Yes, it may take time, but you will prevent future bugs from this domain.
• High development skills. Ironically, high skills do not always reduce bugs rate in production. “Stars” may be strong in some areas (business layer), but may not polish things. It may lead to many small bugs, especially on UI.

Is it a good thing to have "Zero Defects" goal in the sprint? The answer is "Are you kidding me? Zero bugs? It's a miracle!"

Labels: agile, bug management

posted by Michael Dubakov @ 12:10 PM

Mar 13, 2009 / Friday's Digest #10 [Scrumban, Pair Programming]

0 comments

• Why Can't Error Messages Be Fun? Indeed, Chrome does it right.
• Scheduling in Scrumban. "We eliminate iteration planning and instead implement a system of continuous rolling-wave planning." I think rolling-wave planning provides great benefits to Product Owner. Backlog becomes more flexible and adaptable.
• Successfully Adopting Pair Programming. Nice article, describing real pitfalls of pair programming adoption. Beware "Spellchecking Pair" :)
• Sell Your By-products. What by-product you have to sell?

Labels: digest, links digest

posted by Michael Dubakov @ 7:39 AM

Mar 9, 2009 / Agile Bug Management Anti-Patterns

10 comments

Agile development books often skip bug management practice. However, there are several dangerous misunderstandings (anti-patterns) that are quite common. It is better to recognize the problem early and fight to win. Here are three common bug management anti-patterns in a development process.

Bug fixing phase inside iteration

Symptom: You have several days at the end of  iteration fully dedicated to  bug fixing.

If you have a bug fixing phase inside an iteration, you have mini-waterfall in each iteration, which is not a very good idea (in agile we blame waterfall for good reason, don’t we?). Story should be tested as soon as it is completed.

Bug fixing iteration

Symptom: You run several iterations with almost no or little bug fixing and then one (or even two!) iterations fully dedicated to bug fixing.

Bug fixing iterations kill your iterative development. It is even worse than mini-waterfall. Such iterations increase your work in progress dramatically. You have nothing done during several iterations before bug fixing iteration. You have nothing to ship. Moreover, bug fixing iterations reduce motivation, people don't like to fix bugs for 2 weeks or for the whole month.

No “Zero open bugs” rule in user story’s Definition of Done

Symptom: You have open bugs in a completed iteration.

You may have decided that some bugs discovered during story development are unimportant and can be postponed to  future releases/iterations. This is a very dangerous practice that leads to bugs accumulation. Moreover, it may be a reason for Bug fixing iteration (see above). The best strategy is to have “Zero open bugs” rule in Definition of Done for user story.

Labels: agile, bug management, process, qa

posted by Michael Dubakov @ 3:29 PM