dow.ngra.de

I haven’t been interested in Agile (or any other methodologies) for a long while. Partially because I found the things that work for me and my team and partially because there was a lot of marketing BS not confirmed by any actual results. This talk promised to deliver some hard data, so I was interested how it goes together with the consulting mumbo-jumbo.

The talk borrows the title and format from the Mythbuster TV show, where experts recreate the urban legend situations to check if they are plausible or not. Scott confirms or busts one myth at a time using the data from the surveys that come from a variety of sources, some of them more trustworthy, some less.

The reasons I couldn’t take some of the conclusions seriously were:
1) Agile is not well defined. There’s a large number of practices considered Agile, and many people will claim that they’re doing agile e.g. if they have a build server.
2) Surveys only tell what people think is happening. If you want to know how long is the nose of the emperor of China you don’t walk around asking people. Worse a lot of those surveys are conducted only among the Agile or TDD community, which gives them a high bias against traditional development.
3) Comparing Agilists to Traditionalists is like comparing Tide to the Usual Washing Powder. There’s no such thing as a traditional methodology, there’s a bunch of them with very different approaches.

Note that here I give only my interpretation of Scotts talk and data, so please don’t treat it as anything else, but my own opinion.

Myth 1: The majority of organizations are now doing Agile.

Survey shows that 76% of organizations claim they are doing Agile.

CONFIRMED!

Myth 2: The majority of project teams are doing Agile.

In the 76% of the organizations claiming to do Agile only 44% of teams report using it. That translates to roughly 33% of the “popular vote” for Agile. Therefore 66% of developers are not even claiming to do Agile..

BUSTED!

Myth 3: Agile is all about development

In the list of Agile practices the only one that is used by more than half of developers claiming to do Agile is Continuous integration (65%). That kind of underlines the fact

PLAUSIBLE?

Myth 4: Agile is just for small teams

A lot of large projects are confirmed succeeding with Agile. In fact success rate isn’t that different from small ones. Would be interesting to see the breakdown of different practices used in small v/s large projects.

BUSTED!

Myth 5: Agile doesn’t work in a regulatory environment

To clarify here are meant environments where errors are critical and thus are regulated by legal or engineering requirements. Surprisingly 33% are using Agile in a regulatory environment. Again a breakdown of practices would be very interesting as there’s bound to be some difference.

BUSTED!

Myth 6: Agile & CMMI don’t work together.

Only 9% reported using Agile. I think Scott put this as BUSTED, but 9% sounds like CONFIRMED to me :)

Myth 7: Agile approaches are “empirical”.

Very weirdly phrased claim. The survey shown was 51% of respondents collecting manual metrics, 19% collecting automatic metrics and the result was CONFIRMED.

Myth 8: Agile teams are mostly building from scratch.

78% of respondents work with legacy at least on some level. Scott had a pretty interesting digression on how Agile practices handle well the legacy code (with tests, refactoring, etc), but don’t give that much attention to the quality of legacy data.

BUSTED!

Myth 9: Agile certification is a good idea.

78% of respondents think certification is meaningless, wonder how many of them are certified (as does Scott). An interesting digression about a guy having both PhD and Scrum master on his business card, as if their value were the same. Certification that doesn’t require any real effort is greatly overvalued in the community.

However although Scott considers the myth busted, I don’t think it makes sense to ask this question from everyone. If this’d be a poll among hiring managers it would make much more sense.

BUSTED?

Myth 10: Agile teams are colocated.

Survey shows that only 42% are colocated.

BUSTED!

Myth 11: Agile doesn’t do up-front estimates.

In the world of budgets this premiss always sounded ridiculous. And unsurprisingly 89% of teams have to provide an up-front estimate (with different level of certainty).

BUSTED!

Myth 12: Agile teams just start coding.

No kickoff is another thing I never believed. From the data about a month of kickoff is usual before starting the actual coding.

BUSTED!

Myth 13: Agilists follow common dev conventions.

Data is a bit confusing, but seems that 59% have some kind of conventions.

PLAUSIBLE?

Myth 14: Rights & responsibilities are part of the agile culture.

The question is quite weird and the data isn’t really connected to it.

PLAUSIBLE?

Myth 15: Agilists test often and test early.

Only 71% in the TDD community are actually doing TDD. WTF? And agile guys are still having integration tests (45%) and hired testers (36%). This confirms my suspicions that unit tests are not the silver bullet as many suppose them to be.

CONFIRMED!

Myth 16: Agilists don’t do up-front requirements modeling.

76% are doing high-level up-front requirement modeling. This correlates with Myth 11, because to provide reliable estimates you need to have an idea what you’re going to be doing.

BUSTED!

Myth 17: Agilists don’t do up-front architecture.

70% do architecture design. This explains the month of kickoff along with the requirement gathering.

BUSTED!

Myth 18: Agilists write some “interim” docs.

Not sure if anyone doubted it, but 56% agree.

CONFIRMED

Myth 19: Agilists don’t do supporting docs.

The numbers are slightly lower than “Traditionalists”.

BUSTED!

Myth 20: Traditionalists write better docs.

Equally bad at writing docs according to polls. I can believe that :)

BUSTED!

Myth 21: More difficult to add members to Agile projects.

Data shows that it’s slightly easier to add in Agile. Promiscuous pairing makes it even easier, so make sure to be promiscuous.

Frankly I heard the other myth that it’s easy to add members to Agile projects, that is busted as well.

BUSTED!

Myth 22: Agile works better than traditional approaches.

The ultimate question :) Success rate over different approaches:

• Iterative — 71%
• Agile — 70%
• Traditional — 66%
• Ad-hoc — 62%

Personally, I think that the variation is pretty small and it’s more important that 30% of projects fail no matter what than the 4% of difference between Agile and “Traditional”.

However when broken down on specific aspects (like cost, time, quality), it seemed that Agile was considerably better. I’d take it with a grain of salt though, as a lot of people take Agile as a religion and measure success accordingly :)

CONFIRMED?