I won't go into detail about what Extreme Programming (XP) is, since you will find excellent explanations elsewhere on the web. XP has received a lot of hype and, perhaps partly because of the unfortunate name, given some people the sense that it is risky or faddish. I don't think it's either, but neither do I believe XP practices are articles of faith. So I want to offer a brief personal assessment of what worked and what didn't for us. That takes a little bit of explanation of what I mean by XP.
XP is a collection of interlocking and mutually reinforcing practices that promote agility in software development. Agility is a buzz word, but it has clear meaning for me: it's the ability of a team to respond quickly to, in fact embrace, changes in requirements based on continuous input from customers and continuous changes in technology. Agility is a product of prioritizing good communication over rigid up-front designs and plans, and XP is nothing more than a set of habits or practices that people have found from experience cultivate that communication. I like to think of these practices as attempts to weave good communication into the fabric of the developer's work instead of it being a necessary distraction from development. The specific habits or practices of Extreme Programming are not laws carved into stone tablets: they are meant to be adapted to facilitate agility and to improve the odds of successfully delivering complex products in a world where change is the only constant. One of the practices is even called "Fix XP" to emphasize the need for adaptation, which ultimately is what XP is all about. Our team decided to take the canonical "rules and practices" of Extreme Programming pretty seriously and I'm convinced that among them the following made a big difference for us:
Release early, release often
It took us a while before we got to the point where we were doing actual releases every two weeks. We rationalized that it's hard to "release" a complex piece of software when it's still missing huge chunks of required functionality, if only because it's sometimes hard to get customers to look seriously at it. But you really must do it. I recommend doing a lot of prototyping, to make this work at the very start. We did not do this and I think we paid somewhat of a price for this. It was a price that was mostly affordable because we ramped up our effort to involve beta customers later in the game.
We also struggled with iteration planning. One thing we learned is that (aside from prototypes) it's best to deliver a narrow sliver of functionality that works from end to end (all n tiers, plus integration) rather than ship something that's broader in scope, but less complete. This makes QAing each iteration a more meaningful effort right from the start and I think it may also force you to test the assumptions of your architecture earlier so you "fail fast" if you find one of your assumptions or technology choices was wrong.
Failing fast is the whole point of having a short iteration cycle. The sooner the customer gets to see each piece that you develop, the easier it is to fix problems or effect changes. Customers are not always easy to engage this way—In my experience they're less interested in actively participating in iterations than the developers are eager to have their participation. But you do the best job you can to make it interesting and fun for them. For us we tried to make our beta testers feel important and part of something big and cutting edge.
Test-driven development
TDD isn't so much about testing as it is about creating documentation of what your code should do and writing that documentation in a form that it can verify for itself whether your code conforms to it. We were very dedicated to TDD in this project—not just writing the unit tests, but writing each test first and letting the "fail-implement-pass-refactor" cycle drive the development. It was hard because it felt slower, but for me it's easy to see in retrospect that it was a big win that dramatically improved code quality and saved us countless hours. I found that the benefit comes in two ways. First, TDD gives you confidence to change code that you know will have ripple effects through the application. The tests tell you every place that your change causes the software to diverge from its documented behavior. We didn't gain this advantage right from the start, though. It took a while for us to learn how to write tests that documented the essential behavior of a class instead of documenting implementation specifics. We over-used mock objects at the beginning and I learned to recognize this as a "smell" that gave away a fragile test: one that was likely to break if you changed how your code was factored even if the essential behavior was still correct. If you change the name of a method and your unit test fails because it was checking that a method with a specific name was called, you're not writing good unit tests.
The second way TDD saves you time is that it tends to produce loosely coupled factorings that bend instead of breaking when inevitable changes come along. This is not just wishful thinking. TDD enabled us to make some massive changes in the software in very short periods of time. In the first weeks after shipping the 1.0 release it finally became clear that using the Hibernate ORM framework was a mistake for this project. We spiked out a version that substituted iBatis, the software developer's equivalent of performing a heart transplant. Once we determined that this would fix the problem our users were experiencing it took four developers about 8 days to make a transition that potentially affected many hundreds of classes. We experienced almost no regressions from this dramatic change. I think one developer might have spent a single day fixing the few small regressions that weren't caught by the unit tests.
Pair programming
The practice of having two developers work together in front of a single computer is one of the more controversial practices in XP. I was a skeptic at the start, but I've mostly come around. That might sound like faint praise, but to give you an idea of how significant a statement that is you need to know a little more about me. I am one of those people that prizes solitude as an opportunity to get into a creative flow state. I think that the ability to get into this frame of mind is indispensible to software development. I also believe a lot of folks have so many interruptions and multiple sources of stimulus in their lives today that they don't know they are even missing this. I'm enough of a freak about it that I stopped watching TV about five years ago because I felt it took away too much of the time I needed to get into the flow, not just in programming, but in anything I considered important that involves creativity or learning. So when I tell you that I've mostly come around on the subject of pair programming I want you to understand that I place a huge value on concentration.
In a good pairing session you spend a lot more time in flow than you might think. I've heard it said that pair programming is like having a continuous code review. This is totally wrong. If this is what is happening when you pair you need to stop it right now. You program in pairs for the same reason that you have a pilot and a navigator in an F-16. One person is operating at the tactical level, while the other is thinking strategically. Only in XP you trade roles every few minutes. Each person is a resource and helper to the other, like having an extra brain, and a good pair actually helps you stay concentrated. Think about a time when you've had a really good conversation with someone and totally lost track of the time because you were so concentrated on the interaction: that's achievable in pair programming in many cases. There are times when I still work better in solitary silence and there are people who are incompatible as pairs or who simply aren't cut out for pairing at all, so this XP practice is one that I think needs to be exercised judiciously. But I'm convinced that the benefits are huge when you do.
We had a nice semi-controlled experiment to test these benefits when at one point in the project management got nervous that we weren't going to hit a deadline. Pairing was stopped for a time with the hope that a "divide and conquer" approach would increase project velocity. It didn't work. Not only was there no obvious improvement in how fast we completed our tasks, but the quality of the code was reduced enough that we created "debts" during this period that had to be repaid with more development time later in the project.
We found a lot of side benefits of pairing aside from better code quality and lower long-term maintenance costs. Everyone on the team developed a very broad understanding of the application because we rotated pairs instead of cordoning off areas where there were one or two experts. From a management perspective this is a big advantage because losing a team member is more akin to coming down with a bad cold than having a limb amputated. From a developer standpoint it means that you are able to tap into the "hive mind" of the group more effectively. When you open a discussion about a problem you can engage a lot of team members productively to resolve it, and you find that you know so much of the application that the source code itself is a much more valuable reference to you. The benefit is cumulative over time and I would venture to say that the bigger and more complex your application is, the more you have to gain by doing a lot of pair programming.
These were the three XP practices that I think made the biggest difference for this project. I am convinced that this project would have been at a much higher risk of failure had we not employed them. Software defects were discovered earlier because of the combined effect of these practices, and when it comes to fixing defects early fixes are much easier and cheaper than late ones. For the kind of project we were undertaking I think XP was a big win.
Implications for mobile software development
The desktop application we developed for Pampered Chef was a very complex, multi-year effort that I think showed the benefits of XP very well. Returning to the smaller, usually simpler applications that Pikesoft develops for mobile devices it's going to be interesting to think about how we'll adapt these practices. Having frequent, short iterations has been a practice I started long before I ever heard of agile methodology, so that will be nothing new. I plan to make a more systematic effort to involve the customer in iteration planning than I have in the past. With regard to test driven development I expect to see a somewhat reduced role in our mobile projects relative to the project just completed because mobile applications typically have a higher percentage of "GUI logic" relative to unit-testable "business logic." Also, since some of the benefit of TDD is better code factoring (design patterns) and many design patterns are too expensive to use on a resource-constrained mobile device, I think this reduces the benefit of the test-first approach somewhat. Pair programming is something that I'll definitely want to try to introduce where I see that I've got the right people to do it.
On a personal note, I just want to throw out a big thanks to the Pampered Chef Java developer team. I've assembled and led a few developer teams and participated on or collaborated with many others, but you guys are the best. The Midwest grows great engineers and solid people that you can count on when the chips are down.
Posted by cervezas at 10:52:58. Filed under: Extreme Programming
Comments
Add Comment