Last week brought to an end (for now) one of the best software project experiences I have ever had. Over the last year and a half I've had the distinct pleasure of working as a contractor with a team of nine outstanding developers on a custom Salesforce Automation and CRM application for
The Pampered Chef in Chicago. We shipped the Golden Release to 70,000 Pampered Chef consultants in January and have had great feedback and minimal hitches. That's remarkable when you consider that this 3.3 million lines-of-code application was deployed in the wild on home PCs, not in a managed IT environment and that these consultants run their whole business in this application. Sometime soon I will tell you about the smartphone client I worked on while I was on this project. In the meantime I thought I'd share some of the factors that I think contributed to the success of this project, not the least of which was the bold decision by management to let the team try the development methodology known as
Extreme Programming.
I won't go into detail about what Extreme Programming (XP) is, since you will find excellent explanations
elsewhere on the web. XP has received a lot of hype and, perhaps partly because of the unfortunate name, given some people the sense that it is risky or faddish. I don't think it's either, but neither do I believe XP practices are articles of faith. So I want to offer a brief personal assessment of what worked and what didn't for us. That takes a little bit of explanation of what I mean by XP.
XP is a collection of interlocking and mutually reinforcing practices that promote agility in software development. Agility is a buzz word, but it has clear meaning for me: it's the ability of a team to respond quickly to, in fact embrace, changes in requirements based on continuous input from customers and continuous changes in technology. Agility is a product of prioritizing good communication over rigid up-front designs and plans, and XP is nothing more than a set of habits or practices that people have found from experience cultivate that communication. I like to think of these practices as attempts to weave good communication into the fabric of the developer's work instead of it being a necessary distraction from development. The specific habits or practices of Extreme Programming are not laws carved into stone tablets: they are meant to be adapted to facilitate agility and to improve the odds of successfully delivering complex products in a world where change is the only constant. One of the practices is even called "Fix XP" to emphasize the need for adaptation, which ultimately is what XP is all about. Our team decided to take the canonical "rules and practices" of Extreme Programming pretty seriously and I'm convinced that among them the following made a big difference for us:
Since September I've been pair programming about 75% of the time in my work with a Java team for a client in Chicago. Starting in January about half of my pair programming has been done remotely. I blogged
here about how this works using Skype and VNC and I'm a proponent of the technique--with some qualifications. Neither Skype nor VNC are 100% up to the demanding task of letting two developers work on the same desktop. I'd guess the actual productivity penalty is quite small, but the discomfort of dealing with the network lag when mousing and especially when scrolling an editor is substantial. And Skype seems to drop my calls about once every 30 minutes.
Fortunately, one of the top level projects at Eclipse Foundation, the Eclipse Communication Framework, seems to be making progress on enabling some solutions to this problem at the IDE level. Gotta love Eclipse! Riyad Kalla at EclipseZone blogs:
The Eclipse Communication Framework (ECF) continues to make leaps and bounds forward establishing not only a collaboration framework for the Eclipse platform, but a full communication launch pad.
Most recently the ECF project began work on what will become a VoIP API used to create softphones for sending/receiving calls from within an Eclipse RCP application. Have a quick glance at the very preliminary Javadocs here . Skype RCP anyone?
For folks still trying to wrap their minds around what ECF is intended to do, the team provides a slew of great example applications. Two in particular that might hit home for users are the shared text editor as well as Graphshare.
It'll be killer for pair programmers to collaborate on editing a single file in Eclipse within two network-connected IDE installations. What's needed is a way to synchronize not just the text that's entered but the scrolling state of the editor page. You want the option to keep the scroll position of the collaborating editors locked or to sync them up on demand with a key stroke. It shouldn't be long now before someone works this out and releases a pair programming plug-in for Eclipse.
After a stint of four months working on-site for a big Chicago client I'm now able to work mostly off-site, which is a big relief for Terry and me. I didn't really mind the travel itself as much as a lot of folks do--I've been an airport junkie since childhood--but being home only two or three days a week has been a big strain on everyone. Ironically, the move to working remote has coincided with a decision by the client to resume pair programming, which was suspended for a while because a manager thought it was reducing productivity (it wasn't, in my opinion). So how do you pair program with someone who is 900 miles away?
One of the practices in eXtreme Programming is continuous code integration. Since the code is collectively owned by the team (meaning any developer can change any part of the code as needed) your working code needs to be checked in to version control early and often. But I was surprised to run across this statement in David Astels'
Test-Driven Development book:
Never, never, NEVER leave anything unintegrated at the end of the day. If you can't integrate before you leave for the day, throw it out and start fresh in the morning. If a task takes longer than a day to do, either the task is too complex (and has to be broken up) or your solution is too complex (and you need to find a simpler one). In either case, start over.
My first reaction was, cripes, that's rigid and a bit arbitrary, too. Why operate with the restriction that task boundaries have to align with workday boundaries? But as I thought about it I realized it might not be bad advice after all. I think this may be one of those rules that establish a general discipline that is productive even if it seems to cause unproductive actions in particular instances.
Two weeks into my foray into eXtreme Programming I'm finding that the initial shock of Test-First development is starting to wear off and I'm beginning to enjoy myself. I'm still a bit taken aback by the way XP slows the development process. But considering that in XP you typically write more test code than production code and all your development is done with two developers per workstation instead of one, what's remarkable is that the team gets as much work done as they do.
But even if this seems inefficient, there are definitely compensating factors that balance some of the time cost. Code quality is the big one. At first I thought that the main source of improved quality in XP stems from fact that two heads are better than one, four eyes better than two. That's certainly an important factor, as is the fact that you are striving for something close to 100% unit test coverage. But this is only the beginning.
I'll get back to my PalmSource musings before long, here, but in the meantime I wanted to post a reaction to my first encounter with
eXtreme Programming (XP). "Extreme" is a pretty good description of my poor brain's response when its old programming and work habits collided with this unusual (but oddly sensible) approach to writing software in teams. After three days doing
pair programming and
test-driven development, I am mentally tapped out. This stuff is very hard--in the beginning, anyway.
It's been a few years since I worked on a development team, aside from the odd open source project or remote collaboration. I think most developers get very accustomed to their quiet, solitary relationship to the code, but if you work remotely as I usually do you
really get used to being alone and following whatever winding stream of consciousness your little brain produces as you code. Thinking to yourself is often referred to as a "reflective" mental state, but I got a real feeling for how unreflective I am about my programming when I suddenly had to talk to another programmer sitting next to me. I can tell already that just being forced to use the language of object-oriented programming--saying things out loud and with enough precision to be understood--is a good step to becoming a better developer.
But the biggest out-of-the-gate challenge of XP for me is the test-driven development. It took a couple of days to dawn on me that TDD is not about testing your code. It's about designing your code. A test isn't a thing you do after the fact to see if your code works; it is a way to define (in code) precisely what "works" means. In that light,
writing a test before you write a single line of code is perfectly natural, right?
Hah! Let me tell you, it feels anything but natural. They call it "agile" development, but the rhythm feels terribly slow and lurching at first. You constantly have to pull yourself back from writing a line of code before you've written the test to exercise it. And
every little thing you want your code to do must first be specified in a test because the implementation literally is driven by the tests you write. Most of the tests I've been writing are longer than the code they are testing. That's frustrating. While I can really feel the confidence that comes from the practice and see how I am writing better code, it does raise the question whether XP might not be taking its test-driven development aspect to the extreme.
It's way too early for me to judge this, and for now I just want to say that I consider the difficulties of getting started with XP to be the kind of difficulties you encounter whenever you find a new discipline imposed on otherwise undisciplined habits. I'm betting I'll find that most of the pain is salutary.
For the moment I just appreciate being able to work on a team that thinks seriously about the best way to write software. There have got to be better ways to do it than what we developers usually do now, which results in huge numbers of projects failing and and even high profile "successes" having serious quality issues. I'm looking forward to the hands-on experience with eXtreme Programming and expect to make it a regular topic for the blog over the next several months.
