Even a mediocre plan will work if everybody follows it

By Jim Grey (about)

I have been astonished in my life by how few problems are truly unsolvable. I have also noticed that, most of the time, when a problem ends up not being solved it is for one of two reasons: people deny the problem, or they won’t work together on the solution.

There are any number of ways to make software. All of them involve programming, of course; it’s the core activity. But there are other jobs to perform, and the bigger the software being developed, the more people you need to perform them. It becomes necessary to organize the people and form work processes. The ways to do that range from loose and chaotic to structured but flexible to ponderous and formal. They all have their strengths and weaknesses; they all can work. I’ve made software in all of these situations.

One of my past employers sequestered its programmers in a room to hack code together as fast as they could. When they were done, a couple people would play with the product for a week or two in hopes of finding some bugs, which the programmers might or might not fix. Then we shipped it to five customers, who upon installing it invariably found that it failed spectacularly. When we fixed all the problems they found, we chose five more customers. We repeated this process until the software stopped failing, and then announced to the rest of our customers that it was ready.

At the other end of the scale, another past employer had such a heavy software development process that following it felt like slipping into a straitjacket. There were reams of documentation to write and keep with layers of approval at each step. I led the test team. We had to take a screen shot of every test step, print it, and put it in a folder. At the end of a project, we had boxes and boxes and boxes of printouts, hundreds of pounds worth, that we would send to an offsite storage facility. This was in case our only customer, which happened to be the U.S. government, wanted to audit us.

Both companies are still in business.

A third past employer carefully hired smart people and trusted them to know what to do. Because they hired well, this worked for a long time. But as the company grew, this approach became more and more difficult to manage. Product quality became a problem. A big part of the software was an accounting package, and in one fateful release the general ledger simply would not balance. This affected every customer, and to make a long story short, it took the company almost a year to fix it. Several customers quit us.

Too often it takes great pain to drive important change. This pain made us face that we needed more structure to deliver successfully. So we hired a consultant to guide us toward a better software development process. He showed us a way with just enough checks and balances that we could have greater confidence in our work without being too bogged down. We also hired a consultant to help the management team (of which I was a part) work more collaboratively. She taught us how to lead the company through this transition.

It worked. In the very first product release under the new process, quality problems fell off dramatically and we delivered on time for the first time anybody could remember.

The more important hire was the second consultant. The new methodology was good, but it wasn’t magic, and it might not have worked for us if we had not done a good job helping the team through the changes. You see, a few people didn’t understand why the old way couldn’t still work and still others thought our new process wasn’t going far enough. Our task was to overcome the resistance and get everybody singing from the same sheet of music, and it was the hardest thing we did. But we pulled it off.

It’s a rare organization that faces its failings and works to change. It’s a rarer organization still that drives change collaboratively.