Every creative project has three phases: make the thing, polish the thing, deliver the thing. The polish step is where you remove errors through testing, editing, inspection, or other review. The deliver step is where you put your work in people’s hands.
It’s so easy to get stuck on the polish step. You keep checking until you’re sure it’s perfect! This is all about fear.When your work is in the world, they can judge it. They can even ignore it. We want to avoid how bad that feels.
Also, perfection is expensive. You can spend as much time rooting out every tiny flaw as you did making the thing. Those tiny flaws will be embarrassing. But they won’t really hurt anything, and they won’t keep anyone from clicking Buy Now. Crucially, eliminating every last minor flaw keeps you from working on new projects that create new value.
When you’ve applied reasonable polish, when you feel the fear of rejection, it’s time to enter Ship Mode.
In Ship Mode, you single-mindedly do the tasks that put the work in people’s hands. You’re not looking for problems anymore. You choose to think of your work as a finished product. You might notice an error while you’re in Ship Mode, but unless it’s truly egregious, you keep shipping.
My new book was good enough to ship — but it’s not perfect
I write in my spare time, personal stories and essays. I’ve done it for years. I recently self-published a book of some of my earliest work. (Scroll to the bottom for more details!) I did the whole job: writing, editing, creating the print-ready and e-book files, and (now) marketing. I saved money doing it all myself, but I’m skilled in only some of these tasks. Also, there came a point where I’d looked at my book so much I had become blind to it.
I’m a recovering perfectionist and I’m mildly OCD (officially diagnosed). It was hard for me to learn to let go and enter Ship Mode. But I’m glad I learned it many years ago, or I would still be making myself nuts perfecting my book. I have a day job. There’s only so much time to work on side projects. If I polished this one to perfection, it would not be available for several more weeks yet.
Curse you, Page 50!
I saw it only in the last step of the submission process to Amazon: this ungraceful flow on page 50. No publishing company would allow a paragraph on one page to spill three words onto the next right before an illustration.
When I saw it I gritted my teeth. I probably said a four-letter word. But not only is this problem not egregious, but most readers won’t even recognize it as a problem. Nobody will demand a refund because of it. I clicked the Approve button to finish the submission. That’s Ship Mode!
I edited every story as I assembled the book, and then made two proofreading passes. But when my author copy arrived I found two typos in five minutes. How frustrating!
But I will be shocked if you find something really messed up, like garbled sentences or missing paragraphs. During the polish phase I sweated out everything that would have seriously damaged your experience with the book.
And then I got on with shipping it so you could read it and, I hope, enjoy it.
Ship Mode! Because there’s a point past which polish doesn’t pay.
My new book, A Place to Start: Stories and Essays from Down the Road, collects early work from my personal blog. I had just come through a great difficulty in my life, a real setback that left me reeling. Through writing my stories I was able to make sense of what had happened so I could move on. My book captures those stories and shows a way to find your way through hard times to a better future.
I often quip that when you plan and manage a sprint, as a leader you have to channel your inner Donald Rumsfeld.
You remember ol’ Don, don’t you? He was twice the Secretary of Defense in the United States, first in the Gerald Ford administration and later in the George W. Bush administration.
Rumsfeld is famous for having said, “There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns; the ones we don’t know we don’t know.” Here, listen:
He said this in answer to a reporter’s question about a lack of evidence linking Iraq’s government with supplying weapons of mass destruction to terrorists. It sounded so absurd at the time that the media had a field day ridiculing him.
But over the years, we’ve come to realize how much sense his statement makes. In anything you set out to do, you know what you know, you might know or be able to guess some things that you don’t know, and you certainly don’t know many things that you don’t know.
In agile software delivery — heck, in life:
You plan for the known knowns.
You make contingency plans for the known unknowns.
When unknown unknowns happen, all you can do is manage through them.
On a team I once managed, we wanted to build a feature to email end users the day after they abandoned a transaction on our site. We hoped to reengage them to finish their transaction.
We wrote and groomed the stories. Most of them had a clear implementation — plenty of known knowns. They were easy to estimate.
A few stories had known unknowns. One was, “It’s been a while since we’ve been in that repo, and it was written in our wild-west days. I might find some cruft in there that I’d need to straighten out to be able to finish this story.”
Another known unknown was, “I’d have to do a deep dive into that module to be sure, but if does this certain thing in this certain way, I’d have to do considerable extra work to stitch in the code I need to write there.”
In each case we discussed whether that work, were it necessary, would add meaningful complexity to the story and inflate its estimate. Sometimes it was yes, sometimes it was no. When it was yes, sometimes we wrote a dive story to find out and other times we thought the impact would be small enough that we could just accept the risk.
As we planned sprints with those stories, we believed we had accounted for everything we could think of, and were confident in our sprint plans. But two things went wrong we didn’t foresee — unknown unknowns bit us hard.
One story involved writing a job that would check overnight for abandoned transactions. The engineer who picked up that story encountered five or six serious unexpected difficulties with writing the job and with getting the job runner to pick it up properly. A story that we thought he would wrap up in two days spilled well into the next sprint.
Then it turned out that our email service was never built to handle the volume we threw at it, and it failed immediately in Production. One engineer devoted all of his time, and another part of his time, to solving the problem. A lot of our tester’s time went to troubleshooting with the engineers. The situation was surprisingly thorny. The engineers implemented several successive fixes over several days before finally getting past the problem.
Thanks to these unknown unknowns, we missed a couple sprint plans by a mile. Here’s how we managed through them: I worked with the team to prioritize stories and figure out which ones we were likely not to deliver. The engineers watched for stories ready to be tested, and helped our tester by picking some of them up themselves to keep the revised plan on track. Finally, I reset expectations with my management about what we could actually deliver, and how that would affect delivery of other work that depended on it.
You could argue that these should not have been unknown unknowns. Writing a job should have been cut and dried. We should have thought about volume in our planning, and tested for it.
Hindsight is always 20/20. Retrospectives are where you explore that hindsight and make improvements for the future. After you experience unknown unknowns, improvements generally take two forms:
They become known knowns, or maybe known unknowns. We learned a lot about the complexities of jobs in our world. We knew we needed to write a few more in upcoming sprints, so we broke them down into several smaller stories that we could test incrementally. Also, we now expected that there might be challenges in writing jobs we hadn’t encountered yet, and to lay in contingency plans for them.
You can lay in plans to neutralize them as unknowns in the future. We found out that other teams had been having trouble with the job runner. We met with the team that owned that code to figure out how to make the job runner more reliable. They put tech-debt stories in their backlog to handle it. We also researched how we might make the email service more robust and investigated third-party email services. We ended up going with a third-party service.
Your crystal ball is cloudy, but you can sort of predict the future. As long as the future isn’t too far away.
In other words, if you take it in small chunks you usually know enough about the work before you to guess reasonably accurately how long it is going to take.
Back in the old waterfall days, we’d plan months and months of work at a time. The printed project schedules could be 100 feet long. The trouble was, work more than a small number of days away often depended on work that had not yet happened. We needed to know how that work turned out before we could be sure of the work that would follow, and have a good idea how long it would take.
From a straight-up project-management perspective, this is why agile is brilliant. It lets us break big projects down into small chunks of usually two weeks in duration, where ideally we have to give our best estimates only for this short period. This respects that we don’t know what we don’t know about the work that follows.
You might still need to estimate how long an entire feature will take to build. Some companies need some idea when the feature will deliver. But it has to be a projection, not a promise, because your crystal ball is too cloudy.
The current sprint’s estimate matters most. When you plan a sprint, fill it with just the work you are confident you can finish. You want to deliver all of the work in the sprint, barring unforeseen technical complexities, critical customer-reported bugs, team-member illness, or late reviews.
Then check your gut: do you think your team can really deliver all of those stories? If not, please say so, and discuss why. When you leave sprint planning, everyone on the team needs to agree, based on what you know then, that this body of work seems doable in two weeks.
I’m sure it’s happened to you, maybe even where you work now: you wonder why in the heck you’re building a particular feature. It seems to you, and maybe to others you work with, that the company’s time and money would more profitably be spent building a different feature.
You might be right. I hope your company gives good avenues for providing that kind of feedback. But if it doesn’t, or if it does but your appeals failed, then get on with making your assigned features real.
That’s where strong execution begins — when the dust settles, even if you don’t fully agree, you get busy working the plan.
It’s obviously best when your company has brilliant ideas that you then execute strongly. But if you bring strong execution to even mediocre ideas, success usually follows. It’s a great overlooked secret: there is no substitute for everyone pulling the cart hard in the same direction.
Companies with brilliant ideas but poor execution fail. Or so, at least, it has gone at the companies for which I have worked. I was once in a startup with an idea that I thought was brilliant. But there was so much infighting inside that company that neither sales nor engineering could build a strong execution rhythm. That company is still limping along but to my mind its product never reached its potential.
If your company uses agile processes, you’ve hopefully broken your work down into small enough chunks that even if you choose a wrong feature to build, you won’t pull the cart in the wrong direction for too long. You’ll deliver a little software, see how the market responds, adjust your direction as needed, and repeat (and repeat, and repeat). Ideally, this system self-corrects.
By all means, if you disagree with the features you’re assigned to build, provide that feedback using whatever mechanisms your company offers. But after the plan is set, execute it as strongly as you can.
Maybe you work in a shop that executes scrum well. Your team keeps the backlog well groomed, understands its velocity, and delivers working software sprint after sprint. Or maybe you work in an open research-and-development shop where the journey is as important as the result, and projects are always open-ended.
If, like me, you have to deliver software on deadline using a less-than-perfectly executed methodology, this post is for you. It’s going to give you a technique for knowing where you stand against your deadlines – a very useful skill that will reduce the chaos you experience and help your boss steer projects more successfully, both of which are good for you.
It is a method for implementing continuous reestimation, which I wrote about in this post. Read it to know why this will make you and your boss more effective.
Teams that have worked for me and have estimated using this technique have seen their projects come in within estimate about 80% of the time. The other 20% of the time, they learned important things about their work that let them estimate more accurately the next time.
Here’s the technique:
Break down your work into tasks.
Estimate the tasks using a three-point formula.
As you work, reestimate every day.
Break down your work into tasks
You might resist this step because in your core you know you will miss some tasks. It’s okay. This process honors that you don’t know what you don’t know. Just outline the tasks as best you can.
But think small, because you’ll be more accurate when you estimate more small things than fewer big things. Break things down as small as you reasonably can.
Estimate the tasks using a three-point formula
For each task, then, think of the following:
If everything goes right on that task, how long will it take? This is the best case, B.
If everything goes wrong on that task, how long will it take? This is the worst case, W.
If a reasonable amount of things go wrong on that task, how long will it take? This is the likely case, L.
Think for a minute or two about each task, but any more than that is probably overthinking it.
Important: Calculate your estimates in hours. Not days, not weeks, hours. There’s too much slush in days and weeks. Besides, in an 8-hour day you probably work just 6 hours on project tasks. You spend the other two hours in meetings, having hallway conversations, using the restroom, and being interrupted.
Here is where some magic comes in. Just go with me on this. For each task, drop B, W, and L into this formula and calculate the answer.
You’re weighting the likely case, but honoring the best and worst cases – and in one deft motion you take fear out of your estimates. What makes estimates bad is either unexpected things happening or overpadding for unexpected things happening. Thinking best/likely/worst helps you be more objective. And when you add up these three-point estimates for all your tasks, you get a reasonable cushion for the things that could go wrong.
Divide the number of hours by the number of hours per day or week you will work on the project. Lay that number of days out on a calendar; tell your boss that’s when you’ll be done.
As you work, reestimate every day
You are going to discover tasks you didn’t think of at first. And some tasks are going to take longer than you originally thought. When this happens, estimate the new tasks and/or reestimate the task that’s taking longer. Recalculate your end date and break the news to your boss.
Your boss ought to kiss you right then and there, by the way, because you gave him or her information early enough to do something about it – adjust scope, add resources, move the date, or tell you that unfortunately you are looking at some late nights. Hopefully the answer will be a, b, or c more often than d.
And every time you have to reestimate, you’ve learned something about your work that helps you estimate more accurately in the future.
My colleague Matt Block recently posted on his blog a link to an article about how a software shop’s business model affects how well agile scrum works for them. It breaks business models down into emergent, essentially meaning that the company builds product to meet goals such as selling ads or driving traffic,and convergent, essentially meaning that the company builds product that directly serves a target market. The article argues that agile is made for emergent and is a poor fit for convergent. That’s just a sketch of the article; go read it to get the full flavor.
Graffiti found in the town neighboring Monrovia
I’ve always worked for companies following convergent business models. We’ve made our money by selling the software we created, which made it always important to deliver a certain scope by a certain time. When those companies implemented agile scrum, they could never fully adapt a key principle of it: when it’s time to ship, you ship whatever is built. In a convergent world, scope is king; you ship when everything specified is built.
I e-mailed my brother, Rick Grey, a link to this article. It’s great to have a brother who does the same thing I do for a living as we can talk endlessly about it. I thought we’d have a conversation about how to scope an agile project, but instead he had a brilliant insight: What if agile is good for convergent-model companies because it tells you sooner how much your project is off track? He gave me permission to share his e-mailed reply, which I’ve edited.
– – –
What if the companies we’ve worked for and all the other convergent-model teams of the world are doing agile just fine? By “just fine” I mean “as good as they do waterfall,” which may not be “just fine,” but we’ll get to that in a minute. Meanwhile, consider:
Long waterfall project:
No one pays real attention to progress (there’s always next month to catch up)
Engineers go dark, checking out huge sections of the codebase and not merging them back for long periods
Engineers (who are notoriously poor estimators) claim 50% done when it’s really about 25% – and then, as the code-complete milestone nears, they (usually innocently) claim 90% done when it’s really 70%
A couple of days before the code-complete milestone, engineering finally acknowledges they won’t hit the milestone and delays delivery to QA – “but we’re 95% done, for sure”
Under the pressure of already having missed a deadline, developers quietly take shortcuts to make it possible to hit the new QA delivery date
Weeks and months of unmerged changes come crashing in, creating conflicts and compile/deploy problems, further delaying delivery to QA
QA, now staring with a multiple-week handicap on an already-too-aggressive schedule, quietly takes its own shortcuts
QA finds hairy showstopper bugs and so the ship date gets moved
Management is livid, so QA goes into confirmatory testing mode just to get it out the door
Agile project of the same size:
Much of the above happens at a smaller scale, one iteration at a time
You fail to deliver everything planned starting with the first sprint
Instead of spending 80% of the project thinking you don’t suck as an organization and the last 20% realizing that you do, agile lets you feel like you suck every step of the way
Takeaway for management: “agile sucks” and/or “we suck at agile”
I assert that most teams are bad at delivering under a convergent business model. The hallmark pathologies of software delivery under a convergent model are too numerous and powerful for most teams to overcome, but their struggles are masked by waterfall until the end. Agile surfaces the problems every iteration. You feel like a loser by week 4 instead of week 40.
But this is actually a win. You get better project visibility and a tighter feedback loop, meaning you’ve got a better chance to make adjustments earlier to get the most out of your team you have. Embrace the feedback loop as a chance to make things better, and learn not to view it as proof of how much you (collectively) suck.
– – –
I will add that agile also helps you keep resetting expectations within your organization, because it makes it standard practice to keep reestimating what it will take to finish everything. This is just what I was talking about in my last post (read it here).
Not long ago a software developer I work with suggested that building software could not, and so should not, be managed to a delivery date. “You can’t know everything you’re going to encounter while you build it,” he said, “and so it just takes the time it takes to get it done.”
I said that I agreed: you don’t know what you don’t know, and the work takes the time it takes. But I couldn’t agree that projects should be open-ended, finishing when they finish.
Capitalism interferes. Pesky, pesky capitalism.
Most of us make software that achieves some corporate goal about making piles of money. Because the company has plans for that money, it needs to know when it will start rolling in. The money can’t start rolling in until you ship.
Or maybe you’re making a prototype that will be demoed at a trade show in hopes of attracting investors, or you’re fixing a batch of bugs that have angered a key customer who is threatening to walk, or you have to deliver some fixes to comply with some new regulation that’s about to go into effect. Point is, your company has good reasons to ask when you’ll ship.
So you need to set expectations about the size and duration of your work. But fortunately, these guesses don’t have to lock you in and enslave you to your keyboard. That’s because every day you can check your progress, adjust for the things you’ve learned, and see if you need to set new expectations.
You’re driving to a destination
Getting your work done is a lot like taking a road trip. I love road trips! You never know what you’re going to encounter. Road construction, bad weather, mechanical failure, and even fun roadside attractions can all slow you down. You have to steer your way through all of these things.
When you’re on the road, it’s pretty easy to know at any time about when you’ll get where you’re going. It involves answering these three questions:
How will you know when you get where you’re going? On the road, you just recognize your destination. But I’m going to fit this metaphor to your work, where it isn’t always as clear what it means to arrive.
How far do you have left to go? Watch the signs; they’ll tell you.
How fast are you going? Just check your speedometer.
Every time you see a sign, divide the number of miles to go by your average speed, and you have the best case for how long it will take to arrive. You can’t control what might go wrong on the road ahead, of course. But after something on the road does slow you down, you can just recalculate and get a new arrival time.
You can manage your own work in much the same way. But it means you have to build some skill at estimating.
“Uh, two weeks”
Here’s how not to estimate. My first post-college job was as a technical writer in a software company. My first assignment was to convert a thick product manual from an old publishing platform to a new one. It was hardly a mission-critical project, but I was green.
When the boss asked when I’d be done, I had no idea, so I shrugged and said, “Uh, two weeks.” Then the task turned out to be enormous and tedious. I had to put in a ton of extra time to finish in two weeks.
When my boss asked “how long” on my next assignment, I guessed a number – and then doubled it to account for the unknown. Success! I worked no extra time and finished my project before the deadline.
Except that I hadn’t learned anything of value. My “guess” estimating technique became “guess and pad like crazy.” And I still had the problem that if I didn’t pad enough,my only recourse was to work tons of extra hours. I had no way of knowing on any given day whether I was on track to finish when I said I would. I just had to work hard and hope.
It’s okay that you don’t know what you don’t know
If I could tell my 21-year-old self something, it would be this: Ask the boss if you can get back to him in two days. Then use your two days to do this:
Think about what done looks like. What must you do to finish the work? How will you know when you’ve gotten through it all? In this case, I had about the easiest possible situation: “Done” meant running out of pages to convert. You may have a number of requirements to code, or a batch of test charters to execute, or a set of stories to complete – there are any number of ways to chunk the work.
Do some of the work. Pay attention to how many chunks you complete in your two days. Then note how many chunks you have left. There’s your speedometer and your miles-to-go sign. Divide to get your rate: chunks per hour or chunks per day.
Calculate how long the rest of the work will take.Divide your rate into the number of remaining chunks. Lay that number out on a calendar to find your finish date. Tell that to the boss.
And then as you work, recalculate every day based on how much you got done that day. If you need to put in a few extra hours to stay on track, just do it. But when the amount of extra effort needed to stay on track starts to edge toward heroic, it’s time to negotiate.
That’s right, negotiate. You are the master of your own work, after all; nobody knows the reality of it better than you. Your estimate is what you believe it takes to reach Done, and you need to stand up for it. This is true, by the way, even when your boss has given you a due date.
So go to the boss and ask which is more important: the due date or this body of work? In other words, can you change what it means to be done, removing enough work to hit the date? Or can you move the date out to get all the work done? Or can you do a little of both?
Sometimes the answer to this question doesn’t go far enough to make the remaining work match the due date. So ask yourself if you can give some of the work to someone else, enough to hit the due date. If so, ask for people to help you.
What you’re doing is giving the boss and all of your project’s stakeholders as much runway as possible to understand the probability that you’ll deliver the originally defined Done on time. Should that probability drop, your negotiations give them the opportunity to change course, change expectations, or both.
Sometimes, you’ll get no answers to all of these questions. Sometimes, you’ll get yes answers to some or all of them and it still won’t be enough. But asking these questions is your best chance of avoiding working heroic hours.
An estimate should not be a commitment
If no is always the answer, or your bosses just expect you to rise heroically to every occasion, or your initial estimate is interpreted as a contract or an iron-clad promise, you just might be an in environment with dysfunctional beliefs about accountability. Consider finding another place to work.
Your estimate is only your best (and, hopefully, educated) view into when you will finish. It assumes that you don’t know what you don’t know. When you learn more, your estimate is obviously going to change. You’re better off working someplace that has at least an elementary understanding of this.
And now you understand it. Estimate based on what you do know. Go ahead and pad a little (a little!) for the unknown.
But then steer your work to successful completion.
I thank my brother, Rick Grey, who tests software for his supper, for contributing several key concepts to this post.
Jim Grey 