Author: Jim Grey

Writer. Photographer. Leader of geeks.

Categories
Quality

Programming is the number one quality assurance activity

By Jim Grey (about)

Programming is the number one quality assurance activity.

Or maybe it’s design. Or maybe it’s writing good user stories. Or maybe it’s having good ideas for things to build in the first place. Or maybe it’s paying down technical debt.

Test

But it sure as hell isn’t testing.

When I talk to engineering leaders at small software development shops and they find out I make my living in testing, many of them admit that they don’t have any testers yet. They’re sheepish about it. It’s as if I’ll be offended!

I tell them to rock on, and to delay hiring that first tester for as long as they can.

And then I ask them about the quality challenges they have. Too many bugs? Won’t scale? Bogs down under load? Don’t fall prey to the gut reaction “oh my god we need to test,” as if testers are a magic filter through which perfect software passes.

Because if you respond by hiring testers, you’re likely to end up with testing theater. Your testers will do testery things and find bugs, sometimes even good ones, bugs that let you sleep better at night.

But they can’t fix your quality problems. Only your developers can do that. Instead of letting your developers do whatever it is they do and hope a tester can find everything that’s wrong, challenge your developers to get better.

I ask them these questions:

Do your developers have the skills needed to build the software you’re asking of them? If not, help them build those skills or, gulp, replace them with developers who have them.

Are you following good development practices? Test-driven development, pairing, and code reviews. Not only do they promote solid code, they help create a culture of quality among your developers.

Is your team writing lots of automated unit tests and and acceptance tests? (By acceptance tests, I mean thin functional tests at the API or controller level.) Do they run on every commit? This traps basic problems as soon as they’re introduced.

Do you have a well-functioning delivery methodology, at the right scale for your organization? If you’re two developers, that might be a kanban board drawn on a whiteboard. If you’re ten developers, you might use a tool like Pivotal Tracker and have a couple defined but light rules and ceremonies. If you’re 100 developers, it might be some scaled agile methodology like SAFe, backed up with a tool like JIRA, guided by a program management office. Whatever it is, are you following it and is work flowing smoothly through the delivery pipeline?

Are you giving your developers the time they need to do thoughtful work? To design resilient software that performs? To architect it for long-term growth?

Do all of these things before you hire your first tester. Your developers will give you a level of quality that makes it hard for testers to find low-level bugs. Testers will then have time to find more interesting and valuable bugs because basic stuff is seldom broken. This makes testing a lot less expensive, by the way. You need way fewer testers when you deliver software to them where core functionality works.

And then your testers, instead of feverishly testing the basics, can contribute at a higher level. Testers bring a different mindset to your teams, one of critical thinking about how the software is deployed and used. When you harness that mindset, you can improve your designs and your architecture before you write that first line of code.

Categories
Quality The Business of Software

If you want your software to keep producing, be prepared to do some dirty work

By Jim Grey (about)

A woman named Verna built the house I live in. She landscaped it nicely; a sprawling flowerbed stretches in front of my front door and picture windows. Every spring, I eagerly await Verna’s spring color: yellow daffodils, purple hyacinths, red tulips, and finally the giant pink peonies.

Home

I’ve added a few things: lilies, mums, lavender, coreopsis, phlox. I love phlox! But my eagerness to keep adding color petered out pretty quickly because it turns out I hate digging in the dirt.

I don’t much enjoy any of the other routine garden maintenance, either. Mulching. Deadheading. Dividing overgrown plants. Weeding – oh god, the weeding. Does it make me lazy that I just spray my weeds with Roundup and move on?

I just want to enjoy the flowers. But this ninth spring I’ve lived in my home, a few of my plants didn’t come back as strong. A couple didn’t come back at all.

So I asked my mom. She’s the gardener in our family. “When was the last time you fertilized?” she said. “Um, never,” I said. “Ah,” she said.

It turns out that you can’t just ignore the soil, or the plants themselves for that matter. Things growing in it year after year uses up all the nutrients, and crowded plants compete with each other for what little is there. “I’m surprised your flowers didn’t stop coming back a few years ago,” Mom said.

Daffodils

I did some serious fertilizing this season. I also separated some overgrown hostas and moved some of Verna’s plants so they had some elbow room. Not fun, but necessary.

♦ ♦ ♦

I once worked for a software company whose flagship product sold briskly. Version 1.0 was five years in the past, and since then we’d added lots of new features so the product could continue to lead the market. And now here came the head of Product Management asking for more new features

Dan, a quiet fellow, graying at the temples, led Development. “Well, yes, we can add all those features,” Dan said, adjusting his glasses. “This one will take six months. That one will take four. This other one, well, I think that’ll take a year.”

The Product Manager was dumbfounded. “Features of similar scope took far less time in the past, and you had fewer developers then. What gives?”

Dan looked up at the Product Manager kindly, and drew a breath. “Well, we’ve been under such pressure to quickly add features to this product that we’ve not focused enough on its overall design. We’ve also made no time to keep our underlying architecture up to date. These are things I’ve been pointing out all along the way. But we’ve just bolted features on wherever we thought we could get away with it. Now, to add any one of the features you’ve requested, we basically have to unbolt three or four other features, and blend the code all together. And we have to write complicated bridge code to do modern things with our aging architecture, and when that doesn’t work we will have to upgrade some parts of it and test the product well to make sure everything still works. It’s a slow process. And it’s just going to get slower and slower the longer we keep going like this.”

That the product’s design had become cancerous and the underlying architecture had gone out of date were not considered a crisis –- but not being able to rapidly add new features sure was. It focused the company’s entire attention. Their response was to code up a “next generation” product from scratch, which was a disastrous idea for a whole bunch of reasons beyond the point of this story. When the dot-com bubble burst in 2001-2002, they had not yet successfully launched the next-generation product, and they still couldn’t add features to the old product fast enough. Revenue fell precipitously. Quarterly layoffs began, but it was not enough to keep the wolves from the door. That once-promising company was sold; the company that bought it outsourced development to China.

More recently I went to work for another promising software company. They had been in business for about a decade and had sold their software to a number of very large companies. But in the couple years before I’d been hired, the pace of new feature delivery had slowed to a crawl. Adding new features had become increasingly difficult and always broke existing features. As a result, it took longer and longer to test the product, but even then, major bugs were still being delivered to customers. Meanwhile, younger, more nimble competitors were stealing business away from us. As the rate of new revenue decreased, support costs skyrocketed. It was unsustainable, and that company, too, had to sell itself to another company to avoid collapse.

It was much the same story: the company had focused entirely on rapid new-feature delivery and not enough on ongoing design and architecture. After a decade, their soil had gone infertile and the code had become tangled. Nothing new would grow.

♦ ♦ ♦

Software as a garden: to be able to grow more software, to be able to grow revenue with it, you have to keep the soil fertile and give the roots room. The problem is, gardening projects are a hard sell. These are things like refactoring older parts of the code that no longer serve efficiently, or upgrading or replacing outdated parts of the architecture, or redesigning subsystems that work fine today but can’t adapt to things the company wants to do in the future. When you tell executives you need to do these things, what they hear is that they can’t have new features while you do it. New features fuel growing companies.

But if you don’t tend your garden, sooner or later it will stop producing.

Categories
The Business of Software

Stop outsourcing support

By Jim Grey (about)

There’s a trend among the hip companies in Silicon Valley to move tech support departments to middle America. Lauren Smiley wrote about it recently for Medium’s Backchannel, saying that it’s all about cost. You can simply pay support techs less when they’re not on the coasts.

But it has the consequence of blocking support techs from moving up into more interesting — and more lucrative — roles in the company. It makes support a total dead end job. But more importantly to the company, it takes away a key element that can make the software better.

Abandoned National Road and US 40

Support jobs aren’t always an automatic growth opportunity. The supply of support techs usually dwarfs demand for roles they can move up into.

But it’s no wonder support techs want to move up — support is hard. For less money than other technical roles in the company, you have to listen to unhappy users all day and try to help them get the promised value from your company’s products. It’s emotionally draining. I’ve heard support techs joke more than once that support years are like dog years: every year you work feels like seven.

Still, support really can be a good place to grow talent for other teams because techs know the products and, more importantly, how users actually work with and experience them. Support techs spread this invaluable perspective anywhere they go in the company. This is why I love to hire support techs into my testing teams.

But another reason it’s good to keep support in the same building is that it makes the software better.

Here in the Midwest, where cost of living is generally low, the startup, small, and medium-sized companies where I’ve worked don’t outsource customer service. Those workers are already plentiful and inexpensive. And so support is always down the hall or on the next floor. Developers and testers become friends with many of the support techs.

To developers and testers, users can be an abstract concept. It’s a shame, but since we don’t know them and don’t experience our products as they do, we feel freer to make choices that are expedient for us but potentially unpleasant for them.

But we always get an earful from support when things don’t work well! And we don’t want to create difficulties for our friends there. It makes us work hard to deliver software that doesn’t make the phones blow up.

Successful software delivery is a team sport. Don’t cut off a key part of the team just to save a dollar.

Categories
Managing People

I’m so over performance reviews

By Jim Grey (about)

It’s annual review season where I work. I wrote and delivered my team’s last week. But my heart wasn’t in it.

It’s because if I’ve done my job as my team’s leader, they already know what I think of their performance. It’s been an open conversation all along, happening most often in our weekly 1-on-1 meetings but also through in-the-moment praise and coaching. Nothing I tell them in their review should be a surprise.

And we should have also had occasional discussions about what excites them about their work, what challenges they’re facing, how well they’re working with their teams, how they’re experiencing me as their boss, what they need from me to be happier and more effective, how they do and don’t enjoy the company’s culture, and what they want from their career and their life. Hearing their full perspective on their work, the company, and their career, I have what I need to promote the best balance of happiness and productivity for them.

Bump

That’s because as their boss I am only as successful as they are. The more and better they do, the more and better I do and the company does. Giving feedback and coaching for ever greater performance is one of the primary two things a manager is for. (The other is promoting conditions that enable the best possible work to be done.)

And let’s say someone on my team had a bump in the road during the year — an incident where things really went poorly on their watch, or the quality of their work went south, or a particular behavior needed correction. If those issues have been resolved, it is often hurtful to remind them of them at review time. “I gave you a Needs Improvement rating here because of that thing earlier in the year. You know, the one you fully recovered from.”

Bah. I’d like never to deliver another performance review.

Unfortunately, every company that has employed me has based raises on review scores. (Amusingly, a couple of them have sworn review ratings weren’t coupled to raises — but every time I asked for a larger-than-normal raise, the first question asked was whether their review score justified it.) So I still do reviews.

But if you’re doing the valuable (and time-consuming) work of giving feedback and coaching to your team all year, reviews take little time to write. You don’t have to think as hard about their performance all year because it’s been an ongoing topic of discussion. Just summarize what you’ve talked about all along, give some examples, assign appropriate ratings, and you’re done.

Categories
Career Testing

How to hire an entry-level tester

By Jim Grey (about)

Where I live and work, it’s difficult to find experienced developers and testers. Demand outpaces supply. So one strategy I’ve been using to grow my testing team is to hire people who have never tested.

Obviously, that means they need to be taught how to test. It so happens I like doing that! But this means I have to be able to identify non-traditional candidates who are apt to be good students.

I look to my company’s customer-service team for candidates, as those people know the software and understand the customer. But that’s not enough; I need to know whether they can be taught to test.

Faces

Fortunately, I’ve figured out four key traits someone needs to be able to learn to test and to enjoy it. The new tester must have a knack for troubleshooting, which means they are curious and persistent, can create mental models of the system they’re examining, and can develop and investigate theories of why things are broken.

Be curious:  A tester needs to notice when something is wrong and feel compelled to investigate. “Wait, what? Why did the software do that? I’d better dig in and find out.”

Be persistent:  But sometimes the investigations take a while. A tester needs to keep feeling compelled, even when the investigation stretches on. (Bonus points if they recognize when they’ve reached a point of diminishing returns, and choose a different avenue of exploration.) 

Create mental models:  Building a mental model of a system, even if it’s incomplete or partially inaccurate, helps a tester orient themselves to a problem and generate ideas on how to work through it.

Develop a theory of error:  Thinking about how the current problem is creating or exposing a flaw in the system informs the steps taken to prove or disprove the theory — and isolate bugs.

A person who’s not curious won’t troubleshoot. A person who’s not persistent won’t keep going when the troubleshooting gets rough. Troubleshooting doesn’t get very far if you can’t see how the thing you’re troubleshooting is a system and imagine reasons why it’s not working.

So when I interview testers, I ask this question, which gets at all of these things:

Think of a time you had a problem with software or electronic equipment when you tried to solve the problem. It might have been trouble with your home network, some difficulty upgrading your PC or your phone to a new OS, a misbehaving printer, or anything that went wrong with technology you use. Tell me how you assessed the problem and tried to figure out what was wrong, and what steps you took to try to fix the problem.

I ask candidates to tell me two or three stories in answer to this question. Many people can come up with one story, but someone who enjoys, or at least doesn’t mind, getting to the bottom of technology problems will have more than one story.

I’m discouraged when I hear these things in answers:

  • Trying nothing beyond obvious troubleshooting steps, such as rebooting
  • Giving up after a couple steps: “I tried a couple things and then just called support”
  • Trying things randomly in hopes one of them works
  • Fear of trying things because they might break things worse

I’m excited when I hear these things in answers:

  • Developing a theory of how the system works, including applying knowledge from other systems — “I wonder if this thing I’m trying to figure out is like that other thing I know very well”
  • Some logical, systematic method for exploring the problem, generally starting from the simplest, most probable causes and moving stepwise through successively less likely and more complicated possible causes
  • Research, usually via Internet search but also by asking someone who might know
  • Evidence that they learned something about how the system worked, and felt some satisfaction in that

It’s fine if the candidate didn’t successfully resolve the problem. What I’m listening for is whether they systematically eliminate possibilities and move toward finding root cause.

Good answers can be mundane. One candidate told me about sitting down one night to watch Netflix, but his movie kept failing to stream. He was used to experiencing a little of that on Friday and Saturday night; he knew Netflix drew a lot of Internet traffic on those nights. But because it was a weeknight he wondered whether the problem might be within his home. He knew that rebooting the Roku usually solved connectivity problems, but it didn’t work this time. So he went over to his AT&T Uverse modem to check out its blinking lights. “I don’t know much about those lights,” he said, “but I do know that if most of them aren’t blinking, especially the INTERNET light, there’s a problem.” Sure enough, the INTERNET light was out. So he tried rebooting the modem, but the INTERNET light wouldn’t come back on. He found his modem’s manual, but it didn’t cover the situation. He tried a couple Google searches for this condition, but found no useful information. Out of ideas, he called AT&T, which reported intermittent outages in his area.

The candidate’s answer showed that he created a mental model of how Netflix and the Internet work and tried the most likely causes first. I especially liked that he confidently used his incomplete technical knowledge to narrow down the problem. And I didn’t mind it a bit that he ended up calling for help — he’d first done all he reasonably could.

The rest of my interview questions look to gauge technical experience and fit for the company and the team. It’s pretty standard stuff. But without good answers to the troubleshooting question, the candidate goes no further.

So look for these four traits. They give you trainable team members.

Thanks to Rick Grey for his valuable feedback that helped shape this post.

Categories
The Business of Software

To Microsoft: Kudos on the Windows 10 upgrade experience

By Jim Grey (about)

I’m a Windows user. I see how all the cool developers have switched to Mac. I was a Mac user once, so long ago that MacOS was still called just System, and it was on version 6. Work demands switched me to Windows before System 7 came out, and I’ve just stayed with it.

And until recently I’ve never upgraded from whatever version of Windows shipped with my PC. But I bought a cheap Windows 8 laptop a few years ago to schlep to the coffee shop to check email and write in my blog, and I found the “tile world” Windows 8 UI to be enormously frustrating. I took the unprecedented step of upgrading to Windows 8.1 to ease some of the usability headaches.

Except that it took three agonizing tries before it worked. The first two times it spent hours and hours trying to upgrade but failed utterly, I mean black-screen-with-blinking-cursor utterly, and I spent hours and hours restoring my laptop to Windows 8 so that it would function. It was an ungodly, gory mess. I gave up and planned to just stay on Windows 8.

The third upgrade try happened involuntarily. One day my laptop announced that it had downloaded 8.1 in the background, and that the next time I restarted, it would install itself.

Nooooooo! I scoured the Internet trying to figure out how to prevent the upgrade, but found no help. I felt doomed. And sure enough, upon reboot, it went into a long dark night of upgrading.

This time, finally, it came back with Windows 8.1 installed. But hardly anything worked. The display was wonky, my wireless mouse was unresponsive, I couldn’t connect to my printer. I had to update a whole bunch of drivers and change a whole bunch of settings, and finally the laptop worked again.

I was relieved, but resolved: never again!

My favorite sippin' glass
Bourbon: essential to any Windows upgrade

So when Microsoft announced Windows 10, I wasn’t very excited. My laptop was working okay on 8.1, and I was perfectly happy with Windows 7 on my old desktop. But the more I read about Microsoft’s plans for Windows, the more it became clear that Microsoft was changing its ways to not support old versions of their software for years and years anymore. To keep getting good security updates, the day was coming when you’d need to be on the latest version of Windows. The writing was on the wall.

So I reluctantly decided to test the waters on that laptop. I started early on a Saturday just in case I needed to spend the day troubleshooting.

I started the installer. I stood by, my anxiety building. And then an astonishing thing happened: it went flawlessly.

The installer showed me progress every step of the way. It rebooted the computer several times, but each time it came back successfully and kept going. And then it rebooted one last time and there was Windows 10! And everything about my laptop worked properly. Elapsed time, less than 30 minutes.

And then I liked Windows 10. It was enough like Windows 7 that I didn’t have to relearn how to use my PC, but was fast and stable and had some neat new functionality that I wished my old Windows 7 desktop had.

Before long, my Windows 7 desktop notified me that it had downloaded Windows 10 and would upgrade as soon as I gave the word.

Fear stabbed at my heart. My desktop PC was reasonably well tricked out when I bought it — Intel Core i5, 8 GB RAM — but was getting up there in years. I’d installed and uninstalled all manner of software on it. My kids had installed all sorts of games on it and had surfed their way into a couple viruses I had to clean up. So this machine wasn’t exactly pristine. And this PC is the hub of my digital life. All of my photographs, all of my writing, all of my financial and personal files, everything is on it. I have good backups, but they didn’t entirely salve my anxiety. So I waited.

Over the next few months, the popups kept coming: Are you ready to upgrade? Windows 10 is all downloaded and ready. All you have to do is click here! Click the button! The beautiful, shiny button! The jolly, candylike button!

One night I sat down to write. I’d poured myself a bourbon (good stuff — Woodford Reserve Double Oaked). It was busy lowering my inhibitions when Windows popped up another invitation to upgrade.

I was tempted. I poured myself another bourbon, a double. And then, impulsively, I clicked the button.

Instantly, I was flooded with regret and anxiety. But then the install went as flawlessly as on my laptop. It took a little longer, about 45 minutes, but my PC came back up ready to go.

There was one small hitch: my mouse’s scroll wheel no longer worked right in Chrome, and only in Chrome. Updating the driver solved it.

I’m impressed. Microsoft, my hat is off to you. I don’t know how you did it, I don’t know what has changed inside your organization, but this was a great upgrade experience.

Categories
Quality The Business of Software

Don’t piss off your users by suddenly changing your UI

By Jim Grey (about)

Delivering software on the Web is great. Especially with continuous delivery, we can deliver changes large and small anytime we want. And then we can get quick feedback from our users and the market, adjust the software accordingly, and push those updates fast, too. It’s utopia and the Holy Grail rolled into one!

Except that users are not very excited when we change things. They want software to stay as it is. Well, mostly: they want us to fix the bugs that affect them, of course, or even to add this or that little feature. But please, they plead, don’t make it work differently than it does now.

Meanwhile, we face various pressures. Markets shift; new needs emerge and old needs become less important. Technologies shift; old frameworks become outdated, new frameworks enable us to keep pace. Today everything has to not only work on mobile, but feel native to mobile — and all run on a single codebase. This is shifting product direction across our industry.

That’s the backdrop against which WordPress, the content engine behind one out of every four Web sites, rolled out a new editor last week. It was part of a complete rewrite of all of WordPress.com. Their old technologies just couldn’t stretch to where the world was moving. So they threw it out and started from scratch. Their new editor is fast — fast! — and works fluidly, while looking great both in my browser and on my phone.

2015-11-22_2157
Spanking new editor in my browser…

But boy, were users pissed. Pissed! Check the WordPress.com forum: 19 pages of complaints and counting. Sometimes, I swear, users wouldn’t be happy if you sent them gold bars, because they preferred the silver bars you used to send them. But very often users have a point: they’ve gotten into the swing of your software, and now you’ve changed it and they have to learn it all again. Worse, maybe now something they used to be able to do isn’t there anymore, or if it is, they can’t find it.

IMG_4117
…and on my phone

For the record, I was the first commenter on that thread, because I experienced some of those frustrations. I tried to be kind, but several features I use either went missing or were accessed in a way that I couldn’t easily discover. Argh! And I wished the editing space were wider; it felt awfully cramped. I wasn’t alone in any of these complaints.

I wanted to just edit a post. I didn’t want to learn a new interface. But I found that there’s no way to just revert to the previous editor. It is simply gone.

I understand what drives changes like this and know that this is a monumental achievement this is for WordPress. Still, because I’m a heavy WordPress user, more than anything else I feel frustrated. The new editor breaks all of my usage flows, and I’m having to rediscover everything. I didn’t want this.

It’s the same, by the way, with Microsoft Office’s ribbon, which replaced an older menu structure way back in Office 2007. That’s forever ago in software terms. Yet there are still features I can tell you exactly how to find in those old menus, but I have to Google where they are on the ribbon.

Users don’t give a rip about your business or the future of technology. They use your product to accomplish a thing. As long as they can consistently and easily accomplish that thing, they stay happy. Many users learn your product’s nuances and become quite adept with them. When you suddenly change the UI and all of their flows are interrupted, of course they’re frustrated.

So what would happen if you followed Basecamp’s model? Their software helps companies small and large manage projects. Last month, they released Basecamp 3, a ground-up rewrite — yet they received not a single complaint from existing users. That’s because Basecamp 2, and for that matter Basecamp 1, remain fully active. Existing users can upgrade if they want, or stay put if they don’t. There are compelling reasons to move to Basecamp 3. But if you’re a happy Basecamp 1 or 2 user, those products will be there, fully supported, for as long as you want to use them.

Maybe your company can’t do that. But what can you do to ease the transition for your users, so they can stay fully productive? Think this through. It’s more important than any technology or implementation decision you make.

Fortunately, WordPress does, for some reason, still provide back-door access to an even older editor.

2015-11-22_2158
Outdated but highly functional classic editor

I don’t care that this is based on outdated technology: it’s fully featured, and I know how to make it sing. I cut my blogging teeth on this editor when I started my personal blog in 2007. I’ve written over a thousand posts in it. I hope it never goes away.

Categories
Quality Testing

Fast failure recovery lets you take more risk and increase speed

By Jim Grey (about)

I can just imagine the “oh no” moment that rippled through Feedly last week when they learned that their big new release for iOS 9 crashed on launch for iOS 7 and 8 users. Their first response: add a hastily-written warning to the App Store description.

IMG_4066

Their second response: fix the bug, fast. It was ready the next day.

IMG_4069

Apparently, Apple has a way for developers to rush the very occasional critical fix into the App Store, sidestepping the normal, weeks-long approval queue.

Apple’s App Store fast track provides a safety net, and I don’t blame them for using it. Because Feedly could recover fast, hardly anybody will remember this gaffe, and it won’t cost the company very much.

I actually feel slightly bad talking about it, because it keeps the memory of this bug alive. But it illustrates such a simple equation: the faster you can recover from failure, the less perfect your product has to be when you ship it, and therefore the less expensive it is to build and support, and the faster your company can move.

I’m not advocating for delivering buggy product on purpose. Follow good development practices and test for important risks to deliver the best software you can as fast as you can. But when you inevitably deliver a bad bug, being set up to recover fast means you can deliver without worry.

I remember when the best way to get software to users was to mail it to them on CDs. A bug of this magnitude was a much bigger deal then. It was harder to warn users of the problem, and lots slower and more expensive to correct it. In that world, Feedly’s bug could have damaged their reputation for a long time. So back then it made sense to test more thoroughly — and therefore spend more time and money before releasing.

But today, in a world of Web software and 24-hour emergency App Store turnaround, you’ll deliver faster and with less expense when you set yourself up for fast failure recovery. Continuous integration and continuous delivery are usually a part of that strategy.

Categories
Quality Testing

You can’t test it all

By Jim Grey (about)

That new build of software you are about to test? It’s a haystack with some unknown number of needles (bugs) in it.

Hay rolls 3
Have fun finding all the needles!

As a tester, you might think your job is to find all the needles. But how do you do that when you don’t know how many needles are in there? What if there are a lot of needles in there? You’ll never have time to find them all.

You need a plan. You want to find the showstopper bugs right away, and then find as many other bugs that people will care about within the time you have. And then when they come breathing down your neck to stop already and ship, you want to be able to tell them just what badness still might lurk in the code. Give them a reason to think it over.

You do that through assessing risk and targeting test coverage. To assess risk, ask yourself some questions:

  • How stable is the code that was changed? What interactions within the software might these changes break? You’re trying to figure out how likely it is you’ll find bugs.
  • If stuff around these code changes is broken, how much could it hurt the user? How much could it hurt your company? You’re trying to figure out the impact of any bugs you might find.

Risk is the product of likelihood and impact. Test for the highest risk bugs first, working down through the risks. Test more deeply for the bigger risks, more lightly for the smaller ones.

Let’s say they want to ship before you’ve tested through the risks you think people will care about. You can then talk about the risks you haven’t checked for yet, and ask if they’re okay with shipping like that. Do you see the mind shift here? You’re not saying you haven’t run all of your tests yet, which sounds an awful lot like you can’t keep up. Instead, you’re saying that the code might not be ready yet, and here are the specific things you’d like to still check for. It puts you in a much stronger position to get that extra time — and makes it the boss’s decision about what to do next.

Ultimately, it’s best if your developers can and will take great care to not deliver so many needles. That’s always the best case. Click here to read more about it.

Categories
The Business of Software

Personal computers are for content creators; mobile devices are for content consumers

By Jim Grey (About)

Wired wrote last month about the ongoing decline in PC sales. They weren’t the first to notice, but unlike others they made a strong statement: it’s the end for the PC.

It’s hyperbole. The PC’s market share is just shrinking to the audience that has always truly needed it: people who make things.

This seems so obvious to me. Isn’t it to everyone? Or am I missing the boat here?

Because it’s not like PC sales are on their last leg. Wired admits that more than 293 million PCs are expected to ship this year. Mobile devices are more than keeping pace, however. Wired also notes that 71 million iPhones shipped in the first quarter of 2015 alone.

I predict that the cheap PC era is about to end. A few years ago I bought an entry-level laptop for $300 and use it for checking email and Facebook when my son is using my main machine. I also use it to Chromecast YouTube to the big TV in my family room. An iPad or a Surface would work just as well and take up a lot less space. I kind of wish now I’d gone with the Surface.

It’s because mobile devices are simply brilliant for consuming content. I do most of my reading, get most of my news, and watch virtually every video I see on my mobile devices. I actually consume more of this stuff thanks to these devices than I ever did before I got them.

Fixing computers
Me, a long time ago, fixing some old computers. My iPhone is probably more powerful than these machines.

Before phones and tablets became viable Internet devices, the only way to consume Internet content was on a PC. We were pretty happy with that until we discovered, much to our collective delight, that an Internet device in our pocket let us pleasantly wile away the minutes we spend waiting — in doctors’ waiting rooms, before meetings begin, in the john.

Oh sure, on your phone you can type a quick message for the world to read on Facebook, Twitter, Tumblr, even WordPress. You can take a video and upload it to YouTube, or a photo and upload it to Instagram. There are even some apps that let you do some decent photo and video post-processing now.

But that’s all casual content creation. The PC still shines at serious content creation because you get a usable keyboard, a dedicated pointing device, a bigger monitor, and plenty of computing power.

Take this blog post as an example. At 554 words, there’s no way I’d tap this out on my iPhone. I’d hurl it across the room in frustration after five or six sentences. I might get this far on an iPad before feeling fatigued, but I wouldn’t want to write an epic post on one.

And as a hobbyist photographer, I want a bigger monitor when I process my photos. So does my youngest son when he edits the YouTube Poop videos he makes. And when both of us work with large files we’re glad for all the memory and CPU that my 16 GB, Intel i7 PC offers.

Adding accessory keyboards and pointing devices to mobile devices can help, but then those devices become more and more like PCs.

So no, the PC isn’t dying. It’s just found its place.