When I started this site a bunch of years ago I called it Stories from the Software Salt Mines, and gave it the domain softwaresaltmines.com. I thought it would be a group blog, me and a couple colleagues telling crazy stories from the work we’ve done. It would be cathartic and fun.
Instead, it’s been just me. From the start I’ve shared the things I’ve learned along the way about delivering software through leading people well. If you read every post on this site, you’ll have a pretty good idea about how I approach my work as a leader of engineers and testers. This site has become a portfolio of my ideas.
A few years ago when I launched a job search I realized that this blog is a good marketing tool. I changed the masthead to my name.
Now I’m changing the primary domain to my name, too. I’ve owned jimgrey.net for probably 20 years now, so I hung a subdomain off it and am redirecting it to this WordPress.com site.
With that, welcome to dev.jimgrey.net. My old domain is paid through April so I’ll keep it for a while and let it redirect. But one day I’ll pull the plug on my old domain.
The old name will live on in the background, however, as I used softwaresaltmines to register this blog with WordPress.com. If you’re ever feeling nostalgic for the old days, just type softwaresaltmines.wordpress.com into your browser. You’ll come right here.
I hired a software developer right out of college. He had a lot to learn, but he learned it steadily. Yet he admitted to me privately that he wasn’t sure he belonged. He thought that the other developers spoke so confidently and delivered so competently. He compared himself to them and, in his mind, came up wanting.
What he didn’t know was that I was leading developers for the first time. I’d been in management roles in the industry for a very long time, but always of testing and communications teams.
Worse, I hadn’t written a meaningful line of code in about a decade. Even then, most of that code was test automation. That’s not the same as writing product code.
Yet here I was, leading developers. The CTO who hired me wanted my skill in managing people, leading projects, and refining process. But I had so much to learn about modern software development. It was embarrassing to need the developers to explain the basics to me.
I told this young developer this story and admitted that I felt like an impostor, too. But I’d experienced impostor syndrome before. I knew that with effort and time I’d learn what I needed to learn and the feeling would abate. More importantly, even with all I needed to learn, I knew I had something valuable to offer right now. He did too, I told him.
We all figure it out as we go. In time, we build experience that lets us get it right more often.
What I wish I’d told him, what I’ve learned since then, is that there are three kinds of impostors. There are the impostors who don’t know they’re impostors. They’re so self-possessed that they overestimate themselves. There are the impostors who know it but do everything they can to hide it. They live in fear and anxiety that they will be found out. And then there are the impostors who know it, admit it to themselves, and sometimes even admit it to others. They’re the ones who can grow the fastest.
This young developer was the best kind: he admitted it. It let me tell him my own story, which helped put his mind at ease. Then it let us talk frankly about the areas where he felt like he didn’t know what he was doing, so I could pair him with other engineers who could level him up faster.
It’s a foregone conclusion that my company will keep us working from home through the end of the year. There’s no compelling reason to go back. We are delivering as well now as we did before mid-March, when the office closed. But more importantly, no amount of office safety precautions can eliminate our risk of the virus.
A group of voices in our industry has said for years that we would all be more productive and have a superior work-life balance if we all worked from home. Now that we’re all working from home and it’s basically working, they are crowing victory.
I say, not so fast. We’ve been at this only four months, not long enough yet to be sure. Let’s see how we feel in mid-November when it’s been another four months. Given the virus’s resurgence, it doesn’t seem far fetched that this could last even longer than that. Let’s see how we feel at the one year mark, four months later in March.
A software engineer on my team said to me recently, “I really miss everybody. I wish I could get a group together to go to lunch, like we did every day in the office.” I miss that, too. I used to bring my lunch, but anytime I wanted lunchtime camaraderie I knew I’d find some group of engineers going out, and I’d be welcome to join them. Many of those engineers are on projects I’m not involved with and I haven’t seen them in months.
He said he especially missed the in-person dynamic of our seven-person team: “I know I see you and the other engineers on the team on Zoom in our daily check-in meeting and in our regular tech discussions, but I miss our energy in the office. I feel isolated here.” Then he asked, “Do you think it would be possible to get us together, maybe for lunch at some restaurant outside? If people want more distance than that, maybe we could all bring our lunch to a big park?”
I checked in with the rest of the team and they were all enthusiastic about getting together, as long as we keep physical distance. We’ve laid in plans to spend some time together this Thursday, outside, at a place where we can stay spread out.
I also manage the managers of three other teams. Our four teams work toward common corporate goals, but we’re set up so that each team has its own backlog of work. It’s efficient. But part of what makes it work is that in the office we all sit in in adjacent groups of desks. This is deliberate. It lets us see each other all the time so we can stay in frequent contact.
It’s been harder since we started working from home. We’ve done our best to stay in touch, but haven’t found a replacement yet for the random, organic, quick check-ins we were used to. I’ve noticed an undercurrent of tension growing among the managers. Recently, there have been a couple spots of friction among them.
We needed to talk things through. My gut told me we’d be better off meeting in person rather than over Zoom. I asked the managers if they’d be comfortable meeting in a park with masks and good distancing. After they thought it over for a while they agreed — the benefit outweighed the (mitigated) risk. We met Thursday afternoon. We talked through the challenges we’re facing and came up with some collaborative and creative ways to stay aligned, both with each other and with some key peers we work with. All of the managers told me that it was good to be able to see each other in person, and to read body language as we interacted. They wished we could do it more often.
The distributed tech companies will all tell you that meeting in person from time to time is essential. But they tend to do it in big annual or semi-annual whole-company gatherings.
Companies like mine — most tech companies finding themselves with a COVID-driven at-home workforce — have not completed that evolution, if they’re even trying. We are used to the dynamics of working together in person.
Several people I work with closely tell me they’re perfectly happy and could keep this up forever. But the rest of us are still figuring out how to adapt. I can think of a couple people on my teams who might never fully adapt. They might need an office environment to work best.
I don’t look for my teams to meet in person regularly. We are all best off limiting our exposure to people we don’t live with. But during these summer months we can meet in person if we need to. There are plenty of wide-open spaces in our city parks, for example. But what happens when the warm-weather months end? This is Indiana, after all. It’s cold in the winter, and we get a lot of snow. For those of us who feel isolated now, just wait until winter takes away all of our reasonably safe options to connect in person.
All tech companies need to keep intentionally evolving toward more effective ways of working while distributed. I think it’s a long journey, one that is unlikely to be accomplished in the short term. I wish us all luck during the long winter.
I published this on my personal blog yesterday, Juneteenth 2020. It has a message for the people I try to reach with this blog, as well.
My dad was the sole white member of the Dr. Martin Luther King, Jr., Senior Men’s Club in my hometown of South Bend, Indiana. Dad was interested in economic development in the city’s depressed west side, where he had lived as a teenager. He worked with a few west-side groups trying to move initiatives forward, and the Men’s Club was one of them.
Through this, I think, he became aware of the challenges black families faced in South Bend. Whenever I’d see him he’d eventually steer our conversations toward those challenges. He could talk for hours about them. As a dyed-in-the-wool Republican he always advanced conservative solutions. The men of the Men’s Club were Democrats to the core and vigorously disagreed with Dad, but they came to like him anyway.
This is how I first heard of Juneteenth. There were celebrations every year on South Bend’s west side, and my parents always went to them.
I work in technology, specifically in software development. This industry is overwhelmingly the domain of young white men from the middle and upper classes. Where I work now, we have the most diverse team of anyplace I’ve ever worked. That doesn’t mean it’s highly diverse — we just have some women and a few black people on the team. We also have several immigrants from India and China and a few old white guys like me. That’s enough to set us well apart.
The murder of George Floyd by police in Minneapolis was the one-time-too-many that finally captured this nation’s attention. People of all backgrounds are saying enough. It’s time to treat black people with the same dignity and respect that is afforded white people.
At work, leaders have been talking a lot about how to improve the diversity of our team. Because I follow a lot of Indiana tech companies on LinkedIn, I see through their updates that they are having the same kinds of conversations.
I see it as systemic that our industry is made mostly of young white men. Most jobs either require or favor a four-year engineering-related degree. In my experience, that’s overwhelmingly the domain of young white men of some means.
Technology companies should eliminate the degree requirement. Then they should invest in training and apprenticeships. Yes, let’s apply the old trade model to technology. At least in software development, there are coding academies that teach the basics. Companies can band together to create scholarships to those academies, and heavily recruit people to those scholarships who are not young white men. They can then bring graduates on board and show them the ropes on the job.
Personally, when I have an opening on my team I scour LinkedIn for people who aren’t young white men and I recruit them. Plenty of young white men find my opening on their own. I’m still selecting from the pool of people who are already in the industry, blocking people who want to get in but don’t yet have the bona fides. But it has led to my last two offers going to a black software engineer and a woman engineering manager.
Out of the blue, my company this week announced that they are giving us this afternoon off to celebrate Juneteenth. My brother works for another local tech company that has one-upped us: they’ve made Juneteenth a paid company holiday, starting this year.
I’m cynical. Are we just signaling virtue? Will we actually carry our discussions about diversity through to action?
None of the social welfare and justice initiatives Dad was involved in drove lasting change. It was an uphill climb, and it required persistence and cooperation beyond what my father could arrange. The same persistence and cooperation is required to drive diverse hiring in the tech industry. It’s also needed across the United States for the good of our whole nation.
“You are unusually direct,” Elsa said to me. She was one of the first people I hired in my first management role, in the late 1990s. She said this to me on a few occasions as we worked on a large project together. I took it as a compliment then, but with hindsight I see that Elsa found my forthrightness to be challenging.
I say half-jokingly that I was this direct because I have hillbilly and blue-collar roots. My dad grew up in the hills of West Virginia. His family moved to Indiana to find factory and construction work. Dad worked in a farm-equipment factory while I grew up. In the culture I came from, anything less than saying it straight — no matter how much the words hurt — is seen as being untrustworthy.
I was proud of my direct manner. I believed that my forthrightness was good and valuable. It came from a place of wanting good outcomes for the company, customers, and my co-workers. I wasn’t trying to be a jerk, but that’s how I was sometimes perceived.
I’d been a manager for about 10 years when the fellow I worked for at the time said it to me: “Jim. You’ve got to stop leaving dead bodies behind when you talk. Learn some tact.” He told me he’d like to see me move up in the organization, but not while this behavior stood in my way.
That got my attention. I had been pitching fastballs at peoples’ foreheads. That boss coached me in throwing drifts that others can catch. I’ve practiced it ever since, and have built reasonable skill. It has unlocked all sorts of opportunity for me. It has helped me build influence and trust.
It took a long time for more nuanced communication to not feel wrong. It turns out I’m not among my hillbilly family, and I’m not working a blue-collar job. I’m working with midwestern professionals, and the rules are different.
I revert to my natural form when I’m anxious, over-stressed, or very tired. Those are not my finest moments.
But there are times when speaking directly is valuable. Emergencies are one such time. A couple companies ago I ran QA. Production went down while all of the ops managers were at a conference. I was ranking manager, so I dove in and, using my natural directness, led the team to quickly find root cause and get Production back up again. One engineer praised me: “You came outta nowhere and crisply and efficiently drove the train back onto the track. I’ve never seen this side of you!”
Another time is when I think I see something critical that nobody else does, and nuanced communication is not getting the ball across the plate. A flat statement can grab attention and change the conversation. It can also blow up in my face, so it’s a calculated risk. I’m hoping it works because it seems so out of character. “Whoa, Jim is really strident about this one. He’s usually so collegial. Maybe we should listen a little more closely.”
Finally, sometimes you have to say a flat “no” to a challenging request. I try very hard to find a way to say yes while highlighting the tradeoffs I or my team will have to make. “Could you deliver this feature two weeks earlier?” “Yes, if I pause work on this other feature.” Or, “Yes, if we trim scope and accept greater quality risk.” Or, “Yes, if we can flow some of the work through this other team.” But if scope, quality, and team are fixed and don’t support the timeline, I’m left to say no, and I do so plainly.
I will always wish I could be direct all the time. It’s how I’m made. But I care more about being effective than leaning into my basic nature.
This post expands on a comment I left on another post on this subject, on Johanna Rothman’s blog, here.
I often quip that when you plan and manage a sprint, as a leader you have to channel your inner Donald Rumsfeld.
You remember ol’ Don, don’t you? He was twice the Secretary of Defense in the United States, first in the Gerald Ford administration and later in the George W. Bush administration.
Rumsfeld is famous for having said, “There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns; the ones we don’t know we don’t know.” Here, listen:
He said this in answer to a reporter’s question about a lack of evidence linking Iraq’s government with supplying weapons of mass destruction to terrorists. It sounded so absurd at the time that the media had a field day ridiculing him.
But over the years, we’ve come to realize how much sense his statement makes. In anything you set out to do, you know what you know, you might know or be able to guess some things that you don’t know, and you certainly don’t know many things that you don’t know.
In agile software delivery — heck, in life:
You plan for the known knowns.
You make contingency plans for the known unknowns.
When unknown unknowns happen, all you can do is manage through them.
On a team I once managed, we wanted to build a feature to email end users the day after they abandoned a transaction on our site. We hoped to reengage them to finish their transaction.
We wrote and groomed the stories. Most of them had a clear implementation — plenty of known knowns. They were easy to estimate.
A few stories had known unknowns. One was, “It’s been a while since we’ve been in that repo, and it was written in our wild-west days. I might find some cruft in there that I’d need to straighten out to be able to finish this story.”
Another known unknown was, “I’d have to do a deep dive into that module to be sure, but if does this certain thing in this certain way, I’d have to do considerable extra work to stitch in the code I need to write there.”
In each case we discussed whether that work, were it necessary, would add meaningful complexity to the story and inflate its estimate. Sometimes it was yes, sometimes it was no. When it was yes, sometimes we wrote a dive story to find out and other times we thought the impact would be small enough that we could just accept the risk.
As we planned sprints with those stories, we believed we had accounted for everything we could think of, and were confident in our sprint plans. But two things went wrong we didn’t foresee — unknown unknowns bit us hard.
One story involved writing a job that would check overnight for abandoned transactions. The engineer who picked up that story encountered five or six serious unexpected difficulties with writing the job and with getting the job runner to pick it up properly. A story that we thought he would wrap up in two days spilled well into the next sprint.
Then it turned out that our email service was never built to handle the volume we threw at it, and it failed immediately in Production. One engineer devoted all of his time, and another part of his time, to solving the problem. A lot of our tester’s time went to troubleshooting with the engineers. The situation was surprisingly thorny. The engineers implemented several successive fixes over several days before finally getting past the problem.
Thanks to these unknown unknowns, we missed a couple sprint plans by a mile. Here’s how we managed through them: I worked with the team to prioritize stories and figure out which ones we were likely not to deliver. The engineers watched for stories ready to be tested, and helped our tester by picking some of them up themselves to keep the revised plan on track. Finally, I reset expectations with my management about what we could actually deliver, and how that would affect delivery of other work that depended on it.
You could argue that these should not have been unknown unknowns. Writing a job should have been cut and dried. We should have thought about volume in our planning, and tested for it.
Hindsight is always 20/20. Retrospectives are where you explore that hindsight and make improvements for the future. After you experience unknown unknowns, improvements generally take two forms:
They become known knowns, or maybe known unknowns. We learned a lot about the complexities of jobs in our world. We knew we needed to write a few more in upcoming sprints, so we broke them down into several smaller stories that we could test incrementally. Also, we now expected that there might be challenges in writing jobs we hadn’t encountered yet, and to lay in contingency plans for them.
You can lay in plans to neutralize them as unknowns in the future. We found out that other teams had been having trouble with the job runner. We met with the team that owned that code to figure out how to make the job runner more reliable. They put tech-debt stories in their backlog to handle it. We also researched how we might make the email service more robust and investigated third-party email services. We ended up going with a third-party service.
Lots of people in our industry have accomplished much, made a lot of money, and even achieved status or fame. It seems like lots, anyway. It’s probably a small but highly visible percentage. Whichever it is, it’s easy to wonder in this industry of plenty why that can’t be us.
When I remove the incredibly successful from the picture, it’s easy to see that I’ve done well for myself. I’m a middle manager of software engineers making a product you’ve probably heard of, in an engaged and positive workplace. We all make an upper-middle-class living.
Yet I sometimes wonder why I’ve made it only this far, why I’ve not gone farther. And I wonder whether I’ll be able to sustain my career as it enters its waning years.
You see, today marks 30 years that I’ve worked in the software industry. It boggles my mind that I’m only two-thirds of the way through my career — I still have 15 years to go before I’m of normal retirement age.
Fortunately this is all I’ve ever wanted to do, ever since I taught myself to write code in the early 1980s. I went to engineering school to get the credential I needed, but graduated during a recession when jobs were scarce. In a nationwide search I couldn’t find coding job. I managed to land job in the software industry writing manuals and online help, and I was grateful to get it.
I liked the work and became quite good at it, so I kept at it for eight years. Then I transitioned to QA, where I led functional testing, test-automation, and performance-testing teams. I did that for 17 years. And now I’ve transitioned back to my roots in a way by leading engineering teams.
I’ve lived through crazy growth and sudden downturns in the industry. Several times I’ve needed to find a job when one ended unexpectedly; several times I’ve been poached away to a better opportunity. I’ve worked with good people who have taken good care of me, and with charlatans and egomaniacs who stabbed me in the back.
Sometimes I’ve had the tiger by its tail, and sometimes the tiger’s had me in its mouth.
Dozens of people have reported to me since I shifted into management. It’s a small tech community where I live; I bump into many of them a lot. Some of them are still individual contributors, some of them have become managers and leaders, and a few of them have gone on to even greater careers than I’ve had, reaching VP levels or reaping tidy sums when their startups exited.
But I’ll bet every last one of them has had good luck and setbacks all along the way, just as I have. We all sometimes have to figure out how to move forward from here in our careers. We all have to figure out how to stay relevant and vital, especially as we age.
I’ve been in management for 20 years, during which time I focused on being the best leader I could be. It has served me well.
I couldn’t both become a strong leader and keep my technical skills current. I struggle a little bit to swim as fast as my peer engineering leaders, most of whom have written code recently. I can see it’s time to rebuild my technical skills. I don’t expect, and I won’t try, to become the equivalent of a senior engineer. But to have a solid understanding of the technologies involved in my company’s product, to better evaluate the code of someone on my team and maybe even to be able to fix a bug, to be able to write a SQL query more complicated than SELECT * FROM table_name; — it’s time. I’m a little daunted, but in I go nevertheless.
I’ve learned a lot in 30 years. In many ways, I’ve become wise. But I’m still trying to figure things out as my life and career unfold. I can see that this never ends.
Your crystal ball is cloudy, but you can sort of predict the future. As long as the future isn’t too far away.
In other words, if you take it in small chunks you usually know enough about the work before you to guess reasonably accurately how long it is going to take.
Back in the old waterfall days, we’d plan months and months of work at a time. The printed project schedules could be 100 feet long. The trouble was, work more than a small number of days away often depended on work that had not yet happened. We needed to know how that work turned out before we could be sure of the work that would follow, and have a good idea how long it would take.
From a straight-up project-management perspective, this is why agile is brilliant. It lets us break big projects down into small chunks of usually two weeks in duration, where ideally we have to give our best estimates only for this short period. This respects that we don’t know what we don’t know about the work that follows.
You might still need to estimate how long an entire feature will take to build. Some companies need some idea when the feature will deliver. But it has to be a projection, not a promise, because your crystal ball is too cloudy.
The current sprint’s estimate matters most. When you plan a sprint, fill it with just the work you are confident you can finish. You want to deliver all of the work in the sprint, barring unforeseen technical complexities, critical customer-reported bugs, team-member illness, or late reviews.
Then check your gut: do you think your team can really deliver all of those stories? If not, please say so, and discuss why. When you leave sprint planning, everyone on the team needs to agree, based on what you know then, that this body of work seems doable in two weeks.
I enjoy the Signal v. Noise blog from Basecamp. Recently they published a post about the human touch in customer service. Read it here. Its thesis, in short: automating any part of support is akin to saying the company doesn’t care.
I am charmed by the work ethos Signal v. Noise espouses: work at a sustainable pace on things that are interesting to you, and get plenty of rest. It’s very appealing. But sometimes I think they take the contrarian position as if it’s their brand, and this is one of those times.
There’s nothing wrong with a good robot providing routine customer service. As long as the experience is good, and can seamlessly hand off to a human when things go off script, it’s a win.
That’s because so much of customer service work can be rote and repetitive. One of my career stops was at a company called MOBI, which provided mobile-device management services to large companies. One customer was a global manufacturer that had something like 10,000 mobile phones deployed to its executives. MOBI helped them manage cost by keeping each phone on the least-expensive plan for its typical usage, and also provided a help desk with the promise of answering a request in 30 seconds and staying on the case until it was solved. (Try getting that from your carrier.)
But it was too expensive for MOBI to hire a bunch of new customer-service reps every time they took on a new customer. It cut the profit margin too thin.
Moreover, a large percentage of the customer-service work was simply boring and repetitive. After you’ve done a handful of password resets, equipment orders, and plan changes, you can do them in your sleep.
MOBI has devised a number of robots (“Mobots,” they call them) to take away the drudgery.
If a MOBI customer user jumps on a chat, Mobot Audrey answers. She can handle many of the common, repetitive tasks. The minute the customer asks something she can’t handle, she seamlessly transfers the chat to a human, who handles the case from there.
I had lunch a few weeks ago with MOBI’s CTO, my former boss, who told me that the customer-service team was apprehensive at first, but is happy now because they are working on things that tap into their deep knowledge of the product, and that require them to solve problems creatively. As a result, they are more engaged with their work.
This resonates with me from another perspective. I was on the team that built the call-center software that Medicare customer-service reps use nationwide. I led the test team. Our contract required us to do a “full” regression test on every release — meaning we had to run thousands of written test cases. It took almost 24 person-weeks to execute those tests! This was about 15 years ago when automation tools weren’t great. But I hired a couple engineers to automate those tests anyway, and they finished after a few months. It took just 0.6 person-weeks to execute those tests forever after. My functional testers were free to spend more time performing targeted, complex, and exploratory tests, using their experience to find critical defects the automation couldn’t find. It was much more interesting and meaningful work for them.
But back to customer-service robots: we all remember how bad the early ones were. That Medicare customer-service center had an early voice-response robot for callers, and it was awful. It didn’t understand callers right most of the time, it took a long time to navigate, and once navigated the caller found that the robot couldn’t help them and they needed to speak to a human anyway. Callers hated it.
But the technologies have begun to mature. Some of them claim to be able to learn, even, although I’m skeptical that it’s true, full-on AI or ML. At least it’s possible to create a good experience now. MOBI’s Mobots have done it; their user testing bears it out: most users have no idea that they are interacting with a Mobot.
So why wouldn’t a company like MOBI manage support costs and increase employee happiness by automating away the boring, repetitive tasks? I’m for it.