• Luke Hutscal
• Choose boring technology

Allen: Welcome to It Shipped That Way where we talk to product leaders about the lessons they’ve learned, helping build great products and teams. I’m Allen Pike and today’s interview is with Luke Hutscal. Luke is a principal developer at Clio, has previously been a director of engineering at Clio, development lead at Shopify and was one of the early engineers on TestFlight. Welcome, Luke.

Luke: Thanks. Never done this before.

Allen: I’m so glad to be your first podcast as a guest. You and I have had countless conversations, talking one on one on so many topics, zen of building effective teams, using time well, assessing performance, helping people level up, what to do when people aren’t leveling up, all of that. We’ve been having these conversations for so long now. I’m really excited to share one of them with an audience.

Luke: I’ll try to make it a good one. I was talking to somebody about this earlier this week. They’re actually going through a performance issue with somebody they’re managing and it’s their first time. So you remember your first time going through that like there’s a lot to try and figure out. And I think one of the interesting things for folks working in this space is, until you learn to start doing it, I don’t think anyone reaches out for help or build supports or community to actually get better. You might luck into it and fall into a community or you might have a meaningful mentor who’s like, “Allen, you’re struggling and you should go talk to this person about it,” but I think it’s easy to live in a world where you’re not reaching out for help or you’re not learning and you miss it. So I love chats like these. It’s like, “Let’s share because you’re going to learn things I miss, I’m going to learn things and everybody wins.” This is my give back, I suppose.

Allen: Absolutely, and that’s I think one of the big differences in between most individual contributor roles versus being in leadership and certainly when you get to managing managers, is that the problems are so much squishier and fuzzier and less precise and crisp with less exactly correct answers that you become less and less able to simply learn by reading and you have to actually learn by talking through the gritty parts of any particular circumstance, so-

Luke: Yeah, totally. I was actually having this discussion with somebody earlier today. I think you might be underselling the tech side where I think things can be just as squishy, I guess. I was talking to somebody about architectures. There’s that ongoing ever-present monolith, microservice, somewhere in between and I stumbled on a soundbite that now I gift you for your podcast, which was like, “There’s no right or wrong to any of this. Everything will work. It’s are you comfortable paying the costs to make it work, right? Do you know what your costs are and can you pay them? And if you do, great.”

Allen: Yeah, and that’s one of the things that a great engineering leader can do, is talk about things in terms of tradeoffs rather than just, “Well, the right way to do is microservices.” It’s like, “Well, are you sure?” “No. Microservices, they solve all problems.”

Luke: Totally. I have a bit of a personal bias towards more monolithic architectures, but I have had to admit sometimes that there are some places microservices are the best solve and I think it would’ve been solved worse if I went, “Nope, this is the only architecture. We can’t build in that way here. No.”

Allen: So let’s work through. I’ve got a few things I want to toss your way of the soundbites and things that you’ve shared with me over the years, but before we get into that, let’s give people a little bit of context. How would you summarize the Luke story so far, how you’ve built up these various lessons learned and hard-won bits of insight?

Luke: I’ll start at the beginning. I guess in the early 1990s I was born and it’s all been downhill from there, but when I was 14 or so, I wanted to learn to make videogames and I got lucky and I happened to sit next to a kid who knew Perl and he took me under his wing and taught me some Perl. And before he taught me Perl, he taught me HTML. Just, “Hey, let’s get you started by building a website.” And I had set out thinking, “I want to build AAA, the console-type games. I’m going to be the next great anything,” and he coached me through this. For a while, my learning strategy was like go to the bookstore and buy any book that has an animal on the front and a blue rectangle. Just those are your books, cover to cover.

Allen: Yeah, a black and white sketch of an animal definitely means you’re going to learn something.

Luke: Exactly. So that was my school years, was just teaching myself in my spare time and playing around, making a ton of mistakes and just learning and growing from there. That was a lot of my time up until 19, 20-ish. My parents would say, “Hey, it’s time to think about university, what you’re going to do there,” and I didn’t know the word opportunity cost at the time yet, but I was starting to make okay money as homework help for university kids. When I was in high school, I had taught myself just enough C++ to be dangerous because I was going to build the next great MMO and you needed C++ to do that.

Allen: I did the same for the same reason at the same age.

Luke: I’ll get you for a code review on some of it sometime. So at this point I was starting to make money as homework help and I was looking at school and a four-year comp sci degree or something and I was going, “Well, I’m already … They pay me to do this already. I’ve skipped it. This is great. Why would I go spend four years of my life doing that when I already have some of these skills?” And so I didn’t. Through a quirk of things, I’ve taken about four days of postsecondary, which were like a hyper-accelerated asp.net introduction at a technical college here. Then other than that, I didn’t go. I just got into the workforce by hook or by crook and went from there. It’s a decision that, honestly in hindsight, I still can’t tell if I should have gone, should have not gone. There were lots of things that got traded off now that I know the language of tradeoffs, but that was how I got started. But the undercurrent of a ton of that early life and that self-teaching, all these things was like I had to learn how to teach myself and continue to enable and grow what I was trying to get better at, which meant lots and lots of side projects, lots of finding people to talk to. I became a moderator on this programming forum because I was just one of the most active members and I was the most active because I was like, “I just want to learn a ton.” Have you ever read, I think it’s called The War of Art by Steven Pressfield?

Allen: No.

Luke: For a little while, I got a lot of mileage out of reading books about art and ways to create art and be comfortable with art. This was right at that time when it was really trendy to be like programming is doing art and painting. So I was reading books about that. If I recall correctly, I think this story comes from The War of Art. The book opens with this story about a pottery class. This story has been repeated and launched the careers of a thousand Agile consultants at this point. Basically, the story goes, on day one of this pottery class, the class got sat down and they got told, “We’re going to split you into two groups. One of you, your final grade will be on developing the most perfect cup. You’ll submit one cup. That’s the end mark for your semester.” And the other group were told, “Your mark is going to be based on the aggregate weight of cups you produce this semester, right? You’re just going to throw so many cups. Your hands are”-

Allen: “Make a cup, make a cup, make a cup.”

Luke: And you’ve heard the story. I think a lot of people have heard the story. At the end of it, the class that was being graded on weight also had the most beautiful cups because they put in their reps and they got really good. And I happened to stumble on that story at just the right time in my own learning journey to be like, “Cool, output is the way I’m going to learn, right? This is the way that I’m going to get there, is just put in my reps.” And so my early career was all about finding places where I could put in my reps and get really high-volume learning repetitions in and find fast feedback loops. And for a little while, I was looking for rock tumblers, was how I used to describe it, “How can I find the place where I’m going to grow just as fast as possible because this just pushes me to that limit of, ‘You’re going to learn or you’re going to die, right? Just go get it done’?” This is a pretty strong language for tech where I have not truly had to face down a tiger, I don’t think.

Allen: You didn’t literally die because of lack of learning pace, but when you go into, not to spoiler alert, but go into Shopify as it’s growing rapidly, as I understand it from the folks I’ve talked to who were there, if you didn’t learn, you weren’t going to stay there because things were moving fast.

Luke: Yeah. So my career journey has basically been about just taking that and continuing to find, “Where do I think I can learn more, get exposed to something I haven’t grow my impact both on the space I’m working on and the industry and also just on myself in terms of what I get exposed to and have access to?” You touched on the highlight reel of my career. I’ve worked at a ton of places nobody’s heard about, but the move to TestFlight was actually my first experience with high volume web scale tech. Just I’d never worked in an environment where people actually liked and used our stuff that much. People liked and used the stuff at earlier jobs, but TestFlight was the first 24/7, “People are using this and it cannot break and you need to build towards that.” And I was really early, I didn’t know what I was doing, but I was dedicated to learning and figuring it out and I hope the CTO still says that about me. Yeah, so I learned a ton there about just how to work under load and operate under scale. It may be coming through, it may not, but a little bit of that drive to find rock tumblers and to grow and get feedback, at that stage of career, I was an asshole and I didn’t give feedback very well and I got fired a couple times for it.

Allen: I’m surprised to hear that actually. The rock tumblers sanded that off.

Luke: Yes. Yeah, it took a bit though. A good friend of mine called me out on it after one place fired me because I wasn’t getting along with the team and he had shown me this book that was like, “Here’s some techniques. You could learn them. This is neat.” And I just shit on it and he was like, “Hey man, you’re an asshole. I know you’re grumpy about getting fired, but come on.” And it took years to snap out of that, but for a while, I was hustle culture, I suppose, “I’m here to go fast. I’m not here to make friends.”

Allen: “I don’t have time to read a book.”

Luke: Exactly, just pure mercenary, “I’m here to learn and grow and that’s all I’m here to do.” And so I just sought out jobs with bigger and bigger impacts that, moved from TestFlights, went somewhere no one’s heard of and then ended up at Shopify where I was like, “Well, this is an even bigger place to have an impact on an industry and a space.” It took a long time to learn and cultivate towards actually being able to give better feedback and coach and grow and understand that my contribution in a technical organization isn’t just, “I show up, you pay me and I write code. We’re done with the tickets at the end of the day.” That’s not things. But I saw a reflection from somebody about their early career where they were talking about being a mercenary and just how in hindsight they have so much regret about that stage of just, “I missed a ton of opportunity because of that.” And when I look back, honestly, I see some of that. Just like there were connections I probably could have made that I didn’t because I was like, “I don’t talk about my personal life at work.” So it’s been a bit of a long road honestly to find and navigate, “What does it look like to be authentically you at work and then at the same time bring people along, make things better, be that contributor and also be authentic without being toxically positive, I suppose?”

Allen: Or toxically, whatever your valence is for the day.

Luke: Yeah, right. Just Shopify was where I ended up navigating a lot of this. That was a really high-growth, high-pressure environment. There was a lot of really great feedback. There was some pretty aggressive feedback. There was a lot of practicing and learning and understanding a lot of how to work at scale, how to be direct and how to do it without being a dick about it. And then I moved on to Clio, which was at the time a little bit of a smaller company, a little more people-focused than the Shopify I was working at at the time. It was a chance to flex into some of that and get a feel for it. So-

Allen: How big roughly was Clio when you started there?

Luke: About 300. So I joined Shopify at 300. They got to about 3,000 while I was there and then I went to Clio at 300, it was like, “Great.” But one thing I didn’t check in my interview was, when I went to Shopify at 300, they had about 200 devs, I think was roughly the number. So when I was talking to Clio, they were like, “We’re about 300.” I was like, “Great, I’ve been in a 200-dev org. This is going to be awesome.” Not so, really different business, really different industry, really different company layout. So I joined an organization where the engineering team was 45 and got to live that growth and now they’re well past 200 and have been able to be there for a bit of that growth, but there’s some learnings there too, for sure.

Allen: Yeah. So do you want to briefly blurb what Clio is for anyone who is not plugged into the legal tech ecosystem?

Luke: Yeah. Clio is an organization that builds software for lawyers. So one of the big problems facing a lot of people today is access to justice and one of the things that holds people back from accessing justice is not being able to find a lawyer or their lawyer being too expensive for them or similarly everything to do with the legal system. So Clio’s mission is about transforming some of that and making things better. One of the ways they do that is by helping lawyers be more efficient, building software to helps them run their law firm. It sounds funny to be like, “Here, this is tech for lawyers,” and people go like, “I’ve seen Suits. They probably buy a lot of software on Suits.” And the average Clio customer actually looks much, much closer to Saul Goodman in Better Call Saul than anyone you’ve ever seen on Suits. Because a lot of lawyers get into law because they want to change the world or something about it and the law is the vehicle to doing that and then they run face first into the realities of being a small business owner at the same time. So that’s a space that I think Clio goes after making better for them and just everything you need to do to run a pretty good law firm, so you can focus on that side.

Allen: Yeah, let’s talk a bit that’s come up a couple of times in the show over the last decade, this rise of businesses called vertical SaaS where it’s like, “This is the industry you are in. There’s a whole bunch of problems that you shouldn’t have to care about because the thing that you’re good at is not accounting and this and that and paper trails and signing documents or whatever the things are.” And so that’s one of those ways where I feel like it’s just a lot better place to be independently building a business of various kinds now than it was five or 10 years ago.

Luke: There’s a funny story in a lot of tech companies origins where they set out to solve a vertical and they’re like, “We’re going to solve a couple, so we’ll name ourselves something and we’ll have a suite of verticals we’ve solved.” I was there for that stage of the tech industry, “We were all sure that a bunch of these verticals just needed a little bit of spit and a little bit of software and they were good and now we’ve move on to something else.” What a cute thing for us all to think that we can just summarize an industry with three sentences and a little bit of Ruby on Rails.

Allen: It was an idealistic time. So that gives folks some good context of where you’re coming from and some of the things that you’ve seen, which I’m sure we can fill in more detail as we go. One of the things that I was enjoying our conversations, already been happening just popping up, is you have a lot of rules of thumb and pieces of leadership zen that you like to collect and polish so that you can then use them to help level up the teams that you work with. So I’ve been keen to poke at some of them and get your high level on what motivated some of these lessons, maybe some of those stories of how they came about.

Luke: Absolutely.

Allen: So first up, “Find people who intimidate you and spend time with them.”

Luke: This takes me back a little bit to that rock tumbler side, right? Just if you want to learn a ton, you need to go find those people who are like, “I could never be as good as that person.” And then you have to eat them because you learn everything you can and then you get there. Remember MySpace and your top six?

Allen: Yeah.

Luke: Your top six on MySpace, I don’t know if this was actually what drove that feature, but it’s an implementation of that idea that you’re some of the six people you spend the most time with, which I think has now been maybe just proven, maybe not. I’d have to go check.

Allen: Yeah, I think the original research was overstated, but I think it’s still a useful idea.

Luke: And so in a world where you’re like, “Hey, if I’m the sum of the six I spend the most time with, do I want to spend that time with people who I’m not learning from or don’t make me better or do I want to go find those ones who are fearfully better than me, just like, ‘I can’t believe someone is that good at that,’ and then go from there?” And so my path to finding mentors, coaches, honestly jobs even sometimes was just, “What’s a little scary or who do I think wouldn’t ever take my email or talk to me about something and then doing it?” And I have to put a disclaimer here, I understand that I am a white, straight-presenting male in tech just like this doesn’t work for everybody, but I got a lot of mileage out of sending off weird emails to people and just going, “Can I pick your brain about this?” or, “Can I talk to you about this?” or similar and just finding those people who intimidated a bit. And intimidate I think is an interesting word here because there’s a world of intimidate that’s like fear intimidation, right? Somebody pulls a knife on you-

Allen: Yeah, like, “This person intimidates me because they just randomly fire people and verbally berate people and is just a total loose cannon.” It’s like that’s not what you mean by spend time with people who intimidate you.

Luke: Yeah. You should not spend that much time with that person. They’re going to teach you some things, but you learn them all in the first 30 seconds and you’re like, “Great.”

Allen: Yeah, they’re just coping skills.

Luke: Yeah, I’ve learned a lot from some of those people, but definitely, that’s not quite what I mean, but it’s like, “Where do you find those people who almost generate imposter syndrome in you and then how do you get as close to them as you can and learn as much as you can?” So a little bit of my time at Shopify, I had really good access to a couple of directors who were really good at what they did and I built, Lara Hogan calls this like a Manager Voltron, I called it my Coaching Village at the time. I had three or four directors, and essentially what I would do, this was right when I got started into management, it was like just in the deep end, “I don’t know how to do any of this, and boy, is it all just so scary and new?” And so I had a supportive boss and he gave me lots of good guidance, but the other thing that I did was I built this Coaching Village and I got three or four people to invest into me pretty directly for an hour or two every week, two weeks. I forget the exact schedule because it’s been a decade now. That meant that I would have an hour-long one on one with my boss, I’d come away with something I was chewing on and I needed to work through. And then in the course of a week, I would go access 90 more years of management experience as I worked through, call it, four hours of almost hands-on mentorship with these folks. And at the end of it, I would get to distill kind of what my village had told me and go, “Well, I think what aligns with how I want to run this is X, which is actually a little bit of this, a little bit of that and things that I wouldn’t have had if it was just one.” So it was a bit of that top six and all of these directors scared the shit out of me. They were very intimidating from a like, “I don’t even know how you get to that level of doing that thing.” And now having done that job, it’s still a little mindboggling sometimes to be like, “Oh, that was me for a bit,” and it’s one of those things where just like, “Yeah, you got to find those people who you would be scared to email and then you got to email them.”

Allen: “Or chat them up or talk to them at a conference or follow up with them on a thing or whatever.” Yeah, that’s it.

Luke: “Go for a weird coffee in East Bend.”

Allen: Yeah, “Go for a weird coffee in East Bend,” as we did. Yeah, in some ways, that’s the story of this show, is me reaching out to people whose skills and talents intimidate me and being like, “Let’s spend time together.”

Luke: If you go talk to folks who are really good at teaching themselves and learning and growing, I think you find this pattern just executed in just this amazing rainbow of ways, depending on how somebody wants to execute it. It’s like absolutely, you may not like me distilling your podcast like this, but absolutely, this is your execution of the exact same thing.

Allen: Yeah, that feels right to me. All right, next piece of Zen, “Make wrong things Look wrong.”

Luke: I think the credit for this goes originally to maybe Joel Spolsky from some old, old post. One of the things that I noticed when I started working with folks who had a little more experience than I did was that they had a way of solving problems that sidestepped a bunch of issues that I had when I was solving problems. So you would come at a technical problem, you’d be like, “I need to solve X,” and then you’d have figured out your approach and you’d suddenly notice, “Oh, this has a weak spot because it doesn’t perform well here or it’s got this one weird edge case or similar.” And as somebody who was self-taught, I didn’t have the language to describe these edge cases. I didn’t have a lot of the concepts to describe them. So Big O Notation still sometimes I’m Googling just to remember how that all works, but you’d hit this world where you’re like, “Oh, when I go after this as somebody who isn’t sure about something, I’ll solve it in this way and it will have these edge cases.” When I talk to this person with more experience, they’ll teach me two things, “First, here’s a better way to solve it, and second, weirdly, it won’t have any of these edge cases,” that I did because a better choice about how you architect this or what data structure you use or similar leads you to a world of just, “Hey, that edge case isn’t really possible.” I’ve run this software interview for a while. We sit down, we write some code together to do a gift exchange. We pair people up. And in that pairing exercise, I have a tiny list of people and I want to pair them up in a Secret Santa-type gift exchange. Everybody gets one gift. I get paired with you, you get paired with somebody else, that person gets paired with me and that’s the exchange. And there’s a ton of ways to solve that problem. Just like I said about architecture earlier, “What cost you want to pay is what you’re going to get,” and the brute force way is to just generate some pairs and then just loop and loop and loop until you don’t have somebody paired with themselves.

Allen: It theoretically would work. It could work.

Luke: Yeah, it scales great when there’s three. It does not scale great at 300,000, I’m sure, but yeah, and so the first time I had to solve that problem, that’s how I solved it, just fully brute force, “Let’s write a loop and let’s move on,” is that got me places. And a ton of my education and my learning has been about just like, “How do I go get things done? I’m not that fussed about being correct. I’m fussed about how do I impact this, right? Businesses hire me to solve problems, not produce flawless code. If the flawless of the code solves the problem, awesome, but otherwise, I’m here to solve problems.”

Allen: But then you start to be getting to Shopify scale number of items in the list and you find yourself wishing that you’d made a linked list of Secret Santa persons, so one to the other and not randomly permuting the items.

Luke: Yeah, and that’s one of those spaces where if you are able to do that linked list approach and go person to person to person looping around, you never run into the edge case if somebody paired with themselves. You never loop over or you only have to process everything once. I don’t know the Big O for this, but it’s much better than the brute force approach. But the thing about this, that shows up with experience sometimes, but I think making wrong things look wrong is a really powerful way to think about how you build the systems you’re working on and what you’re working in so that folks hopefully can be steered to the right thing. I used to work with somebody who had an interesting approach to product, I think where he said, “Our job for the folks that we’re building product for is to make the right decision every time for them that we can just. If there’s a clear right decision, we just do that. We don’t do anything else. And for a place where it isn’t, we make the right decision most of the time and we’d give them a chance to choose what they need something to be.” And I think it’s really easy in the tech industry particularly to look at your coworkers and be like, “These people suck. They make terrible decisions. They can’t build worth shit. What are we doing here?”

Allen: Especially because your attention is drawn not to the median coworker or the best coworker, but the person who wrote a bug that is now in production and you’re getting paged about or whatever.

Luke: Yeah, right. The guy who has silently just deployed stuff that works and never taken down production and has been here eight years, he’s probably working over in some corner and nobody knows that he exists. But I’m grateful to him, but he’s probably not the person where I’m like, “Oh, man, here’s Allen again.” But yeah, and so I think there’s almost this growth curve of how you look at your coworkers if you were an asshole back in the day, which is you start from, “These people are not good and they’re making bad decisions and they just don’t know what they’re doing.” And then you get to know them, you start to understand that the people around you are people weirdly. And then you get to this world of like, “Well, everybody makes the best decision they can with the information they have, right? You don’t go to work to make good bad decisions today hopefully, and neither do I. So between the two of us, if I think you made a bad decision, probably I’m missing what the context was that made that seem like the right decision. So then I get to making wrong things look wrong and what can I do to position things so that the wrong thing doesn’t seem like the right decision?” And that might mean making something a little harder to change, right? When you’re first learning to work in Ruby on Rails, everybody goes through the stage where you’re just fighting the hell out of the framework because you’re not thinking in the patterns that it wants you to and you’re just fighting it. And there’s this moment of clarity that shows up when you grok it, where all of a sudden you’re like, “Oh, this is … I get it now. It’s all falling into place. I don’t feel like I’m fighting it anymore,” and that’s the same thing. So sometimes wrong looking wrong is make it so something has to be fought because the fight might be the signal.

Allen: That reminds me of many, many years ago I was working in this JavaScript framework that’s supposed to be a single page app framework that was built by a team that was inspired by Rails and they’re like, “Oh, let’s make the Rails of JavaScript,” and this was 15 years ago or something. And we were building on it. We were one of these internal teams at Apple that was using beta versions of the software and we were just trying to get things working and we were just recipients of this framework that wasn’t super documented, wasn’t super fleshed out yet. And we were hitting some problems and we were working around those problems by kicking certain parts of the API service that I think it was all under documents. It wasn’t totally clear what was and wasn’t intended to be public API of the framework. And we were having a problem that got escalated to the creator of the framework and we were showing him what we were doing and he was looking at the code and he’s like, “I didn’t know if someone could do this. This is very bad. Not meaning your code is written poorly, but it’s bad that the thing that you’re doing is possible.” Because we were just poking parts of the internal things in the framework to work around browser bugs or whatever the thing was at the time and didn’t look like just plainly you look at the code and you’re like, “Oh, okay, well, you go into the structure of this thing and you modify it or whatever.” It’s like, “Well, that was supposed to be encapsulated and it just wasn’t because it’s JavaScript, and the API service, it didn’t have underscore, underscore unsafe or something like that in that variable in the method name, right?”

Luke: Yeah, no, for sure. That’s like, “Oh, this looks incorrect or something is wrong.” Just to trigger more reflection, I have a couple of friends who are just entering that new parent stage, right? Baby is here, they’re very, very sleep-deprived and they’re figuring out how to navigate that. And apparently, one piece of advice they’re starting to give new parents is like, “You will accidentally forget your kid in the car. You need to not do that, right? That’s not a good thing to do.” Reasonable advice, I think. But the advice they’re starting to give is, “Make it impossible to forget your kid in the car,” and so-

Allen: “Put your wallet in there, car seat or something.”

Luke: “Put your wallet in the car seat, take your shoe off, just something where it’s so obvious you have to pause and engage your brain,” but it’s another execution of making wrong things look wrong. When you get out of your car and you’re like, “Oh, I don’t have my wallet. I don’t feel it where it should be,” that’s what pauses you and slows your brain down. And I hate friction on engineering teams. I don’t think friction is healthy a lot of the time. Things should be smooth, and everywhere we can, we can make things smoother, but the wrong things look wrong, just pause and engage and survey and make a choice I think can be really powerful.

Allen: Was there the thing where they have the two keys to launch the nukes or whatever?

Luke: Yeah, totally.

Allen: With the red button, with the flap and the iris scanner, whatever they have.

Luke: I had a friend who works for, I forget where he was working, but he worked on nuclear power plant control systems and they had a red button that was full plant shutdown. They called the million dollar button because people would just sneak past and scooch and then they’d accidentally push it. And so they did eventually put one of those flip covers on it. He was like, “Yeah, that didn’t happen until there were six shutdowns.” It’s amazing to think this probably 20¢ piece of plastic solves this in a way where that problem never has to be thought about again.

Allen: Yeah. Well, we’ll remember that next time we’re designing a system about how to modify DNS for an application. So we touched on Rails, which I know that you built Shopify and Clio, both are very Rails-centric teams. I’m curious what you think about the aphorism, it’s not something that you coined, but I’m confident you’ll have an opinion about it. What do you think about the term, “Choose boring technology”?

Luke: If I ever encounter the guy who coined that, I will buy him a beer. Just like I think it’s a great approach to take with caveats. So I talked earlier about learning through side projects and lots of building and everything else and I think there’s a world where trying out a technology that’s not boring for a little hobby project or a side thing or somewhere where your goal is actually just to learn or understand something, I think it’d be really powerful. The first time that I built something using MongoDB, it really shifted how I thought about things. And I didn’t deploy MongoDB to production for that project, I did Futurelife, but it was something that reshaped my brain and helped me understand something new. But I think the choose boring technology’s maxim is fantastic as a way to scale a team and work in a world where you’re growing and there’s this great talk from this guy at Etsy, his boringtechnology.club or something, he’s just made a little website for it where he talks about their rationale around choosing boring technologies, but the overhead benefits of having a boring set of technologies are amazing. I worked at one place where we had this tiny JavaScript framework, like what you were talking about, and the dev team for that framework was like four people. It barely existed. It was a really interesting technology. There was lots to learn and we would’ve had a way better time if we had just shipped jQuery. It’s the age of this project because jQuery was big and well understood and boring, but the tech lead on this project was like, “I think jQuery is boring. Let’s do this,” which at the time I’m like, “Cool, get to learn something new. This is awesome. I still love learning new things,” but with the benefit of years and years of hindsight now, I should have flagged really hard on something going, “I think this is boring. Let’s use X.” Because I feel like I repeat myself a little bit about this, if businesses are hiring me to solve problems, me getting to learn as part of that is a really happy accident and one of the things I think is amazing about this job, but they’re probably hiring me to solve problems in a way that’s boring and that doesn’t involve them spending a lot of time mitigating or having exciting things happen because it turns out that technology was really interesting, but maybe not well understood or not well supported or too esoteric to be able to deploy in a configuration that isn’t hellishly expensive.

Allen: Yeah, I like the original phrasing, I think in the essay that coined that, “Choose boring technology,” that talked about the idea of innovation tokens, which I dislike the word innovation because it gets misused so much, but basically, you have as a team a capacity for doing novel technical things, to your point, that you’re actually trying to solve some underlying problem. And so there’s a certain amount of novel technology that you can learn and maintain and make good use of and hire for and all these sort of things. You have a certain amount of budget for that and you want to spend that at places where you’re going to get a really high return. So it’s like, “Oh, okay, we have an exceptional use case that most teams don’t have,” or, “We are really uniquely positioned to be able to go off the beaten path with this thing,” or, “For some reason or another, we think this going to be competitive advantage.” And doing that 80/20 where 20% or less of the time, “Okay, we’re going to actually try and do and take a bit of a risk on something that could have a big payoff,” and then 80% of the time it’s like, “Okay, well let’s just do the thing that we can easily hire for that everybody understands well, that we are not the largest user of, that we’re not going to be having to make custom patches to the database adapter for this thing.”

Luke: I think this comes out of the lean startup movement a little bit, but there’s that idea that you should outsource everything that is in your company’s core competency, right? If you’re like, “We build invoicing software,” don’t build your own log aggregator. Use Splunk. Don’t build your own exception tracking. Use BugSnag, like all the way down, just all of these things you can use so that you get to focus on invoicing, I think is the example I’m using here. And boring technologies is the same thing. “Do I want to spend my time on the way the ZooKeeper gets configured so that it can coordinate the Kafka cluster just so or do I want to focus on, ‘Hey, we made a better way to send invoices and it’s not that exciting’?” Yeah, I would actually love to see more devs and more of the industry talking about this. I think you don’t get conference talks out of boring tech a lot of the time and I would love to see more like, “Check out this really boring solution to a complicated problem,” which it turned out used 10-year-old components because it didn’t need more than that.

Allen: Yeah, obviously, there’s a bit of a cycle there where attention in our media, whether it’s conference talks or blogposts and stuff is generally goes to what’s novel and interesting and like, “Hey, did you know this unexpected thing?” or, “Here’s this new thing that just came out,” and then people have FOMO for it. And so you have this huge drive towards, “Well, what are people wanting to promote at the conferences or write blogposts about or upload on Hacker News or whatever your source of information is posted into the company Slack, is the new, exciting, interesting thing that sounds novel?” and we don’t yet know all the reasons why it’s problematic or not as good as we would like it to be. How though do you think, although I do generally agree with this, “Choosing boring technology,” is it a good 80/20? How do you think about that flags where you are potentially choosing a technology that is establishing or entrenching what’s becoming technical debt, something has going past boring and starting to become meaningfully and eventually problematically just no longer the best thing, is going to be becoming hard to hire for the community for, is shrinking? You might think of that is like maybe Java or PHP in 2024 if you’re starting a totally new startup. Are you using those? Certainly, they’re boring. Certainly, they’re well understood. You could, but it’s not going to doom your company probably, but is there a point? How do we know when it’s gotten too boring, I guess is the question?

Luke: I don’t know. Honestly, I might director you here a little bit, but honestly, I think comes back to tradeoffs a little bit and I think the state of the world is amazing for boring technology. Do you remember the C100K problem with Nginx?

Allen: Yeah.

Luke: Do you remember when Nginx was ascended?

Allen: Yes. Yeah, and that’s like trying to get it, so that it could support 100,000 connections at once.

Luke: And when Nginx first came around, it was like, “This is a massive issue and this is one of the only things built to handle it.” And everybody who was running Apache and not sure Apache would scale to that was like, “Oh, I better learn Nginx.” That was when I learned Nginx. It was super cool. It absolutely saved my bacon at TestFlight a couple of times, but C100K, it was like a Y2K-type problem, just like the internet is becoming popular and we’re not going to be able to deal with it. And now in 2024, I can say to you, “Do you remember the C100K problem?” And you’re like, “Kind of,” because none of us were talking about.

Allen: Yeah, that’s a problem that we had to worry about.

Luke: Yeah, and so I think one of the amazing things about the state of this industry is there’s lots of things that suck, there’s lots of things that could be better, there’s lots of things we wish could be better, but if you pause and step out and look around, a ton of it is getting so much better all the time. And so yes, Java in 2024 might be a boring choice, but it’s a massively mature ecosystem and the tool chain is so good, and as one person, if I think back to what writing code in 2005 looked like compared to what writing code now looks like, I’m not going to call myself a 10x dev, maybe like a 4x, but you can go so much farther on one person, two people. It’s not, “I need to hire a team of a thousand. We’re working in Java and got to get this done.” It’s like, “Honestly, this all just works. I need three people.” And so you’re in this world where these things, and maybe this isn’t boring tech, maybe this is matured tech, but I think the tool chains have gotten so good that it’s like, “Oh, this is not so bad. Even if it’s boring or stale or everybody thinks you should use Postgres instead, just it’s pretty good,” which gives me a lot of hope, honestly. I think it’s going to be really neat to see small teams do things that previously you needed big teams for. Just yeah, it’s a neat thing.

Allen: Yeah, that’s going to be an interesting thing to watch over the next few years. One of the classic arguments that Gail will get brought out, whether it’s by the CTO or by the ICs that are angry about a mandate that, “We’re going to use something boring,” is, well, if we use this thing that’s so boring, whether it’s Java or PHP or something that people feel is on the decline rather than just stable, we’ll say, “If we use that, we’re going to have trouble hiring and retaining people because people are not going to want to learn the technology, so we’re going to be like, ‘Oh, I don’t know why I have to learn that thing. That’s not going to be transferable to other places,’ or because they’re just feeling like, ‘Oh, if I stay here and all I’m doing is continuing to use this thing that’s on the decline, then my resume or my job marketability is going to go down.’” And I do think that’s a meaningful thing to consider. I think that that would be my go-to answer of, “When is a boring technology maybe getting too boring?” but it’s an interesting counterargument that you posit there is like, “We are on a trend where a small team can get more and more done,” and so you can certainly make an argument, which we don’t know exactly how true it is that especially as we get into it, we are already in the age of co-pilot where me, someone who’s been a manager of managers that my engineering skills are fairly rusty, I can be far more productive on quickly hacking something together today than I was two years ago using the skills I already have just because those little roadblocks that you would hit where it’s like, “Oh, what is exactly the syntax for this?” or, “Why is Jem angry at me? They get resolved in 30 seconds or two minutes instead of 20 minutes or an hour. And that’s just me, right? And obviously, people who are also senior and not rusty are going to have even more ways of accelerating themselves with this tool. So I’ve been seeing some people make arguments, “It matters less what technology you’re using because five developers are going to be able to get where the previous generation of companies had 50 or whatever,” and obviously that can have a whole bunch of interesting ramifications.

Luke: Yeah, no, I remember in the Rails too days, the big talk track was like, “Rails doesn’t scale because you needed a cron job to kill and restart your Rails all the time because it just wasn’t doing great.” And that perception has persisted. To this day, I meet candidates who are applying for jobs and they’re like, “I heard Rails doesn’t scale,” and you’re like, “There are multibillion dollar businesses built on this. It is capable.”

Allen: Shopify is still primarily Rails, isn’t it? Isn’t that one of the most scaled companies that there are?

Luke: Yup. It’s one that always gives me a bit of pause of … I don’t think it makes you look as smart as you think it does in an interview to be like, “Hey, I heard Rails doesn’t scale,” and you’re like, “Have you done any research?”

Allen: “Did you see a Hacker News article 10 years ago or”-

Luke: Yeah, right. I think the discussion of calling it resume-driven development is an interesting one where you’re like, “We’re not going to be able to keep people because we work with stale tech,” or, “We’re n0ot going to be able to hire them because this is such an esoteric thing,” right? One of the things that I’ve been noticing as I’ve talked to folks is that, absolutely, there are some people who are resume-driven developers. If you’re not the latest greatest hotness, they’re going to go somewhere where they can. And there are some spaces where you need to do that, right? If you want to work on the cutting edge of graphics programming and your organization isn’t using whatever is cutting edge in graphics programming right now, you got to go. Go find the intimidating graphics programmers. But I think a lot of the devs I talk to, when they get really excited about the work they’ve done and when they talk about the impact of the work they’ve worked on, it’s extremely rare for me to encounter someone who’s really excited about, “I use this tech and that was so cool.” And typically, there’s an impact statement that shows up. “I used MongoDB and it meant we got this into somebody’s hands in three days instead of 30 and that was amazing. Mongo’s the best.” And you can repeat this all the way down, but most of the folks I talk to who really like writing software and enjoying what they’re doing, they talk about the people they impact, they talk about the problem they solved and how good it feels for somebody to actually use something you built. And that feeling, I don’t think is going away whether you’re in COBOL or Java or Rails or anything else, but it’s just, “Hey, is this the group of people you want to be helping and are you still happy that they use this?” So totally, I think you can have a piece of text that’s getting a bit stale and people might leave over it, but other people who want to solve the problems, you are in a space to solve might go, “Hey, this is my chance to go after it.” But yeah, if you’re the last organization using COBOL, you’re going to have to figure out how you train people in COBOL, I suppose.

Allen: Yeah, and I think it’s another one of those tradeoff things and just figuring out how much. I think most people acknowledge at the limit being … No one wants to be the last person using a technology and do try to avoid being in that circumstance. And so just how much does it matter that the thing is a bit on the decline or versus something that’s really over? I’m going to still say no to COBOL. I don’t know. You’re going to have hard time to convince me on that one, but yeah, no, I like that mentality of focusing on the impact. Before we run out of time, I want to touch on something that we’ve a few times had some good conversations and we talked a little bit about already today, leveling and performance in our organizations, assessing it, trying to improve it. Really important, fiendishly difficult, what do we do about it? What have you learned? Share your secrets, Luke.

Luke: I think this is going to make a lot of new managers mad, but it’s important. I was talking to a new leader recently who was like, “Why does everything I do in this job take so much time? Why is it so draining to do this job and why is it so hard and everything is slower than I think it should be?” And I think the answer for that is because it’s new, right? The first time you do anything, you’re not very good at it. If you don’t have a coach or a teacher alongside, you’re figuring it all out as you go. So first cheat code I have, get those teachers or that community of practice to help make you better, but I think the core of doing the job of leading people and being that involved is all about time and how you manage where it’s going and what you’re doing. And I don’t think there are that many shortcuts. Somebody sells you a shortcut, they’re looking for your money, they’re probably not helping you. Back in the day, I was a newly minted senior manager and I was trying to hire for my team. And my boss at the time had a connection to this conference that was called like The CTO Summit or something. It was for aspiring CTOs. It’s in San Francisco, so each one of them was the CTO of an organization like six, but otherwise it was a CTO conference. And I was running a team of I think 10 or 12 at the time, but I was like, “Hiring is the worst thing I’ve ever done. Just talking to all these people and sending these messages and not hearing anything back or being like, ‘Hey, you and I have totally misunderstood what this job is and we shouldn’t go further.’” And sometimes you find great people, but it is daunting to actually go proactively hire people for your team even if you’re trying to do that. So I went to this conference being like, “Surely in a room of 600 CTO types, one of them will have cracked hiring, at least one, maybe even three if I’m really lucky.” And I was sitting at this conference room table and I’m sitting with the CTOs of, I don’t even remember, some household names of San Francisco startups at the time. I was like, “Hey, tell me how you do hiring. How do you recruit and how you go after these things because I’m a manager who is having to learn this for the first time, and boy, it’s just crushing the life out of me figuring out how to do this because it just seems like just this massive time investment that doesn’t seem like it has a very good return.” I think there were eight people at that table and two of one, all they said was, “You put in your reps. Every morning, I spend a half hour on LinkedIn and I send some messages, and every evening, I spend a half hour and I reply to some messages or I send some more. And I just do that every day because I know, in 18 months, I’m going to want to hire someone and it’s going to take me about that long to find the right person. So I just put in my reps.” And I think it can be really tempting in management to go, “How do I spend less time on my people? How do I think about them a little less? How do I make it so I don’t have to pay so much frigging attention to what these people are doing?” And the honest answer is, absolutely, you can build feedback loops and mechanisms to get you insights and help direct you to things, but if you’re trying to spend less time on your people, I think you’re not actually being a very good leader to those people. So it’s really, “Can you understand what they’re working on and can you build that understanding?” and it takes a ton of time and it’s figuring out how you balance that with all the other things you have to do. Yeah, I don’t have a shortcut answer for you for leveling. I’m not like, “Well, if they use a linked list, they’re clearly senior.” It’s, “Can you spend enough time to understand your people and really get a feel for where they are and have that understanding?” And if you don’t have the time to do that, you’re going to have a really rough time assessing level or even deciding.

Allen: Michael Lopp on Rands in Repose says, “Spend an hour every day for each open rec you have. For each job you’re hiring, spend an hour a day.” And he’s saying that purposely, he knows that most people or a lot of especially new managers going to be like, “We can’t do that. That’s an hour every day. Look, what am I getting? Isn’t that the recruiter’s job?” And the answer to that, “Many, if not most, all of the senior leaders that really care about this stuff converge on is like, “Yeah, you might not get the full hour in every day, but it’s a good goal because what matters more than getting the best people.”

Luke: I have a pretty firm belief as a leader that your working life gets better if you build really good relationships with all your supporting functions like your HR person, your recruiting team, everyone else, just how do you connect and get that trust and that mutual relationship to a point where you’re working really well together. I have a friend who is a recruiter and we talked about this like, “I don’t want to do an hour of reaching out. That’s recruiting’s job.” And recruiter friend was like, “Yeah, we hear that all the time, but the manager who shows up to us and is like, ‘Hey, I’ve been doing these reach outs. Can you help me?’ we’re so excited like falling over ourselves to go help that person because they’re doing a little bit of the work and they’re showing up with something and we can help them turn that into something.” But yeah, I think it took nine months for me to make that first hire after trying to go out and hire. And I was doing three, four hours a week at that point and it was rough. Just I would not make it as a recruiter. I think that’s an exhausting job, but I’m so glad someone does it who isn’t me.

Allen: Yeah. Well, there’s a part of that that you didn’t mention is that the recruiters, they’re blasting out how many messages and they get whatever response rate, whatever quality of responses. But if you get a message from a hiring manager or someone who you look up to from their technical … They’re one of those people that intimidate you because you would want to learn from them, the response rate and the quality of responses you’re going to get from those messages once they’re dialed in and you figure out how to make them, which might take a few months, right? But that response, it’s just no recruiter can ever … They can be the … Yeah, I don’t know, maybe there’s some recruiter who is also a TED speaker or whatever something that people are so to respond to them. But for the most part, that’s going to be how you’re going to get the really high-quality responses. So unfortunately, putting in the reps, it’s the non-fun, non-trick answer that actually gets you there.

Luke: I had a summer once where a recruiter and I, we’re in a competition, just, “What can we get for a response rate on our reach outs?” Because it was like, “Well, I’m a hiring manager and you’re a recruiter. Let’s go after it.” And by this time, I had dialed in how I was reaching out and I was doing pretty well. And I was actually running into a problem of too many responses. And so my recruiting team supporting me were like, “Let’s send out some more reach outs. Let’s go find people,” and I was going, “I don’t have the time to handle all the responses coming in.” And they were like, “That’s not a thing. That truly isn’t a thing.” And so this recruiter and I went head to head on just like, “We’ve both got our own sets of candidates. It’s enough to be statistically significant how this goes. Let’s go.” And I think his response rate was somewhere in the 5 to 10%, and mine was like 20 to 25. He never gave me shit for it after that. It was just like, “Hey, man, I’m busy. They’re all responding, but yeah, it means so much more coming from you.” But even if it means more coming from me, I have to give some credit to the fact that it took a village of recruiters to make me that good at reaching out and getting replies. And so even if they’re not doing the reaching out, they were huge factors in me successfully hiring. And I think that’s a space where we underrecognize some of those groups doing that work that makes you any good at what you’re doing. A manager performance, managing someone for the first time hopefully is getting heavily coached by an HR person on how to do it kindly and humanely to this person. And probably no one knows about all the, honestly, it’s emotional labor that HR is doing to just go support them through this. And it’s good to have close relationships with these folks, make a huge difference.

Allen: Another item on the list of people worth learning from. All right, well, thanks Luke. This has been excellent, as I expected it would be. Where can people go to learn more about you and your work?

Luke: Okay, that’s the hardest question. I have a very seldomly updated website that I’m sure you can put in your show notes, lukehutscal.com, I think.

Allen: It is lukehutscal.com. I will have it in the show notes for folks.

Luke: Perfect.

Allen: And thanks for being on the show.

Luke: Yeah, thanks so much for having me. It’s been fun.

Allen: It’s Shipped That Way is brought to you by Steamclock Software. If you’re a growing business and your customers need a really nice mobile app, get in touch with Steamclock. That’s it for today. You can give us feedback. Follow us on social media or rate the show by going to itshipped.fm/contact. Until next time, keep shipping.