• Jaycee’s website, jayceeday.com

• Jaycee’s original article about her onboarding workflow, How Solving Our Biggest Customer Complaint at Blinkist Led to a 23% Increase in Conversion

Allen: Welcome to It Shipped That Way, where we talk to product leaders about the lessons they’ve learned helping build great products and teams. I’m Allen Pike. Our guest today is product designer, Jaycee-Ann Day. Jaycee is an app designer, startup founder, creator of a world-famous onboarding workflow, and is currently a lead designer at GitHub. Welcome, Jaycee.

Jaycee: Thanks, Allen. Always excited for a chat. Excited to be on the podcast.

Allen: Yeah, it’s been too long since we talked, and I figured, why not record it?

Jaycee: Exactly.

Allen: Today we’re going to talk about product onboarding, the aforementioned famous flow, and some of the interesting trade-offs that we face when we’re building first time user experiences. But first I’d like to start with some context. Give folks a chance to understand who you are, and how you got here. What’s the short-version recap of Jaycee’s story so far?

Jaycee: Yeah. So, I first got into tech by, I was a receptionist at a real estate company, and I had a bunch of times sitting there. I didn’t have much to do besides look pretty and greet guests. So, I found out about coding and startups. So, taught myself how to code that way while I sat at this front desk, and through coding, made websites for this company that I was working for, and realized quickly that I was much more of a designer, not a great coder. So, that’s how I first got into design. From there, then worked for a startup doing product design, UI/UX design. Then, while working there I found out about Steamclock Software. And it was one of these companies that I really wanted to work for, I remember, and I was, “Yes, I’m gonna work there someday.” Then, you reached out to me, as you know, had this interview, and then I got the job at Steamclock.

Allen: And that was, I don’t know what, eight years ago now? It feels like a long time ago.

Jaycee: Seven years ago?

Allen: A few couple of lifetimes back.

Jaycee: Yeah, totally. Yeah. Steamclock was great. Outside of work, I needed a big change in my life, so decided to move to Berlin. So, at that point then I started doing a bunch of freelance work for quite a few years, and then founded my own company, which was Avocado Bots. And that was chatbots for conferences and events. And with that it was going pretty well, but then we couldn’t find product market fit, and my co-founder left me. So, it was a bit sad, but we decided to not continue with Avocado Bots, but I continued with some clients. One of them was Blinkist, and I was working on conversion, onboarding flows, subscriptions, and stuff like that there. And I absolutely love the team. They offered me a job. So, I started there at Blinkist, which is, I think, the main thing we’ll talk about today. Get into that later. Now, I’m with GitHub for the past year, and I’m working on communities and discussions, specifically, on GitHub.

Allen: There’s lots of little pieces we can dig into, but I think, like you say, the Blinkist one is a good place to start. And to give people just a bit of context before we start talking about the work that you did there, do you want to recap a little, for people, what Blinkist is like? It’s quite a successful business, but it’s in a certain niche. So, you want to kind of describe it a little?

Jaycee: Yeah, for sure. So, Blinkist is bite-sized books basically. So, it’s book summaries within 15 minutes. It’s on Web, but also an app. I worked on the app side of things. It has over 20 million users. Pretty cool app.

Allen: And this has a subscription service. You subscribe and you can get a certain number of summaries or usage per… Do they have a free plan at all or is it just trial and then you have to pay it to keep using it?

Jaycee: Yeah. So, there’s a free version where you just get a blink per day. So, Blinkist decides these random books basically per day that you can have that are free. But the premium version has access to everything that’s on the app.

Allen: Which is a pretty common consumer prosumer-oriented SaaS model. You let people play around with it, but that it’s paid. So, the business is driven by getting the subscriptions which, people don’t like paying for things. So, it’s one of the core challenges of a business like that. And I know you did a project there, or at least one, maybe multiple, working on conversion on onboarding. But one in particular that you did there, it was so fun to see the whole internet lighting up and people that I follow, blogs and things like that, talking about this project that you did redesigning and having a pretty major impact on the conversion of their onboarding flow for getting people to subscribe, and in a non-evil way, which is a part of what made it fun. So, often it’s, “Oh, we found a way to make people subscribe by just hiding the unsubscribed link,” or something like that, and you did the opposite. So, tell us about how did that project come about?

Jaycee: Yeah. And just to add to that too, I definitely focus now, I want to stay focused on conversion. So, it’s definitely always ethical conversion. And this is what made me get really excited about that during the time at Blinkist. But this project came up, basically, since I started at Blinkist, I saw that in the app reviews, this was the top thing, that people were complaining about was that when they sign up for the trial, it’s a seven-day trial, and then they’re charged that last day, which is a pretty standard thing across apps. But a lot of people would forget to cancel, forgot they signed up for the app, and then they would be charged on that seventh day, and then be, “Where did this come from,” and write these reviews that were very angry and feel they’re being tricked into this. And when I brought it up at the beginning, it was just people saying, “Well, this is just a normal thing in the industry. Every app does this.”

Allen: What can you do? It’s just the way it is.

Jaycee: Yeah, exactly. “Everybody’s doing this. This has helped us for conversion for so much. We can’t go back on this.” And I was, “Let’s just get rid of this subscription model entirely. Let’s just make it free.” But that was way too in one direction, of course.

Allen: No revenue.

Jaycee: Yeah, exactly. And as a designer, for the user, that’s what my perspective was when I first joined there and saw this, but then realized that I had to take a different approach, like, how do we balance the business case for the company, how do we still make revenue but also make it a good experience for the user? Then, that was a big aha moment for me in trying to pitch that in a different way.

Allen: So, your goal was, let’s try and decrease the number of complaints that we’re getting from users or increase customer satisfaction, basically, by improving this in some way, and so you set out to find a way to make it better, that hopefully the business wouldn’t reject because it would ruin the revenue?

Jaycee: Yeah. Exactly, yeah. And that’s exactly how I pitched it, basically. I was, “We should focus on this,” because if we reduce those customer complaints, growth will be better because we get more people using the app. Also, of course, it’s ethically better. So, just internally, we’re going to feel better about the product that if we do some work on this, we don’t know what it is yet, but the main goal was just to reduce those customer complaints. That was the first goal.

Allen: So, was there skepticism of, well, it sounds like you already said there was some skepticism, but I guess you and the folks on your team were able to get some backing to get at least a certain amount of design and development time to trying this out?

Jaycee: Yeah, exactly. So, we didn’t think of an increased conversion, but we just didn’t want it to go down. So, that was the goal. And I just pitched it to some stakeholders that we should work on this, and then brought it to a workshop with the team and some other stakeholders as well and said, “Here are some problems we can focus on.” But basically, unanimously, everybody said, this is the problem that we should be focusing on now, and then we came up with potential solutions altogether in that workshop with my team.

Allen: It’s not a core topic, but I think that’s interesting. You mentioned running a workshop. How do you think about and approach when you want to run workshops like this, because I’ve seen huge variance in how different teams, even what they mean when they say, “What is a workshop.” So, what does that mean to you in the general sense, and how did you find it helped at Blinkist?

Jaycee: Yeah. So, I love running workshops to come up with ideas to get everybody involved. I think that was a big part of it, just to get everybody on board so it’s not just coming just from me. But with this workshop, it’s starting out with a bunch of research, usually, or what we did for this one was a bunch of research, and then we come up with the problem statements, basically, from this research that we want to focus on, potentially. Then, we all voted for those, the problems that we thought were the top things we want to tackle. Then, usually get everybody to do some sketches. So, get everybody to do some ideas after voting on what things we want to focus on and the solutions. So, we’ll come up with some solutions and then sketch those.

Allen: Are you trying to get the non-designer stakeholders that are in there to sketch too?

Jaycee: Yeah, I get everybody it to, like engineers, product managers, whoever else is there. It’s good.

Allen: One of the things, I have seen some teams struggle with over the last couple of years is that they were in-person and now they’re remote, and how workshops change, because the idea of, oh, let’s pull everybody into this nice boardroom for four hours, is slightly more appealing, I think, to a lot of people than, let’s try to be on a four-hour Zoom call. So, have you done anything like that, that in this remote era that was as successful as what you were doing when you were able to do it in-person?

Jaycee: Totally. Those ones too were half remote, some were in-person as well. So, since the beginning, basically, it’s been doing a lot of remote. I keep them to two hours now. I have them pretty fast and well done. I usually use the same general format, and it’s just keeping the energy up and being really quick, timing people and making sure that it’s done within that two hours. If you don’t go over, people are usually pretty happy.

Allen: Okay, yeah. That’s good to hear because sometimes I feel a bit, not bad, but we on our teams have been pretty strict about short meetings. We try to make them half an hour, 50 minutes is common, and then occasionally you’ll be, “All right, let’s go wild and do a two-hour workshop,” or something like that. But I hear about other teams who were, like, “Well, we had a six-hour workshop,” or all-day meeting or two day meeting, and I want to die a bit thinking about being on Zoom for that long. So, hearing others, successful teams saying, “No, no, no, you can do it in two hours.” We just need to be disciplined and then people will not resent you. I’m sure there’s some things that you can’t get done in two hours, but I don’t know. I’m “team two hour” max for sure.

Jaycee: Yeah, totally. Not to slam design sprints, I think design sprints for a whole week is just way too long. So, I just took the best pieces of that and then squish them into two hours. Also, what comes out of it is, why are frames done for me, because everybody does sketches. So, it’s skipping that step, which is always nice.

Allen: Nice.

Jaycee: Yeah, getting people to do my job.

Allen: So, you do your two-hour workshop, you get everyone, including the product people and engineering people and whoever’s involved, to do some sketches. Then, where did it go from there?

Jaycee: Yeah. So, the number one thing everybody wanted to do was have some sort of a screen that lets people know that they can cancel and when to cancel within the trial, and explain to them more what the trial is. It was just to have more transparency. We also talked about having trial reminders so that after they subscribed, they can sign up for trial reminders, and then a day or two before their trial ends, we just send them a notification that says, “Hey, your trial is ending. Do you want to cancel, do you want to stay, or you maybe can check out the app and find something more.” So, that was generally the idea that everybody was excited about.

Allen: And I imagine that was something that was a little concerning to certain stakeholders who were, “What, you’re gonna remind people to cancel their trial? That sounds like a bad idea.”

Jaycee: Yeah, totally. That was obviously a concern. And everybody thought that conversion or cancellation of trials would go way up, and we said, “Okay, it could go down by 10%, but this is good ethically, so we do want to have it, but that’s the limit.”

Allen: You could also imagine a decreasing churn where fewer people convert to paid, but then maybe more of the people who convert to paid will stay, will still be around after six months, you could imagine?

Jaycee: Yeah. At that point we didn’t really think that way. We just were, okay, we just want to reduce these complaints because we want to get more people to be using the app.

Allen: So, you went in and you did a design pass, I imagine, spent six months perfecting it so that it would be perfectly beautiful?

Jaycee: No, definitely not. So, for the first test that we would do is the trial reminders. So, before we do this new subscription screen, let’s see how many people actually cancel. With this, I did design using basically stuff we already had, so it was pretty quick. I also coded it myself because we were using this tool that you could easily update the screen and put it in there. We were able to do this quite quickly. And yeah, we did have problems, but we were able to test this. So, we wanted to run this as an AB test, of course, and we had this AB tester set up. We AB test everything all the time. So, we put this out into the world to see what would happen, and we actually saw that 4% fewer people canceled, which was surprising when we thought there would be 10% more people canceling. So, that was a huge surprise. Then, we also saw push notification go from 6% to 74%, which was a huge bump.

Allen: So, a more than 10 times increase in a core-

Jaycee: 1116% increase, or something like that. Because of these results, of course, we decided, okay, let’s roll this out. So, we fixed it up a bit and rolled it out to everybody.

Allen: That’s amazing.

Jaycee: Yeah. Then, from that we were, “Okay, where do we go from here? This is amazing results. Let’s keep riding this wave. What do we do with this?” Then, we went back to this idea of changing the subscription screen to letting people know that they have this option of the reminder, and letting them know that we’ll remind them two days before so they can cancel. So, we wanted to continue with this. So, I did some mock-ups for that I did some different ideas with swipe things, some other design, I don’t know, different wire frames. But we ended up wanting to do a timeline, so having three steps, and showing what happens at each step, like, today you get access, in five days you’ll be reminded by notification or email that your subscription is ending and you’ll have to pay. Sorry that your trial is ending, that you’ll have to pay for the subscription.” Then, in seven days, if you don’t cancel, you have access to everything as you still do now. So, that was the design that we wanted to test

Allen: Just a visual representation of what the next week is going to look like?

Jaycee: Yeah. Then, we designed this. I did some user testing on this, of course, and people really loved it. So, we showed them the app and we showed them the subscription screen and asked them their favorite part, and they were, “This subscription screen, it’s just so transparent. I love this, even more than the app,” which was very weird.

Allen: So, you did a whole app usability test and they were excited about this thing that’s trying to get them to get a subscription?

Jaycee: It wasn’t the whole app, but we still showed them the discovery screen and what the app’s about afterwards. So, that was clear, “Okay, let’s definitely do an AB test for this.” And there were some small fixes that I made to the design, and decided to roll this out. Again, it was an AB test to see how this works. Our goal at this point was 10% increase in trial opt-ins-

Allen: Which would be huge.

Jaycee: Yeah. That’s also big. That was usually our goal for each of our tests. So, that was our goal.

Allen: And what did you see?

Jaycee: After rolling this out, we checked back, and it was 23% increase in trial opt-ins, which was way more than that 10%, so that was amazing.

Allen: That’s amazing. 23% trial opt-in increase. So, that’s more people are opting into the trial. Did it decrease how many people are sticking with it afterwards?

Jaycee: Yeah. So, that would still be the 4% before because we still had the screen that you could opt in to push notifications for the trial reminders. So, the retention we still saw was still 4% more, and then the trial signups were 23% more.

Allen: So, in the aggregate, how many person weeks have gone into this project at this point?

Jaycee: Very little, to be honest. I was thinking about this for nine months until we started to actually do it, but I was not actively working on it. Then, the actual time, we’d usually run these tests for two weeks. Development time was usually two weeks, and then I would work on it for maybe a couple of weeks. So, maybe everything together was a month or maybe a little over a month, just waiting for the results for the test before as well.

Allen: And as a result, you have a product that has millions of users, has 25% times, plus another 25,%, plus a 67%, a 1,000% increase of this metric. The aggregate impact of revenue had to have been at least 50% increase to the business. It was wild.

Jaycee: Yeah, it was crazy huge. So, this is obviously a huge one. The original goal, when we go back to that, was decreasing of the customer complaints.

Allen: Oh, yeah, the customer complaints. I’ve gotten all excited about revenue, and you’re, “But what about the user?” Do they like it?

Jaycee: But this is because we had to wait a few months, of course, because we had to see if they’re still complaining or they stopped complaining and stuff like that. So, we waited a while for that, then got the results back of that, and we saw that there was decreased complaints by 55%, which was also amazing. Yeah.

Allen: Wow. And your goal was 10% or something like… Oh, man. So, 55% decrease in your biggest customer complaint, all the metrics going for the roof, Jaycee is now the CEO.

Jaycee: Totally. Yeah, exactly. Should have been. So, it was amazing. And a lot of those complaints that were still coming in about this, even though it decreased by 55%, a lot of the complaints were still from three years ago somebody was scoring and they got charged, and they’re still-

Allen: Oh, and they now noticed.

Jaycee: Yeah. Let’s just keep on complaining everywhere about it forever.

Allen: So, it probably just continued to decrease since then. That is so awesome. That feels like the kind of thing you’d want people to know about when you’ve had such an amazing discovery.

Jaycee: Once again, I think subscription and conversion stuff can be just a lot more ethical, like we talked about before, and I really wanted other apps to do this. If every app is doing this and there’s 55% less people being screwed over by this, then that’s great for the world. So, I wrote this medium article about it and posted it, and then it just got a lot of response from this.

Allen: Yeah. That was the point where I was exploding all over my various social timelines. And I even saw it on LinkedIn. I try not to spend too much time on LinkedIn, but it’s, “Oh, wow. Jaycee’s everywhere. This is awesome.”

Jaycee: Yeah. So, it was really cool. That’s exactly what I wanted because I want people to be implementing this. I think, now, every second app I download and look at, they have this timeline screen and I’m, “There’s my designs.” So cool.

Allen: The ones that you hacked together in a week with this weird tool.

Jaycee: It’s actually terrible too because the actual timeline design, I really didn’t. I still am not super excited about the UI because I had to fit it into this certain box and stuff, so it was really a lot of constraints. But everybody’s copied it, so I guess it’s not so bad.

Allen: Well, maybe that’s an opportunity for you, at some point you end up working on another onboarding subscription flow and you’re, “No. Now, my goal is to do even better than that. I’ve got to one-up myself.”

Jaycee: Yeah, totally. It’s my time.

Allen: So, what was the most surprising… You’ve got all this attention, and then you had people blogging about it and linking it and talking about it and celebrating this discovery, basically, this thing that people could have found years ago, or maybe no one might have found for more years, but you’ve discovered and popularized this thing that’s better for users and better for businesses, which is one of those rare, just, okay, everybody should basically just do this if you have a business like this, what was the most surprising thing that came from all that attention and movement?

Jaycee: Yeah. From that, it was really cool to see people remixing it and doing their own version, adding steps one, two, three, and a checkbox. I think that could also be better if that was there and people tested that, which is cool. I also had this oma, she emailed me and said, “I’m a 80-year-old oma,” or something like that, “And thank you so much for doing this screen because I always get charged unexpectedly, and this is so great.” And I was, “Oh, this is so sweet.” So, that was awesome.

Allen: That’s great.

Jaycee: And that just makes me feel good, getting these emails, people try it out and just hearing from them. Then, of course, the biggest thing was Apple. Somebody tagged me in this on Twitter that Apple is now recommending this to apps and how to video with the screen, that they now recommend to everyone. And this was one of my goals, and still is, to somehow get into Apple guidelines, so Apple suggests this to everybody. But it was really cool to see, but kind of startling too because it was exactly my designs, but they just changed the name of the app. But I still think this is better for the world. But that was the biggest surprise.

Allen: That’s very cool. I think that there’s a lot of people who wish they could impact what Apple’s policies and preferences are and what they advocate for developers, and it’s fun to come across a thing that was, you can actually come up with something that is a big enough impact that they’re, “Okay, this is the way the app should be now”.

Jaycee: Yeah, totally.

Allen: As much as our frustration is sometimes, it can be difficult to steer them.

Jaycee: Yeah, that was really good. It’s good for Apple. They’re going to be making a lot more money off of it and-

Allen: Yeah, I haven’t thought about that. They make 50% of all of all of these-

Jaycee: Exactly. And I’ve seen people go up 20 to 50% increase in conversion. So, for Apple, figuring out that, how much money they’re making from it, it’s quite a bit.

Allen: Yeah. I was thinking about how much money you made for Blinkist, but you probably made Apple so many tens of millions, if not hundreds of millions of dollars, maybe billions. I don’t even know the scale. Yeah.

Jaycee: Yeah.

Allen: Well, congratulations. Are you waiting on your check now?

Jaycee: Yeah, exactly. So, when I first saw it, I was really excited, and then I was, “Wow, I could have made a lot more money from this.” Anyway, it’s still good for the world. I still wish they had reached out before they did post it, but they never did, but otherwise, it’s cool.

Allen: There’s something about our industry that I find fascinating that there are things like this that are discoverable. Like I was saying, someone else could have, in theory, tried this and tested this out at any given point. And if you hadn’t, then it might have been longer. But then, once we discover it or pull the refresh is one of those things. There was originally the Twitter app that was, “Oh, maybe if you pull down then you can load more tweets.” Then, people were, “Oh, that’s clever. Oh, that’s kind of cool.” Then, eventually, everybody was kind, “Yeah, I’m pretty much, I think we should just do this. I think this is just the way software works now.” So, it’s interesting to be one of those little building bricks of getting us there.

Jaycee: Totally. Yeah. I think that’s peak of my career, so that’s great.

Allen: So far, so far. Now. You’re going to make tens of billions of dollars within your next-

Jaycee: Hopefully.

Allen: On Blinkist, one last thing I thought was interesting is this. So, you wrote this medium post, and I think some companies have a bit of a tendency too when there is some discovery of a design change they’ve made, especially something as profound as this, to maybe be a little skeptical of, “Oh, I don’t know, should we share this? Is this competitively advantageous to us to not share it?” So, was that a difficult sell for you to pitch or was it a no-brainer of, yeah, this is going to get… We’re all talking about Blinkist now, and I’ve been celebrating this product. So, it ended up being a win for them, I think. But was that a difficult decision?

Jaycee: No, not really. I was really lucky with Blinkist that they had this policy, you can share whatever. As long as it’s live, you can share anything about the process or what you designed. It’s totally fine to do any of these use cases. So, I asked about it. I also asked if it should be posted under Blinkist’s medium or something, but they said no, it’s going to get bigger reach if you do it under, I think I put it on some other publication, UX Planet or something. So, they said, “Yeah, just do what you want,” basically. So, I was lucky.

Allen: Yeah. There’s something that, we’re getting to the point now at Steamclock where I used to be the only one that would write about what we were doing. But one of the things over the last couple of years we’ve been getting more and more successful with is having other folks on the team write about it. And we don’t have a lot of policy other than, if we’ve shipped something, then, “Yeah, let’s write about that.” It’s always heartening to me to see these examples of a big win when it’s, we did something good, do great work, and then tell people about it is the loop that I like. So, I think it’s a great example of that.

Jaycee: Yeah. I wish more people did that too so we could all learn from each other, more of these kind of cases like this big ones and stuff. I agree. I

Allen: Agree. Yeah, it’s scary though that a lot of people are able to convince themselves that they can’t write, and that writing is some dark art or something that you were born able to do or not when… I don’t know. My thinking on it, as someone who went from objectively not being able to write and now writing okay, it’s practice and time and investment and re-editing and re-editing.

Jaycee: Yeah, totally. And it’s a lot more about, like in this case, in my case at least, it’s a lot about the content as well than just showing this design and stuff.

Allen: Yes.

Jaycee: Your writing is great. You’re a really good writer.

Allen: Oh, thank you. So, onboarding, more broadly out, do you have any projects that come to mind that you’ve worked on that you’d be interested to… anything that you remember, lessons learned, or difficulties or challenges or even just frustrations on other onboarding projects you’ve worked on either at Blinkist or other products? I know you and I both, in between the two of us, have worked on dozens of onboarding flows and projects.

Jaycee: Yeah, for sure. I think the most I worked on is Blinkist. So, I think it’s most interesting about the Blinkist stuff. With that, we just made this whole onboarding flow. We discovered, through data, that most subscriptions come from that first subscription screen. I think this is pretty standard across apps. So, everything that was working towards subscriptions should be before that. You could invest after, but a lot of it is better that it’s in the onboarding before. And we had to find this balance too between, okay for conversion but then also for engagement for people to actually be onboarded into the app. And I had the vision before. At the beginning there wasn’t really a long onboarding at Blinkist. Then, we decided to make it just really short. And that increased conversion. Like getting rid of the signup in the flow, for example, I think increased conversion by 10%. And I was under the impression that longer onboarding will decrease conversion, but we realized that it actually didn’t change much if we add a bunch of screens before, which I thought was really interesting.

Allen: Oh, interesting. So, was this one of the flows where they force you to go to the end or you have to quit the app in order to get out of it, or could you cancel, could you X out of the onboarding flow?

Jaycee: We had that you could skip on the first part of the onboarding flow, but then after that you had to stay in.

Allen: Okay. So, if you went one or two screens in, then it’s, “We got you.”

Jaycee: Then you’re in there, yeah.

Allen: And that’s interesting because a lot of the onboarding flows that I’ve worked on, there’s either no skip at all or there’s skip at each point. I’m not an onboarding expert, which is why I’m talking to you. But we have often seen metrics where you make them longer and then more people skip. You lose certain people at each step. So, what you’re saying, at least for Blinkist was that, if you don’t have a skip on the first screen, obviously that’s bad because there’s some percentage of people who are not willing to go through, so they’ll just bail on the app entirely, but if they’re willing to go to the second screen, then you can get away with, “Okay, now I’ve got you.” Then, you have to, not in an unethical way, now I have half an hour of onboarding screens. But now, if you have six or seven, they’re going to go through them, probably, and there’s not too much difference in between five and seven at that point. Or I assume that’s the range of numbers you’re talking about? You’re not talking about 20 screens?

Jaycee: Yeah, exactly. And we looked at the data as well. So, on that first screen there would be a drop, but then, all the other screens after that, it was very minimal are no drop at all, and people would stay through the whole process, even if it was seven screens. So, it’d be, “Whoop,” and then they just stay.

Allen: Okay.

Jaycee: So, with this, it just gave us a lot of room to experiment with more ideas and put it into the onboarding, and see what affects conversion, for example, what people in onboarding, this sort of thing. So, I really focused on that then.

Allen: What did people like? What were the things that you saw that were the biggest positive impacts other than reminding people about your subscription, which was world-class, 10 out of 10.

Jaycee: So, with that one, yeah, of course, that was really good. We said to people… One thing that came up was, we would ask, “How much time per day would you like to learn?” And you give them some options, like five, 10, whatever minutes, and then we had a screen after that would say, “You’ll save 56 hours of reading in this week.”

Allen: That would be a lot.

Jaycee: Yeah. Yeah, it is. It’s a bit ridiculous, but anyway. But we found this is the main use case. People really wanted to save time and stuff, so that’s why we rolled with that, because that’s the user feedback from interviews and stuff that we got. So, we actually did 56 hours as the AB test, just put 56 hours all the time, not matching what they entered there, like 15, 35, whatever.

Allen: During the AB test just to see-

Jaycee: Yeah, just to see. Then, we were, “Okay, this is not super ethical,” because it’s this random number, but whatever, let’s just see how it goes. I think it increased conversion by 90%, or something like that. There was an increase. So, we’re, “Okay. Well, let’s build this actually real, that it actually matches what time that they chose.” And when we did that, conversion went down.

Allen: Oh, no. Because they were, “56 hours! I can’t afford 56 hours. I need to subscribe.”

Jaycee: But then there was some that were higher. I don’t know. People just liked the number 56. There wasn’t really-

Allen: 56?

Jaycee: It was, that’s the number.

Allen: Conversion pro tip.

Jaycee: That’s my point of the story. But with that, we still ended up, I think it balanced out in the end that it wasn’t decreasing or increasing, or maybe it was 2% increase or something. So, we decided to role it out with the actual numbers just because we were all about ethics now. We don’t want to be lying to these people what these numbers actually are. So, we decided to roll it out even though the 56 would’ve been better, but it still is good for the user.

Allen: Yeah. I like that mentality. And obviously it’s complicated when you get into a business, especially as the business starts to scale where there are little levers that you can pull where you can make money. And the bigger you get, the more money on lever. You make a billion dollars by telling what they would maybe call a white lie, but then, the people who are then spending money on the idea that they’re saving 56 hours but it’s actually 11 hours or one hour, or maybe it’s not so white to them when real money is at hand. One of the examples I came up recently as I’m reading that, that Lyft and Uber misestimate how soon your driver will be here. It says that they’ll be here in one minute for the last two minutes.

Jaycee: Oh.

Allen: And they both do this. And it’s one of these, “Oh, this is unethical.” It’s, “Yeah, but we calculated, we’ll lose a billion dollars of people not being there for when the ride is there or ride’s just not completing or people not being willing to…” So, it just basically decreases all the numbers by one minute.

Jaycee: Oh, my goodness.

Allen: And it’s one of those, if the Congress made us both fix it, then we could do it, but if our competitor does it and we don’t, then… Or at least that they tell themselves maybe there’s an ethical solution to it. But it’s way harder to measure the mistrust that that generates when it’s, “I’ve been standing here for two and a half minutes,” because sometimes it’s wrong. “I know it’s been way more than a minute and I still don’t know where my driver is,” or whatever.

Jaycee: Yeah, totally. I didn’t even know this was a thing, but it makes sense because I feel it’s always longer when I’m waiting, especially here. But I think as well, to add to that though too, I think before you even launch these tests, I think it’s really important to think of the ethics and what could come out of this that you rethink different directions that you could do that still help the business case but are ethical. There’s definitely things that I’ve seen too where it’s, “Okay, let’s just do it as a test.” But no, we don’t ever want to do this. Thinking about the ethics as early as possible in every project that you do I think is so important.

Allen: Were there any other things from the testing and onboarding process, that work that you’ve done, that come to mind, either as interesting wins or things that were a net win or things that were, “Wow, this totally didn’t work. What a waste,” or, “Let’s not do that one”?

Jaycee: Yeah, yeah, totally. Another good one too was saying “cancel” a lot on the subscription screen. So, the more times we said “cancel” on the subscription screen, so on the button and an FAQ in the timeline, you just write, “You can cancel everywhere,” we saw conversion go up every time, and then at some point we’re, “Okay, we are saying it way too many times. We have to put a limit on this because all you can see is ‘cancel’. Just ‘cancel, cancel’.”

Allen: It’s getting ridiculous. You cancel cancellation for your optimal canceling experience.

Jaycee: I want to experiment with that one. That’s a good one. So, that was an interesting one, just making sure it’s really clear on your subscription screen. I recommend people to do that.

Allen: Did that come out of talking to people about what feelings they had when going through the screen or was it just a guess of, “Oh, we think people maybe like being reminded that they can cancel”?

Jaycee: We also saw feedback in reviews or talking to users and things like that, that they’re, “I don’t understand how to cancel. How do I cancel in the app or an app store.” So, that’s why we added this FAQ to say, “This is how you can cancel.” And I wanted to also put this in settings as well, but for now we just had it on the subscription screen to tell them how they can. So, that was definitely from users as well.

Allen: So, this is in the onboarding. User installs the app, they’re going through, they go past the first screen where they can get out, and then they’re in their seven steps. And one of those seven steps has an FAQ before they’ve even started using the app?

Jaycee: No. That’s actually a good idea to try that out as well, but this was on the subscription screen. We made it a longer scrollable screen, that under the timeline we have a section that says how to cancel, and then you go to here and press this. And it was actually on the subscription screen. There was definitely a lot of failures, for sure. There was one screen that I really wanted to bring some delight into this because it was all pretty static, answering questions, going through. So, I was, “Oh, I wanna do this scale thing of how many books you read to time,” and I don’t know, just axis and you could drag it and it would have this effect. Then, I was really excited about this, and we spent all this time developing this thing, and it still didn’t even work perfectly well at the end. And I was so sure of it and I was so attached to it, but then we put it out to users… I didn’t use or test it first before either, which was a mistake. I always user test, but we saw conversion go way down because of this.

Allen: Well.

Jaycee: So, I did a postmortem user test on it to figure out why. And people just didn’t get it. It just didn’t make any sense. And it was hard to use and it was too complicated, so they just ditched the app.

Allen: We had a few of those. We built a fun side project, this spy game, as you know, that it took us a few iterations to get it to the point where… Because we built this little strategy game, and it’s, “Oh, this is cool science and simple,” but the rules behind it were a bit more complicated than it seemed at first because you couldn’t see what your opponent was doing because it’s a spy game. So, that seems like a cool idea for a game. But next time, if I ever make a game, I’ll make a game again one day, but my next game will not have this component where you can’t tell what your opponent is doing and your opponent can’t tell what you’re doing because it makes it way harder to teach the game. You look at a game Monopoly and you can just see, okay, this is the board and this is what’s on it, where it’s, oh, but you can’t see, or maybe you can. It’s mysterious, which, mystery does not help with user understanding in those first five minutes. So, we had so many user tests and so many experiments and so many onboardings and tutorial things and changes of the rules and simplifications. And of course, all the people who did understand the game were, “Oh, you’re ruining the game by simplifying it this way,” and the new people are, “I have no idea what’s happening. I’m just dying repeatedly to the other players.”

Jaycee: Oh, man. I remember the spy game as well when I was there at Steamclock. I was working on it, and I was, “I don’t even really get it.” But…

Allen: Yeah, you were one of the first user testers where it’s, “Okay, you’re helping us design this thing.” It’s, “Oh, can you test this out?” And you just make a move and I’m, “Oh, no. The fact that you thought that that is a move that would be reasonable means that I’ve failed in my game design.”

Jaycee: So good. When I saw it recently, I was looking at it. It’s been doing pretty well, hey? I was, “Yeah, it’s funny.”

Allen: It’s had almost a million players now.

Jaycee: Oh, my God!

Allen: Which is what I never expected. Yeah.

Jaycee: That’s so exciting.

Allen: So, it’s not quite a billion dollars. We did not make a billion dollars for Apple, but we had a million people play it, which is cool. And we learned a lot. Now, we’re getting way off-topic. It’s fine. That game taught me something that I knew intellectually but I now have more in my soul, which is that business model is the product. It’s not the only part of the product, but a product with without a business model, and going in with the idea, “Oh, wouldn’t be great if Blinkist just didn’t have subscriptions and it was just free for everyone,” like it would be more ethical. Well, it wouldn’t really help users because then there would be no Blinkist and then they would have no-

Jaycee: Exactly. Yeah, yeah.

Allen: So, it wouldn’t exist anymore. And that was what we did with this game, which was, we work on mobile primarily, but we don’t do games because we had this idea of, oh, well mobile game market is all just about tricking people into buying 10,000 gyms. And that’s not something that appealed to us, and we really wanted, if we were going to work on a game, we wanted to do a mobile because that’s where we enjoy working, and we wanted to do a game that was not like that and that it had ethical and really user-centric purchasing model. So, we designed the whole game with that not in our heads about, okay, well what is that purchasing model? Then, it’s, “Okay, it’s time to launch the game.” It’s, “Well, how’s it gonna make money?” It’s, “Well, I guess we’ll sell cosmetic items into a two-dimensional spy strategy game,” which I’m pretty confident was the least bad monetization strategy for the game that we developed. We tested a bunch and we did a bunch of evaluation, did a bunch of user, look at different ways, like we could just charge upfront for the game and a bunch of things. But I think we maximized our revenue given the game that we designed and built, but we designed and built a game that can’t be profitable. So, we put ourselves in a hole, and it’s, “Ah, I’m stuck sucking this hole.” But that was a product design failure, and I knew that intellectually before. If someone had come to me for advice and they had said they were going to do what I was going to do, then I would had told them all that, but I didn’t feel it, to my own soul of, no, well… Therefore, good game design and good product design is finding compelling products that also are profitable and can pay for themselves. So, that’s obvious, but now I know it in my bones.

Jaycee: Yeah, totally. I learned that it’s similar to this project as well. I also learned that too. I think it’s just something that, I don’t know, we say it’s obvious now, but I don’t think it is obvious. A lot of designers are just, “Okay, just for the user, we have to do this for the user,” but you have to think of this business impact as well and how you make monetization strategies and all of these things too. You have to do it for both, so…

Allen: Yeah. I think sometimes that mentality comes out of the adversarial culture that can leach into certain companies. And obviously, you want to do your best to try and not let this happen in your company, but sometimes you get companies where there’s the business, which I dislike that as a phrase, like, “Oh, well, what does the business want?” The business is individual people. They’re trying to make good decisions. You start referring to them as the business, you’re already getting into adversarial… But oh, the business wants this thing that I think is unethical or not good for users, like claiming 56 hours or whatever. Then, people will get into the mentality, it’s, “Well, I’m the designer, and so I’m gonna try to protect the user, and the business is gonna try to get evil business outcomes, and then we’re just gonna try to compete as good versus evil.” That mentality can seep in when it’s a really large company and you don’t have necessarily a great understanding of how the people on the “business” side are making decisions, and it creates the zero-sum mentality sometimes where you end up with, then dumble down the business. It’s, “Oh, all these designers are trying to make the subscription free,” or, “design a game with no business model, and so we have to ignore them because…” and then everything just corrupts down into… It’s almost like a self-fulfilling prophecy when you start behaving that way.

Jaycee: Yeah, exactly. Totally. A 100%.

Allen: So, here’s to a new generation, well, the current generation, but the continued growth of this product and business, working together in harmony, finding huge wins and making billions of dollars for everybody, hopefully.

Jaycee: Yes. Hopefully, yes. That’s what I want to see in the world.

Allen: It’s been wonderful having you on the show, and thank you for making the time.

Jaycee: Thanks for having me.

Allen: How do people find more about you and your work?

Jaycee: Yeah. So, you can find me online as 2average on Twitter and Macedon, or whatever. I’m Jaycee Day on LinkedIn. I’m thinking about getting back into conversion, so thinking about doing some consulting. So, if anybody’s interested in that, then they can email me at hi@jayceeday.com, or if you just want to chat about anything, you can email me.

Allen: Awesome. Well, we’ll link all those up in the show notes. It Chipped That Way is brought to you by Steamclock Software. If you are a growing business and your customers need a really nice mobile app, get in touch with Steamclock. You can give us feedback or rate the show by going to itshipped.fm, or @itshippedfm on Twitter. We should get that Mastodon going as well, although you can actually find steam clock on Mastodon now. We’re @steamclockonhackeederm.io, which I will not spell out. It’s Mastodon. It’s just the Wild Wild West right now, but it’s fun. So, check us out, give us feedback, tell us how you feel about the show, whichever way works for you. And until next time, keep shipping.