Ep. #6, Building a Developer Community

John Sheehan: The one overarching theme that we really tried to instill in every evangelist that we hired, that it was make developers successful, period.

There was no question anymore about what the value was of evangelism. It was just a matter of, what are the lead times gonna be?

If we can't get 10 signups on a business day, what are we even doing here?

One of the things we quickly discovered was that you couldn't actually replicate the Twilio model. You can maybe replicate the philosophies, but you can't replicate the tactics.

Yaron Sadka: Hi, I'm Yaron Sadka, senior sales engineer at Runscope. You're listening to Road to Growth, a podcast about startup sales organizations brought to you by Heavybit, a nine-month program for developer-facing startups.

Road to Growth is a bi-weekly series that breaks down SaaS sales organizations one piece at a time, from the first person to kick off sales at a company, all the way down to the partnership and cohesion with the marketing and product teams, we'll take you through what it takes to build a powerful, fine-tuned sales organization.

If you're interested in being a guest, have a topic for us to discuss, or a role you'd like us to dive into, send an email to roadtogrowth at heavybit.com. On this episode, I'm joined by John Sheehan, CEO of Runscope. We discuss the philosophy behind building a developer community and how every product requires a different set of tactics to yield positive results.

From Visual Basic to Twilio

Alright, so today we're joined by John Sheehan of Runscope. Welcome, John.

John: I only listen to your podcast. It's hard to actually look at you in the eyes while we do this instead of just hearing your voice.

Yaron: We can put you in another room next door.

John: We need like a divider or something.

Yaron: Yeah, so welcome, thank you for being here. We normally start the show off with a two to three minute introduction of yourself and kind of your history in the tech world and what kind of brought you to starting Runscope and how things are going now.

John: All right. So what brought me to tech? Well, when I turned 13, I asked my parents to buy me a copy of Visual Basic 3 for my birthday. It was really, you know, the really hot item that year for 13-year-olds.

And so the reason that was is because in my eighth grade class we actually started programming BASIC on Apple IIGSs or IIG somethings. And I knew BASIC and I thought, hey, now I can make programs if I get Visual Basic at home instead of on this really ancient Apple computer.

And so that's when I started programming, was right around twelve or thirteen. And I went from Visual Basic three through four, five, six, and then the web started growing popular, and so I was looking for web language, and since I knew BASIC I went to VBScript, since that was basically the same, allegedly same base language, that I started writing web apps.

And by the time I actually started writing web apps, I was sort of already on my second company. So then I started doing more and more web development along with that. That company was called Wise Guys.

And then after high school, I went to college for a semester and I got really bored with that. Kind of moved a little too slowly for my liking. So I decided to start another company because it was 2000 and the dot-com boom.

And so I actually started a site that helped people try to find a new church when they move to a new area. So finding your church is actually sort of like dating. And so the site I actually created, I realized in hindsight, was really a dating site.

So like, I'm looking for something with families and that's close to here and has certain belief systems and all that. It's almost identical to dating. Somebody should really make a Tinder for church locating, you know, swipe right, swipe left. No, no one has ever uttered that sentence before, I guarantee you.

And so I did that for a little while, actually raised a little bit of money from some friends on that, did that for a year, ran out of money, went horribly broke. As you imagine, church is not a big market. Not a lot of money available to vendors there. Anyway, so I gave up on that and then I started working at a couple different jobs.

I was bouncing around doing entry-level help desk support and systems administration and then sort of working my way up there, I ran the website for the Vikings for a while, for the Mall of America, a bunch of other big institutions in Minnesota, survived the Randy Moss trade server-wise. Our servers were on fire and it stayed up, thankfully. Static file and caching, great strategies that I learned when Randy Moss got traded.

And so on the side I was running the softball league and I was doing a whole bunch of, this is the longest background, by the way.

Yaron: Yeah, we're sorry. We're delving it into cut a little bit.

John: He said go back as far as I wanted.

I was running the softball site and it was really popular and it actually was helping grow the league. So we had full stats for every player in the league and everyone was really getting into it and the league was getting popular just because of this website. And so I decided to start a company offering that as a service to everybody else.

And so I decided to quit my jobs and do that again full-time, sort of working with a partnership with the Minnesota Twins to offer it to all the youth teams in Minnesota, but then being able to sell it on my own as a service.

So I did that for a little while and that was sort of my first software as a service thing and the first company that actually made a little bit of money, even though it didn't really ever take off the way I hope to at the time.

And so I left that job and went to another agency who was also doing sports league websites and started building stuff with them, doing other more websites for Minnesota companies. Then we started this thing called Screenfeed as a product, trying to get out of consulting, and that was basically delivering high quality news content to digital signage.

So if you're in San Francisco and you go in the Montgomery BART station and you see these big video boards that they have up now, and then they show you these full HD pictures and on the bottom there's a headline over a semi-translucent black background. That's Screenfeed. And so I built the first version of that. It's now on over a hundred thousand screens in the US and it's still growing really fast.

And at the same time I came across Twilio basically while we were working on that. And so a lot of the stuff we were doing was starting to involve wanting to interact with the phone, or for the sports stuff, there's a lot of call-in hotlines for weather updates, that sort of stuff. And so I found Twilio and immediately fell in love with it and started building Twilio apps just for fun on the side.

So I was running around going to conferences, talking about it and writing blog posts and doing screencasts and doing basically all the things that a developer evangelist does, but just because I loved the service and the product. And it was fun telling other people about it.

Twilio noticed this, and when I happened to run into the founder, CEO Jeff, and Danielle Morrill in Vegas one time, they were at a trade show. I was at a trade show. Danielle introduced me to Jeff and said, hey, this is John. I'd like to hire him as our first developer evangelist, which was news to me on multiple fronts. One, I didn't know they had it open, and two, I didn't know that she had noticed enough to want me for that job.

A couple things happened, but a few days later I basically got an offer letter to start as an evangelist for Twilio. So at the time there were about six or seven employees right around then after the co-founders.

And so the first year was basically spent running around the country helping developers know about Twilio. It was basically you'd go to an event and everyone would be like, what's Twilio? And then we would sort of tell them, but we were also there really to help them just be successful at whatever they were doing.

So we were like a global Startup Weekend partner, and what we went to do when we went to a Startup Weekend was not to actually tell them necessarily about Twilio, although that was sort of a benefit. It was actually to help them try to win Startup Weekend. That was our goal. We didn't care if you used Twilio or not.

And that just really started to build up this brand amongst developers. And so that team just took off growing that I was on. And then it was about five or six people, moved out to San Francisco, formally took over the team, built it up into a formal program.

And then right around that time, an opening opened up for a developer experience product manager, which was essentially taking the last year and a half of talking to developers, watching them in that first five minutes of building their apps and trying to turn that into better product.

And so I leaped at that opportunity to get that job, switched to that, and then did that for about six months and then took off from there and went to IFTTT and helped build new channels on the platform there, again working with more APIs. And then from there kind of got frustrated with all the API problems we were running into at IFTTT repeatedly that were really hampering our ability to create a quality end user experience. And so that's where the idea for Runscope sprung from. So now, Runscope chapter two.

No, I'm just kidding. So it's a long and winding road, but it actually is really interesting for me every time somebody asks me that, which is rare, to trace the path how I got to where I got to. Like starting with BASIC on the Apple IIG, you can trace which technologies I ended up choosing all the way along the way pretty much until I got out and started our own company based on just my eighth grade class happen to have that as the programming language of choice for the computer club that we were in.

So I find that path interesting, but hopefully your listeners do too, because it's long.

Evangelism at a Startup

Yaron: Yeah, so you've had tech in your life this whole time. Really, what a lot of people who know you, know you for your time at Twilio. Which was taking this company from a six, seven person team, creating this whole new revolutionary position kind of almost and using that to really catapult it to where it is today.

When you first started, you had no idea what this kind of position was, or did you? How much did you know about what a developer evangelist did?

John: I knew of developer evangelists because being involved in the Microsoft community, it's been a long-standing position within the Microsoft community. But most of the companies that had developer evangelists historically were like Sun, Microsoft, giant huge companies who could afford to pay people just to talk about their software, right, and not build it themselves, right?

It was sort of unknown to startups, I think, largely at that point. I don't remember going to an event and running into another developer evangelist, I think, until the SendGrid team started hiring them. And then by the time I was done, it was like there was an army of us at every event.

Now there's developer evangelist meetups at every event I go to. Evangelists actually get together and meet up on their own at every event. Which is crazy to me because no one was doing it at the startup stage when we were doing it. At least it didn't become apparent to us.

So the model, we struggled with a little bit with the title, because evangelist obviously has connotations outside of software, and it's basically like spread the word, right? Let people know. And that was actually never our intent, was solely to go out and just spread the word about Twilio.

What we wanted to do is, basically the way we looked at it was like, we're gonna become developers' best friends. We want every developer in the world to know who we are and that we are a reliable, trusted source and a resource to them to help them be more successful, whether that's shipping an app or getting funded or getting a raise, getting promoted. We wanted to be an asset and a catalyst for people to achieve those outcomes.

Because we knew that eventually communications was gonna touch every application in the world. And so if we had all those relationships, when it came time to add communications to their apps, the path to Twilio was very short, right? You already knew somebody there and then that's where you would go. And I think that that generally worked out.

And I think one of the reasons the Twilio developer community grew from less than 10,000 when I started to now approaching a million, about six years later, is that it's primarily first about the developer and not primarily first about Twilio, right?

And that was sort of the one overarching theme that we really tried to instill in every evangelist that we hired, that it was make developers successful, period.

Not make developers successful with Twilio, it was make developers successful.

Yaron: So you take this idea that was huge in the enterprise space and you bring it to a startup, especially a team as small as Twilio was when you first joined, how did the convincing aspect work to kind of be like, okay, we're gonna give John all this money and we're gonna send them all over the world, but we're not gonna see returns for like a year or two, or at least we don't think we will? How do you sell that and actually get the buy-in from those around you?

John: Sure. So I think that really started with the founders being technical and them being essentially the first developer evangelists, right? They knew they were building something that was going to be where developers were primarily the audience, and that traditional marketing was not necessarily going to work because they were developers first and they hated being marketed to in traditional forms, right?

And so usually, if you're wondering at an early stage in your startup how to act, generally the first thing you do is, how would I buy this, or how would I react to this, or how would I want to be sold this product or interact with this product. And that's usually a pretty good gut check on whether or not a certain tactic is good or not.

And so the Twilio founders basically said, we're developers, we don't want to be marketed to, we want to go self-serve, sign up. If we don't ever want to talk to somebody, we don't have to. But we'd like to have people who are sort of on our side and are there to provide us resources in a non-threatening sales environment, basically, right? That are going to be there to help us when we want it.

And so I think they saw that model early on. I guess you'd have to ask them why they thought that should have been their seventh hire or whatever it was, or sixth or seventh hire, but I think it really came down to, they wanted to build an outreach program that they would like to have experienced when they were buying other tools and products.

Measuring Evangelism

Yaron: So apart from traveling and doing all that stuff, what were your other job responsibilities that received tangible outcome that you could kind of get metrics on and look at and be like, I affected that?

John: Sure. So the first year was really crazy because I was on the road about half the time going to events. The other half of my time was split between doing content for the blog, running our developer contest and support tickets. So I was doing about 50 tickets a day on top of everything else, which is a full-time job almost on its own.

And then I was tasked with creating a blog post. Our team was tasked with creating a blog post every day of the year. Basically we wanted to get the

Yaron: Just the evangelist?

John: Just the evangelist team, yep. So that started when it was me and Danielle. We're like, we're gonna write a blog post every day of the year, right? So we're not gonna go to an event if we can't get some output from that that's going to result in blog posts.

That was the first benchmark, was, is this event worthy of writing a recap post for it? If it wasn't, then we weren't going to go there, probably, right?

And so the tangible benefits early on were not really present, I guess, in a way that people would traditionally look at. I think the thing we looked at is that the registered developer number kept growing and the rate that it was growing kept increasing, right? And so it was kind of hard to argue with that.

And that was basically the sole number that we were largely measured on, but nobody ever told me we have to get to this number or we have this quote or anything like that, right? But that was the one, as long as this rate is up and to the right and it's tilting up faster, then this is successful, right? And so that was the biggest thing.

Tickets was surely about quantity and quick responses. Eventually they hired, thankfully, a support team for that. Content was really, we would look at traffic to see which posts were resonating with people, but then we also tried to find ways to just increase our throughput.

So we actually ended up coming up with a way where we could turn out a customer story in about five minutes. Mostly getting the customer to write their own content without them realizing it.

We actually had a backlog of about three, I think it got up to at one point three hundred. I may be overstating that. It was in the hundreds of posts where we had the content ready to go if somebody wrote an intro and a conclusion to it, but we couldn't get it out fast enough, right? There wasn't enough room on the blog.

And they all started to sound the same too. That was the other problem. But we got really good at high velocity content, right? And so all of those things, though, just every time we wrote a blog post about somebody new, they'd go tell their friends, we'd get a bunch more signups.

And so it just kind of snowballed on top of itself. It wasn't really until, I think, there was a defining moment, sort of untangible result to evangelism, and it really comes down to the GroupMe story, which within Twilio is probably the most infamous evangelist story that exists.

So Danielle and another Twilio employee were out in New York for TechCrunch Disrupt hackathon, and they were hanging around helping teams, and the GroupMe team basically came up to them, said, hey, we got this idea for group messaging app, can you help us build on Twilio? They sat down, they spent the first hour prototyping the app with them.

And then you fast forward 18 months, they got bought by Skype, or from Microsoft, for 75 million dollars or whatever it was. And along the way, they were consistently amongst the top Twilio customers. And they got there very quickly and it kept ramping and ramping and ramping.

A lot of work was going into scaling just to support them and a couple other really high growth customers, but that was, you could trace the origin of that back to an evangelist was in the right place at the right time to help them be successful. They didn't even win that hackathon, by the way, they got second behind something else, right?

But after that happened, there was no question anymore about what the value was of evangelism. It was just a matter of, what are the lead times gonna be? Is it gonna be 18 months till we get the outcomes with these customers? How quickly are they gonna ramp, that sort of stuff?

And they've had plenty of other customers since that have shortened that timeline and ramped even faster and gone to even crazier scales than GroupMe was doing. And they may not have been directly sourced by evangelism, but even if they saw GroupMe and thought, oh, we'll just use Twilio for our thing too, because of that story, it all goes back to that again, having the right person at the right time.

You Can't Replicate Twilio

Yaron: Right. And so you go through this, you see all the growth, you are now the CEO of your own company. At what point did you decide, all right, I don't have enough time to do this anymore, we should bring somebody in to do this?

John: So when we first started the company, it was basically me and Frank, and then we hired three other people and none of them were on outreach or sales, basically. It was all engineering and product. And that was for pretty much the first year of the project.

So the first year I was still doing all of the blogging, all of the events, all of the open source stuff. But one of the things we quickly discovered was that you couldn't actually replicate the Twilio model.

In fact, when people ask me, hey, how do we do what Twilio did at my company, I say, it's impossible, you can't, the circumstances are not the same, you are not Twilio, either your product's not as good or the environment has changed in a way whereby when you go to Invent now, you're competing with sponsors like GE and Microsoft and Dell. And at the time we were competing with nobody, right?

You can't actually replicate what Twilio did. You can maybe replicate the philosophies, but you can't replicate the tactics.

And so we very much had a similar revelation. It was like, we don't have a million dollar event budget to spend traveling and sponsoring everything this year. We need to find something else. We need to find another avenue to get access to developers to help them be successful that does not involve basically the event strategy that we sort of patented at Twilio in a way.

Community Projects as Outreach

And so about six months in, Jeff Lindsay, I think, was the first one who said, basically, he was running RequestBin. So Jeff Lindsay created the term webhooks. He worked at Twilio with us. He built localtunnel, which eventually was sort of superseded by ngrok, but a lot of these early API tools that were sort of open, free community stuff.

RequestBin was having a lot of reliability problems and he was kind of ready to move on to other problems in his life. And so I said, well, why don't you just give us RequestBin? We'll take care of it. We'll keep it up. We'll put the resources in to make it stable. And we'll give you a little something for that.

And then you don't have to worry about it anymore. But we promise we'll keep it free and open and no sign up and that sort of stuff.

And so he said, okay, that sounds great. Problem solved. Now I don't have to worry about this. Now it's your problem, right? Washes his hands of it.

And so we basically took it over, we did a very, very slight rebranding of it, we just basically polished the design a little bit. Again, we didn't meaningfully change the functionality at all. And we put our logo on it and right away we started to see an uptick in signups.

So people were coming through from that project and signing up directly because that helped them debug webhooks, and now they associated that with Runscope, and now they had something, maybe other API problems that they wanted to solve.

And then we started putting, we were starting to get into do a little bit of experimentation around retargeting with our own audiences. And we thought, well, what if we put the retargeting tag on our community sites? And then we started running ads against those and they started converting like crazy as well.

And so we just realized we had this super highly targeted audience that had that same goodwill feeling towards us because we helped them solve a problem and gave us the benefit of the doubt when it came to try Runscope.

And the best part was we didn't have to fly anywhere to do that. And it didn't take up that much time, is basically what I was getting at. So it was basically inexpensive or free. And so at that point we're like, okay, we gotta go get more of these, right? This is working, let's go get more.

But our original sign-up traction, we had definitely hit the trough of despair after launch. We had such an exciting launch. We got a whole bunch of signups and we're feeling good about things. We got our first paid customer from our private beta testers the day before we launched.

And so we're riding this high and then you go two months later and I think that July of 2013, we had a weekday where we signed up two people, right? It was like, if we can't get 10 signups on a business day, what are we even doing here, right? And so we're really starting to feel the trough of despair that most go through, and that's when the RequestBin thing happened, and then it just started taking off from there and never really slowed down, right?

We added Hurl.it. We took that over from Twilio because Twilio again, it was having reliability issues. I'd actually acquired it while I was at Twilio from Leah Culver and Chris Wanstrath, who had built it as part of a Rails Rumble, and I don't know. Either way, it was going down. Twilio was like, great, thank you for taking this problem off my hands, right?

Over time, I think we're up to about a dozen now that we've either acquired and operate, sponsor costs for so that the author doesn't have to pay for hosting fees or certificate fees, that sort of stuff. Some we just give sort of a yearly stipend to as a sort of support of the project, and then we've actually built some new ones ourselves too. So API Changelog was originally a Runscope community project, Embed Curl, and a couple others.

And so we tried to basically build up this portfolio of touch points with developers so we could get access to them. And the total audience across all these developer sites is hundreds of thousands of developers a month. It's scale that we could never even have imagined reaching doing events at Twilio, for a fraction of the cost. And they're very highly targeted and they convert at a great rate when they land on Runscope.

And so basically the idea was the same core philosophy, help developers be successful. How do we reach a lot of them, but do it in a way that is our own and unique? And I guess that's sort of where that ended up.

So after about a year doing that all myself, going back to the question that I just remembered, it was when we first hired our first developer advocate. And so that was in April of twenty fourteen. So we'd basically been launched for like thirteen months at that point. And then we built up over the next year of that, head of developer relations and then communications lead, which was sort of part of that group as well, working on reaching developers through content and stuff like that.

Content for a Single-Purpose Product

Yaron: So are you still employing the new piece of content every day, even at Runscope, or has that been thrown out the window in lieu of quality content that comes out maybe once a week?

John: So we never really got to that level of velocity for content. But I also, the use cases for Twilio are infinite, right? There's every kind of app that you can write for it, so there's infinite number of stories to tell on what you can do.

With Runscope, it's sort of a single purpose, maybe a dual purpose too, for testing and monitoring, right? But within testing and monitoring, there's not a huge variety of different cases that you're trying to necessarily harp on. That's like, okay, you can monitor a REST API, now you can monitor a SOAP API, and you're sort of out of interesting angles to go there, right? Occasionally customers come by and do interesting things and we can try to tell those stories when it happens, but

there wasn't the inspiration problem that we thought we had at Twilio, which was everyone would come to Twilio and say, this looks awesome. This is a great API. I love SMS, but I don't know how I should use this or why I should use this in my app.

And so one of our big motivations was to give people ideas for how they should be using it so that the next time a project started to say, oh, hey, I remember this other group messaging app was using Twilio, I want to do that, or notification app, or even marketing teams would be like, overhear conversations about wanting to replace the call-in, like the toll-free number for the company.

And there's a restaurant chain based in the Twin Cities, pure coincidence, that was basically, the marketing intern was like, oh, I could write you a new call-in toll-free number phone menu in like an hour, and we can have it customized by all of our customer data based on what phone number they call in and all this stuff. And they did. And that ended up replacing the whole company's big toll-free dialing number.

That was because he had been subjected, for lack of a better word, exposed to these types of ideas from other customers and had been following along and then was able to apply that back to his job when it came time. And so that model does not translate to Runscope at all, right?

It tends to be a much more technical question or technical thing that people want to understand when they're using Runscope. How do I deal with this specific instance? How do I handle my OAuth APIs and stuff like that, instead of inspiring the use case in the first place.

Yaron: Yeah. So you've been able to kind of take that community piece, or the value of community that you learned at Twilio, kind of repurpose it into how the users of Runscope would interact with that product, and bring out this whole new suite of ideas around these community projects and I guess tools that they can use that aren't necessarily being charged for it.

John: Sure, right. Yep. Usually what people, again, when they say, how do I replicate what Twilio did, I always say, go your own way, you've got a different model here, and thankfully we didn't get really, really stuck on trying to replicate the Twilio playbook again, right?

And so being really, really, really conscious to what those cues are from your community is gonna lead you in the right direction much better than listening to anything that anyone from Twilio from 2010 to 2015 said about working there or what they did that worked. You just gotta know your community. And if you're not, first of all, part of your own community, you better find somebody to hire very quickly who is, and then trust them to make the decisions based on what do you want first, right?

Outreach Meets SaaS Marketing

Yaron: So was the person you hired the same as you, in the sense that they were just constantly talking about Runscope, or was it more of just a known connection that was just kinda apparent?

John: Yeah. So the first person we hired to do developer outreach, the task was actually really not explicitly to talk about Runscope, right? So we consider that person a domain expert. We wanted them to write authoritatively about things that they knew that were related to what we do, but they didn't have to be explicitly about the product itself. And well, again, going back to, we wanted people to just see us as a resource to them to help them be better at their jobs or whatever else.

And so I think the thing that in hindsight didn't work out is that, because of that sort of difference I was talking about, about inspiring use cases versus how you use Runscope versus how you use Twilio, that was one case where we absolutely tried applying the old playbook to the new model and it didn't work particularly well, right? We had a lot of good content out of that, a lot of good resources, things I'm really proud of, but ultimately that was not the right model for us, right? And so we had to sort of shift from that.

One of the things that we were finding was that for Runscope, a lot of traditional SaaS marketing tactics actually work a lot better than developer outreach tactics.

Now, our goal is to have both working really, really well, because when they're both working well, they sort of build on top of each other.

But we've been really trying to ramp up the amount of traditional, when I say traditional, I talk about the SaaS era since 2005 maybe, traditional marketing tactics about sourcing leads that are decision makers, not necessarily solely implementers, which has been the developer tool SaaS model.

Making sure they're equipped to make the buying decision, making sure they understand what the implications are for the broader organization if they bring it to more people, and then ultimately helping them with their key objectives, because the developers' key objectives of shipping a reliable API do not always necessarily directly correlate to the business value of having reliable APIs. So we want to relay both of them. What is the technical value? What is the business value?

And so we've been definitely doing more in the traditional marketing sense, more recently, as we've sort of learned that that can pay off a little bit better than maybe a Twilio model evangelist.

Yaron: Yeah. So it seems like what I'm hearing, and from my experience as well, is when you're young, you wanna build that mind share. You wanna make sure that people know about you, and the people that are gonna be using you, not the people that are gonna be necessarily signing the checks, but the people that on a day-to-day basis will be on the Runscope website building their tests and things of that nature.

And as you grow and they start finding either new jobs or what have you, they can then use that tool, relay to the actual business decision maker, hey, this is something we want to use, and kind of get growth from the bottom up instead of kind of that top-down growth model where it's just like, we bought this, now you use it.

Yeah. And I've seen that work very successfully as well for a lot of B2B SaaS companies in kind of acquiring that mindset, though they all do it very differently from a marketing perspective in terms of how they build community and the different tools they use. So definitely a lot of good parallels drawn from that.

Thank you very much for joining us. We really appreciate it. And best of luck to you with Runscope.

John: Thanks a lot.

Yaron: That's all we have time for today. Questions? Feedback? Contact me at roadtogrowth at heavybit.com. Thanks again to Heavybit for sponsoring our program.

To learn more about Heavybit's nine-month program for developer-facing startups, visit heavybit.com. And while you're there, make sure to check out their library. It's packed with great educational talks from developer company founders and industry leaders. Thanks again for joining us. Have a great week.