Ep. #3, Feat. Russ Smith & Guy Podjarny

Guy Podjarny: The best way to become CTO is to found a company and call yourself a CTO.

And I find the challenge with entirely distributed is this notion of culture. We have the two locations where you're growing them. None of them is necessarily the center.

Hiring is very different depending on stage that you're in.

Hopefully you don't make many mistakes, but you are going to make mistakes and the worst thing is to sort of cling to your previous decision.

Ted Carstensen: Hi, I'm Ted Carstensen and I'd like to welcome you to Caveat Founder, a regular series featuring founders sharing their experiences building developer-facing companies. Gain insight into what it takes to build a successful developer-facing company by hearing about big wins and epic fails directly from founders themselves.

In this episode we host Russ Smith, co-founder and CTO of Rainforest QA, and Guy Podjarny, CEO of Snyk.io, previously CTO at Akamai. Russ and Guy discuss the realities of distributed teams. They share their methods and learnings around hiring for startups, and finally, they discuss the many different roles that fall under the title of CTO.

To learn more about Heavybit, visit Heavybit.com. And while you're there, check out our library. It's home to over 75 talks from other developer company founders and industry leaders.

Securing Open Source Dependencies

Russell Smith: Hi, I'm Russell Smith. I'm CTO of Rainforest QA. I'm here today for Caveat Founder with Guy from Snyk, the new startup I think he launched early last week, right? Or the week before?

Guy: Yeah, we had our sort of official launch post. We've been around, we're sort of in open beta for a month or so, a month and a half.

Russell: So Guy, I had a quick look at your website, but I'd love you to tell me what you do in a bit more detail.

Guy: Yeah, so Snyk, you know, I get a lot of comments on how you pronounce it. It's short for So Now You Know. So fundamentally what we're building right now is we deal with the security risk that your open source dependencies introduce. Over the last decade or so, the way we consume open source has changed substantially.

And we've gone from consuming it from sort of this select small number of well-trusted providers like Apache or Linux or MySQL and we just sort of trusted them and we use them. And today it's more of a crowdsource, more of a marketplace style environment. NPM has 60,000 contributors.

Russell: It's super default to just start using a bunch of open source gems or modules that you don't really know where they come from.

Guy: Precisely, and it's awesome in terms of functionality, right? It lets you actually sort of build the functionality that you need and you just have these amazing sort of boosters.

But at the end of the day, you're taking some stranger's code from somewhere around the internet and you're running it today fairly blindly in your system.

Russell: So how are you actually telling the security risks? Are you looking at the individual modules and working out if there is one, or you're relying on external people to report the bugs?

Guy: No, so I guess there's this sort of the long-term plan and the short term. So in the short term, what we're doing at the moment is more of an inventorying exercise. We maintain a database, we open sourced it, of vulnerabilities.

These are known vulnerabilities. We didn't find them. We kind of inventoried them. We created metadata and we helped you match your code against it, as well as guide you at how to fix them either through upgrades, and we also created patches for the cases, it's very common, you can't upgrade away. So that's what we did today.

And long term we'll sort of build more functionality that deals with, in general, tackling this sort of security threat of those components.

Russell: And so I guess eventually you'll charge to get fixes before they're publicly available?

Guy: It's a good question. So we definitely will charge eventually. Right now the whole thing is in beta and free. So a part of it is a mission to sort of secure software.

I've been a little bit on a journey of trying to get developers to embrace security. And an aspect of that does require sort of a free component and an attempt to sort of help make open source more secure.

Eventually there would be components here that are premium. They would usually revolve around kind of the enterprise manageability of it, you know, monitoring components. Probably the patches themselves and the tests are more likely to remain free.

From Blaze to Akamai

Russell: Okay, interesting. So before you started this, this is not your first rodeo, right? You've done a whole bunch of stuff before. So can you tell me a bit about, you were at Akamai for a number of years, right, as CTO?

Guy: Yeah, correct.

Russell: And then before at Blaze.io, right?

Guy: Yeah.

Russell: I'd love to hear the few minute version of what that was and how you started out at Blaze and what happened in Akamai and the transition between the two.

Guy: Sure. So I'm actually Israeli and I worked in a bunch of startups during which I moved to Canada. So I worked at Sanctum, that got acquired by Watchfire, they got acquired by IBM, left IBM. Leaving security, all this time was in web security, web app firewall, web security scanner, and then sort of felt like the cutting edge technology in our ability to understand websites, sort of analyze, automatically analyze websites, and find and fix flaws in them was much further along in the world of security than it was in pretty much any other sort of discipline.

So looking at other alternatives for it I found performance and left IBM to found Blaze, Blaze.io, which dealt with automatic front-end optimization. So making websites faster, working as a proxy in line.

Russell: It's dead now, right? It's totally shut down.

Guy: It's totally shut down. You can Google Blaze .io and you will get to Akamai. We got the .io domain before it was cool, mostly because we couldn't get Blaze .com. Yeah, so Blaze was a relatively short run, about two years, or just a little over two years, even less kind of from the funding of it.

And it was a very good run. We started off thinking we're gonna be more of a performance for the rest of us, kind of make the web fast and deal with the long tail. But really a lot of my co-founders' connections were much more enterprise related and we found ourselves more naturally switching to making it more of an enterprise product going after sort of the big guns. We had sort of big deals queued up and then Akamai acquired us. That entire market consolidated very quickly.

Russell: That was a few years back, right?

Guy: That was about four years ago now, coming up.

Russell: Okay, interesting. Yeah, I just had a quick look on Crunchbase, February eighth, twenty twelve, apparently. Precisely, so yeah, that was before I even started Rainforest, I think. Actually, no, that's right.

Guy: Around the right time.

Russell: Yeah, around the same time.

Guy: So it was good. It was a good run at Akamai. This product kind of made its way into the sort of flagship product, Ion, into the top web performance product, the premium one, which was good. The team grew, the team is like three or four times the size now in Ottawa, everybody's very happy. And I took the CTO role for the web performance business, which was an interesting, very interesting experience.

You know, Akamai is a great company. It grew substantially while I was there. I think it roughly doubled itself during those years, despite the size of it.

Russell: So they had CTOs per branch of the business or per major product?

Guy: Yeah, basically Akamai has two big business units and then a bunch of sort of smaller ones. Web performance is one of the big ones. It's sort of a six-seven hundred million dollar a year business unit. It was a very educational experience, also very enjoyable to sort of learn working with kind of the people there.

And despite being 15 years old, there are aspects of it that like to sort of think they're a startup, there's still a little bit of internal battles. And for me, I've had a bit of a jump from Watchfire, which was a 200 person company, to IBM, which is some gazillion people company, to Blaze, to Akamai, which is now a 5,000 person company.

Russell: How big was Akamai when you were CTO?

Guy: So Akamai as a whole, I think when the acquisition happened, Akamai was in the sort of high two thousand, maybe three thousand people. And it was about five thousand when I left earlier this year. So yeah, very sort of a good experience.

The Developer Tooling Challenger

Interestingly, in Akamai, I've actually sort of come across the developer tooling scene as a competitor with Fastly, which was kind of interesting play as well to see the

Russell: Fastly's super small compared to Akamai.

Guy: Oh, it is tiny and there's many differences. It was an interesting experience to sort of see the pros and cons, I guess, of a developer tooling company from the perspective of the incumbent.

Russell: Yeah, 'cause Akamai is way more enterprisey, if I get it right. Whereas Fastly is developer first, right? You can just sign up and start using it.

Guy: Yeah, absolutely. And I think, you know, in some respects the fact that Akamai is an enterprise company has sort of helped. And for many businesses, for something as mission critical, as sort of critical path as a CDN, as something that is delivering your content and is in charge of your scaling, the notion of a startup, let alone one that is sort of intentionally more sort of scrappy looking, and the way we sort of build dev tooling companies, was a no-no, it was sort of a non-starter.

And for other audiences it was quite the opposite. It was so much so appealing that it almost didn't matter what the product was.

Russell: I mean, that's the thing, at least from a small startup point of view, Akamai seems so unreachable to be used. I would never think to try that to start with. But Fastly, or in fact Rainforest uses Cloudflare, but arguably similar difference. I know they're different products, but they have similar uses.

Guy: Yeah, they cater to a similar audience.

Russell: They cater to a similar audience. But there's no way I would look at Akamai right now. I don't know whether that's wrong of me, because I don't actually know how much they price. I just know that it's not listed publicly.

Guy: Yeah, Akamai is definitely not tuned for that type of audience. And the question is, you know, when you get to the point where really the price tag is not an inhibitor, can Akamai sort of win you over with the relevant functionality?

So it was just sort of interesting to see it. And in fact, when I look at sort of the learnings in Akamai, I think they demonstrated both the sort of the strengths and weaknesses of developer tooling company as a competitor to an existing company.

Russell: That's an interesting thought actually, I wonder if Fastly have a graduation problem. At some point you start as a startup on Fastly and then you get to the sudden point and then you're like, "Ah, actually I need to move off Fastly and I need to find the next thing because Fastly just doesn't have the features to support me." Does that make sense? They may have that issue.

Guy: Yeah, they may. I think when you go into a market that is similar to the CDN market, Akamai is very much the gorilla there. And for the most part, it's everybody against Akamai, like really your primary playbook is to either get those players like Cloudflare did, which do not use, so you're building some new technology that sort of democratizes, right, makes it suddenly feasible for a startup to have a CDN which before was out of reach, or you try to basically take away business from the existing bigger players.

So Fastly seem to have been somewhere in between those two components. So oftentimes it's less about the graduation and more about, can a customer be taken away?

And the same goes, like my job, my folks in Akamai, Akamai has kind of made major leaps during those three years in terms of all these things that as a developer tooling company you build up front. So being API driven, being very sort of self, like investment in self-service. Everything real time. It used to be before continuous deployment, the notion of a four-hour propagation time when you push a configuration was fine because you ship your software once every two months. It wasn't great, but it wasn't...

Russell: Now that's totally not.

Guy: Yeah, today it's like, you know, unfathomable. So there's this constant stream to do it. There's the notion of openness. A lot of my job is focused on opening Akamai up and reaching audiences.

So it all comes back to the change in how we develop software and to the empowerment of developers and how the decision factors change. So the big companies can change.

I guess kind of my input for a developer tooling company coming in and trying to sort of go up against an existing giant is that at the end of the day, it's harder to sort of move. It's just like these shifts take longer to perform. So the question is sort of how nimble and how quick you are. And you know, there's still the issue of focus.

Russell: Akamai is firmly moving towards continuous deployment, delivery, acceptableness, right? That's what you're saying.

Guy: Yeah, but there's a gap. I think Akamai is drastically today, but if you looked at where Akamai was two years ago, the gap between it and some of the sudden new competitors in that specific realm of how quickly to sort of roll out software was far bigger.

So Akamai moves more slowly, but it has more of that sort of existing set of technologies to build on.

Russell: Did Akamai start as enterprise first or was it, do you know? I don't know the history of it.

Guy: Oh sure, kind of going back, yeah, this is like during the dot boomer. There's a whole, that's probably like a podcast by itself. Akamai has gone through this sort of schizophrenic... It was like the fourth biggest IPO after Google and then it got delisted from Nasdaq.

And one of the co-founders died in nine eleven. It was a whole story to Akamai, Akamai's begin, which is quite...

Continuous QA as a Service

Russell: Let's save that for another podcast then.

So interestingly, you're talking about having to embrace continuous delivery and integration in a modern startup. That's one of the reasons we did Rainforest at all. I think we didn't talk about this when we're actually recording, but Rainforest is a continuous QA as a service. So it's a way of you doing some of your QA process in a continuous delivery pipeline, which up until today is basically impossible to do. So had you seen us before or not?

Guy: No, and I feel like Rainforest is tapping into a very needed space. It feels like the world of QA and testing has always been one of those that's sort of desired but hasn't been getting as much loving as it should have, and the notion of working with it continuously is a lot of interest.

I'm curious, how did you... So when I've had some conversations about QA when I was dealing with performance, when I was starting Blaze, generally the feedback I got was, "There's no money in QA, you're not gonna be able to, those markets are not sexy." Did you come across that? How did you get over it?

Russell: How did we get over it? We just didn't believe that that was true. Mainly because people care about quality and the amount of money they spend, they as in companies, spend on doing QA with automation is a lot because of the humans.

And even QA with manual labor is really expensive when you take into account the head cost.

And that's actually normally ignored by people. They're like, "Hey, it's not very expensive to do, but it's really slow." But then they ignore the fact they've got five people doing it that cost 70 grand a year.

So turns out people do actually spend a lot of money on it. And the reason why is because they want to ship a quality product. And having a human QA team normally gives you good quality, but is generally slow, and then automation gives you also good quality, but it's slow in a different way.

So it's difficult to maintain a large suite of tests with automation. You just end up with more and more debt. The other thing is you'll find that it slows down the whole process of delivering something to production, which is obviously not ideal if you're trying to ship stuff many times a day.

Guy: No, absolutely not. I'm just curious, so in Rainforest would you say that the audience is more of the developer that almost sort of wants QA done, but doesn't want it sort of breaking their flows and their continuous practices, or is it the testing team that is looking to sort of broaden?

Russell: So today it's not the developer. It's mainly the testing team or the product team that want this. So we find the most interest from those two audiences.

So QA want to optimize what they're doing. There's several types of QA. There's the QA where you're just checking. Somebody's already written the tests. And you're running through on five or ten browsers, just going through a flow, checking that it's working correctly.

That we call checking. That's what Rainforest does. It removes that workload from the QA team so that they can focus on much more leveraged things like exploratory testing or writing new tests or figuring out what the test should actually be. So yeah, that's what Rainforest does, is running through flows, functional testing.

Guy: For me it's sort of interesting. I've always sort of been very much in sort of pure software. I kind of get the difference between the sort of desktop where I've had experience building downloadable software versus sort of this transition to the world of hosted software, of ASP turned into SaaS and that world.

But this sort of notion of human automation, of sort of invoking the power of sort of humans around the globe almost feels like a new breed of it. How much do you feel like the standard, if such a thing exists, methodologies and approaches of sort of SaaS businesses apply to your business?

Russell: It's an interesting question. At least maybe ten years ago, people were like super excited about the crowd. And a lot of platforms have come up, so like Mechanical Turk or CrowdFlower or there's a whole heap of other ones, but most of them are horizontally scaled, so they're like, "Hey, here's this generic platform of humans that you can do anything with."

And so far I've not seen any of those be hugely successful, probably maybe with the exception of MTurk, but it's arguable whether that's hugely acceptable, it's definitely like the bastard child of AWS. But I've seen more success recently with the verticalized ones. So the ones that use humans that are trained in a very specific way to do a specific thing.

So like Rainforest is an example, but something like LeadGenius is also an example. They're crowdsourced, partly computer, sorry, partly machine learning based system for finding leads for companies. But there's many other examples of people doing the vertical version.

But I still don't see it becoming a common pattern. People tend to ignore things that can be done with humans in a nice way.

Guy: It's interesting though, like, you know, this is the enterprise flavor of the sharing economy. And you've sort of seen the same thing in consumer products, right? You've seen these sort of TaskRabbit or whatever, you know, I don't know if you've even heard of them, right? All these sort of again horizontal, generic personal assistant who would do all sorts of stuff.

I don't know if they're sort of successful, but it feels like they're not as successful as, you know, Uber or Instacart or sort of examples that are very, very focused. They're also sort of engaging, they're also doing some sort of time arbitrage of somebody who has a service to offer and kind of enabling. So it's interesting to sort of see, to me that aspect, once you bring it up, feels very natural analogy to what we've seen maybe a few years ahead of that in the world of a consumer.

Russell: Yeah, I hope more people will figure out that this is a good thing to do.

Remote Teams and Time Zones

Guy: The other topic, you know, when we chat before, one of the things that are sort of interesting to talk about is just sort of location. So, you know, you're based in San Francisco, but you've got a team around the world. How do you see that working? You know, whether it's related or not to the actual type of business, you know, would it work for everybody? What do you see kind of working and failing there?

Russell: Yeah, that's an interesting question. I do believe that remote teams work pretty well. There's definitely trade-offs compared to having a fully local team in San Francisco or actually wherever you're based, but it's definitely been easier to hire really great people than it would be here. Sure, there's really great people in San Francisco, but A, it's super expensive and B, there's a whole heap of competition.

Whereas if you're like a really great developer in say Germany or in Portugal or in Tokyo, like it's way, way different to being in San Francisco.

But I mean, so we've hired people I think on most continents apart from Australia now, actually we don't have anyone in Africa either, but we have about 13 people on the dev team and 11 of them are remote today. So they're literally all over the place working from their own houses.

And for people that have families as well, there seems to be a massive win. So like most of the, I say most, like maybe fifty percent of them, the people remote, have families and with children and stuff. So it makes it much easier for them to have a flexible, we also have flexible time by the way. So they'd like take their kids to school and then come back, go and pick them up for, have lunch with them or whatever, and then pick them up from school and then work later in the evening after they've gone to bed.

So it us fitting around their life they seem to really love. So anyway, we can give people a different type of job than just come into the office and work nine to five.

Guy: And how do you deal with time zones? I've sort of worked between sort of Israel and the East Coast and from Israel with the West Coast and, you know, now my team is between London and Israel.

So like time zones are generally a bit of a pain. London and Israel is kind of easy. But you know, how do you deal with people all over the globe? You know, how do you have a company meeting?

Russell: So mostly we do everything over Slack, but also if we ever have to do a team-wide meeting, we would do that at like nine or ten AM PST. And that seems to at the moment be a common time when everybody's online. Yeah, it might not be normal working time, but it's working time for the people. Yeah, it's so flexible here, it works pretty well.

As for like larger meetings, one of the things we do is bring the whole team to San Francisco three times a year. So like Q1, two and three, everybody flies into San Francisco for a week. And then Q4 we normally go somewhere off site. So that's the entire company.

So we use that to get to know each other. So like the local sales, dev and marketing team will get to know the remote developers. We found that to be really awesome, especially 'cause the remote developers don't get to work with each other face to face either. So like everyone bonds and it's really good.

Guy: Yeah, I feel like face-to-face is a sort of a must in building the culture. I've sort of had a variety of experiences here. You know, first I was in an Israeli company that got acquired by the Canadian one and when I moved to Canada, you know, this was sort of a total of a two hundred person company.

And working sort of with a seven-hour time zone difference and a slightly off sort of work week, with Israel being Sunday to Thursday versus Monday to Friday, it was tough and really the primary solution there was, the teams were big enough already to have pretty much separate projects in those sort of two locations.

Russell: I can imagine that's massively difficult to do when you didn't start that way.

Guy: Yeah.

Russell: But I think we've had an advantage because we started as a remote team. So we had one employee in San Francisco and then we hired remote people and then just never looked back from there really for developers. So I think that really helps.

And even when it was just Fred and I, my co-founder, we still used Slack. Well, I think HipChat back then, before Slack was cool. Even though we were like sitting next to each other, we'd still be on Slack. Because we'd be headphones on, listening to music, writing some code, so to not interrupt each other. So I think we're kind of the odd ones out on that one.

Guy: Well, and I agree with you that there's this lack of correlation between the talent of an individual who is located in a less central location and the opportunities in front of him, basically. You know, you can get sort of better people, might not be quite as competitive if you're set up that way.

And we're, you know, very much working in the same way. We're only seven people now, but we've got four in London, one in Brighton, which may sound like London, but is actually an hour and a bit south, and two in Israel, and we're sort of...

Russell: So the people in London in one office?

Guy: Yeah, so there's four of us. So we're building those out.

And I think, you know, we're putting the same type of emphasis on in-person meetings. You know, the folks from Israel came over twice now, the entire team is going to Israel, the London team, in January. And we plan to sort of maintain that and expand, but we do want the sort of the two centres in London and Israel to have a certain critical mass.

Russell: So you're not going the fully remote, you're going the offices route.

Culture Across Locations

Guy: Yeah, and I find the challenge with entirely distributed is just this notion of culture. So I guess my bet here a little bit, it's sort of left to be seen that it works, is we have the two locations, we're growing them. None of them is necessarily the center. They're both sort of equal weight. And for me specifically, it worked well with talent.

We're building developer-focused security company. And it's between sort of my network and just sort of the talent around, it's easier for me to sort of find really good sort of developer tooling type talent in London, while in Israel I can find really good security talent. And both of those are kind of more hard to find in the other location.

Also, there's good tech talent in both locations. Israel is a much more sort of mature startup environment.

So competition there is almost as fierce as it is in the Valley. In London the competition is more, have to do with people's kind of orientation towards working for a bank or some big company. The risk appetite is a little different. So it's, you know, those things sort of hedge and allow me to sort of tap into talent of different varieties.

Russell: It's interesting that you are worried about culture for a fairly remote team, as in a fully distributed team. Because I strongly believe we have a good culture even though we're fifty percent totally distributed. Though we do have an office culture and the remote culture at the moment.

They're similar but not quite the same. But I still believe both are good and that's been quite difficult to curate, I guess, but is definitely possible. If you're doubting that, I believe you're wrong.

Guy: No, and I totally believe that it's doable and I'd also like to sort of tap into this opportunity of sort of people everywhere and good talent, you know, we've had some really good hires in Akamai in that fashion. And you know, I definitely want to build that out.

It feels to me like there's a difference when there's no sort of core five or six people in at least sort of one or two of those locations, when you talk about sort of attracting people that are nearby. I could be wrong here, so I guess I'm kind of rolling with those.

But what I do feel is that because the two teams work very closely, and we actually have three locations with Brighton as well, that everything happens over Slack, you know, everything is kind of documented digitally, the hangouts and the like, everything is sort of video hangouts. So it's easier for us to sort of embrace somebody else now that is not local, which sort of opens up that opportunity.

Russell: It sounds like, based on what you're hiring for at least, that you'll have a pretty good split of the project between the two locations, right? Or the three locations. And if so, doesn't that itself affect the culture? Because you'll have a different culture in both offices, presumably.

Guy: Everybody works actually with everybody else. It's just about the sort of the talent pool. There's no value in security knowledge without sort of a good display of the security information or providing the information for it or having a system that triggers the right events once a new vulnerability is up or a front end that will display it.

There's no separation between the teams of the ownerships, and it helps that it's just a two-hour time zone and it's an easy kind of hop and a skip, maybe compared to, you know, it's more East Coast.

It's sort of like having two teams on the East Coast and West Coast of the US.

So there's no intent to do it, but there's a different talent pool.

So I guess there's a lot of roles that I would just love to get people in either of those locations, but there are some specific types of roles where I would look in both locations, but it's much more likely that I would find security researchers in Israel and much more likely that I would find people with existing developer outreach in London.

Russell: That makes sense. You've got a very small office in Brighton, right?

Guy: Brighton is not. It's a one-person office. So he works from, Remy Sharp. He's actually fairly well known in the JavaScript world. So he kinda works as a, you know, once we have two or three people in the same spot, if they want it, we will get a local office for them.

Russell: Interesting. So you may end up with a little hub of just separate people for a while.

Guy: Yeah, and I think that's okay as long as, I feel like camaraderie is very important. So you could do that remote. I think you can be entirely remote.

The other thing that I do think doesn't work, and if you started remote then that's probably how you're doing it, but my sense was the worst thing is that when there is an existing core, if you talk to people that work with even like at Microsoft at their size, or like anybody here in London, you know, when you work about all these sort of US-based companies that expanded to the world, the Twitter, the Facebooks, and everything is like, yeah, the core, the heart, all of those conversations happen in, you know, location X, in Redmond, in San Francisco. And then everybody else is sort of a second grade.

In Akamai it was really hard. Akamai is very much a US company and it expanded. The field activities have a lot of autonomy, but product activities in the different locations had a hard time. You know, like Bangalore eventually got to a critical mass where it's sort of big enough on its own. Krakow is working on it.

Russell: It's very bad if you end up with the remote people being a second class citizen.

Guy: Yeah, and it's not intentional, but you need to make sure that there's not a large weight to sort of hallway conversations and the likes, right?

Those need to be digital hallways.

Russell: You have to be super deliberate about making that part of your culture. Like that, yeah. And we've been pretty careful about that, but also we're forced to because of the nature of having that many remote people.

Guy: It's interesting to stop and think that the tech stacks have sort of enabled this, almost all sorts of things like, you know, Git or GitHub.

Russell: If we didn't have GitHub or equivalent, and also Slack of course, it'd be very difficult to do this.

Guy: You could just sort of imagine doing this when somebody could check out and lock a file remote in a different time zone and you couldn't edit it in the meeting.

Russell: That would be massively painful. But luckily we have Git, so yeah.

Guy: And video hangouts, and there's all these sort of technology advancements we now take for granted that have enabled this type of functionality. There's cultural things that we need to sort of adapt and to build out, there's some best practices and they grow.

Interestingly, I have a friend who worked for Slack in the early days and then eventually sort of stopped working for them because they pretty much discontinued their remote teams. I don't know if there's sort of validity in that statement, but there was a little bit of, you know, that's super bizarre for Slack, cobbler goes barefoot. So, you know, it's one person's input, so it may be that he got the facts wrong. But it was an interesting observation.

Hiring Early and Always

Russell: So I guess moving on to another question that we had, or topic we had rather, is hiring. So we've been through like a whole journey of changing our hiring process and like learning how to hire. I've run a company before but this is my first actual startup. And I know you've obviously been CTO and founder of, well, like two companies now. So I'd love to hear how you guys hire and how that's evolved and what it looks like, especially for developers.

Guy: Yeah. There's no sort of secret sauce, unfortunately or fortunately, depending on... I feel like hiring is very different depending on the stage that you're in.

In the early stages, my preference has been, and that's actually one thing that I felt I did well, is maybe sort of two things. One is targeted, so reaching out, explicitly talking to people, sort of understanding that good people are in good places and that you're going to get turned down nine times, not because you're doing something wrong or because it's not that appealing, or they might even sort of like the idea, but generally good people are in good places and you need to also happen to get lucky with the timing.

So there's some sort of thick skin that needs to be developed to still continue and talk to them.

And whenever you have that type of a turn down from people that you sort of consider to be the top tier, ask them about others and then proceed from there. So I feel like in the early days that's necessary.

And then the sort of the second aspect, I would say, when you're still sort of at the small team, then it's the kind of always be hiring type of motto. You need to constantly keep doing it and sort of have a constant pipeline, and then you should be devoting, especially as a founder, like some person on the company should be devoting a good sort of twenty, thirty percent of their time probably to finding good people, and at some point even more.

When you're big, when you're like Akamai, that's a whole different model, right? You need to hopefully have built, if not, you need to kind of work hard to build kind of a good brand outside in terms of an employee. You want your developers, or your team, to be able to sort of go back and bring others inside. And it's a lot about sort of presence and brand, and you want to have the postings and the opportunities and the kind of nurturing, you know, there's sort of the opportunity to bring people when they're young. So that's a whole different ball game.

I think the key challenge is sort of in between, which is not a small gap, where you are not quite big enough to be devoting entire teams and structures to just be sort of hiring and building those out, but you need more people than what your sort of personal network and outreach can get.

So I think in those, for me right now, from what I've experienced in Blaze and what I'm sort of starting to get into soon here, it ends up sort of being a little bit of a, you should ignore the possible direction. So a little bit of both paths go.

You need to always be hiring, have a big pipeline, hire good people even if you don't have a role for them.

And at the same time you need to start thinking about brand awareness. Are you open sourcing some tools that are general? Are you present at events as a speaker or as a participant to sort of demonstrate your brand? And it's just an evolution. You're gonna get from personal network to sort of this big brand base, and you need to be kind of dancing on in both worlds for a little while.

Russell: I think it's true. The interesting one is, at least for us, as in Fred and I, we both moved from the UK to San Francisco, or Mountain View at the time. So we didn't actually have a personal network to hire from, either from university or from previous roles in the US, because we had never done anything here.

So that was actually quite an interesting problem. But we ended up hitting on some of the things he said, so like events and also conferences seem to be like a win. But luckily we went through Y Combinator in twenty twelve. So they have this awesome thing where you can list jobs on Hacker News. And that's been awesome for us so far.

How to Interview Developers

But yeah, I'm curious how you actually interview people after you found them. Because that's been one of the things that we've evolved the most at Rainforest. But I'm always curious to hear other people's processes.

Guy: Yeah, so I'd love to sort of hear about your sort of processes there as well. In my end, I feel like a lot of it is almost like iteration. My approach here is that this is similar to the iteration process in almost the software itself.

You know, you have to start, when your team is even at sort of 20-30 people, personality fit, and you know, sort of starting from scrutinizing whether this is a person that is sort of pleasant, that you want, that fits the theme, is almost sort of number one and definitely is sort of easy.

It's more intangible, but you know, you get that sense, right? If you put that person in front of multiple people, then, you know, if people have doubt of it, it's one of those thresholds that you should, you know, be patient and not hire that person. From a technical skill perspective, I like two approaches, and we're not that sort of structured yet in how we define them, but we try to suss out, on one hand, just sort of go deep into one or two of the topics that the person should have skill in, figure out, is this a mobile developer that is really building on top of platforms, or does he understand, you know, how does the components work, right?

Is this a front-end developer that just uses React, or does she know, you know, what are the sort of fundamentals of HTML5 and what are the sort of the new technologies that are coming up? So those components matter a lot. And so that's one type of interview.

And the second type of interview is more of a free form, ask a question, sort of see how the mind works. There's one question that I've always found appealing, which is, you know, you open a browser, you put in the URL, you hit enter, a page shows up, what happened in between? And there are a thousand different ways to answer that question and none of them are right or wrong, but it shows you a mindset of how that person thinks.

Russell: Yeah, for me that one's a classic. It's more for me is what questions they ask about the question. Like how deep do they want you to go.

Guy: And I also like, sorry to kind of chime in, I guess sort of wanted to throw in two things there additional. One is I really like asking what project are you most proud of and why. Which I also find elicits a good conversation.

And then kind of the last topic is on hiring in diversity and sort of hiring women, hiring kind of people of different groups that are not sort of white dudes.

And the approach, I'm, you know, really kind of working hard. I feel like totally messed that up in Blaze, you know, sort of a group of dudes there, and feel like it's a very important topic. It's near to my heart. We're working hard on it in Snyk. And you know, we're sort of in decent shape right now, but keep doing it.

And the one kind of guideline I liked the most, and we had a lot of conversations about this in Akamai, who is coming up from behind but is kind of making good progress there, is the notion of having a woman finalist. So when you have roles and you have them open, make sure that you do not hire a person for that role without having sort of a woman candidate that has sort of crossed the line. It's a little bit of a narrow version of diversity, of just sort of looking at women, but I feel like that forces you to look longer, but it puts you in a position where you're gonna make kind of better decisions.

Russell: Yeah, for me the main problem we've been having, so diversity wise, like Rainforest is, for programmers, is 100% male, but is I think somewhere under 50% white male. So we're diverse but not on a sex level, if that makes sense. Or gender rather. But I think we've realized that one of the biggest problems we're having is where we're listing jobs.

We're like only mainly listing on Hacker News, which is probably 99. 5% male. So that's one of the things I'm looking at fixing at the moment, is like where can we source more diverse candidates? That is an unanswered question for us, but one that we're taking seriously.

Guy: And there are groups, by the way, like specifically for sort of women in tech, there are groups that would help kind of circulate your resumes, so they can do a lot of the work for you.

Russell: Yeah, we're on the case. So let's see what happens in twenty sixteen.

So going back to your interview, do you do much technical interviewing? Is it whiteboarding or is it practical or how does that look for a candidate? And how long's the process, end to end?

Guy: Yeah, so we're not kind of that structured in the process. We haven't been in Blaze. It's more about sort of hitting these three types of conversations to have. So you have to sort of get a fit on the personal front. You have to do at least some conversations and sort of go deeper into a tech topic.

And you have to sort of have some of those sort of broader technical but more sort of open conversations to see if a person can actually describe and understand a system that they've kind of claim to have been architected, can you get down to the detail level? And oftentimes that would flush out people that sort of talk the talk but don't actually understand what happened behind the scenes.

As far as which technical skills, I guess I personally, and it was aligned with others when we interview, don't feel like you can really suss out sort of full technical depth. Well, not full technical kind of mastery of a topic that's really heavy. What I feel like you can suss out is technical depth or sort of inclination, right?

If you talk to somebody who's building in Python, you can talk about CPython versus, you know, how do the internals work? How are different objects implemented? If you talk to them about, you know, Java, you can talk about, to see if they know about sort of IBM versus Sun. If they talk about the browser, you can ask for their opinion about some of the latest trends. And those sort of flush out a depth, sort of a level of desire even to understand.

And by the way, those don't disqualify, like the fact is somebody can't sort of go all the way down to the metal, it depends on the role that you're hiring. If they say they did and they cannot, then that's a problem.

And then the other aspect is just probation. Like, you know, you need to cut people loose if they start off...

Hopefully you don't make many mistakes, but you are going to make mistakes, and the worst thing is to sort of cling to your previous decision.

Had a couple of those cases in kind of startups before and it was hard but very important.

Russell: Is that as difficult to do in Israel as is in London for employees? So obviously like in California it's at will, so it's like you can instantly terminate someone. But in London you can't.

Guy: Like at least legally you can't. So generally you'd set up a probation period during which it's easier to do. Letting somebody go afterwards, it depends. When you're sort of a small, in a startup, with the relevant notice, the primary difference is the notice period is the key thing.

Russell: And the more formal process, right?

Guy: There's a more formal process. I guess usually, for right or wrong, usually not as scrutinized for smaller companies. It's more about the sort of the larger employers. I guess you're just not much of a lawsuit target.

The Hack Hour

Russell: Yeah, so talking a bit about Rainforest hiring process, it's pretty structured because we want to try and get consistent, fair results, even if it's done by different people. So originally the process was run by one person, which was me, and then handed over to somebody else. But now it's actually distributed between the three teams that we have at Rainforest. So each team will hire for their own roles, but the process is basically the same.

So we have a very simple coding challenge that you can actually do online yourself. It should take you like under two minutes to do, just to filter out the total noob. Well, I guess before we had that, we had so many applicants that it was very difficult to filter them out before spending a lot of time doing an interview.

Guy: Also filter is interest level, right? Because if somebody's not willing to do that, they're basically...

Russell: Yeah, sure. There's an argument that that's bad, but personally I think it's good for the reason you said, it's like if they're really interested, they'll spend the two minutes doing this. And so far that's proven to be totally true.

But so after that, if you get through, we check out your CV, we read it and check out your GitHub and stuff and then put you on to the next stage, which is a short technical interview. And we ask two or three questions. Very simple coding that we watch you do over Hangouts normally. But I think those two aren't the most interesting ones.

The most interesting part of our process so far is we do something called a hack hour, where we would set up a time when convenient over Hangouts and ask the candidate to do about an hour's worth of work on one of their side projects or something open source that they know well, and add a feature or fix a bug in front of us. And that gives us massive insight into them and what they're capable of and what they think they can do in an hour.

But it also shows us stuff like their tooling choice and how they architected their project, what technology they chose and why they're doing it in the first place. So yeah, that's been the best part of our interview process.

Because I think the other thing that we realized was that the whiteboarding style of interview, or even the asking them to solve a random problem kind of interview, is actually a significant amount of pressure for the candidate.

And this actually removes a lot of that pressure because you already know what you're gonna do before the hour, because you're allowed to think about it. You already know the tooling and the code because you wrote it, or you should have. So yeah, that for us has been the most interesting thing to have uncovered on the journey of interviewing.

Guy: No, it sounds like a very interesting thing. Do people push back in terms of, like, do they get uncomfortable having somebody digitally stand over their shoulder as they're...

Russell: So far no, and we also generally will help them. So if they're stuck... None of the interviews we do at all are closed, so even the first one, which is like solve an arbitrary problem that should take you a few minutes to do, you can Google, you can open Google and open Python docs or Ruby docs or whatever you need if you've forgotten the method.

I don't believe in doing a closed book test because it's not a realistic setup. Like I'm opening Google like 20 times a day trying to remember some random method on array class I've like totally forgotten. So I don't think that's fair at all. And that's very common in other people's interviews, is to do a totally closed whiteboarding, like unrealistic interview. Maybe it shows stuff, but I've never found it to be good, just massive pressure.

Guy: So I generally agree with that statement. The only sort of caveat I'd have is when I ask somebody to sort of, you know, drilling in and trying to sort of understand about how does this work and how does that work, I don't really care to sort of see kind of their level of depth in it, of like whatever, how some structure works, but I do want to see whether they were interested in checking, and also sometimes in them answering a straight up... Like really the most common scenario is that they would make up an answer, right? Some bullshit. And those are kind of the best flush outs of like, okay.

Russell: Like what we're doing doesn't at all preclude that. Like we do that in the hour-long interview. We're like, "Hey, why did you choose Redis for this?" Or like, "What is Redis? Like I've never used this before."

Well, most of the time it's not that. It's like some random framework that they're using that I've never used. And so I'll drill into why they picked it and what it is and how this individual thing works.

But yeah, it'll be on code that they know and that's in front of them normally. So yeah, I totally agree that you should dig in until some deep runs out of knowledge. And then hopefully they say I don't know, which is a valid answer.

Many Kinds of CTO

Guy: So I guess, you know, kind of maybe clinging on to this one and maybe tapping another topic we're hoping to cover is this notion of sort of CTO, or sort of this role of a technical leader. And how does that transition? So you mentioned that you used to own this and now you've transitioned it to somebody else. And you know, how was your sort of journey there in terms of the CTO role change? Same title, different role.

Russell: Well yeah, same title. The CTO role, it seems, I've talked to I don't know ten, twenty CTOs of various stages, trying to find mentors for myself. And none of them are doing the same thing. Like the roles are so different.

So anywhere from like a two-person startup CTO, which is basically doing whatever is needed, to, I think the biggest, somebody with like 250 engineers reporting to him was the biggest CTO that I've talked to recently, the role's totally different. And even the guy with that many people, his role is still changing week to week, what he's doing. He's still learning new things all the time.

So it's pretty crazy. The closest thing I've come to find as a common pattern is that there's three main types of CTOs. There's like the technical leader ninja CTO that prototypes everything. There's the marketing focused one, so Vogels, Werner Vogels-esque.

And then there's the engineering leadership one, which would be the sort of more product-focused version of CTO. So I'm still trying to decide which one is the best for me.

Currently I've done all three and been variously successful at some. So I actually really love doing the marketing side of stuff. So running events and talking at events. But currently I'm doing the technical leadership stuff. So I'm working on the more either difficult to do or bleeding edge stuff for Rainforest.

The management stuff has been a massive learning curve for me because, well, I've never really managed a team of engineers before. So it's been learning from other people and then making a bunch of mistakes and trying to not make those mistakes many times. And then realizing that actually I'm probably not the best person to manage like the day-to-day stuff.

So Rainforest actually moved to a teal management structure, if that makes sense. So we have individually managed, self-managed teams. So there's a role which does planning on each team. So yeah, I don't do day-to-day planning of what people are up to on the dev team, which is an interesting transition.

Guy: Hopefully even as a manager in a larger organization you don't do day to day planning.

Russell: Sure, but even week to week for me is not a thing. I help people on the longer term stuff, but that's advice, not demand, if that makes sense. It's not "you must do this."

So yeah, that's been an interesting transition. I personally, and totally personally, believe in teal being like the right way of doing it for us. It's definitely made the developers more engaged and I think much happier, and from talking to them I believe that to be true, and from observing them too. So yeah, they have much higher ownership now of Rainforest and what they're working on, which has only been a good thing for us so far.

Guy: I guess for me, I kind of agree with your classification. I'm on this email list of CTOs in London. There's maybe two hundred CTOs on it. At some point somebody threw out this sort of question of like, you know, what is a CTO, triggered like 60 or 70 different, kind of a pretty heavy email thread, like a really interesting one.

So to a certain extent it boils down to the three categories you talked about. It changes when the organizations get big. So there's also the notion of sort of inward-facing. So in Akamai I was a marketing, maybe kind of focused on that, but a lot of my effort was actually kind of inward focused, not doing the prototypes, but rather investing in driving sort of innovation vehicles. And innovation is kind of an abused word, but making it easier for others to prototype and to sort of bring, not just prototype, but move from prototype to market.

So there, you know, the variety, the inconsistency of the definition works. You know, I got into it when I was a product manager in some earlier part of my career, I asked a mentor, "You know, is this the right path to CTO?" And so, "Well, the best way to become CTO is to found a company and call yourself a CTO." So eventually that's what I did and just sort of rolled with it.

So I, you know, I've sort of personally always been more on the, you know, I've gone from sitting in a kind of in a cave or something in my bedroom and writing code for six months to gradually shifting out of that and being very much the sort of outward facing, hopefully sort of vision driving, yeah. What might probably fall into that sort of marketing CTO you've defined.

Russell: I think that's a good role. Like it seems like a really fun role.

Guy: I like it. There's some road warrior components to it every now and then which are challenging, and I think it's different people, but I feel like there's definitely a need for you to sort of define what is your forte, in part for your sort of, you know, personal sanity, but also for kind of a complementary of the team, because at the end of the day in the team you need sort of a strong presence in each one of these three functions.

Somebody needs to be more the face of the company, the outward facing, maybe it's the CEO, maybe it's not. Somebody needs to manage engineering and somebody needs to sort of drive and be a bit of a technical expert.

So if you're gonna choose one of those, you need to make sure you have somebody else filling the other roles.

Russell: Yeah, I've also heard it described like that, as basically all the things the CEO doesn't do that are roughly technical. And that seems to be pretty true too.

Guy: I think it depends on the roles. Like I feel in developer tooling companies specifically there's an expectation from the CEO to at least talk tech well, if not actually sort of be tech, given the sort of the audience is kind of, well, I guess in security it depends on who you're talking to. But I think there's value there.

Russell: You personally have both bases covered, so...

Guy: I, you know, I feel like I've gone from the sort of, you know, hardcore developer, I've actually written very little code in Snyk, and I am sometimes happy about that and sometimes sad, depending on the moment, but it's also one of those things where you can't... So I've gone from CTO to CEO. And it's a bit of an evolution and I've yet to sort of know whether I will be, you know, happy as a CEO of a hundred person company. But at this phase it works out well for me.

Russell: It's definitely been, so Rainforest is about twenty-eight people, just for size, and it's been interesting watching my co-founder's evolution as CEO compared to how we were like a few years ago. And it's definitely a totally different role to CTO. So I wish you luck with that one.

Guy: And I think, like for me, I enjoy the broader puzzle, the company building puzzle, and I take a little bit of enjoyment in everything. You know, I've got a bit of a weird enjoyment sometimes in even sort of reviewing a legal document, and I enjoy kind of structuring the finance plans. I don't enjoy it as much as coding and I don't enjoy it as much as sort of product definitions, sufficiently to sort of fill in the role.

Russell: Yeah, I used to do COO style stuff as well, until we hired, thankfully, Sharon, who joined us to solve all those problems of me spending too much time doing that. But yeah, it's fun. Like, but it's not productive, at least for me to be doing that now. I should be much more focused on tech.

Hiring Yourself Out of a Job

Guy: And that's sort of a big aspect in general, I think, when you build a company, which is, you know, it depends a little bit as well on who are your sort of co-founders and what are the different skill sets, but, you know, there's many roles that do not exist. There are many roles where basically somebody needs to fill them, and usually those fall into the CEO, because at the beginning there's a lot more sort of tech activity going on.

So the majority of the team is technical. So there are many people who do all sorts of technical things in the company and they need to be led, they need to be sort of driven, and that's the CTO role.

And there are all these other sort of things. There are a million of them, ranging from sort of, you know, finance to indeed sort of marketing aspects, to the legal components, to finding an office, to, you know, all sorts of related to the pricing elements, to the sales. So many of these things, depending on the individuals, fall into the CEO. So, you know, you need some generalists there.

But I think also that's a bit of the secret of a successful founding team, which is the kind of people that enjoy doing different things.

Russell: I think is also understanding when to hire for those people, hire away those roles, has been a big thing, like focusing on what's actually high leverage for individual founders.

Guy: Yeah. And I, for me right now, look to sort of hire myself out of a job. I mean my top candidates, above and beyond sort of the product definitions, is just to look at what is taking up the majority of my time and hire somebody to take those on.

I mean right now a lot of my time goes to recruiting, and a lot of my time, which is probably going to continue, but still I probably need people that focus more time on that. And a lot of my time goes on what I would define as developer relations and outreach and, you know, kind of broad, like writing O'Reilly Radar posts and 24 ways posts and the likes. And those are two sort of big chunks of time that I probably need a full time person doing. Not a full time recruiter, but somebody who's kind of more team building than acting. So you know, those would be priorities for hiring.

Russell: I'm stalking you on your about page and you don't have a CTO listed.

Guy: I do not have a CTO listed.

Russell: You have three founders, security and ops, and you, why do you not have a CTO role? This is the ask a question.

Guy: So to a certain extent I've got three. Like to a certain extent myself, Danny, and Assaf all three sort of fall into that category, and because of that none of us are. There sort of hasn't been quite a need and we're, you know, sort of reserving the flexibility in terms of what we did. So it may be that either Danny or Assaf or somebody else on the team would sort of take that role. It may be that we find sort of the right person to sort of fill in the role.

So right now it's a little bit of a, you know, an ace up our sleeve that we can use as sort of some extra leverage for the right person coming in. So, you know, yeah, it was actually frequent conversation. I feel like at this phase we've got a ton of technical leadership. So while we don't have that sort of bullet items we have there, and we have a fairly technical CEO. So no, we get by, but it was an interesting conversation.

I also need to make sure that I don't hire, I think one of the bad things to do, I think it was in Marc Andreessen's book where he was talking about, or in one of his gathering of posts, when he was talking about how founders prove to be worst at hiring somebody for their own sort of previous skill set. So to a certain extent I feel like I will need to make sure that when we bring in a CTO, or somebody takes on that role, I know what responsibilities I'm shedding, and actually sort of set up that person for success, as opposed to sort of be constantly kind of fiddling and micromanaging them.

Russell: Well yeah, I guess you've got to work out whether you're gonna hire someone to do it in your image or in their own way.

Guy: Yeah.

Russell: And that's gonna be difficult to hand off, I guess.

Guy: Well, maybe. And I feel like today we can, you know, I guess we've got the luxury of not needing to make that decision quite yet.

Russell: Well, I guess that's one way of looking at it. The other way is that it'll become much more difficult the bigger you get. So maybe you should do it now.

What Titles Are For

Guy: Possibly. The other title, by the way, that we don't have is a VP R&D. So it's one of those things, to talk about a self-managing team. We've got a team of senior people right now, you know, all capable and many with experience of managing small teams. So there sort of hasn't really been a need to do it.

To a certain extent, that mention that I said about sort of having somebody who's focused on building the team, that might be that person.

So I think titles matter. They matter for intent. When you hire a person, they matter for communication outside.

But also when you're hiring, titles are also a parameter in communicating that intent and your sort of commitment. If you hire somebody and you give him a VP R&D title, you're not going to hire another person the next day and give him the same title. It's a given. So it's a declaration about their role in the company, which while flexible, I think is not pointless.

Russell: Titles is a whole nother conversation that we do not have time for today. But yeah, okay, I think we have to wrap this up now because I have a hard stop in about two and a half minutes.

So that was great. I'm Russ. You can follow me on Twitter, @rhs. Or you can check out Rainforest at www. rainforestqa.com.

Guy: And I'm Guy Podjarny, GuyPo. I'm @guypod on Twitter, G-U-Y-P-O-D. And you can check out Snyk at snyk .io. So now you know, that's the acronym, a good way to remember it. Okay, catch you soon.

Russell: Thank you very much.

Ted: Thanks for listening to this episode of Caveat Founder, brought to you by Heavybit. Head over to Heavybit.com to sign up to be notified when the next episode is available. And while you're there, check out our library. It's home to over 75 talks from other developer company founders and industry leaders.