Ep. #21, Continuous AI with Chad Metcalf of Continue

Chad Metcalf: So I've had a bunch of different careers.

One of them was a software engineer and I worked for a company that did work for DARPA.

So we did a bunch of robotics and we did distributed systems and we had these tiny little embedded devices that were called StrongARM.

That was the name of the chip, the device was called NetWinder.

And my favorite thing about the NetWinder is one of the engineers recorded his five year old daughter saying "NetWinder."

And whenever you would hit boot it was hard coded into the ROM and then it would pipe it to the system speaker but it had no memory, it was just an early, early ARM.

And so builds would take 24 to 36 hours to build the entire software suite. And at one point I snapped.

And so I went out and I bought a GCC book and I figured out how to do cross compilers because that was before cross compilers were just in the package repositories, you actually had to build them from scratch.

And then that sent me down this eventual hill of moving from software engineering through what now would probably be called a platform team.

So when I moved out here I started doing startups and one of the first startups I was at was a company called Cloudera.

That was a big data company and I had a team called the Kitchen.

And the whole name of that, why it was called the Kitchen was that's where all the stuff happens. It happens in the kitchen.

And Hal Varian had a quote for this from Google.

It's like, "part of what makes Google awesome is our kitchen is better than your kitchen. So we can cook better food, not because we have better ingredients, but because our kitchen's fantastic and we have everything. What you need is where it is."

And so we called it Kitchen and we were responsible for build and release and we were responsible for all of the pieces of taking 70-something open source projects and putting them together into a distribution.

So not quite as big as Linux, but it was a big data distribution.

That was back when Jenkins was Hudson, before it split and before, I don't know if you know this, but Jenkins used to be called Hudson and that was the open source community.

And then that somehow got sucked into Oracle and then they did things that they typically do and then it forked out and created a new--

The original project team left and created a new thing.

I joked with somebody the other day when we were chatting about CI that my career started before what we consider CI.

I can remember the very first thing that I used in the space was a python project called Cruise Control. I don't even know if like--

John McBride: Oh, I've used this before.

Chad: Yeah, yeah, I mean that was the first. Right?

It's basically, yeah, it's on SourceForge. I don't even want to talk about how long ago this probably was but it was a long time ago.

Brian Douglas: Yeah. So I realized we should probably just welcome the podcast through the intro and then we'll jump back into this journey.

So welcome back to another episode of Open Source Ready? John, are you ready?

John: I'm ready. Brian, are you ready?

Brian: I'm very ready.

I'm super excited to talk with Chad, CEO of Continue.

Continue.dev is actually a platform for-- Actually it's a place I work my day job.

So we're helping folks build customers coding agents, but also build and deploy them at scale using open source software.

So Chad, you were already talking to us about your journey, your history, but do you want to catch us up on what you're doing today?

Chad: Yeah, so I joined as the CEO of Continue in May. Before that I was an advisor for Continue. So I met Nate and Ty at an Ollama meetup.

So in the summer of 2023, I was super into Ollama because I know that team from when I worked at Docker and I was playing with it and trying to figure out useful things.

And then I go to a meetup at the Replicate office and Nate and Ty got up there and they did the demo and I had some of that same like, "ooh, very cool," sort of developer tingle experience that like, I could figure out something to use for this.

And in the time between them showing the demo and, I already had Ollama going. So then I was able to get the whole thing up during that demo and I thought, "this is pretty awesome. So like, how can I help? This is one of the first things I've seen that's really useful for this and I want to be part of it."

So, yeah, I just started helping however I could and then fast forward a little bit and I joined the company.

John: Yeah. And how has it been going so far?

Chad: Oh, it's a lot of fun. So every CEO's journey is probably very different.

Mine is I do a lot of things I didn't expect to do, but one of those things is actually write code.

So, I think Continue has helped me knock enough rust off where I can, you know, I get a question from a potential customer and I can ask Continue to help me understand why this is happening or what's happening.

In one case, like two weeks ago, we found a small bug and was able to do a two line patch and then send it out to engineering to get it fixed.

Did not have that on my CEO dance card, at least my initial one. So, yeah, it's fantastic.

So the team's growing pretty quickly. I think it's more or less doubled since we joined.

And yeah, it's a lot of fun. It's a really interesting time. There's so much happening in this space.

Brian: Yeah, I mean, that's an understatement because, I was just doing a talk in Nigeria last week which sounds like a weird flex, but, yeah, it was a lot of work.

But someone was asking, "hey, I'm actually kind of new to AI."

I was like, "well, good news. We all are. Everything's brand new."

We didn't have MCP until last fall. We barely had agents until maybe this spring. So we're in this weird renaissance right now for development.

So there's a lot of questions around where AI can unlock and unblock things like CEOs trying to unblock customer calls, but then also, where does this fit with like established engineers who are like, "hey, I've been actually like handwriting my Python for years. Like, don't take my Moonlander away from me. Like I'm going to be, I will be like Vimming or Emacs my way into like solving problems. So, AI all my beer."

Chad: Yeah, the way I describe it is: I like typing curly braces. Don't make me stop typing curly braces.

I was actually a biologist before I became a computer scientist.

So when I came in it was partly because I just had more fun in this particular space.

One of the first things I ever did was I got a job where the first thing that you did was make sure your dev environment was up and the way that you would do that was you would basically flash an embedded board.

And the test was literally making a red light blink. And I remember making the LED blink and thinking about the Robin Williams, like infinite cosmic power-- the genie sort of moment.

So, yeah, I wouldn't want to give that up either.

John: Yeah.

I was actually just having a conversation with a coworker who has a degree in computer engineering, which is a lot more chip focused or kind of the intersection of computer science and maybe electrical engineering.

But he had taken all these classes on like PCB design and all this stuff. I was telling him, I wouldn't even know where to start, like designing a chip or building a PCB or like building a piece of hardware.

So I would have failed the blinking red light test. But it sounds fascinating today.

I mean, I'm very curious what Continue has been kind of moving towards because I think when I initially looked at it, there was a bunch of stuff and integrations with Ollama, the VS code plugin and stuff.

But I've now seen some of the stuff you've also been talking about with continuous AI and now, uh, the continuous CLI.

Give me the elevator pitch of where continuous headed.

Chad: Yeah. So Continue open. So open source, but also open in terms of an ecosystem.

So you bring the model that you want or use the models that you want. But I think more than that, it's what problem are you trying to solve?

So the CLI came out this week. This is a big week for us. You'll see lots of stuff that we'll talk about.

But we've got the IDE that most people would expect. And that comes in VS Code or JetBrains. Like, what do you want to do? Like, you know, I don't want to.

I know a lot of IntelliJ users who say you'll pry the IntelliJ from their cold dead fingers.

And so what are you trying to solve? Where do you want to work? What models do you want to use? And then I think the other thing is it's not just enough to have an IDE, it's not just enough to have this in the generation space. You actually want these tools to help you throughout the entire lifecycle.

And so that might be in CI, it might be in code review, it might be in build and release.

And so I think partly what we've done is really push to get the CLI out so that you can start to have workflows that go across, not just codegen but sort of the downstream components of software.

Because I think if CodeGen makes software go so much faster, but the downstream, you're testing your builds, your code review, if that can't scale to meet that, then you won't get any value out of any of it.

And so I think this is where the whole space will have to go because eventually, and this is where I think we like the continuous AI moniker because it harkens back to original CI, which is like you need build to go.

You know, you start with CI then you maybe get to CD, maybe you don't.

But it's an iteration sort of process where-- I wrote a blog post that the title is, Intervention Rate is the New Build Time.

So if you think about like when we started with CI, build time was the metric that you used to make sure that you were always getting better. Like, are our builds getting better?

And then you might say, look, oh, actually this build is huge. So actually what I need to do is I need to split this build and I'll take the part out of it and I'll make it two things and then I'll have a pipeline and then this one will run all the time and this one will run less frequently.

Or you just have an entire choice or I'm going to split it across platforms or whatever because you're trying to get as much feedback as quickly as possible.

And I think we're going to use the same thing, the same sort of approach, which is, how often did I have to touch this workflow?

How often did I have to intervene?

How often did I have to actually look at the output of the test logs and go figure out what was wrong versus an automated process going in and telling me what was wrong and then I take it to the next step.

And then eventually I'll build enough trust that I'll just trust that it can do the next step and then I'll intervene there and then I'll get to the point where I trust that next step and then the whole thing will be something that can be automated and I think that's kind of the direction that we're headed.

Brian: Yeah, I've been working on, and John you've had eyes to this like Python thing that I've been building, this like data pipe and I just recently because I had to go to Nigeria and I knew I was going to have some pretty bad Internet so I had this service essentially collect GitHub data and then generate some insights around that and then deploy it to a front end experience and needed to do some, like--

One of our friends at Continue is actually dltHub so I wanted to do some like DLT pipeline that would use to GitHub archive and do like some pretty cool things.

And I knew I couldn't run this on my machine on the airplane and I couldn't run it on from the conference on the conference wifi.

So I deployed it to fly in like a fly box and or uh, what do they call these like dev dev boxes, like droplets on digitalocean.

John: Oh like a workstation, a cloud kind of thing?

Brian: Yeah, yeah. So I deployed it there but also I'm like I'm not actually, I'm not a DevOps professional.

Like I've been in rooms where things seem to get fixed or like things were planned so I kind of like know how things are working.

But I would never be full time like managing a server or anything like that.

But I had enough wherewithal of like continuous AI where if something goes down open issue, it at-mentioned me.

So like I'm not gonna-- No century tracking, no Grafana dashboard, but like at least I know something breaks.

So for me it's like okay, if I can get that opened, what if I just label it as like highest priority, P0, and like that's another thing, with a GitHub action, yes you could also automate every issue open. If it's this, label it.

But the beauty of like having an agent to like do that first step and that first touch then you can now I'm like, at the point of like, oh, what if an agent did go ahead and triage with that first?

So it's opened, something happened, and before I even wake up, I already have a description of like, hey, went through on like a run loop or whatever, within an environment and test it out and here's the mitigation to like fix this.

So at that point, like, I haven't done this yet, but like, hey, it's opened up the comment.

Could it also open up the PR based on like the response of what it's first touch?

So then when I have the PR open, we've already had the concept of like PR review with agents.

What if the next agent basically reviews that PR before I wake up? So like, it's already opened the issue labeled it has a comment on best take on how to fix it.

PR is open, PR reviewed, it gets reviewed, it passes the test, new tests are written, documentation's written.

Now I'm like, okay, human in the loop. Let me just catch up to what I missed and I'll hit the merge button.

So that's the pipeline I'm currently working on what my continuous AI pipeline.

I think that these small, little quick wins tend to turn into like, how do I hand this over to the next thing to validate?

So like, when I do have to wake up at like 7am or get that page, by the time I roll out of bed and get to the workstation and like turn on the lights and like get the coffee going, there's some extra things that have already been done.

Chad: That feels really familiar because that's how like-- back in the old days, I rolled out Puppet to two different organizations.

And when we would lay out the automation to do the infrastructure, we basically said, what sucks the worst?

What's the thing that we spend the most time on or that drags us down the worst? If we fix one thing right now, if we could automate one thing and get it out of the way, what would have the most impact?

And actually, so I went into sales engineering for a brief period and I was actually at Puppet.

And my very first sales call, I was talking to this really gray beard, grumpy sysadmin and he definitely didn't want anything to do with Puppet.

I remember asking him what sucks the worst? And my sales guy's eyes went big.

He's like, "you know, we don't talk about what sucks in sales. You need to make fond memories of us with them. Like, don't ask him what sucks."

And it's like, well, this dude doesn't want to be here, so let's, let's just meet his energy.

And he's like, "you know what sucks? Ssh. Keys suck. I have to constantly update them and developers always change them and I have to do this."

It's like, well, would you like to solve that with Puppet? And we sat down and wrote a Puppet module.

It took about 15 minutes. He pushed it out to a bunch of servers and then he was like, "oh."

I was like, "Yeah, now you don't have to do that anymore."

Well, that's great. And I think that's the same thing that's going to happen here. It's like, what is the workflow that drags you down the most?

And it could just be because you do it a lot. Isn't there an xkcd?

There's probably an xkcd for everything, but there's one about automation and like if it takes you a minute to automate it and you do it a million times is what it saves you.

But if it takes you a year to automate it and you only do it twice in your life, it's not. I think, you know, we should think about it that same way.

Brian: Yeah. John I'm curious to hear your thoughts because it's just a bit of your world as well.

And I'm curious like, if you have any follow up questions about continuous AI, but also like how, how are you using AI today for your day job?

John: Yeah, I mean it's a good question.

I do kind of ride the waves of just the emotional roller coaster of how I feel about AI day to day as I feel like maybe a lot of professionals in the industry do.

It's funny having now had years of experience in the industry and starting to see trends come and go and like things come and go and then getting swept up in one of them and feeling like at times like I don't know how I feel about this and I wonder if people with Web3 or even like the dot com bubble were like, you know, feeling similar things.

Maybe, probably. But not a wax poetic take is that, yeah, I think the, the obvious direction that some of this goes is like the software continuing to eat itself.

Like that's sort of been the way software has always been is that we've always found ways to make higher level things.

At some point some of these things, yeah, definitely break down and it's like I don't know if anybody's going and building the assembly to make the light blink.

You'd probably write a little bit of C or C++ or even Rust today or Zig or something. Keep going higher level up and up and up.

And I don't know if I would use AI to go and do something like that but. Or at least have like there's probably an agent you could build to like go and blink the light.

But it makes sense that if you have these big, high level tasks that are like oh my service is down. Like can you start some investigation?

Like you have it linked up to your log service and your metric service and a bunch of your like other cloud services and can just start looking for you like while you get the coffee going.

Like you were saying, Brian, like I think that like high level task kind of putting it in the context of like DevOps and continuous integration, continuous development, continuous AI, it starts to make a lot of sense.

I'm curious how you start to think about and solve for the problems of non determinism or you know, just how these things can break in weird and goofy and strange and bizarre and scary ways.

Like how do you start to tackle that if you start to kind of lift and shift a lot of this more into higher level dimensions.

Chad: Yeah. So you know, my favorite thing in the entire world is to write a bash script. A really good one.

Like something that just, you know, you've got some redirection in there, you got some pipes in there.

John: Put a little color in there sometimes.

Chad: Yeah, for sure. And I'll give you--it's not quite a great example, but it kind of goes in that direction.

I have a little tool that helps me write or actually draft the commit message for a Git commit.

And that's non deterministic but I can gain trust that it's increasingly better and generally I review it. And so it's a very short thing.

We use conventional commits. I want to do a conventional commit, look at the diff, you know, and basically I have, I know my way enough around the porcelain in Git that I can set this up and then I've staged it in a Bash so that I've got the commitment, get the diff, pass that to the tool, give it a little bit of A prompt for help me generate a good, short, concise, conventional commit and then pop it out and then review it.

And I think part of it is that the more that you ask it to do, and especially the more steps that that takes, then potentially you need to have some tooling around that to help with that. But also to some extent you just need some amount of trust. And I think the way that you do that is this incremental step where you check it.

And so like what Brian is saying, like, get it to make a PR. So one of the things I use Continue for is we have lots of partners that have models that update all the time and we have Hub where we update the blocks so that you can, it's more like just making it really easy to just consume a model from Anthropic or whoever, or Ollama.

And one of the things is some of them have those models on an API, which is fantastic. And we can make that fully deterministic and just write a Python script.

Some of them though, and I won't name names, but some of them won't and I've asked them to.

It's like, give me a list of your models and put it on an API. And so instead what I do is I actually ask, instead of writing a scraper that breaks, I ask the tool to go figure it out.

Like go find this list from this library page that they've got on their website, get the models that I don't have, and then make a PR and push it with the new models that I don't have and I'll look at it.

And then the thing is like it's now running and it runs in the background.

It runs as a GitHub action and then it creates the PR and then over time I will gain trust with that. But it's also, it's very scoped, right?

Like you're going to get some HTML over here. You have some amount of knowledge of what this should look like because we have examples that exist.

And I want you to find the things in this table that don't exist and then make them look like these other ones.

And so I used to do this with like a web scraper Python script, but it would break because they would change the webpage all the time.

And so it was like, okay, well, and this one, it actually does a pretty good job. Potentially the web page is changing, but I haven't noticed yet because the machine seems to be working.

So I think there's places where it works. But for my other partners where I have, I can hit an endpoint and get a JSON blob, I'll have the tool help me write a Python script and that'll be deterministic.

John: Yeah, it's interesting.

I mean it really is a first principles thing where like breaking problems down into small consumable pieces is always going to be easier for AI for a CICD system for programming.

I mean it, it sort of just starts to make sense that way.

Chad: I mean what sucks? Is it the build? Like what's taking your time? You have hammers and nails and screws and screwdrivers, like go hit them with something.

And I think sometimes, you know, you just like-- I like writing software.

I still write lots of things that I never publish that scratch an itch for me and, and then eventually die.

And I maintain a GitHub action that generates a change log between two references and I now keep it up because like, I don't know, 150 people use it and it doesn't take me very much to keep it going and I just enjoy doing it.

But there's other things where it's like--

You know what, I don't super enjoy scraping websites, but I need that to get done and so like, let me go do that. And this is where I think there's so much tedium that this kind of tooling is built for.

You know, like you think about like at Cloudera we had this. So it was a big data shop, right? And we had all kinds of PhDs that were good at data processing.

So we would take these tar balls from customers where you would hit a support button and they would go pull a bunch of logs and zip it up and send it to us.

And then we would drop that into a place and then you know, through some automation it would rip it open, it would compare it against, basically a database of support tickets we had seen before, and do a whole bunch of machine learning things.

Well you can do that same thing now. And so then that initial tedium of if you're a CSE or something and you get an inbound from a customer and then something rips it open for you and says like, well here are the three most likely things.

Well that's automatically better than what it was before, like you having to go like grep inside of it and maybe those things are wrong, but at least like you have like that initial step and then you can build some trust and then you can go to the next thing and then the next thing .

And like, nobody. Nobody. Like, oh, maybe I don't say nobody. Some people probably really get into, like, grappling through error logs and like, aha, I found the trace that has given me the answer.

But I think for other folks it's just like, let me just answer this question for this person and move on.

John: Yeah, I've definitely worked with some of those people who loved finding smoking guns in like, gigabytes of logs and writing really crazy scripts and things to go through cloud watch and figure it out and stuff.

So my hat's off to them. I don't know if that's how I like to spend my time.

Chad: Yeah. And so then what do you want to do? I think this is the other part.

I think as an industry, by and large, we will look back at the massive hype that's occurring right now and say either that's silly or that that hindered adoption because we went so far to say it's so magic.

It is magic. The first time it actually worked for me, it's like, wow, that's great. That is, that is so cool. But it's also--

I'm not convinced that I won't, or that the three of us won't, have a job in six months. And I'm a CEO of a company that's building these things.

I think for us it's, how do we, you know--We all do stuff that we'd rather not do and that the machine could be more capable of doing for us. And so let's go get those things.

I think it's interesting because the startups talk about how many billions of lines of code they generate, but if you listen to how Microsoft and how Amazon and how other folks talk about it, it's, "hey, we migrated Java 8 to Java 11 really quickly."

I think those are the ones for big companies that, that's magic.

Like, oh, you have a code base with a thousand applications which are on Java 8, and you want to move them to something more modern and you want teams to be able to do that more quickly. That sounds awesome.

Brian: Yeah, I was going to say, the tedium is like really the thing that like, checks the box in my head, which is like, I know how to do enough things. I know how to like.

So, for example, I was a longtime MacVim user and never quite converted over to Neovim because I was like, ah, there's like so much muscle memory within my Janus setup from like 10 years ago that now I want to translate that into Neovim.

So I ended up using our CLI actually a couple weeks ago to just go to my Vim config and be like, I want to get to Neovim, look at my current config, what are missing pieces that I can use a Neovim plugin for?

And that would like take a bunch of googling or jump in the forums which is like, again, people do love doing that.

I don't like spending a lot of time with a thing that's just like my, my set of dot files and it got me pretty much 100% like, sorry, I'd say like 95% of the way there.

There was one plugin that I still need to fix that doesn't quite convert over to what I need.

So it will require some human in the loop to like debug and try to figure out what's missing in my setup.

But I mostly am using Neovim now because I was able to like get all the muscle memory out I've used for years from my normal Vim setup.

And like things like that are setting up my tmux configuration.

Like those are things that again I just don't have a lot of muscle memory on tmux.

But I can leverage AI and like drop in a blog post which I think, the higher level, like what we're seeing in the industry is like we had vibe coding earlier this year.

We have this like new thing which is context engineering. And like I'm dropping in context to then unblock a thing that I'm trying to accomplish.

And it might not be like, you know, scalable infrastructure and software because that's not my need, but it is like checking a box for my day to day.

Like can I be successful and useful at my job?

Chad: Yeah, you know, that sort of tees into this thing.

So at Continue, there's a value that's called "know what kind of tree you are." Which is to say that it takes an entire forest of different things to build a company. And I think as an industry we've kind of lost sight of this. There are different kinds of software people in the world.

Like there are systemsy people who really love going all the way down to the metal and basically living in the machine metal interface and that's great.

Then that's where you should be and go live there.

And then I think there's a bunch of folks who sit at the entrepreneurial or the product side where they love the prototype, they love that initial thing.

And so like the vibe code side of it really just resonates for them. And then there's an entire spectrum in between that.

And I think one of the things that gets lost in the hype that's going around is that developers are some monolith where we all enjoy the same sort of things.

And actually, I don't know, I've been around for a long time.

I actually kind of hate Full Stack because I used to be like--I was in infrastructure, I was an expert in the thing that I did.

If you think about any of these things, there used to be entire jobs for the people who were the infrastructure folks and the folks that, still is database but like, you know, and then we started smushing them together.

It's like really, do we like there's, there's entire careers worth of learning inside of any of these one things.

So I think one of the tricks is that like these tools should be monolithic in the way that you use them.

It's like, no. No developer is monolithic. Some developers love distributed systems and that's a very different developer than the developer that loves writing delightful user experiences, which is a very different developer than the people that like to do the systems level program, like operating systems or whatever.

So I wish that we would be more like as an industry more around what are you trying to do?

Oh, I'm trying to write software. I like it.

You know, maybe you do like writing. I tend to like writing the C style because that's where I came from.

So I play around with Zig a little bit now. I definitely write some Go. You know, my favorite language in the entire world to write is actually Ruby.

I really enjoyed writing Ruby back in the day. And so I probably where I'm going to spend my time now that might not be career wise.

Turns out that I don't have to worry about that part anymore, but that's where I want to spend my time.

So I think the way you use AI should flow from what you're trying to do.

Brian: Cool, so I'm curious, what are you excited about? What's next in the continuous AI front?

Um, like this thing's moving pretty quickly. And I think we're seeing a lot of folks naming it differently and attaching continuous AI to their workflow.

So what's exciting coming up in the next quarter, year, 10 years?

Chad: Everybody likes a five minute example.

So I think right now it's early enough that there are a handful of examples that exist in the world where people are thinking about it.

But we have a hub. I came from Docker, so Docker Hub is in my head of like there should be a wealth of examples.

And so like Dockerfile, you should just publish them and then people can go look at them.

And then like, oh, this is pretty close, but I would do do these two things differently. Right.

And so then, great, we'll take that Dockerfile, make it your own, take out the two lines you don't like, add the five lines you do like, and then publish it again.

So I am hoping that actually-- Brian, I was thinking about when GitHub Actions first came out and there wasn't very many actions and then suddenly there was a lot of actions and then maybe too many actions and then that contracted back down as, as like the best practices and people figured out what to do and how to do it.

And I'm looking forward to that because I think you're going to get some really creative uses. And then also I just like getting work done.

And so if there's going to be a handful of these things that other people write that I can take advantage of and just integrate them into my workflow, well then read the prompt first.

Don't just curl pipe shell. But I think that's going to be great because you're going to see this explosion of ideas and workflows and also potentially this early stage where the best practices the "How do you take care of day two?" I think that stuff is all still really new and it's really exciting.

Brian: Cool. Well, what's exciting is actually we have quite a bit of reads.

So Chad, I have to ask you the question, are you ready to read?

Chad: Sure.

Brian: Excellent. So John you have quite a plethora of reads. But I did have one which is actually pretty quick.

So I'm going to sneak this one in first, which is our friend of the show, former guest Mitchell Hashimoto opened up a PR to the Ghostty repo saying that AI attribution required.

Uh, and I guess this is coming out of the, you know, there's a lot of open source happening.

A lot of folks are using AI to do their open source contributions.

I actually just found a bug in Vite, so the front end build bundle tool for React and front end libraries.

And I was very tempted to use AI to go solve it, but I was like ah, I'm going to go look into this manually and when I have time, but I probably will chat with an AI to unblock my discovery.

But now you have to attribute that you leverage AI to either generate the code or the commit messages or the PR description.

So I'm curious both of you, what are your thoughts on that?

Do we see this as a trend that we're going to now need to have a link of like CN or continue or Clause Code generated this PR moving forward?

Chad: Yeah, I think you will. I think the interesting thing that I--

So I've chatted to a couple people about it and I've seen strong reactions on both sides and I think the interesting one is for the folks that have a strong reaction where they don't want to attribute, I think it's interesting. Why don't you want to attribute?

And I think, not to call out any motivations, but like I think it's interesting that there are so many people who are like oh, I don't want people to know that I used a tool to do this.

You know, maybe that's why people want, you know, there's some accountability to like, you know, what was--

The Curl guys did a talk at a conference just the other day where they talked about just how many--it's almost to the point where they're almost ready to shut down their bounty program because it's just so much.

And I think even before this we talked about how burned out maintainers were just on the number of human tickets that came through the door.

I think one of the things that also can help for this is so many people want to be part of open communities and want to get in. They just don't know how.

And so it's easy to think like, well like the way that you do that is through sheer contribution.

And so I'll just use a tool and I'll be able to like work my way in versus like there's other ways to do it.

So I would definitely think, you know, and I also participate in the Ghostty community, I think they do a great job of helping people understand how to participate and it's a welcoming community.

So I don't think that that's a challenge, but I think that that's one thing that you're going to start to see a lot of.

John: Yeah, I agree with all that. I mean I sort of land on three different things.

The first is just the licensing stuff. Like I think every open source project that's permissively licensed probably needs to have a disclosure process in the contributions because frankly like code that is generated off of stronger copy left stuff like off of AGPL things or whatever could very well and very easily land inside your repo from AI generate.

And we just don't know. We just, I mean it's not been legally tested. I mean this is like such a new paradigm.

Like all this is just a sand trap around us and in a few years if and when it becomes a thing that like, oh sorry, we're going to like uphold these strong copy left licenses that were used to generate from these AI agents or coding assistants or Cursor or Claude or whatever, you now have a copy left product, you now have to go and open source it.

It's a big bummer and I understand why Mitchell would want to do that because a lot of the other terminals in the space--

Yeah, I'm thinking of the Windows terminal, I think like WezTerm maybe is GPL.

These things-- it's a trickier space for Ghostty I think versus maybe like front end where just like so many of those libraries are permissibly licensed.

The second thing I go to is exactly what you were saying Chad, about the Curl people.

Just the burden it puts onto maintainers to have to go through and like try to let alone like good contributions from like normal people.

Before the age of AI, that still was a big burden. There was a lot of burnt out developers trying to contribute free time into open source and yet still couldn't keep up.

And now that has just gotten worse and worse.

Which brings me to the third thing which is the incentive structures in open source were already broken and now are like ridiculously broken because we had people who were going unpaid with a lack of tools and a lack of resources and a lack of infrastructure, maintaining critical projects, critical resources and like not even code maintenance.

Like projects that require a lot of community support, interacting and interfacing with companies out in the wild.

Like I think about a lot of these things in the Linux ecosystem.

It's way less about code and way more about like hey Red Hat, we're not going to break this thing.

You know, hey Canonical, we're going to make sure that we don't break your thing too.

And those incentive structures from the maintenance side are just completely bonkers.

And then you look at it from the contribution side and I think the incentive structures for a long time were like yes, I want that badge of honor.

I want it, I want to get a code commit into the Linux kernel or I want to like, I don't know, have the cool little collaborator badge or contributor badge on GitHub or something or the green squares or something. And now all that is just meaningless.

Like it, it basically doesn't mean anything, to get a small chunk of code into a project.

Where those incentive structures really break down then for contributing code is just like putting all that burden onto the maintainers.

So I don't know there's a lot there and I have many feelings about it obviously but I think it makes sense to have this kind of disclosure and protect the project at the least.

Chad: Some of it though, you'll feel it.

I mean I've seen some really good bug reports in my life or some nice PRs where people have put a lot of thought into it but now with these tools like even, I mean I, I use, like I said, I have a tool that generates these things and I look at this and I look at some of the, the commits it makes and they're perfect. They're like yeah, picture perfect.

Like I don't know who wrote that, it wasn't me. But at least I have the understanding to say like yes, this is actually what this is doing.

And usually what I have to do is tell the thing to be quiet, or make that half as long.

So I think that there are places where like Git commit like these are places where I have less, I think the code gen side like that's the part where now yeah it gets a little more--

What are you trying to do? I also think that like what are we, what are we trying to do as a career?

If you want to get better at developing, if you want to become part of a community, well then do what Brian said he's going to do. Do it the hard way.

Have the tool tell you where the code base is, have it help you find the file that you need to do and that's great.

I think especially in a new code base, like use it as a pair programmer. Great. But then learn the code base and then learn how to do it yourself.

And then if you want to, you know, use it to get a code review before you actually put it public.

I mean I definitely think I would do that. Back in the old days, I was always terrified of having my code reviewed.

Like having a private code review in advance, like, oh, that's great, I'll fix that, then send it up, then do that.

But I think, you know, why do you want to contribute? You know, I think there's definitely, I see it a lot in our security inbounds where we get a bunch of people who clearly just had Claude look at the website and say like come up with something that's wrong and it's like we'll get these emails and you know, they're very clearly written by an AI and it's very clearly like you have a, you know, bug here.

Where's your HackerOne login so I can get my 50 bucks or whatever, right? That's not, that's not useful.

So I think, you know, you all remember this. Like, usually somebody would come out with like one of those things that maybe was a badge or something.

And then, you know, they've tried to get a community to go out to all the open source projects and get their badges in and like I worked with some of the dev environment folks and like they definitely had a challenge where sometimes our communities would go out and try to get our file format in somebody's repo when those people didn't use that particular dev environment tool.

And then people get grumpy like why are you trying to put this file in here?

It's like the same thing, like know your audience, like why are you trying to do what you're trying to do?

And if you want to get involved in an open source project, then go watch it a little bit, figure out where their--

Chances are, they probably have a "good for newbies" or whatever their particular label is or they have a Discord and, or everybody will be happy.

If you find issues in docs. Like I guarantee that, you know, docs patches are definitely going to be things that, that folks are, get excited about because that doesn't always.

Like there's lots of ways to do it. Why are you doing what you're doing?

Yeah, and don't just do it to get a badge do it because like you actually want to participate and so then go figure out how that group actually works.

John: Yeah, exactly. And I think in the end if there was some wide adoption of this, it would really probably look similar to a developer certificate of origin, which is pretty common in the Linux ecosystem anyways.

And it's really just saying that like yes, I am the developer who made this patch and I certify this to be distributed under this license.

Somehow every episode we do goes back to open source licenses.

Chad: This is going to be a DCO like 2.0 which is like I attest, you know, it's basically I attest that I have the right to commit this and that I either didn't or I am disclosing the way that I used a tool.

John: Yeah. Or just even the disclosure because again it does go to just how dangerous that could be for certain projects that--

I'll give you another good example, the proxy space.

Like some of these, like I think Envoy proxy is relatively permissively licensed but there's like tens of years of these projects that probably the code bases are like out on the interwebs, were like proprietary licensed.

I'm not sure if NGINX is one of those that's more strict, but I don't know.

It's one of those that it's like you start to touch things that maybe it was trained on and then all of a sudden some developer at like Oracle sees that you did that and they're like "that chunk of a hundred lines looks really close to this thing" or like flagged to this thing.

So just being able to disclose like hey, I had this fully generated by AI I didn't review it is probably just as well for some of these more like high touch projects .

Chad: Or for some of the high security ones. Right?

Like, I think actually what you'll see. So you'll see I think that there'll be disclosures.

I think you'll also see people who just outright reject large patches , because it'll be--

You know, if you think about a tool that can generate thousands of lines of code.

Like if you AI generated this and it's a thousand lines of code, I have absolutely no idea what's happening inside this.

So like you know, I'm going to want to see five line patches that I can really understand or.

John: Right.

Chad: I think it'll be interesting or it will become even more of a club where like you have to get in the club and only the people that are in the club--

Like I remember, Cloudera was an open source but we played within the Apache Foundation.

Like most of the open source projects we were in Apache.

And so I mean I think lots of people are used to like the sort of the open ended permissiveness of like GitHub versus the more rigid, very focused way that changes happen in a foundation like Apache where only certain people have commit bits and like you have to generate a bunch of support to get Apache--

Like will the whole industry move to this? Much more like not.

I mean it's not as quite as open. I mean you'll still be, you'll still be able to get through the front door but you know the bar to get a patch even just really visualized by somebody will be higher .

Brian: Yep. With that, John you've got some Reads. Do you want to run through those?

John: Yeah, some of them are maybe not as relevant or the order of interest as we go down.

So I'll go fast through some of these. But the most interesting one was this crazy, crazy, I would even call it maybe like breach or like ginormous vulnerability that happened in Code Rabbit.

What happened, I'll give kind of the TLDR , is some security researchers found a way to execute remote code on the CodeRabbit's runners which if you're not aware CodeRabbit is one of those like AI review tools that runs inside your PRs, public repos and your GitHub enterprise, really wherever.

But they were able to run a little bit of Ruby but because one of these tools that CodeRabbit could run had a Ruby, I don't even know what it was like a Linter or something that could execute codes.

Chad: It was like RuboCop, right? Or something like that.

John: Yeah, yeah, yeah, exactly. And it can run Ruby as part of a plugin system on that basically.

So they were able to dump like all these environment variables and like everything was in those environment variables from the system.

Like the exploit isn't actually that interesting because it's sort of like script Kitty 101.

Just be like, "oh, go get the environment variables if you're on a system to see what's in there."

But it was everything. It was like the tokens and PIMS for all the inference providers they use, a bunch of like database secrets and stuff.

But the really big one, the like gold mine was actually a private key to their GitHub app, which in theory gives anybody with that access to every single repository that's ever installed CodeRabbit onto it.

Because this is also maybe a problem. I'm curious Brian, your take on this because this is kind of a problem with how GitHub does apps, right?

Like if I just go and install an app and then that private key gets out there. I guess I'm just trusting the apps to like do the right thing?

And like this is definitely going to make me pay way more attention to what permission access I'm giving to apps that I install on my org and on my profile, because it had everything. It had write and read on everything on the repo.

Brian: Yeah, so I spent my tenure at GitHub and GraphQL is actually why I got hired and then very quickly got to support the app team with doing Devrel.

So I spent a lot of time doing that before we switched over to action.

So I have a pretty popular YouTube video about how to build GitHub apps that still gets comments and I'm like, guys, go talk to GitHub about this. I don't have any more answers.

John: Move on.

Brian: But yeah, it is basically the PIM is a secret and the trust is that one you just never leak that.

A lot of times tutorials would be like oh yeah, just commit this in your Git history. And a lot of people just have that exposed.

So for rinky dink side projects and tutorials, that's whatever.

But if it gets any sort of usage and they get access to that PIM and you happen to also have your permissions to be like, oh, write access to this, this and this.

So like with the review tool you want to write access to like comments or write access to like your secrets.

Because I think Claude's a good example where you install Claude's action from the command line installs the app so it has right access to secrets and issues and et cetera.

Claude's pretty locked down for their interaction. So like they're, they're good.

Hopefully on the front we'll knock on wood on that one.

But yeah, basically a lot of folks around 2018 were building individual GitHub users to basically say hey, this is a bot and this bot will do all this stuff and like this bot does all these things to like generate new repos or whatever.

And GitHub app was the answer that you didn't have to take up an entire seat. Back when GitHub was focused on seat based pricing.

You use GitHub apps to like not take a seat for a bot. Now it's all usage based pricing.

So now these bots are actually driving up your usage with all these action minutes and stuff like that.

But yeah, yeah, you do not want to expose your PIM. Yeah, TLDR: never do that.

Chad: Yeah, I mean I think--

One of the interesting things about this industry is so much of this industry is you learn on the job.

Like it shocks me at the amount of like, oh, I didn't know that was a thing.

Or I didn't know that, you know that you could get blown up that way because you know, I've definitely talked to folks about environment variables and like, yeah, we saw this movie the first time in the CI.

This was a way that you would go blow up CI. You'd leak variables in your CI builds.

The challenge is this is a pretty well known way to get blown up.

And there are pretty easy mitigations for this, which is one, when you spawn these kinds of workers, you don't need to have them inherit the environment.

And so this is pretty bad. And so we go back to my first thing: There were reasons that people specialized in stuff.

I think if you think about back in the day, there were folks who were really good at running apps and that was their job.

They ran them and then there was folks who wrote them.

And these, the skill sets were slightly different and you know, the folks who wrote them would, would do things that made it easier to write them and then the folks who ran them would do something else because it made it more secure to run it, but it made it harder to develop it.

And there were definitely challenges to that.

I think we learn through stories and through collective screw ups.

John: Yeah.

Chad: And so hopefully a bunch of people learned a lot about--like I definitely think that the Continue team got a large lesson in environment variables this week from me and some other folks to make sure that we understood that like this is a thing.

John: Yeah, it's a pretty wild thing. I mean, "things happen,"is the other thing to remember. So hug ops to the CodeRabbit team because that's rough.

Chad: I mean Brian, you, you probably heard the--

My favorite story in this line is the early GitHub folks talked about the time where they were at a conference and they were working and then suddenly GitHub disappeared.

It was just gone. And at some point they realized it was because they had run the tests and the connection string was still connected to prod. It was not connected to the local test.

And the first thing that tests do is they drop the database.

And so, you know, the original GitHub team had a moment where, yeah, they ran the test and dropped GitHub's database.

And, they were able to get it back and they were able to recover and, and clearly the company is still around.

But I think the way that you learn these things is through people sharing stories about the time that they blew something up.

John: Wow, that is wild.

Brian: It's a popular, popular piece of the GitHub lore, for sure.

Chad: Yeah.

John: And that's just like how some of those Ruby, because, you know, GitHub was originally written in Ruby, so that was just how he did it in Ruby, right? Was you had these migrations that ran and--

Chad: And who would, who would think they would do that?

Like, I guess I didn't even. I think the joke was I didn't even think it would be possible for a tool to be able to do that.

And somehow, you know, that connection string, which was likely in an environment variable which just hadn't gotten switched, blew it up. So, yeah, interesting day.

John: Crazy. Well, I'll do one more because we're getting close on time, but this one was something I stumbled on, on Hacker News.

It was on slash dot.org, which is kind of like one of these forms, and somebody was commenting that Steve Wozniak has like over $100 million or something, and he's like, you know, not living in the poor house or something.

Which is maybe one of these, like, kind of weird cultural things people think about Steve Wozniak, who is early at Apple and was pivotal and actually having the early Apple computers work.

And then Steve Jobs was there as more of the face of a lot of these things.

And the lore seems to be that, you know, he was maybe not so financially successful.

Anyway, so then he just appears in this forum to comment on that.

So Steve Wozniak said, I gave all my Apple wealth away because wealth and power are not what I live for. I have a lot of fun and happiness.

And it's kind of a short blurb, but the quote that I really liked was, "life to me was never about accomplishment, but about happiness."

And that for some reason was just like a huge breath of fresh air for me this week. I think I get rather exhausted by--

I don't know what you would call it, like, the egos or the sort of like, "we gotta win, like, all the time" in tech. Um, and just having some humility to say, like, no, I do some of these things.

I did it for fun and happiness and like, yeah, he's rich, but, like, at some point, maybe that doesn't, um, actually matter that much.

So. Steve Wozniak What a guy.

Brian: Yeah. I mean, it really puts things in perspective too, as well.

I've had some opportunity to talk to, you know, some millionaires, billionaires, in my my tenure, which it sounds like a weird flex, but it really isn't.

I really enjoy how down to earth people can be at that level.

And it's nice reading this about Steve Wozniak because he's always eluded that sort of presence. At least in the tenure I've known of him.

John: Yeah.

Brian: Yeah, because I wasn't around when Apple Computers was created, but I grew up with one, at least in the school I went to.

Chad: I also think-- What are you trying to do? Like, make cool things?

Like, I worked at Docker for seven years and part of that was because it got to have a tremendous impact on people.

And people come up and be like, I love Docker. It's like, "Great. I love working on it. Because you love it."

And so, I mean, I talk a lot to my team about how do you charge your batteries?

Like, what particular thing puts energy into you and what takes it out of you.

And you should spend more time putting energy in than out, because you know, if you've got things that put energy into you 3/4 of your time, well, then you can do that forever.

But if you're draining your battery 75% of the time, you can't do that that long.

And so, you know, what's going to put energy in? What's going to take it out of you?

John: Well said.

Brian: Cool. Well, I'd love to take us out of this podcast, because we've been talking for a bit.

So, Chad, thanks so much for coming on board. Chad Metcalf, everybody, who is CEO of Continue.

We appreciate you just having the conversation about continuous AI.

Folks, check it out and stay ready.