<div class="heading_h4" style="grid-column: span 2; margin:15px 0">Introduction</div>

Cam Saul

It’s honestly kinda exhausting with languages like JavaScript, keeping up with whatever the framework of the week is, right?

Vadym Kostiuk

What do you think are the most critical reasons that could lead a company to leave Clojure?

Cam Saul

Convincing everyone that we would want, that it would be a cool idea to rewrite three hundred thousand lines of JavaScript to ClojureScript is, like, a big task.

Artem Barmin

Yeah, it’s a common story about Clojure startups, when a very small team can deliver big results.

Cam Saul

I was able to kinda port a Python application to Clojure very quickly, I kinda sold the rest of the company on it. You know, it’s not just us looking at a code, it’s potentially anyone in the world looking at it, so little bit of extra pressure but I think that’s probably a good thing. I think that’s one of Clojure’s superpowers. We won't be able to hire people because no one's gonna wanna write all these parentheses.

Artem Barmin

Welcome to the tenth and final episode of our podcast, “Clojure in Product: Would You Do It Again?”

Vadym Kostiuk

And today we’re excited to welcome Cam Saul from San Francisco, the founding engineer and chief architect at Metabase, one of the most well-known Clojure-based open-source projects.

Artem Barmin

This platform offers a user-friendly interface with features such as query-builder, interactive dashboards and SQL editor, allowing user to explore data without extensive technical knowledge.

Vadym Kostiuk

Also, our previous guest, Alexander Johannes, who is a Head of development at JustOn, asked a question: “Did adopting Clojure require changes to your development process? Were there any unique challenges or advantages that let you or your team to change approach or methodology?”

Artem Barmin

So let’s begin, map over question and macro-expand them into real-life Clojure experience.

<div class="heading_h4" style="grid-column: span 2; margin:15px 0">Full text</div>

Vadym Kostiuk

Hello, Cam. Thanks for joining us on our 10th episode of “Clojure in Product: Would You Do It Again?”

Cam Saul

10th and last episode, right? Hey, how you guys doing?

Vadym Kostiuk 

Yeah, thanks for joining us. It's really pleasure. 

Cam Saul

Great to be here with you guys. I'm excited, you know. Happy to talk about Clojure, one of my top 10 favorite subjects.

Vadym Kostiuk 

That's really nice to hear. And we're excited to hear what you have to say about Clojure. It's going to be a nice conversation. And actually, to kick off, I wanted to ask you if you could maybe tell us a bit more about your background, like how you first encountered Clojure and what is it actually that attracted you to functional programming and Clojure specifically. It would be nice.

Cam Saul

I had been doing all sorts of other stuff. I got into Clojure maybe about 10, 12 years ago. I've been doing other stuff, mostly mobile apps actually. I met different startups. And then I quit the startup I was at to start my own startup. And I heard lots of things about Clojure. I think the things that really sold it to me, where Paul Graham's speaking on averages, which is not about Clojure specifically, but it's about Lisp-like languages, right? The most important take on point is just kind of like the power and how effective or how quickly you can kind of ship new features. And that really appealed to me as a person, the only technical person out of like a two person startup, right? So that Clojure was kind of like the hot new Lisp of the day.<br/><br/>So there's a lot of hype in Clojure. I gotta learn Clojure, gotta learn Clojure. So I kind of like picked it up, picked up some of the books. I think like it was like “Programing in Clojure”. I don't remember which ones were the ones at the time 12 years ago, but I kind of learned it and then used that for all of my backend stuff when I was doing my own startup. And I had a really positive, really good experience with it. Like it's definitely a different way of looking at things. Like, cause I was mostly writing the JECC and Java doing mobile apps. So it's definitely a little bit of a learning curve. But once you get used to it, I really was able to kind of crank out a lot of code really quickly. And that kind of sold me on it. I was like a Clojure evangelist after that point. <br/><br/>Fast forward a couple years, my startup didn't make it. Otherwise, I'd still be there. But I wouldn't got a real job. <br/><br/>And I did bounce around between different companies doing different things. I was working on another mobile app, doing the mobile app and also doing the back end in Python. I actually knew the CEO, founder of Metabase was writing a dashboard for us. We were all at the same startup incubator together. And then he was like, if I write one more analytics dashboard, that's it. I can't take it. So he wrote a tool to make analytics dashboards. And he convinced me to come join his company. And so the initial version of Metabase was in Python, and back 10 years ago, I that was before, I think Docker had just started existing, but nobody had heard of it, at least not in San Francisco, maybe if you go to Y Combinator, they heard of it, I don't know, but I didn't know what it was. So deploying a Python application 10 years ago was like a gigantic hassle, right? So I'm trying to tell, especially non-technical people, you gotta set up virtual env, you gotta do this, you gotta do that, was basically a non-starter, so.<br/><br/>We were really attracted to something that would be easy to deploy and that basically meant something on the JVM. Java dash jar, whatever dot jar is as simple as it can get, you know? So we looked at Scala, I'm like, I've used Clojure in the past and let's try doing this in Clojure. And I was able to kind of port the Python application to Clojure really quickly and that kind of sold the rest of the company on it. And we kind of never looked back, so.<br/><br/>And then I've been full-time Clojure ever since.

Vadym Kostiuk

You said that Metabase started initially with Python. 

Cam Saul

That's right.

Vadym Kostiuk

Were you already with the team when Metabase started or not?

Cam Saul

Yeah, so it's kind of like a interesting situation. Like I was with the team. Well, I sat next to the people on the team. We were all at the same startup incubator, but I was not like on the team at first. But so they kind of started working on that. And then I transferred over to that company pretty shortly after they started, you know, maybe a few weeks later. So I didn't write like the first lines of Python, but I did write the first lines of Clojure once we switch to Clojure.

Vadym Kostiuk 

What was the period of time that you actually were working on Metabase on Python and then you decided to switch to Clojure? Like how long have you been developing Metabase with Python?

Cam Saul

Yeah, it's all a blur because it was like so long ago. But I think it was like maybe six months or a year, somewhere in that range. It was in a ton of time. And we hadn't launched publicly yet. There was maybe 10 or 20 companies at the startup incubator. So we were sharing it with these other companies internally. And they were using it and kind of dogfooding it that way. But we hadn't launched publicly yet.<br/><br/>It was still very early and we only had like a very limited feature set. We switched over pretty quickly because we realized, you know, if we want to launch publicly, we're going to have to have an easier deployment story. That was like one of the things we found out actually by kind of sharing it with these other companies at our incubator is that we got to go help them figure out how to even spin it up. So we wanted something that'd be easier to deploy.

Artem Barmin

It's very interesting to hear about this criteria first time I hear about it about deployment. What other alternatives have you considered? You mentioned Scala, aside from Clojure, what else was on the table?

Cam Saul

Yeah, so I mean the other big alternative would have been just to write it in Java, right? But I had written Java previously, like I was saying, I did Android apps and those were all in Java back in the day. I don't think Kotlin even existed back then. Or if it did, we hadn't heard of it. So that was kind of the only alternative. But I had written Java and the other engineer at the company had written Java too and we both kind of looked at each other and said, you know, we would be happy to never write Java ever again. Yeah, so that kind of narrowed it down to Scala was cool, Clojure was super cool. So we want to pick one of these two, right? Yeah.

Artem Barmin

And I'm curious, how did you tell other part of the team about your decision of the Clojure? Because they were Python engineers, as I understand. Some of them had experience with Java and was they happy? Were they happy about this?

Cam Saul

Yeah, so I mean, I don't know if it was so much me telling people as me making a case for it and everyone was like kind of sold on it. The biggest thing was once we started looking at porting it, I was like, I could go, I could port this to Clojure in a week. And I actually kind of got pretty close, I think. I sat down and cranked out a mountain of Clojure code that replicated 90% of our app. I think that really sold people right there. Just how quickly, yeah, just how quickly we can get stuff done. I think that's really one of Clojure's superpowers – with someone experienced with the language, you could really crank out a lot of code really quickly. It's really a fast development flow. I think you hear a lot about, immutability, macros, blah, blah, blah, but it's really, I think, kind of that REPL-based development, that fast feedback loop, you you're just sitting there, you could really kind of plug into the matrix, so to speak, and just crank out a ton of code really quickly. So that was what really sold people, you know, like with Scala, I think after a week, I don't know, we'd have like one REST API endpoint up or something, you know, we'd have our type hierarchy figured out. But with Clojure, you know, it was… We had code, we were like, wow, this is it. could like switch right now. So I think that sold everybody.

Artem Barmin

Yeah, that's really impressive. Several years of development in Python, but rewritten in Clojure in one week with one person.

Cam Saul

Yeah, like, I don't know if it's several years, maybe like half a year to a year. But yeah, I think of course you have that second system effects, right? Like you already know what you're writing, you don't have to figure it out again. So I wouldn't say if we had started in Clojure in the first place, we would have only taken a week to replicate the same thing. But just kind of porting code that does the same thing was pretty quick.

Artem Barmin

That's not the first time when I hear the story about porting existing code base and choosing by creating prototypes. So we had another story when they created prototypes in several languages and Clojure was a perfect winner in this challenge.

Cam Saul

I guess that's a good way to sell it. If you try a bunch of different languages in Clojure, gets you the furthest quickest or something, right? Yeah.

Vadym Kostiuk

That's interesting. And in those early days, you prepared the initial version of the Clojure-based product for Metabase. And the team was impressed, I guess. I mean, this is a delivery in such a short term. But have they maybe told you about any concerns or maybe challenges that they were expecting because of Clojure? 

Cam Saul

Yeah, so absolutely the biggest concern of Clojure at the time was that it would be hard to hire people to do Clojure. Not necessarily because it would be hard to hire people who know Clojure because we weren't too worried about that. thought, well, people don't really know Clojure, but smart people can pick up new stuff, right? So we weren't worried about that. It'd be that people would not want to do Clojure.<br/><br/>There's this big concern that if we switch to Clojure, we won't be able to hire people because no one's gonna wanna write all these parentheses and stuff like that. So this was kind of when Clojure was relatively new and whether this had been decided one way or another was not really, that was still kind of open question. So the jury was out, so to speak. So it was kind of like a leap of faith on our part. We're gonna jump in and do this enClojure and hope that if we want to hire 30, 40, 50 people down the road that we'll be able to do it. <br/><br/>We didn't really have too many other concerns of that because I kind of like watched the other people at the company through how to do it and they kind of picked it up without too much drama. It looks kind of intimidating if you come from an Algol syntax language like something from the C family, JavaScript, Python, whatever. Something like where you do x plus y as opposed to plus x y, right? It looks kind of intimidating with all those parentheses, I think, especially if you have someone who knows, who's written a little bit, you can pick it up without too much trouble. So I don't think people were worried about learning it at our company, but we were worried about would people want to learn it when we hired.

Artem Barmin

You mentioned the challenges that you anticipated, but what was the actual problems or drawbacks that you faced during this Clojure journey for the 10 years? Was the hiring the problem or maybe something else arrived that you haven't anticipated?

Cam Saul

Yeah, so hiring I think actually turned out to be a big benefit of Clojure. We've had so many people apply and want to work at a company because of Clojure. They kind of, I don't know, drank the same Kool-Aid I did. And they're like, I would really love to work in Clojure. Or I heard Clojure's cool. I want to find a job in Clojure. I played around with it. So at the time, like I actually had done the same thing before I ended up at that company.<br/><br/>I wanted to find a job in Clojure and I couldn't, so I settled for another mobile job and a job where I did back into Python and iOS apps and Objective-C and Android apps and Java because I couldn't find a Clojure job. So we've actually hired a ton of people just because they wanted to do Clojure. <br/><br/>And Clojure, think, is actually a little bit self-selecting. People don't go to coding boot camps to learn Clojure. Generally, you learn Python, you learn JavaScript. So you kind of have to go seek out Clojure. It doesn't find you the same way. You're not going to fail a class because you don't know Clojure generally. I think I've talked to maybe one person who did Clojure in college or university. So people who are kind of curious and go seek out Clojure I think are the kind of people we like to hire. So we've had good success. We've hired a bunch of people who didn't know Clojure, but we're curious about it want to learn it. And they picked it up pretty quickly on the job. <br/><br/>So that hasn't been a big thing at the end of the day. I think just, you know, one of the challenges has been we have a lot of third party database connections. So we have Databricks, we have all these people, like we don't develop the connection with Metabase itself, but they are working on the code. you know, so trying to get these people who write, like, I don't know, they work on their, maybe they work on the JDBC driver for whatever database and they do that in Java all day and then they have to go write a little bit of Clojure code, but it's like 1 % of what they do. So that's kind of been tricky. If we were in Java or Python or something, I think it'd be a little bit easier for third party people to kind of come in and contribute little bits of code. So it's not been hard for people at our company, but harder for external contributors, I guess.

Artem Barmin

But you still have external contributors as an open source project.

Cam Saul

Yeah, we do. Yeah, so as an open source project as a whole, I think that's probably been a benefit that it's in Clojure. I think if we were in Python, we get all sorts of random people submitting pull requests that would fix one bug and then break five other things. But it fixes my bug, right? And that's always kind of you got to say to people, I appreciate you taking all the time to do this, but we can't merge this because it's gonna mess everything else up, right? So we don't have to deal with that as much in Clojure. Some people come try to write Python code in Clojure or whatever, but not as many as if we had a Python project, I think. Yeah.

Artem Barmin

And it's interesting, have you converted some of external contributors in your team members? Because it's basically hiring pool for you. You can talk to these people.

Cam Saul

Yeah, we've actually hired, I'd say, four or five people that way. Yeah, we've actually kind of gotten big enough now that people have been using Metabase at their job, and then they need some kind of fix for some kind of issue they have. And then one of the engineers at their company says, I'll submit a pull request. They kind of, it's usually people who are kind of like, heard about Clojure, and they think it's neat, and they want to like, play around with it, and here's an excuse to play around with it.<br/><br/>We've hired a few people who have submitted us pull requests to fix some bugs. Actually, sometimes we'll get really impressive fixes like, wow, I don't know how you managed to figure out how to fix that. I've been here 10 years, I wouldn't wish that upon a new hire, let alone someone who doesn't even work here. That's kind of touching the deep cobwebs of the code base. And then after that, we say, we reach out. We're like, wow, if you could figure that out, we want you to come work here. And we've hired a few people that way.<br/><br/>I think that's one of the cool things. I haven't worked anywhere else on an open source project, but I'd imagine it's probably the same. Yeah, it's good way to kind of find people.

Vadym Kostiuk

Yeah, and the learning curve, the onboarding is minimal. Basically, the person already is aware about the code base, about the specifics, and just can start right away. It's awesome.

Cam Saul

Yeah, working on an open source code base is definitely interesting and fun, I think, and kind of makes you just kind of more aware of everything you're doing, because there's all these eyes looking at it. When we first announced on Hacker News, people were like, I'm going to go read your code base. And I'm like, wow, that's a lot of pressure. So that kind of motivates us to write things clean manner or document stuff. Make sure, you know, it's not just us looking at our code, you know, it's potentially anyone in the world looking at it, so a little bit of extra pressure, but I think that's probably a good thing.

Vadym Kostiuk

As a founding principal engineer, you're now a chief architect at the Metabase. Can you please tell us what are your responsibilities as the main architect of the company, of the project? And how involved are you personally actually in the coding process daily?

Cam Saul

Yeah, so we're still relatively small. We have, I think, about 80 people now. So I still wear a bunch of different hats. I kind of jump around, do whatever I'm needed. I spend a lot of time reviewing. If we're building out some new system, and one of the other engineers writes a technical design document or whatever, like an architecture document, I spend a lot of time reviewing stuff like that, reviewing pull requests, trying to keep everyone on the same page and stuff like that.<br/><br/>But I still jump in and write a lot of code. I especially write custom linters with condo and stuff like that to try to enforce certain rules about the code base. An example would be we want to, our code base, think, is 300-350,000 lines of Clojure if you count the tests these days. So we're trying to make sure we keep it modular and not have kind of dependencies between the internals of this part of the code and this other part. So I was working on a custom Condo linter, for example, that would say, don't use this namespace from this part of the code base outside of it. Use the API namespace of it. So I spend a lot of time on stuff like that, working on linter to kind of enforce things that you would normally point out in a pull request review but computers can do that for you, And with Clojure, it's especially super easy to use something like Condo to write your own custom linter and kind of enforce your own rules about your code base. So I kind of do different things. I'll do that, review architecture docs, write architecture docs myself. I think earlier in the company's history, I was doing all sorts of stuff. I was doing, I set up Zendesk and was doing… answering customer questions and stuff like that, but now we have a team that does that. So I think as our company grows, my list of different things I'm doing, I think is shrinking and that's a good thing. So I could focus more on kind of engineering related stuff now.

Vadym Kostiuk

Yeah, definitely, definitely. And out of those 80 people that you mentioned you have in Metabase, how many engineers, like how big is the actually engineering department in the team?

Cam Saul

Yeah, so I think we're 40 engineers now. Don't quote me on that. But I think that's like the rough number. And then 20 of them aren't Clojure. Then we also are front-end. I should have mentioned this, but our front-end is mostly plain JavaScript or TypeScript. So we have about 20 JavaScript people, 20 Clojure people.

Artem Barmin

Why don't you use ClojureScript?

Cam Saul

Yeah, that's a great question. I think the main reason was I didn't know how to make it work back when we did switch to Clojure. when we were, the original version of Python, our frontend was in JavaScript back in the day. So I guess it was Python plus JavaScript. And we were using Angular back then instead of React and still using jQuery and stuff like that. So it was a little bit different. But I think the quickest path to switching over to Clojure was just switching the backend stuff to Clojure. And if I could wave a magic wand or use a time machine, I think it would have been probably better to do all the front end stuff in Clojure script just so you could have just work in one language on both sides of your application. But we hired a bunch of people to do JavaScript. And I think at this point, it may be too much to port. We also have 300,000 lines of code JavaScript.<br/><br/>I don't think we want to port that all to ClojureScript, but we've been doing little bits. We've been doing more and more ClojureScript. So, yeah, so now we're kind of a mix. You we got some plain JavaScript, mostly the UI stuff, React, and then we have ClojureScript for kind of stuff that's shared between our front end and our back end. Yeah.

Artem Barmin

Are you trying to convince the JavaScript team to learn ClojureScript or are you trying to attract backend Clojure developers to do some frontend stuff?

Cam Saul

Yeah, I don't know if I'm trying to convince anyone, honestly. We have a few of the JavaScript people who really want to learn ClojureScript. We have some times we're doing some new feature. And we're like, it'd be easier to build out this API if we build this part in Clojure C files that compile to both JVM Clojure and ClojureScript. And then we could kind of share this code between frontend and backend. And everyone's like, yeah, we don't have to write this code twice. We don't need to know about the different options for this thing or that thing. We could just write it once and then have it available in both places. <br/><br/>So I think it kind of sells itself in that way. No one likes to write code twice usually, so if you could do it once. I think convincing everyone that we would want that it would be a cool idea to rewrite 300,000 lines of JavaScript and ClojureScript is a big task. It's not on the top of my to-do list, honestly. If everyone else wakes up tomorrow and wants to do it, then I'm not gonna say no. What we have right now actually works pretty well. We're just running with it.

Artem Barmin 

And do you have cases when engineers decided to switch from JavaScript to Clojure and come to you and say, I want to learn and I want to write something in Clojure, some kind of internal migration of developers to the new technology.

Cam Saul

Yeah, we don't have any kind of rule that says, well, we hired you as a JavaScript developer. You can't write any Clojure or vice versa, right? So we've hired a couple people who have done Clojure and JavaScript, actually. And they kind of jump around and do whatever needs done. So some people jump around. I'd probably spend maybe a couple percents of my time writing JavaScript, mostly just like signature, TypeScript signatures for stuff, so other people can kind of go write code that uses it. I write a little bit, you know. When I started, I was doing almost half of my time writing frontend stuff in Angular. <br/><br/>But it's really, I think it's one of those things where it's easier for people to kind of focus on one area. If we have 300,000 lines of Clojure, 300,000 lines of JavaScript, it be very hard for someone to know 600,000 lines of two different languages, right? It's actually kind of hard just to know 300,000 lines of one, right? So it's easier for people to kind of be a specialist in one kind of one area of the code base as opposed to jack of all trades, master of none, right? So usually people just focus on one area. But if they're comfortable jumping around, more power to them.

Artem Barmin

Now I want to talk about the flexibility of Clojure and flexibility is usually treated as a good property, but in some contexts it can be very painful. from what I've seen on Clojure projects, sometimes I very strange over-engineered decisions that need to be refactored. For example, I've seen very big domain specific language written in macros and the actual program was written in this DSL, not in Clojure itself. And I'm curious to hear your experience because you started from the days when Clojure development practices was not so developed as it now. And maybe you faced it some kind of situations like this.

Cam Saul

Sure, so I think things like macro usage are definitely a matter of taste, right? You can definitely overdo it, but you could overdo something in any language. in C++, the language has so many features that a lot of people are tempted to use every single feature when maybe they're not really adding anything to your code base. <br/><br/>We strike a nice balance. There are some cases when a macro would be very beneficial, it would really kind of abstract out kind of repeating code patterns and really make the code. You're looking at what's different about this example of doing something as opposed to some other example, as opposed to just a bunch of boilerplate. So I think a really good example would be when I used to write Java, and this used to be when people used Eclipse, right? And you could right click and say generate getters and setters, right? You do this in like every class, right? So that's just like repetitive boilerplate, that's kind of like a good use case for a macro enClojure. You would say like have a class but with automatic getters and setters or something, know, generate them automatically. <br/><br/>So you certainly can write crazy macros that do crazy things, but you have to balance that against people who are new to your code base having any idea what's going on. So there's a such thing as too much magic. And we like to lean towards having it be kind of obvious what's going on.<br/><br/>Someone who's relatively unfamiliar with the code base or someone new. It's maybe someone who already knows Clojure but doesn't know our code base should be able to kind of jump in and figure out what the heck is going on, right? So I've heard of people working at Clojure companies and they've joined us and were like, yeah, my company completely banned macros, no macros because they were so worried that if you write one macro, your code base becomes unreadable, right?<br/><br/>So it's one of those things, just because it's there doesn't mean you should use it, but tasteful usages of it in certain places really can make a big difference. So a little tiny fraction of our code base is macros, but we use them in very tasteful places that I think really kind of abstract out a lot of boilerplate code. <br/><br/>We try pretty hard not to do anything too crazy. Sometimes maybe you could come up with something crazy that would save you one line of code, but nobody will know what's going on besides you. We try not to do stuff like that. We try to adhere to the Clojure community style guide. It’s been a big resource that we refer to a lot. If things don't need to be macros, if there's no good use case for making it a macro, if it can be a function that takes another function, we usually go towards that route.

Artem Barmin

Maybe you can still remember some cases when you need to do refactoring overengineered parts of a system.

Cam Saul

We try to lean towards leaving the door open to engineer something more, but not over-engineer it. So I think one of those big challenges of working on a startup product is sometimes you just got to get something out there and see if it's even what you want to build in the first place. Get the MVP, the minimum viable product, or whatever it stands for. So we try to figure out. We want to build, we think we want to build something like this, but how can we build one little chunk of it and kind of get some feedback and then that will set us up to potentially build the next chunk or the next chunk. <br/><br/>So sometimes we have some stuff that would be potentially over-engineered or kind of written with step five in mind when we still haven't gotten to step one. So I think that's one of those engineering team culture things where we've all gotten pretty good about When we're doing pull request reviews Saying let's take a step back and just see like we don't need to figure out all this stuff right now. You know, let's just build something that'll let us try out v1 and you know The door still open to make it much more complicated in the future. <br/><br/>So I think we I think it's If you avoid over engineering stuff in the first place, know, of course, that's the easiest way not to over engineer and pull request reviews are a good place to do that stuff. We've been really good about, especially lately, writing out technical designs and then getting the whole team together to kind of review these designs and kind of, someone will point out, you know, we can simplify it if we do something like this or something like that. shipping just like the simplest thing that would work. You build stuff, you realize that you did something in a way that was more complicated than it needed to be. <br/><br/>We were using Core Async to do event dispatch. If you change a password, then we send a user change password event, and then some other handler somewhere else would record that in our audit log. Having this whole Core async event system to kind of do little tiny things like that actually just made it way more complicated than we needed it and actually made it way harder to test too because you want to test whether we recorded a password change event in the audit log. Now you've to write some code that will kind of wait for an async event to happen. Just writing the tests for it was harder. <br/><br/>We changed that to happen synchronously like a synchronous event system a while back. I guess that's like a little bit less over-engineered and made everything easier to test so yeah, so I don't know you kind of go in you look at stuff, learn lessons from writing a system go back and clean it up. You know keep up with your tech that you see a better way to do stuff and then go and update it.

Artem Barmin

Thank you for sharing the experience of managing the engineering culture. From what I hear from previous podcasts, it's kind of a very important part of managing, I think, any product, but for Clojure specifically.

Cam Saul

I don't really know if it's like Clojure specifically because you could absolutely over engineer things in any language. Like enterprise, have you guys seen enterprise fizz buzz? Like it's like a repo is like, they're kind of like joking, like if we wrote fizz buzz, but we did it like the way people write enterprise Java applications, it would look like this, you know, and it's just like this enormous code base where it's like 10,000, 20,000 lines of code to write fizz buzz, right? They have fizz buzz like adapters and factories and all this other stuff. And it's just kind of funny, know? So I think that's one of those things you just gotta kind of, especially like if you're coming from like a corporate culture where that is how things are done, you've gotta take a step back and see, do we need all this like ceremony to write like an API endpoint or something? Usually the answer is no. You can always add it later, right? As long as you leave the door open.

Vadym Kostiuk

I have a question I normally ask all our speakers about this matter. So we've seen the recent tendency for Clojure engineering teams, for Clojure products, that some of them, they are actually moving away from Clojure for various reasons. It can be sometimes because of the external push, investors just make them to do this, or maybe internal push, maybe some leading engineer left the team and you know it's hard for them just to actually maintain the system and develop it or find someone who will replace this expertise and will actually keep going. Can you please tell me in your opinion, maybe in your experience you've been talking with different engineers, you have a lot of Clojure engineers on the team. What do you think are the most, I guess, critical points or what are the reasons that could lead the company to leave Clojure, in your opinion, like nowadays?

Cam Saul

Yeah, so maybe you guys have talked to a lot more people than I have, but the only time I've heard of people doing it really are if you're working at a smaller company that gets acquired by a bigger one, that's not a Clojure shop. And so maybe you're team of like 10 people working on Clojure and you get hired by a company of a thousand people, that's mostly like a Java shop, then the pressure from the new company is kind of to rewrite it in Java or whatever, because that's what that company does, right? So I've heard of that happening. <br/><br/>But most of the people I've talked to, like I was just at Clojure clonj last week, and I talked to a lot of people, and there's kind of like this push to write little chunks of things this team wants to do their work in Clojure.  There's always kind of a team that does something in X, you would prefer to see everything done in X. And that's true of any two languages probably. But I haven't talked to a ton of people who have been pressured to switch things away from Clojure. We certainly have zero plans. I think, you know, where would we go, right? There's nothing to this day Clojure's still the best fit for what we want to do, so.

Vadym Kostiuk

If at some moment you will decide to actually move away from Clojure, why do you think it can happen? I mean, you know everything about Metabase.

Cam Saul

Yeah, mean, that's really the biggest thing, you know, is that there's this external pressure to switch from somewhere else, usually a company that acquired you, I think if we, if there were some other reason, you know, we weren't acquired, but we're still like an independent startup and we decided to switch away from Clojure, you know, maybe if the language kind of ceased to exist or stop being developed, but it doesn't seem like that's about to happen anytime soon.<br/><br/>And it's relatively stable too, right? Even if Clojure never got another release, it's not really stopping us from continuing to build on it, right? It's on top of the JVM, so you still get all those JVM libraries and JVM's constantly improving. So as long as it was compatible with new releases of the JVM, you'd still be able to upgrade and get better garbage collection or stuff like that. So if I became the only Clojure developer in the world and we couldn't hire anybody or whatever, maybe I think that would maybe motivate us to switch away. Or, if another, Clojure 2.0 came out and was better than Clojure, if one of Paul Graham's Lisp dialects ever takes off and it's even better, it makes Clojure look super lame and we're like, this is way better than, we'd have that discussion. But I think these are all unrealistic possibilities.<br/><br/>I don't see us moving away anytime soon. As long as we can continue to hire people and we have had no problem with that and as long as the language continues to develop no matter how slow it happens, I think we'll be on Clojure for a long time.

Artem Barmin

Speaking about ecosystem and speed of language development, do you think this stability of Clojure is a good thing or bad thing for community and for the project itself? Because for example, JavaScript 10 years ago and JavaScript right now, it's basically two different languages. But if you take Clojure everything that was written 10 years ago is still working and applicable and even tutorials are very up-to-date. Nothing really big happened.

Cam Saul

Yeah, I think this is definitely one of the good things about Clojure. So I'll give you an example. I was trying to do a demo at an internal company conference, or we had a company offsite, I guess, where I was showing people our original Python prototype of Metapace. And I'm like, cool. I'll just spin it up and show people how it worked. And that turned out to be almost a solid day project just getting it out to work, because we wrote it in Python 2, can't even install Python 2 on my MacBook these days, you know? So I'll just go update everything to be Python 3. I don't know. Even Python, which how much can Python change? It's changed a lot in 10 years. But Clojure, I also showed people our very first Clojure prototype and I was able to spin that up with like zero changes. So that's kind of a good thing, I think. <br/><br/>Clojure doesn't really need to change that much, right? It has everything it needs and because it's a Lisp language with macros, you can add whatever features you want on top of it. Like the example I always give people is that in JavaScript it took them 10 years to add the for each loop, right? They started talking about it. They're like, whatever other language has for each, you know? So let's add it to Java. it took 10 years from when they started talking about it to when they had all their meetings and then spec’d it out.<br/><br/>And it made it out into the release of the JVM, right? So it would literally take you five minutes to do that in Clojure if you wanted to write that kind of thing. So you can extend the language to add new syntax or query sync is basically just like go style stuff in Clojure, right? And someone went and wrote that library or whatever. So yeah, anything you want to add to the language can be done in a library.<br/><br/>So it doesn't really need anything, you know, as long as it keeps up, as long as it still works with new releases of the JVM, you're getting those kind of upgrades to garbage collection or performance or other things that happen to the JVM. And your old code continues to work, you could add whatever the hot new feature of the week is. And it's a good thing. It's honestly kind of exhausting with languages like JavaScript keeping up with whatever the framework of the week is, right? It's kind of nice to not worry about that with Clojure.

Artem Barmin

Do you have this problem in your frontend team? So keeping up with JavaScript and TypeScript development.

Cam Saul

We're not too far out of date. We're using reacts now instead of angular. we're not using jQuery anymore, but we're still using Redux and a lot of people move away from Redux and using like React Actions or whatever, and we haven't gone there yet. I don't know if we're like the only people still using Redux or I don't really have a great feel for that since I most of my time in Clojure, honestly. There's kind of more of a pressure there to, you know, people aren't doing things this way, or especially when we hire new people who are doing things the new way and we're still doing them the old way. And the pressure that is just like absolutely nonexistent with Clojure. You know, we're doing things the way, there's a few different ways of doing like a REST API endpoint routing or something. And we're using Ring and Composure, which has been around for a long time and there are certainly some other ways but this is not like a dead way of doing things.

Artem Barmin 

Yeah, I agree. I remember maybe 2011 when I first started doing Clojure. It was already Composure and Ring and it's still, it's still applicable. And I'm curious, if you had an opportunity to go back 15 years ago and change one aspect of Clojure as a language, what would you change?

Cam Saul

Would I change if I can go back in time? Yeah, good question, good question. I'll be honest with you and I'll get in trouble for saying this with the Clojure community, but I'm kind of a static typing person, know? I did C++ for a while. I did Java and Objective-C and I kind of missed that. know, kind of, if your code compiles, you don't even have to write tests, right? You could just pretend it works, right?<br/><br/>I think it'd be cool if Clojure had some way you could opt into that. Maybe not required, but you can say, just like a core language feature, this function takes a string. And then if it knows you're calling it without a string, because you're calling it with some other function that's typed, then you can get a compiler error. <br/><br/>Of course, there's like typed Clojure, which is working on that. Some of the stuff with Molly is able to say, or spec tool, there's like all these different tools that are kind of working towards that, but if that was a good core language feature, that'd be neat. So I kind of miss that. And of course, there's all the benefits of not having to fight the type system and stuff. So I wouldn't want it to be mandatory. I think something where you can opt into it would be neat. But that's like a minor thing. The way it works now has served us well.

Artem Barmin 

And are you enforcing, if you switch some schema solutions in your code base?

Cam Saul

Yeah, we use Mali everywhere, actually. We love Mali. We were using Schema. We were big users of Schema for a long time. And then when Mali came out, we switched to that. And yeah, we have some custom Mali or some custom macros that let you do like, def in with Mali. I guess there's actually stuff like that in Mali these days, Mali.experimental. But yeah.

Artem Barmin

Yeah, it's kind of experimental, I think.

Cam Saul

Yeah, I think we were trying to use that, but we ran into some problems. So we kind of rolled our own. And I think at some point we need to pull request that because I think we have like a couple, a couple improvements over stuff in my experimental, but we use that everywhere. And so in all of our tests, when you're in the REPL, then we do all the type checking and then production, we turn it off. So you get all, you don't have to worry about performance at all because it's not affecting anything in production. But then when you're working on stuff, it could really catch stuff really quickly. Like I wrote this function and it's supposed to take this map with this key and then you write some other code that calls it and it doesn't have that key and then right away you know that geez, my code is not doing the right thing and then you can really quickly fix it. So it really kind of amplifies the quick feedback loop that you get with Clojure and lets you catch stuff really quickly and kind of lets you write down assumptions you have. <br/><br/>It actually serves as documentation I think. You can write down that this function expects a map with this key. And even if you don't write a doc string or you don't update it, people can go look at it and say, yeah, okay, I understand what this function needs and what it returns to. So yeah, we use that everywhere. And we also use our REST API endpoints. We got a little macro that uses Mali schemas to do coercion. So if someone, our REST API is all JSON based, right? Passing something as a JSON and we wanna convert this string to a keyword or whatever, we just use molly schemas to do that. It does it automatically. We say this is like a required parameter in the request body. It has to be one of these three keywords and then molly coercion will just take it, well it's a string, we'll convert it to a keyword automatically and then we'll check, you know, is it one of these three keywords and if it's not, then we have it set up to error automatically, you know, 400 invalid parameter. So, yeah.

Artem Barmin

Very interesting. We actually had a question about how deep the schema usage should go in the code base on the boundaries or use inside. And it's really interesting that you use it everywhere.

Cam Saul

Yeah, so let me give you another example of we do with it. We actually generate documentation for our REST API based on these schemas. We leverage, like Molly has, I think it's like molly.experimental.describe, but we have a little bit of tooling built on top of that. So we generate all of our API documentation based on these schemas. Yeah, so I guess it does face external boundaries. I think what we want to do next actually is generate like Swagger documentation or whatever, something that'll be maybe like one of those little things where you can spin up a web server and try out the API and then it'll tell you what the parameters are. That's kind of on our to-do list. So one of the cool things about Clojure is that it's so easy to walk the code or walk schemas and do things like this. Yeah, so the world's kind of your oyster with Mali schemas and being able to walk your code.

Artem Barmin

Yeah, I really like this picture of the Clojure. Abstract syntax tree as a first-class citizen, I would say. And, you know, given all this information, all the details that you've provided, can you tell, would you choose Clojure again for the new product right now in 2024?

Cam Saul

Yeah, I think absolutely. Just how quickly a few people can get stuff done and just all the power it kind of gives you, the ability to walk the code and write custom linters to kind of make it, I don't have to ever go pull request review, do a pull request review and say, well, please follow this pattern, write a linter that says that for me. How easy it is to walk the code and generate documentation, all these things really make Clojure a really good tool for what I want to do, especially if I'm starting something new. And just as a team of one or a team of a few people, Clojure really kind of serves as a force magnifier or whatever, like an amplifier. It really lets a small team get stuff done really quickly. Yeah, I think it's a great fit for the job. <br/><br/>And I'm struggling to think what would be even a viable alternative, you know? The options we have today aren't that many more than we had 10 years ago. If you want to write a web application, know, Rust is not probably a great choice for that. If you're writing some kind of system stuff, you know, maybe you'd pick Rust or whatever, but it's kind of the same choice as you could write it in Node and JavaScript or you could write Python or you can write some of these other languages, but I think the advantages Clojure had then are still ones that has now, so.<br/><br/>Unless like all the Clojure, people who want to learn Clojure vanish off the face of the earth, you know, I think I'd still pick Clojure.

Artem Barmin

Yeah, and you know, I'm curious to hear a bit of a story. Maybe it should be asked half hour ago, but half year, half hour. But can you tell how many people were at the start of Metabase? So what was the team size?

Cam Saul

Yeah, so I think we had four people. We had the CEO who himself was a Python engineer, which is why Metabase was in Python originally. We had another engineer that he hired who had written Scala previously, which is one of the reasons why we looked at Scala. And then we had a designer who was also good at doing CSS and a little bit of JavaScript stuff. And then I kind of came in at that point and I was, like I was saying, I was doing mobile apps, but I also doing backend stuff from Python. So it was, I guess, the four of us.

Artem Barmin 

And with this team composition, how long have you been moving here to?

Cam Saul

How long did we have that size? Yeah, actually, we had a really small team up until maybe like three years ago, I think. Only three years ago did we get to like 10 or 20 people. Yeah, we actually spent a few years working on the product and dogfooding it with a bunch of companies before releasing it. Really kind of trying to find the right fit. We raised the seed round kind of from the beginning, so we had time to figure things out.<br/><br/>So we spent a long time kind of iterating on the product and really kind of nailing it down. The other competitors in our space, like Tableau or Looker or Power BI, are very complicated tools, I'd say. With Tableau, people go into LinkedIn, and they go get Tableau certified, and they go put that on LinkedIn. So the way we were going to differentiate ourselves is you don't need to go get Metabase certified. You can spin it up in five minutes, and then you don't need a NoSQL. You can get the data you want, the information you want, make a little bar chart. So we really kind of iterated on that a lot. And we grew pretty slowly. And we had an office in San Francisco. And it was maybe a little bit harder to hire people in office than it is remotely. So we grew kind of slowly and deliberately. then, yeah, I think it's been pretty recent that we've grown as much as we have.<br/><br/>The next longest tenured engineer at Metabase, I think has been there like four years. And we hired him when we had like 10 or 15 people. So we've grown, I guess it's like if you look at the graph, it's like kind of like a hockey stick, you know? Yeah. Yeah.

Artem Barmin

I see. Yeah, that's a really common story about Clojure startups when very small team can deliver very big results. And that's also optimizing the burn rate. You can live more on the same seed round. Yeah, that's cool. That's cool. 

Cam Saul

Exactly, Yeah, think some of our competitors, their teams are probably, I don't know, 500 people, but Clojure really let us do a lot with a tiny team. So we didn't have that pressure to expand out to 500 people early. We were able to ship a lot of stuff with a small team. Yeah.

Artem Barmin

That's really encouraging me because I'm starting a startup right now in a low-code space. I think the Clojure will help me to make it lean. 

Cam Saul

Nice, congrats.

Artem Barmin

Thank you. And now I want to ask you to help us to improve our podcast and maybe suggest a question that we should have asked, but haven't.

Cam Saul

Oh man, suggest a question you should have asked but haven't. Yeah. I would ask people, you what do you hope to see in Clojure and the Clojure community in the future? Where do you want to see this language in this community in two years from now, five years from now?

Artem Barmin 

Can you answer this question?

Cam Saul

Yeah, you know, so this is one thing that Clojure, I mean, we've all drank the Clojure Kool-Aid, right? Otherwise, it wouldn't be here on this podcast, but I think Clojure is just one of those things that if more people knew about it, I think more people, if they knew what we knew, then we'd have more people using it. <br/><br/>So I would love to see the community grow a little bit, but it doesn't seem like it's shrinking to me, but it's at least not growing like 10, 12 years ago, Clojure was like the exciting new thing on Hacker News. And now it's just, know, just like in the language, it's like stable, but not like, I would, I'd be really surprised if I woke up in two years and it was like 10 times the size as it is now. So I think kind of getting more people into the language, that's kind of like an open question. How do we do that? I wanna actually shout out to some people I ran into at Clojure Conj doing something called, I think it's called Clojure Camp. And they are pairing people who know the language well with people who are learning it or want to learn it and helping kind of bring more people into the language. So I think efforts like that are really good. And I'm happy to see that happening in the community. And I definitely want to see more of that.

Artem Barmin

For some people, it's not a problem at all because it's good to be niche. The Haskell guys saying avoid success at all costs because it makes them able to change language, to be lean, to be fast. Yeah. Thank you. Thank you. We will record this question.

Cam Saul

Yeah, yeah, mean, it never hurts to have more people in your language, right? You know, I think one of the things I'm jealous about with JavaScript is they just have so many people working on it. Like they have these like huge, they have like meta or whatever is like, well, we're gonna come up with some crazy new framework, you know, and the Clojure kind of doesn't have that, which is I guess a blessing and a curse, but, or like if you have like Google and Meta and all these other companies using it and developing tooling for it, you know, you get to use that for free, so.<br/><br/>It never hurts to have more people use your language, I think. The Haskell people, I don't know, guess you'd have to go on the Haskell on product podcast, which would have like one episode, right? it's not like a, it's more of a academic language than a building serious products language, I feel like, know, so maybe.

Artem Barmin

Yeah, I heard about examples of when Haskell is used for production, but much, much less than Clojure.

Cam Saul

Yeah. Right, yeah, so Clojure is like a, Clojure is the Common Lisp side of things in the common list versus scheme debate, right? It's a language for building stuff, not a language for proving that hello world is correct or whatever, right? So yeah, I think it never hurts to have more people.

Vadym Kostiuk 

We're now at the part of the podcast where I'd like to ask you a question that our previous guest left for you. So our previous guest, Alexander Johannes, he's a head of development at the company called JustOn. And he left you question, which is, you partially actually responded to it during our podcast, but still.<br/><br/>The question is, have you experienced a need to change your development process when integrating Clojure into your technical stack? Especially, did Clojure introduce unique challenges or advantages that prompted you to adapt your team's approach and methodology in any particular way? So it's a long question, but yeah.

Cam Saul

Yeah, so actually, I guess maybe in some ways, we used to target Heroku as a deployment target, which I think gave us like 300 megabytes of RAM. So I'm sure if we were written in plain Java, or especially if we were written in like some sort of, if like C++ or something. We wouldn't have to be memory conscious, but we do have to be kind of careful of not... Clojure would be happy to eat a ton of memory if you don't be too careful. We stream results from a database, and if we're getting like million rows, do that with transducers and stream it one row at a time. Don't get them all in memory at once and then do some kind of post-processing and then spit that all out. That's a recipe for eating up your 300 megabytes of RAM you have on Heroku, especially if you're doing that on 50 different Ring threads or something. So I guess maybe that's something we had to think about. It's not like, well, I mean, that would have been something we'd have to worry about with Python or most of these other languages as well. But you do have to, I guess, be a little bit careful not to do, not to yourself in the foot doing kind of dumb things but that's not like a Clojure-only problem. I can't think of a lot of Clojure specific problems, honestly.

Artem Barmin

That's a good sign.

Cam Saul

Yeah, I guess it is, isn't it? 

Vadym Kostiuk

Here is the part where you can leave the question to the next guest. 

Cam Saul

Well, my question was, where would you like to see Clojure and the Clojure community in the next few years? If you took a little time machine into the future a year from now, three years from now, five years from now, what would you like to see? How would you like to see Clojure and the Clojure community develop? What would you like to see different? Or should it be exactly the same?

Artem Barmin

Great, thank you. And actually that's the end and it was really a pleasure talking to you in this episode and thank you for sharing all this information, insights and the story behind the Metabase. That was really interesting.

Cam Saul

Yeah, thanks. I had fun too. If you guys want to have me back, but you don't want to get the same guest, I can do my hair differently and pretend to be someone else or something. It's been fun. Yeah. Cool. Looking forward to seeing this published. Thank you for having me. It's been a pleasure.

Vadym Kostiuk 

Yeah, thank you very much and special thanks to your assistant.

Cam Saul

Yeah, this is Scrambles everyone by the way. 

Vadym Kostiuk

Yeah, and to our audience, if you'd like to learn more about Metabase, we'll be including links to their website below this video.

Artem Barmin 

I think our audience know a lot about Metabase. Thank you. See you. Bye.

Cam Saul

Thank you guys. Bye, thank you guys.

Vadym Kostiuk 

Thank you. Until next time. Bye.