Podcast: Clojure in Product /  
Episode 06: Discovering Clojure – True enlightenment begins
Artem Barmin Portrait

Artem
Barmin

Co-founder
of Freshcode
Vadym Kostiuk Portrait

Vadym
Kostiuk

Clojure
Partnerships
Hosts
Freshcode logo

Yehonathan Sharvit

Software architect at CyCognito
Guest
Guest company
Read transcript

<div class="heading_h4" style="grid-column: span 2; margin:15px 0">Introduction</div>

Yehonathan Sharvit

For the first 10 years my life as a programmer was not that enjoyable.

Artem Barmin

The oldest our Clojure programmer works in a company for 10 years.

Yehonathan Sharvit

And then truly I discovered not the light but enlightenment, when in 2012 I discovered Clojure. It’s a language that makes you productive.

Artem Barmin

And Clojure has a lot of concepts that are not widely popular in usual languages, but all those concepts slowly go inside the JavaScript ecosystem. Even immutability, you know. I started with Java first, spent one or two months, and quickly switched to Clojure, did much more in a week than I did before in two months.

Yehonathan Sharvit

I think that some people left the company because of the language.

Artem Barmin

CV-driven development

Yehonathan Sharvit

Very-very painful.

Artem Barmin

Hey everyone, welcome to the sixth episode of the podcast “Clojure in Product: Would You Do It Again?”

Vadym Kostiuk

And today we’re excited to have Yehonathan Sharvit from Israel joining us. Yehonathan is the author of the “Data-Oriented Programming” book. And he’s also a speaker at different tech conferences. He’s very passionate about sharing Clojure principles with non-Clojure developers.

Artem Barmin

And we also have a question from our previous guest, lead back-end engineer at HolidayPirates, “How to handle state in Clojure projects? Specifically, how do you keep stateful parts of your codebase minimal and separate from the rest of the logic, especially when stateful code starts spreading across the functions?”<br/><br/>So, let’s begin to map over questions and macro-expand them into the real-life Clojure experience.

<div class="heading_h4" style="grid-column: span 2; margin:15px 0">Full version</div>

Vadym Kostiuk

Hello, Yehonathan, and welcome. Thank you for joining us on our sixth episode of “Clojure in Product: Would you do it again?”

Yehonathan Sharvit

Hello

Vadym Kostiuk

Yeah, we're really excited to have you, and we are looking forward to the conversation that you'll be having. But to start off, I'd like to ask you if you could share a bit about your background and about your journey as an engineer from C++ and Java engineer to discovering Clojure. How was it actually?

Yehonathan Sharvit

Alright. Okay, so I graduated around 2000, so 25 years ago, something like that. And for the first 10 years, my life as a programmer was not that enjoyable. I coded in C++ mostly, in MATLAB even, before, and a bit of Java.<br/><br/>I was not great at any of these. And I always had the feeling that I have to learn how to please the compiler. And I'm not a good pleaser. So I always struggled to remember the inheritance, is the constructor inherited, is it not inherited, what runs before? And it was really, really a nightmare for me to be a C++ developer.<br/><br/>I started to discover the light in 2009 when I joined the startup here in Israel, web ad network startup. And they developed a JavaScript plugin for showing advertisements. So that's the bad part that it was an ad company. But the good part is that we were using JavaScript and I was not aware at all at 2009 that there was something different than C++ and Java. And suddenly I discovered that what usually takes a page of code in JavaScript is two lines. And that I can express myself in a much easier way. When I have to express business logic, I can simply express the business logic.<br/><br/>And for me that was really, really a game changer. So 2009, JavaScript after 10 years of suffering. It was a company where they were using many languages. They were not afraid of testing new languages and new technologies. So in 2011, I discovered Ruby and Ruby on Rails. And until today, I don't know if I prefer Ruby or JavaScript, to be honest, because Ruby has more power of expression in a sense, but Ruby has the problem that it's very easy to write, but then it's difficult to read. <br/><br/>In JavaScript, it's much, you have less, less ways to do things. For example, in JavaScript, you have just an anonymous function. At least today, you didn't have the arrow notation. It was just function, curly braces. And in Ruby, you have, I don't know the name, a lambda, a proc, a context, this and that, and there's a subtle difference that nobody remembers. And you never know which one you want. And if you are used to one, when you're a colleague or somewhere on, you read code from somebody else, you cannot read it because you're not used to it. <br/><br/>So it's a write once program like Perl used to be for me. But much, but more powerful also. So I think I really liked the JavaScript of the early days of the web. I didn't like the CSS part and the HTML, but the JavaScript and the business logic was really, it was really a big change for me.<br/><br/>And then I truly discovered not the light, but the enlightenment when in 2012 I discovered Clojure. And then it was wow, wow. And since then programming has been a huge fun for me.

Vadym Kostiuk

Well, that's interesting. Back then in 2012, what is that that attracted you actually to Clojure?

Yehonathan Sharvit

Okay, the first thing that I tried to do in Clojure is the purity of the language in terms of the elegance of the language. I was really... I have a second degree in mathematics. And I was really interested in the foundation of mathematics. And somehow I got to lambda calculus and then to Lisp. I didn't learn Lisp at school. I discovered Lisp via mathematics. But for me, I was sure that Lisp was like the Greek language. It's very interesting. Nobody uses it. <br/><br/>So when I discovered that something that looks like Lisp is used in production, wow. I was really amazed. And as I like to say, “so I have a degree in mathematics, but I'm not, I don't think I'm a mathematician.” That's why I didn't go for a PhD because I don't think I'm smart enough to do a PhD. So I'm not smart enough to be a mathematician, but I'm smart enough to be a good developer or maybe a little bit better than average, somewhere in the middle, you know, between the genius of a mathematician and the ingenuity of engineers, I'm somewhere in the middle. <br/><br/>So when I see that something theoretical is applied in software, I'm really excited. So that's what triggered my curiosity to Clojure. <br/><br/>But then I discovered totally different things. It's a language that makes you productive. It's not just a language that is fun to use. We are much, much productive in Clojure. And also to tell you about my journey in Clojure. So I left this company in 2013. And when I left, the guy that introduced Clojure to me at this company asked me what will be my next venture. So I told him I'm joining someone to build a startup together. He told me, you will use Clojure. 2013, no way. Nobody uses Clojure. And also our strategy will be to recruit junior developers, fresh from university. So there is no way that we could teach them Clojure. But he told me, no, you should try it. And he was right. So we tried Clojure. In fact, it was ClojureScript, 2013, of 2013, so 2014, before Reframe, it was something called Home back then. It was the early beginning of ClojureScript with, you know, bugs and new features every release and excitement and performance issue and Google Groups was before Slack. We sent a lot of messages in Google Stack Overflow. <br/><br/>But the really interesting thing is that fresh developers from university have no problem to learn Clojure. No problem at all. And it feels very natural to them to use, know, like what we used to say, the weird parenthesis. It's not weird at all. And the immutability, it's not weird at all. And the REPL? It's great.<br/><br/>So that was, and it allowed us to develop very, very quickly because we did something quite complicated in the front-end side with a lot of data manipulation and complicated logic. And for us, it was really a, really an advantage. We could develop very quickly the product. And we were, at the beginning, two developers, and then three, four, five. And all of them were juniors.

Artem Barmin

That's very similar to our experience. We also started our company in 2012, I think. And I started it with Java first, spent maybe one or two months, and then quickly switched to the Clojure. Did much more in a week than I did before in two months. And I was really impressed. I also had experience before Clojure with Common Lisp and I'm very thankful to the books that I found about Common Lisp, “Practical Common Lisp” especially. And I want to ask you a question. You wrote actually two books, “Get Programming with Clojure” and “Data Oriented Programming”. By the way, I’ve read “Data Oriented Programming” and it's pleasure to meet the author. Can you share some inspiration? Why you decided to write this book?

Yehonathan Sharvit

Okay, yes. So first of all, here is the book. Unfortunately, I cannot show you the other book that you mentioned, “Get Programming with Clojure”, because actually it was never finalized. And I think that's the best failure that happened to me. So let me tell you what happened with “Get Programming with Clojure” and then with “Data Under Programming”. And I think it's a nice story that we could easily extrapolate to other domains, not only to books. <br/><br/>But before that, I have to tell you why Manning contacted Manning the publisher, why they contacted the guy like me that nobody knows and asked him, “Would you like to write a book for us?” So first of all, when it happened, was, wow, wow. That's a dream, a dream comes true. <br/><br/>A year or two years before they contacted me, so we were in 2017, I guess, 17, 18 somewhere. So two years before, 2016, we were using ClojureScript in production. And from time to time, we asked ourselves, how this piece of code in Clojure is transpiled to JavaScript, which is what the ClojureScript compiler does. So most of the time, we don't care. But from time to time, we are interested, if I wrote a code like that, how would it be transpiled? <br/><br/>So you can write it in your project, and then you see the result. But sometimes you want it just a small expression and to see how it looks like in JavaScript. And actually, I developed a tool that does that using a function in ClojureScript that is called compile. And it was very easy. It took me, I don't know, two hours to put a website where there is a form. You put Clojure code, you press a button and then you see the transpiled JavaScript. And I call this tool Clips. It sounds like Lisp. And in Hebrew, Clips is the thing that you use to to attach the clothes when you want to make them dry. <br/><br/>So I developed this tool and then I say, okay, I can, and it really, was nothing. I put it online. And then I also add the possibility to see the result of the evaluation, not only the transpilation. So it's kind of online REPL. Client side, not server side.<br/><br/>And then I started to write a blog about my findings in Clojure and ClojureScript. And I used this tool inside my blog. So it was, I think, the first ever blog where you could see pieces of code and see the results of the evaluation. And the reader could modify the code and see instantaneously what is the result. No server side involved, no security risk, nothing, no cost for hosting. So that was all good. <br/><br/>And I wrote, I don't know, 10 or 20 articles about macros, protocols, about performance issues and stuff like that. And that's how Manning, the publisher, contacted me to write a book. They liked my blog and they asked, would I like to write a book about Clojure? I said, sure. So I wrote.<br/><br/>The book was supposed to have 15 chapters. I wrote three chapters. And after three chapters, in Manning, they published the book for early access. So they are very agile in that sense. And people, you know, they purchase the book full price and every time there is a new chapter, they receive a new chapter.<br/><br/>So I was super excited. On the first day we sold, I don't know, we sold maybe 100 copies in a day. But then almost nothing for a few months. So I kept writing new chapters, but nobody purchased the book. So after, I don't know, six chapters or something, Manning told me, your book is maybe great, but nobody… Even if you finalize the book, will buy it, so let's stop the project. And they asked me, do I have another book project because we liked each other. So I said, thought about that and said, okay, if I cannot convince people to get interested in Clojure, maybe I can share Clojure principles to people who don't want to learn a new language.<br/><br/>So I asked myself, what makes Clojure unique? It's not the fact that it's a Lisp, because there are many Lisps. It's not the fact that it has a REPL, because every Lisp has a REPL, and JavaScript kind of has something like a REPL. And it's the fact that it's data-oriented, that it's really good at manipulating data. <br/><br/>And actually, I found a mention of the term data-oriented in the announcement of Clojure at the beginning. say Clojure is a data-oriented language. So that's how I came to this title, “Data-Oriented Programming”. And in fact, it's data-oriented programming is how to apply Clojure principles in JavaScript, Java, Python, C++, whatever. It was really an interesting intellectual effort to extract from Clojure the syntax of the principles and to be able to express them in a language-agnostic way and to convince people who have no idea about Clojure why it might be a good idea to use those principles in their own language.<br/><br/>To my surprise, this book did very well in terms of sales. Today we are around 7,000 or 8,000 copies.<br/><br/>The same thing could happen, not only in a book. Many of the JavaScript front-end frameworks are inspired by Clojure. React is inspired by Clojure. Redux is inspired by Clojure. The latest feature request to JavaScript language comes from Clojure. Many of them. Someday we will have immutable data in JavaScript. So if before an array you had the pound sign, then the array is immutable.<br/><br/>And they are all those requests come from a company. I forgot the name of a huge American company and the guy that used to be a Clojurist, Clojurian?.. And what he likes from the language, he simply suggests ECMAScript to add to JavaScript. So even if Clojure does not become a huge success in terms of adoption, it will be a success because it will reach though JavaScript.

Vadym Kostiuk

Yeah, and I wanna ask you about your work at CyCognito. So you joined the team, I guess in 2019 and the product like widely known for its cybersecurity purpose. So can you please provide us with a short overview of what actually platform does?

Yehonathan Sharvit

Yes, what the platform does, they build the attack surface of a big company. So imagine you are a huge company like Apple or Nestle or Hilton or whatever.<br/><br/>You have subsidiaries, the subsidiaries have subsidiaries, and each of these companies inside the big organization have digital assets, millions of digital assets and you want to make sure that everything is okay in terms of security. <br/><br/>So once you have an asset to check that the door is locked properly, it's easy. The problem is that those companies don't know what door they have. So if you are the CSO of Apple, the Chief Security Officer of Apple, and someone asks you, what is the status of our assets?<br/><br/>Probably the CSO would say, I hope it's okay. But I even don't know what is the map of the assets and the relationship between them. It's like you are the manager of a hotel chain and somebody asks you, are all the doors of all the rooms locked? You don't know what hotels you have, where around the world and how many rooms and how many floors in all the hotels. So CyCognito brings you a map of all your doors, of your digital assets, and then they check the vulnerability, the security status.

Artem Barmin

So it's done in automatic or semi -automatic?

Yehonathan Sharvit

Yes, yes, it's 90% automatics.

Artem Barmin

Very interesting. It's a hard task to find all the assets, all the doors, all web servers, VPNs, everything else. I can imagine. Can you tell a bit more about the role of Clojure in this project? 

Yehonathan Sharvit

It's almost all the code is written in Clojure. The data pipelines are written in Clojure. The app is written in Clojure and ClojureScript. The API is in Clojure. Almost everything is in Clojure. There is a bit of Python also because many security modules are available in Python.<br/><br/>It's hard for me to admit, but now they want to migrate part of the code to TypeScript.

Artem Barmin

Can you tell a bit more history maybe when the project started? Who was responsible for choosing Clojure and why?

Yehonathan Sharvit

Okay, actually it's a very kind of a funny story. First of all, how I got to this company is because of the book that I mentioned that was a failure.<br/><br/>Somehow the CTO of this company read a few chapters from the book and somebody else also read the book and he connected between us. So, you know, when I told him I'm the author of the book, he said, come on, you can... He didn't do a technical interview. So for me, that was great because I really suck at technical interviews. And they started, I think, one year or two years before I joined, so around six or seven years ago. <br/><br/>And this guy actually worked as a consultant at a company, for the same company as I worked, where I discovered Clojure. So there was a guy at the company, a Russian guy, that introduced Clojure in 2012.<br/><br/>I discovered Clojure in 2012 at this company. I left and this guy, the CTO of CyCognito joined as a consultant for the same company but after I left in 2015 and he liked Clojure there. So when he built his own company, he said, sure, I will use Clojure.

Artem Barmin

That's very typical story about Clojure projects because we have a lot of clients and we support their projects and it always starts with a very enthusiastic CTO that really likes language and just decides to do this in Clojure. There's usually no real kind of reasons to do this, it's just passion.

Yehonathan Sharvit

I think the majority of the developers, they don't really like Clojure.

Vadym Kostiuk

Do you mean within the company, yeah? Within the team?

Yehonathan Sharvit

Yeah, inside the company. Inside the company. Yes. They are okay, but they are not excited about Clojure. They either like it or they say, Because in Israel, there is something sad. The developer community, are very hype-oriented. And Clojure is not popular at all in Israel. And they have this irrational thinking.<br/><br/>They say, if now I want to join another company, I won't be aware of the latest JavaScript, React, Webpack module. And probably I won't find a job because of that.

Artem Barmin

CV-driven development, yeah. We need to put as many new frameworks into CV to make it  more expensive.

Yehonathan Sharvit

What they don't know is that anyway, when they will be in an interview, even if you are at a company that use Webpack, when you are at an interview, they use already the Next, the, I don't know, NPX or PNPM or Yarn or whatever. Then, and even if you know already after you start, you will need to learn another one. And so framework is not something where expertise is valuable.<br/><br/>You need expertise, but you can reach it very quickly. And that's not what you bring into a company, especially nowadays with the AI and everything. You don't bring knowledge, you bring mind, you bring analytical skills, bring intuition.<br/><br/>And Clojure teaches you all these things. Clojure the language. Another thing that I really like about the language is that it is, don't know, like transparent. The implementation of the language is very, very transparent. I was really amazed when I discovered that 90% of the code that you use in Clojure, actually in Clojure Core, it's something that is implemented in Clojure. It's not something at the level of the compiler. <br/><br/>So in order to understand what is a map or what is filter, what is map, what is a record, you need to read Clojure code. You don't need to read Java code. Maybe in Clojure you need, but in ClojureScript, everything almost written in itself. In Clojure, it's not really, since they had to bootstrap, it's not exactly true what I just said, but...<br/><br/>Conceptually, it is true. The compiler, the core of the compiler, not the Clojure code namespace, the core of the language is really, really a small piece of the language. So for us, the non-smart guy, I cannot read compiler code. I'm not that smart. It's very hard. But to read implementation of a function in Clojure, usually I can. Not all the time, but usually I can. And it allows me to understand also how the language works under the hood.<br/><br/>And so that's why I'm saying Clojure teaches you to become, if you are a bit curious and you practice Clojure for a year, you after that year, you are a better developer. Even if after that you go to another language. That's, I think that's a really, really great gift for a programmer.

Artem Barmin

That's true. And Clojure has a lot of concepts that are not widely popular in usual languages. But as you mentioned before, all these concepts are slowly going inside JavaScript ecosystem, even immutability. If we take Lodash in JavaScript, it's very popular library. It borrows a lot of concepts of immutable transformations, chains, threading operations, and such kind of stuff.<br/><br/>And you know, I want to talk a bit more about team and about people. You mentioned that not a lot of people really like Clojure. And when you hire for something, when you find the new talents, how do you approach this problem? Because you need to sell the idea of Clojure to newcomers.

Yehonathan Sharvit

No, not really. We don't go into that. First of all, we almost never hire Clojure developers. We hire just developers, and they learn Clojure. And either we don't mention the language, or they ask, and so we use Clojure and say, OK. Or from time to time, someone will say, no, I don't want Clojure at all. But usually, it's not an issue.<br/><br/>But the thing I think where we didn't find a way to...<br/><br/>We are not good enough, I think, is that… If you are not curious, then you can stay an average developer in Clojure, and that you don't understand how things work. And then you complain and you find work around and you write bad Clojure code and you dislike the language because you are not using it properly. And I think that's what is a little bit missing. The language allows you to be productive even without knowing it perfectly.<br/><br/>So there is a combination of this. And also there is the problem that sometimes you pick the wrong framework. But the framework is stable, and it's kind of forward compatible. And then the framework is deprecated, and you don't rewrite your code because it's usually a big price. And then you are stuck with a very, very legacy framework.<br/><br/>That's what happened at the company I mentioned where I created the company and we hired junior developers. We used OM, which was a wrapper around the React. was the early days of React. And then OM was deprecated into OMnext, but OMnext was not compatible with OM. We couldn't migrate to OMnext without rewriting everything. So we stayed with OM.<br/><br/>And until today, this company is 2024. They have 50,000 of Clojure Script code based on OM. And they don't like it. And I think they cannot really, I don't know, they cannot upgrade all the libraries and they, it's a bit of a deprecated OM. The way you are using home is not the modern way of doing it. So they are stuck with OM.<br/><br/>But why they are stuck with OM? Because home was stable enough. There is like a paradox. If we were in JavaScript and you pick the wrong framework, after a month you cannot do everything. You are frozen. So you are forced to choose another framework. Because Clojure and OM is kind of stable enough, there is never a rush to upgrade the framework. And then you don't upgrade, you don't upgrade. And after 10 years, you find yourself with a framework that in 10 year old.

Artem Barmin

I often see that libraries enClojure are pretty, I would say easy to support because I have a lot of experience digging inside the libraries, doing some new features, even they long, long, long time ago abandoned.<br/><br/>But that's not blocking me from bringing some new technologies, new features into the libraries. Don't they support OM for their organization? Maybe maintaining their own fork of a home.

Yehonathan Sharvit

They could have done that, but I don't think that those developers have the level of… to be able to support a framework.<br/><br/>So that's one thing. In CyCognito, there was something different. In the front end, there was a very smart guy who built his own state management framework instead of a Reframe. Maybe it was before reframe, and he did something like Reframe, but not Reframe. And also the company, this guy has left, and the company is stuck with this front end framework. It's not fun to use the framework, but it works. So again…<br/><br/>The fact that it works is the best feature of the language and it's also the thing that explains the lack of adoption somehow. Like, you know, when they asked Rich Hickey, don't remember, two or three years ago, “What's next in Clojure?” And he said something like, nothing. Like, we are done. We don't need to add new features, new things. You can do everything in the library at the core level.<br/><br/>Maybe besides ClojureSpec, which was not really a success, but besides that, what do they, okay, they fix bugs. What do they need to add? A new function, a new thing? The language is stable. It solves 90% of your problem. But it's not cool to be a language with no evolution. It's cool to have new features, new tools, tools that do the things that solve problems that you don't have in Clojure. You know, in JavaScript, all the tools, they solve problems that you don't have and it makes you feel amazing. <br/><br/>All the JSX thing in React, which is amazing, that you can write template code inside your code with backtick and the ID supports it and I did it properly, although it's HTML inside Java, it's amazing. But we don't need that. We have a Hiccup, which is just data. We don't need templates. Oh, you don't have templates. <br/><br/>I don't know if it's the same in other countries or maybe it's specific to Israel. I'm not sure.

Vadym Kostiuk

By the way, I have a comment. It's strange to hear about this, like not really about the engineers that are not really satisfied with the choice of Clojure within your company, because as far as I know, there are really like a bunch of companies that are using Clojure in Israel. So like from the top of my head, I know AppsFlyer. It's a pretty big company. They're using Clojure and there are others.

Yehonathan

No, I don't think there are others. There is AppsFlyer, is Acognito, there is the previous company that I mentioned. There is a company that used to be named MadLand also. There is another one, I think, but that's it. When we were organizing meetups, we were seeing always the same people. There is a guy, Daniel Slutsky that he's a solo contributor and he pushes a lot to data analysis via Clojure. But I don't think that companies in Israel are. And AppsFlyer also, they have moved a lot of code to Go, I think.

Vadym Kostiuk

But there is also like Wix as far as I know, they use Clojure. Wix, yeah.

Yehonathan Sharvit

No. Wix? I'm very surprised. Maybe they have, you know, a side project in Clojure, but I don't think that something at the core is written in Clojure. I would be very surprised. 

Vadym Kostiuk

I have a question regarding actually your current position at CyCognito. So you're a chief software architect, am I right, within the team?

Yehonathan Sharvit

No, not chief. I'm an architect or infrastructure manager. What we do is interesting.<br/><br/>Okay, let me mention the problem we had and the way we solved it. So we are using microservices architecture. At the beginning, we had no data schema at all. And we have, you know, like business entities that travel around many services. <br/><br/>And when one service modify or change the name of a field, it always cause issues because the team forgot to tell the other team, or they said something and they did another thing, or they misunderstood, or they deployed the version before or after. So there is a lot of issues around data. <br/><br/>And actually, I think this problem is not related to Clojure. It's related to the fact that you want to share data schema between across boundaries, not inside the program. Inside the program, probably if you are using a strongly typed language, it helps inside the program. when you move to another program,<br/><br/>I'm not aware of a way to share a class or a type between programs.

Yehonathan Sharvit

A runtime, it's a way to, you can share your schema to a human, but then you have to implement it in a program. Even if you have, right? You cannot, I don't, can you, or maybe you can generate code from a swagger schema. Okay. Okay. So the way we did it is to, we did something interesting using Mali because Mali allows you to specify your schema as data.<br/><br/>And so we have like one repository for all the data types. And this repository is like a library, so it's versioned. And each service requires declaring this library as a dependency and requires the data types. And then that's how we share data schema between, across boundaries of the programs. And we are pretty happy with that. <br/><br/>What we are not really happy with is inside the program. First of where it works really well is at the API level. Because we have middlewares that allows you to specify, here is the schema for this route, HTTP server libraries for that. So that's great. <br/><br/>But after it has passed the middleware or the handler layer inside the program, we are not really using types. So it's just maps like we like in Clojure. And sometimes it's okay, sometimes it's not okay. But I, so I wanted to mention that the single repo for data types, I think it's an interesting thing. Something that don't really work well is we don't have a way to visualize the types. We don't have a type catalog, it's only code, you know? The integration with Swagger works really well because Mali is well integrated with Swagger. So if you look at the Swagger of our services, you will see JSON schema, although the schema is expressed in Mali, so that's great. But if you ask yourself as a human, not as a program, “What is the schema for an organization?” You have to read code. And because Mali is very smart, it has lots of references to other schemas. So you have to traverse the code, which is not really fun. Or you have to open a REPL and to print the form of the schema. But we are thinking of making tools for schema visualization, schema catalog, and stuff like that. It's doable.

Artem Barmin

We had a previous in one of previous podcasts, the question about using schema inside the code base. And you mentioned that you use it only on boundaries of a system. Why don't you use it inside? Why don't you use it inside the application?

Yehonathan Sharvit

We use it inside, but in a very explicit way and in very, very local. So when we have some critical function, we check explicitly. We write code that checks explicitly. Does this data conform to the schema? And before we send the response of the function, we check that the response fits the schema.<br/><br/>We don't have like synthetic sugar for that, where you declare the types of your schema like you can do in Mali. And also with that, have to, it caused us a lot of performance problems because we are not careful. Because if you don't pre-compile your schema and the function it used at scale, like a thousand times per second, then the schema has a performance issue.

Artem Barmin

Yeah, I can imagine.

Yehonathan Sharvit

So we always have to remember to pre-compile schemas.

Artem Barmin

But Mali has these features of defining functions with schema inline.

Yehonathan Sharvit

We tried and I don't know. And somehow it has an integration with Condor also. But we didn't do the extra mile of really integrating it. We could, but I don't know. It didn't feel, maybe I was lazy or maybe, I don't know. I tried it, but somehow we got rid of that. So we use it explicitly in some key functions.

Artem Barmin

And from this point of view, I want to ask you about team management and code support. Do you have some problems with the refactoring the whole code base wise and...

Yehonathan Sharvit

Before Condo, yes. But since we have CLG Condo, it's really you can traverse your code and refactor very easily. Find usages and yeah, that's really great.

Artem Barmin

Okay, and now I want to dig inside the topic of pushing to migrate from Clojure and you mentioned this before. Can you tell a bit more why this discussion even started in a company? So what were the triggers?

Yehonathan Sharvit

Two things. One that I mentioned is that developers are not happy, or they say they are not happy, or they are jealous of their friends that work at companies where the tooling seems more modern. And the other one is recruiting a concern.<br/><br/>Okay, I'm going to say one thing and the opposite of it. On the one hand, when I hire someone, I don't care what is their language of expertise. If you are a good developer in whatever language, I can work with you. Especially if you have a language that is similar to mine. <br/><br/>But on the other hand, sometimes we do need expertise. And at CyCognito, we have, I don't know, maybe two or three, I don't know, four maybe really Clojure experts, but we need more. Because when we have problems in the language, so 90% of them, you don't have problems in the language. You express your business logic and everything is fine. But when you do have a problem, if you don't have an expert near you, you might get in trouble. And...

Artem Barmin

What kind of problems you mentioned? Problems in the language itself, in compiler?

Yehonathan Sharvit

We have many, many services, and we have also many reports for internal libraries. <br/><br/>A year ago, I was working on a service that requires a library and the library requires another library and it didn't work. I had like a compilation error, wrong number of arguments or the function is not defined. And indeed the function is defined in the latest version of the library. And when I check what is the version of the library that I declared, I see that it's the proper version, the latest.<br/><br/>When I check with line depth three what version I got, I have the correct version. But the compiler says, no, the symbol does not exist. How could it be? I'm knocking my head, knocking my head, knocking my head. And then someone, not me, found the problem. Can you guess what is the problem?

Artem Barmin

I think something with class pass in JVM. Class pass, maybe you had two different versions of a library in the same class pass and one of them were actually...

Yehonathan Sharvit

Yes. But how? How could it happen? How could I have two versions of the library in the class path? You're right. That's what happened. But how? And here is the thing. The reason was because, so remember, I have a service A that uses a library B, and library B also uses library C. But the service uses a library X that also uses library C. The problem was that library B in the middle was not only a library, it had also a CLI inside to run the function from the library, which is nice. You could run from the lane run something or even something smarter. <br/><br/>So the library was also runnable. And by default, when you do that, it's ahead of time precompiled, which means that even when you consume the library as a library, it's precompiled with the dependency inside. But it's hidden from you. You have no way to see what is the version of the library that is.

Artem Barmin

I can imagine that sounds very painful to debug.

Yehonathan Sharvit

Very, very, very painful. And you know, if you are lucky, you ask on Slack and someone will tell you. If you're not lucky, you will not ask, or you will ask on Slack and nobody will help you and you can. So at CyCognito, we are fortunate to have someone that was an expert enough in Clojure and he solved this problem.

Artem Barmin

I would say this is maybe not Clojure-specific, but I would say Java's piece your JVM specific.

Yehonathan Sharvit

Yes, if you are a Java shop, have thousands of Java experts that work for you. If you are a Clojure shop in Israel, there is no Clojure experts available. So that's like a hiring problem.

Artem Barmin

So you mentioned the hiring problem.

Yehonathan Sharvit

Tooling, I would say tooling the developer experiment.

Artem Barmin

Were you planning to move from Clojure? Or are you still thinking how to tackle this problem

Yehonathan Sharvit

So we are right now, we are refactoring to TypeScript one of the services. And actually it's interesting because I discovered that in TypeScript, by the way, TypeScript is, I think it's really an amazing language because it allows you to be statically typed only what you want. You always have a way to say here, I don't care, which is great. But the team that is responsible for the migration to TypeScript, actually they use not the type from TypeScript, but the type from a library in TypeScript called Zod that does type that is very similar to Mali that checks type at runtime. For example, it allows you to parse a JSON string as a type, meaning to check or to ensure that it conforms to the type.<br/><br/>And all of this happened at runtime. So it's kind of, it's a mix of, and I think it's the few, for me, if there is something I don't like in Clojure, or that I would like somehow to improve, is to find a way, and maybe TypeScript is the way. <br/><br/>Because sometimes you do want to be statically typed. There is value. Sometimes you want to write a function, check IP, you want your code to assume explicitly that the data that you get is an IP. And if somehow by static analysis you can detect that there is a path in your code that cause dysfunction and where the IP is not an IP, I want that. And I think that will give you the ability to do this and that. Like to cut the cake and have it too.

Artem Barmin

Yeah, so you plan to move everything to the TypeScript.

Yehonathan Sharvit

No, if the migration to TypeScript is a success with this one experiment, maybe more will move. I don't know.

Artem Barmin

So this is backends TypeScript project running on Node.js.

Yehonathan Sharvit

Both, backend and frontend. So time will tell.

Artem Barmin

So the main reason from what you said, as I understand, is the hiring problem.

Yehonathan Sharvit

Yes, that's the main reason. I think hiring and also I think that some people left the company because of the language. That's my guess. But now I don't know if these people I want to keep them or not to keep them. Maybe it's good that they left.

Vadym Kostiuk

Just a quick question. I'm wondering where you're thinking about finding an expert in Clojure maybe somewhere in Europe or US Because they're like the talent pool is wider from what I see.

Yehonathan Sharvit

Yes, yes, we have that. We are using a bit of it, but we are mostly a non-developer team. It's mostly in Israel. So we have one like that, and that's the one that found the issue, how to solve the issue I mentioned before with the mismatch in the class path. But it's not enough. We want one of these in each team and the...<br/><br/>Because when you have one like that, also it makes the other better. Otherwise, it's not only to solve issues when someone has curiosity and love the language and he's source of knowledge and expertise, then also the level of everybody raises. If everybody has learned Clojure on its own and didn't know Clojure before, somehow it has the problem of everyone becomes average.<br/><br/>I'm not saying that this is what happens at my company, but something like that is happening. It's very hard to bring someone that will push everyone forward.

Artem Barmin

You know, that's interesting because in our company, a lot of people want to switch to Clojure from JavaScript or TypeScript. And I constantly see this passion. Sometimes I run internal courses for our employees and we had some students and there is a plenty of people that actually want to move to Clojure. Maybe because they are tired of usual technologies and they want more fun.

Yehonathan Sharvit

The real question is what will happen to them in a year or two? Will they want to stay in Clojure or will they prefer to go back to...

Artem Barmin

Yeah, yeah. Most of them are staying from what I can see. Yeah, staying maybe five years. The oldest our Clojure developer works in a company for 10 years. He started as a student and I teached him and he's still working in a company and he don't want to switch to other technologies.<br/><br/>I want to ask you a big question: Would you take Clojure as a main technical stack for your product right now if you would start?

Yehonathan Sharvit

Yes, absolutely. No doubt. Yes.

Artem Barmin 

Yes, even after the stories about migration to the TypeScript. Can you elaborate a bit more? That's interesting.

Yehonathan Sharvit

That was not my decision to try TypeScript. That was the decision of my manager. I am not the one that took this decision. I'm a bit sad.<br/><br/>So time will tell, maybe he's right, maybe we can discuss again in a year and I will tell you was this project a success and I don't know. But for me, the problem is that I'm kind of in love with Clojure. So maybe I'm not rational also. I don't know. I'm not sure.<br/><br/>Back to my book, I even don't know what I want, because on the one hand, the purpose of my book is that people can adopt Clojure principles without adopting Clojure of the language. But on the other hand, I really want people to adopt Clojure the language. So I'm kind of schizophrenic, I'm split from inside.

Artem Barmin

You would say it one more time to start a new project. Yehonathan Sharvit<br/><br/>I would take it for sure, yes. I would take it for sure. I would take it for sure.

Artem Barmin

Yeah, I see. And you know, I have this specific question to your podcast and I want to ask you, how do you as a specialist in cyber sec, how do you think?<br/><br/>This stability of a Clojure ecosystem is a good thing for security of Clojure  products. Or it's a bad thing because a lot of libraries became abandoned. Some dependencies became abandoned and maybe supply chain attacks can arrive from this situation.

Yehonathan Sharvit

I discovered also last week a bug we had in production because some of the... Not in production, in local environment. Because one project uses an old version of MongoDriver, and actually, we wrote our own Mongo driver. But this one uses Monger, like the famous driver for Clojure. And something didn't work around the JSON manipulation. And in order to debug this problem, I realized that there was inside the dependency tree or inside the call stack, I mentioned of Clojure Works. And I never heard about this project. I never used this library. And I was curious, where did it come from? And it was brought by Monger. And Clojure Works has this amazing, very scary feature is that it detects whether you use, in your project, you Clojure DataJSON or Shasher. And then it rewrites the implementation of write string so that it works fine for dates. <br/><br/>And all of this happened under the hood. And you will never see any direct call to this piece of code. It's the protocol extension dynamic thing that you can do is Clojure, which is amazing when it works. But when it doesn't work, you can knock your head until you find where does it come from.<br/><br/>Because nowhere in the code, nobody mentioned ClojureWorks. And also, ClojureWorks is 12-years old. It has not changed for the last 12 years, or at least this part of the project. So it's a very old thing. Probably it works. But somehow, ClojureWorks, Shasher, DataJSON, some version of all these matched together caused a very weird problem.<br/><br/>Until now, I'm not sure what is the problem, but it caused the problem. And when I upgraded or downgraded, remember, or removed one of the dependency, then it worked.<br/><br/>Is it good or bad that Clojure Works has not been touched for 12 years? What does it mean? Does it mean it's perfect? Does it mean it's deprecated? Does it mean it has bugs? Does it mean it has security holes?

Artem Barmin

You tell a lot of scary stories. As a developer, I can only imagine how to tackle such problems because it's impossible to find these parts of the code that modify other libraries and this extensibility, monkey-patching and everything else can lead to very hard situations.

Yehonathan Sharvit

Something I really like about Clojure, let me mention dependency, is the fact that when you declare a dependency version, you declare it precisely. And you don't have, like in NPM, greater than 5.2 or 5.2, almost 5.2. And you don't have the packaged JSON and packaged JSON log. I think it works well for us.

Artem Barmin

Yeah, that's kind of bugs that hard to catch with any with any...

Yehonathan Sharvit

Hard to catch, hard to debug… 

Artem Barmin

We can move to the last part of our podcast is the chain of questions and we have a question for you from our previous guest, Jeffy. He is Leadback and engineer at HolidayPirates and he asked, “How do you handle state management enClojure? How do you ensure that state remains contained and doesn't spread across the code base, especially in large system?”<br/><br/>And I can add from myself, from what I understand from our previous discussion, that's the question about the side effects and mutability. So for example, how to isolate this pure part of the application from the parts of the application that have side effects and how to keep them kind of pure separation.

Yehonathan Sharvit

Actually, there is two or three chapters in my book about that, about, yes, about state management. When I started to think about the book, I thought that, you know, data immutability is great when you do mathematics, when you have to do just calculation is great. <br/><br/>But when you have to do real stuff, and real stuff usually involves state management, then it becomes a problem. And you have to find ways to, because state is mutable, state changes, and how to express state changes with immutability. But in fact, yes, it's chapter four in the book. In fact, I realized that it's exactly the opposite.<br/><br/>Immutability is the best way to manage state. It's not that it's okay, it's great when you have no state and you have a state it's a problem, it's the opposite. Of course when you don't have state it's great, but when you have state it's even greater.<br/><br/>Let's take, for example, game, a chess game. Every time you move a piece, the state changes. But the fact that you have moved the piece, it doesn't mean that it doesn't change the fact that a movement ago, the state was this. So actually the state is like a succession of snapshots and the snapshots are immutable. So you have snapshot A, snapshot B, snapshot C.<br/><br/>The state is just a pointer to one of these snapshots. That's the state management. State management is to decide across all the values and imagine you do undo, redo, backdo, et cetera. You could have a tree of snapshots, which is not that easy to manage. And the state is only a point, a decision about managing the pointer, exactly like in Git. In Git, everything is immutable, all the commits are immutable, but a branch is a pointer to one of the commits in the tree. So it's exactly the same. And the fact that Git is immutable is an advantage for Git to manage the state. It's not a drawback.

Artem Barmin

That's true. But that's true only for the state that you manage inside the Clojure application. But if we talk about side effects like updating the database, for example, Postgres or MySQL, we can't think about state in this way because it mutates.

Yehonathan Sharvit

No, but if you do that, the state is managing the database, then you don't care. The thing I’ve also discovered in the book, that state management is usually more relevant in the front end than in the back end.

Artem Barmin

Yeah, I agree.

Yehonathan Sharvit

The database manages this problem. But in the front end, you don't have a database to manage the internal state of the app.

Vadym Kostiuk

What question would you like to leave to our next guest?

Yehonathan Sharvit

Okay, so I thought about four questions. One, two, three, four. I give you the choice of the question. So give me a number between one and four.

Artem Barmin

Three.

Yehonathan Sharvit

What's the part of Clojure the language aside for macros that you consistently find challenging to explain to a teammate?

Artem Barmin

Very, very interesting. I think we can add this question to the list of our usual questions in the podcast.

Vadym Kostiuk

Later on we will ask you to share the other three questions. Perhaps we'll be asking them regularly.

Artem Barmin

It's been a pleasure to have you on our podcast today. Thank you for a very, you know, controversial kind of topics that you've covered about moving from Clojure, about all these problems, bugs and all this stuff. These details that we are looking for in our podcast. And I am really pleased that you've shared everything with us. And thank you very much.

Vadym Kostiuk

And to our audience, if you're curious to learn more about Jonathan's work, about his books, and about CyCognito, we'll be including links to all of that below this video. So please make sure to check them out.

Artem Barmin

And thank you, Yehonathan, and thank you to the audience and see you next time.

Vadym Kostiuk

Yeah, until next time. Thank you. Bye.

Clojure in Product.
Would you do it again?

I truly discovered the enlightenment with Clojure, with Yehonathan, CyCognito

Episode 06
January 20, 2025
58 min

The sixth episode of our podcast is here! Join us in speaking with Yehonathan Sharvit, author of "Data-Oriented Programming: Reduce Software Complexity." This episode is packed with inspiration and practical advice from Yehonathan, who currently works in cybersecurity, where Clojure plays a crucial role in CyCognito's tech stack. We hope you find valuable insights for your product's journey in our podcast.

Yehonathan shares his transition from C++ and Java to Clojure, highlighting some challenges he experienced early in his career with C++ and MATLAB. We explore how Clojure's elegance and productivity reignited his passion for development, inspiring him to write books and create tools to simplify language learning.

We also emphasize the importance of having seasoned Clojure engineers on a team to navigate complex challenges and enhance junior developers' understanding of this technology.

Watch more episodes