My name is Ilya Dozorenko and I am Clojure Lead at Freshcode, one of the IT-companies with expertise in software development in Clojure. I've been working as a programmer for over 11 years and during this time Clojure was probably one of my brightest discoverings. In this article, I want to share with you the reasons that formed my opinion. Also, you'll find here some real-life cases and tips for beginners on how to start with Clojure.
History of Clojure
Rich Hickey
Clojure was invented by <medium>Rich Hickey, a geek-programmer and kind of a cult figure<medium> among the Clojure community. Before realizing creating Clojure, he taught C++ at New York University and developed information systems, mapping real-world processes and information to regular models.
For instance, in music scheduling, trying to decide: whether artists have songs or songs have artists. In 2005 he took a sabbatical for working on personal projects, and after 2 years the first version of Clojure appeared.
Today <medium>Rich Hickey and Cognitect provide commercial support for Clojure.<medium>
Rationale, Reasonability
Here is Rich Hickey's brief overview of the reasons for creating Clojure:
{{rr25-1="/custom-block-to-blog/two-page"}}
You can read the detailed story in the article "History Of Clojure" written by the language creator.
<medium>Clojure has much fewer new and unique approaches than you might think<medium>. Of course, it seems to be quite an experimental language compared to Java (to say the least). But in fact, Rich Hickey has designed language specifications based on time-tested ideas and concepts only.
<medium>Clojure is a dialect of Lisp. In turn, lambda calculus is the core of Lisp with its first-class and high-order functions<medium>. In other words, functions are treated as values, they can receive functions as parameters, and return functions as a result.
Lisp was designed around the idea of code as data. The fundamental unit of code in Lisp is the s-expression (symbolic expression):
<medium>(A (B 3) (C D) (()))<medium>
<medium>S-expression<medium> is a list composed of values and other lists. S-expressions are written in prefix notation where the first element is commonly an operator or function name and any <medium>remaining elements are treated as arguments<medium>, eg (+ 1 2 3).
A program in Lisp is isomorphic to its abstract syntax tree (AST). This property is called homoiconicity, and it creates new opportunities for metaprogramming, in fact, it allows Lisp program code to be easily modified on-the-fly.
Clojure also brings to Lisp some major innovations related to better adaptation to real-world tasks. For example, it supports<medium> new data structures<medium> (maps, vectors, sets) in addition to lists.
Shortly after the initial release, the concept of macros was added to Lisp. It provides an additional macro-expansion phase to the interpretation of Lisp programs. Macros provided a faster path for programmers to extend Lisp.
However, the power of Lisp is also its curse (The Lisp Curse), which resulted in a fragmented Lisp community, a surplus of implementations, and a lack of unified standards.
Since Lisp creation, <medium>a lot of dialects have appeared with the most popular being Common Lisp<medium>; nevertheless, they didn't gain much popularity when compared to modern script languages like Python or Ruby. Thereby the important goal of Clojure is the popularization of Lisp, and it is well achieved by using the JVM platform and supporting the programmers' community.
Statistic of using Clojure
According to the Clojure 2020 survey results, last year we saw growing use for work and enterprise applications in particular and an ever-evolving community of users.
<medium>Clojure language is steadily attracting newcomers<medium> — 15.78% of the interviewees (393 persons of 2491) started to use it for the first time in 2020. So, learning Clojure seems to be a timely idea and promising trend in the software development field.
In the foreword of "Clojure Applied" Russ Olsen says that for most people the <medium>process of learning Clojure proceeds in three stages<medium>.
On the initial stage you are <medium>studying the fundamental syntax and principles<medium>, i. e., how to deal with parentheses, what are square brackets needed for, what is the difference between a list and a vector, etc.
The middle stage is when you learn <medium>how to fit everything together<medium>, for example how to assemble all high-order functions into working code, or how to work with immutable data structures.
Finally, once you have an understanding of the language you enter the third stage and start <medium>exploring the Clojure ecosystem<medium> — libraries and applications, using your new knowledge to work out what other people have built.
And that's when the real fun begins!
Introduction to Clojure: getting started
<medium>Selenium Tests<medium>
I first met Clojure in 2013 while I was working on a Java EE project (> 300K LOC) in the telecom sector. The main web application has just been migrated to Java 7, and it included try-with-resources and NIO but still, there were no arrow functions, <medium>Java Stream API or jshell<medium>. It's no secret <medium>Java is considered a programming language with quite a verbose syntax<medium>. IDEs provide some assistance to hide this verbosity with varying degrees of success, and so do the new APIs and syntactic sugar in the latest Java versions. Java 16 is the actual version to date. However, according to JetBrains <medium>3/4 Java-developers were using Java 8 in 2020<medium> regularly. More than 80% of production applications used it, in accordance with New Relic research.
Java is also not quite suitable for script tasks. That's why <medium>Clojure was proposed for covering the application with Selenium tests<medium>, as well as for a bunch of other reasons:
stable and familiar JVM platform
integration with Java — the ability to reuse code and avoid duplications
REPL and interactive testing without a need to recompile code
simple syntax (for instance, compared with Scala or even Java) which might allow QA engineers to edit test cases by themselves
Opinions were divided regarding the last point. There were concerns that shifting to a functional programming paradigm would be difficult even for experienced developers. But the <medium>team was full of enthusiasm, and the management was tolerant to innovations<medium>, so it was decided to take a risk and try Clojure.
Firstly, helper functions were developed using clj-webdriver in order to navigate through a website and control misc UI components. Then work on the actual test cases started.
REPL benefits hardly can surprise anyone today, but it made a splash at the time. <medium>Test-cases were written on-the-fly directly from the console, with no need to recompile or restart a script, and all this ran on the JVM<medium>. Unbelievable!
Here is an example of one of the test cases:
Final test-cases in Clojure were quite easy to read and looked familiar and... imperative! DSL was covering up to ¾ cases written by QA engineers. For other cases, we had to involve developers. At the time I had a minor role which was assisting QAs with writing helper functions.
The first stage of learning Clojure is quite simple thanks to its minimal syntax. The <medium>fear of parentheses that affects beginners is highly exaggerated<medium>.
There are <medium>paredit plugins<medium> for all major editors and IDEs. They help to keep parentheses balanced (adding parens in pairs and avoiding accidental deletion) and also support structural editing (keyboard shortcuts for manipulating s-expressions).
Structural editing can be applied to Clojure code:
Java interop is straightforward, you just need to remember a set of syntax rules, and get used to the fact that function calls stay first in a list.
For example, this Java code
can be represented in Clojure as follows
By the way, Clojure offers wrappers for most Java libraries and modules. So you can rewrite the same code without calling Java, just by using byte-streams:
As for integration, it is worth mentioning that the whole JVM toolkit will work in Clojure. For example, you can use <medium>YourKit<medium> for profiling Clojure apps too.
So, the first stage is passed, syntax and key structures are learned. But mind you, dear reader, <medium>when you "get hooked" on Clojure it's hard to stop<medium>, and you will keep delving into it!
Intermediate phase
<medium>Collections of problems<medium>
Usually, the second stage of studying is the most difficult. I got to tell you, that <medium>switch from imperative Java style to Lisp may be quite challenging<medium>. Despite its minimal syntax, I puzzled over the different structures initially.
That is where <medium>a large number of resources and books, as well community support will help you a lot<medium>. With its elegance and simplicity, Clojure is similar to the puzzle game, <medium>something like Rubik's Cube<medium>. It confuses you with its complexity but it's a big pleasure to understand how it works.
You can find more advanced learning resources, with gamification of the process, beautiful achievements, and real-world tasks. But as for me, the main motivation was the interest in working with Clojure, which forced me to look for answers and analyze them.
<medium>Functional approach<medium>
The first thing you notice while creating a data model in Clojure is the <medium>lack of familiar OOP.<medium>
In Java, we use class fields to describe data and class methods to describe logic. <medium>Clojure functions don't belong to data<medium> (as opposed to class methods in Java), <medium>they process data<medium>. A basic encapsulation unit for function definitions is a namespace that usually corresponds to a single file. Clojure allows using impure functions but encourages a functional approach by providing high-order functions (e.g., map, reduce, filter, remove for working with collections) and function compositions (partial, comp, juxt).
<medium>Data-oriented approach<medium>
In this piece of lecture «Clojure, Made Simple» Rich Hickey explains the benefits of a data-oriented approach in Clojure.
In Java data is usually accessed through accessors/mutators or simply getters/setters. <medium>If you've ever worked with large POJOs<medium> you know how tedious it could be to create lots of boilerplate for accessing your data or searching for appropriate methods to access private properties. Well, IDEs indeed provided <medium>autocomplete to simplify the search<medium>, and at some point, <medium>boilerplate autogeneration<medium> became possible with Project Lombok plugin; a similar approach (Records) was implemented not so long ago in Java 14.
However, we are talking about <medium>essential libraries which are still in use<medium>. For instance, take a look at the methods of javax.servlet.http.HttpServletRequest class:
Interfaces for accessing the <medium>3 different fields here are represented by different colors<medium>. Notice how specific they are. Different naming logic — getParameterMap vs getHeaders, methods remove and set are used for the attributes, while they are not available for parameters, etc. In addition to a <medium>cumbersome interface<medium>, we get an<medium> inability to reuse data manipulation logic and complexities with creating test stubs<medium> (remember Builder pattern). Clojure contrasts this with the data-oriented approach where objects are just maped with public access to its values:
The idea here is that you may <medium>apply generic sequence manipulation functions to any data<medium>, for example to the map above, or to the map of headers, or to any other collection.
There are dozens of functions, but it's not hard to learn them. These <medium>functions are more general and more valuable<medium> because you can apply them more broadly.
<medium>Threading macro<medium> -> allows to rewrite function calls in a more convenient way resembling the Stream API in Java.
Using a small set of data structures together with a functional approach <medium>facilitated the data manipulation process<medium> immensely. And, ultimately, it improves the programming process, including program readability as a result.
<medium>Immutable data structures<medium>
Clojure data structures are immutable and persistent. This means that each "modification" operation yields a new collection. With that in mind, algorithmic complexity of operations doesn't change significantly (e.g., accessing values of vector or hash-map is O(log32(N)), memory consumption is also near-constant because each successive version of a data structure shares data with its source.
The <medium>concept of object equality takes on a new meaning<medium> — if two structures are equal, then they would be equal always, and not just at a certain point in time.
This approach greatly facilitates working with multithreaded apps. You don't have to remember to use thread-safe versions of collections. Your only real option to store the application state is to use <medium>mutable containers which are thread-safe<medium> (Var, Atom, Agent, Ref). Clojure supports <medium>transactional modifications of Refs<medium> via software transactional memory (STM) system, but that's a subject for another article.
<medium>Lazy evaluations<medium>
Clojure supports <medium>lazily evaluated sequences<medium>. This means that elements of the sequence are not available ahead of time and produced as the result of a computation (also "evaluation" or "realization"). "Lazy function" is a term used to describe a function that returns a lazy sequence. We can use lazy sequences to describe infinite sequences. A case in point is Project Euler Problem №10 — finding the sum of all the primes below two million.
Definition of prime? has been omitted here for short. In fact, primes defines whole set of prime numbers, because iterate and filter are lazy functions. primes sequence won't be evaluated until we access collection values (also, if you enter primes in the REPL, it will stop responding on the evaluation stage).
Take-while is also a lazy function, it's only reduce that <medium>finally evaluates the sequence and returns the sum<medium>. Of course, the mentioned problem can be solved without lazy infinite sequences, and you can choose another approach, for example, using for. But let's agree, this one above looks elegant!
{{rr25-2="/custom-block-to-blog/two-page"}}
Advanced level
<medium>Web development in Clojure<medium>
After a while, I had a chance to try Clojure in web development, and thus proceed to the third phase of studying, which is exploring the Clojure ecosystem. Ring is a web development standard in Clojure; its analogs for Python and Ruby are WSGI and Rack respectively. Ring repository contains:
specifications of request/response/middleware and base code to work with them
basic middleware, an adapter for the Jetty web serverdocumentation
documentation
All modern Clojure frameworks ensure compatibility with the Ring standard. This enables running applications on the different platforms without applying any changes (Jetty, http-kit, Immutant, and others).
At the time the <medium>ease of using Ring has become almost a revelation to me<medium>. The idiomatic approach around the concept of Middleware based on functions is used in many programming languages and frameworks today (for example, Express in NodeJS).At the time I had several years of experience in Java Servlets and Spring. Therefore, I primarily compared the middleware approach to Servlet Filters and Spring Boot. For instance, compare this example of a basic REST-service built with Spring to the equivalent code written in Clojure.
My first Clojure-based REST API almost looked like the above example; the only thing, I used a compojure library instead of a more advanced compojure-api. <medium>Combined with REPL, Ring facilitates turbo development<medium> of applications.
Compojure is a routing library for Ring which allows describing handlers in form of an easy-to-read list of routes:
The great benefit of Hiccup is that it uses plain old Clojure data structures (vectors and maps), and they can be manipulated with the same Clojure functions, with no need to use any special syntax like enclosing expressions in curly braces.
The same format is used by Reagent, an analogue of ReactJS and one of the most popular ClojureScript libraries.
<medium>Web frameworks<medium>
The Clojure community prefers to use <medium>custom compositions of popular libraries instead of classic «all-in-one» frameworks<medium> for facilitated web development and more flexibility. Nevertheless, different types of popular templates and more advanced frameworks are available for Clojure web development:
1. compojure-api — compojure "on steroids" for working with RESTful-services; classic compojure was 'enriched' by clojure.spec support and Swagger, as well as with other options.
2. Liberator — library for working with RESTful-services, using an approach similar to Erlang's webmachine. It represents data with the concept of resources. Resource is basically a request handler, which is invoked after request has passed the decision graph. Such a strict approach ensures compatibility with HTTP RFC 2616 standard.
3. yada — like Liberator, it provides compatibility with HTTP-standard. It includes Swagger, Websockets (aleph), and other services. It's interesting that yada is generally compatible with Ring, but doesn't use Ring Middleware, offering a data-driven approach.
4. Pedestal — web framework by Cognitect, quite powerful (besides the documentation), but also opinionated (e.g., it uses its own router). Pedestal has a high degree of optimization and compatibility with web servers (for example, the ability to deploy to AWS Lambda).
5. luminus — project template with a specific set of libraries for working with REST services (Reitit, Swagger), Websockets (depends on the selected server), DB (HugSQL + Migratus), templating (Hiccup, Selmer), i18n (Tempura), etc. Luminus is actively supported today.
6. Jetty (adapter provided by Ring) and http-kit — as basic web servers for Clojure applications, support working with websockets too. They are standalone-libraries, i.e. they will be packed into the jar file with the application
7. Immutant — a suite which includes Undertow web server and libraries for messaging (HornetQ), caсhing (Infinispan), and scheduling (Quartz). It can be used as a standalone-library, as well as inside a WildFly container to enable clustering. Immutant is supported by JBoss Community.
The current list of the categorized libraries is available in Clojure Toolbox.
<medium>Extensibility<medium>
Clojure's extensibility plays a significant role in the process of software development because plenty of popular features of the language were implemented not in the core but in external libraries via macros.
Thus, for example, <medium>pattern matching<medium> was implemented in the core.match library. You can match any Clojure data structure:
… or extend match for new patterns and data types:
core.async is a library for async communications via queue-like channels and using thread pool & inversion of control patterns. It's close to channels in Golang; the main difference is that Clojure channels are not embedded into the language core, but provided through a library.
Go and go-loop blocks (the last one combines go macro and loop function) define a code block that will be executed asynchronously in the separate thread and returns control immediately.
Function <! parks a thread inside the go-loop block, i.e. when a message is received, the loop-worker handler is invoked, then the block execution stops waiting for the next message.
Blocking function >!! passes strings to the log-chan channel, which are printed to console by loop-worker.core.async maintains a default fixed-size thread pool (8 threads) for the go blocks.
core.async is actively used in ClojureScript too. cljs implementation contains no blocking operations and provides an idiomatic Clojure approach to solve the callback-hell problem.
<medium>clojure.spec and Dynamic typing<medium>
Clojure primarily owes its dynamic typing to Lisp. Rich Hickey also is a supporter of using dynamic typing by default. For years, there have been long and complex debates about typing, thus I describe here only my personal, maybe quite subjective expressions. Indeed, using dynamic typing it's easy to shoot yourself in the foot, but on the other hand, shooting becomes much easier per se :) One might say that this is a more pragmatic and result-oriented approach.
Lack of the «static» is offset by the ability to test separate components as well as the whole system "on the fly" in REPL, and also by spec'ing data with core.spec. It makes use of a data-oriented approach by <medium>representing data schema with Clojure data structures<medium>. Clojure makes a bold hypothesis that usually you will have no need to track types from start to finish of the function chain. Instead, specs can be leveraged to describe API, validate a web form, or even for a test-data generation. In the example below spec is used for the request/response validation and type conversion, as well as for the generation of the swagger docs (for further information, see the full example):
After several years of using static typing in Java, at first, I was feeling insecure while working with the dynamic information model in Clojure. Of course, type check errors and intellisense are certain benefits of static typing. But for my money clojure.spec <medium>really helps with the everyday cases of web app development<medium>. It's also true that sometimes there is a reasonable need in adding static typing to a complex algorithm or a specific module. Here Clojure extensibility again comes in handy, take for example core.typed library. It provides an optional type system for Clojure that can be added either to the whole program or to a selected piece of code.
Other Clojure options
Clojure is actively used not just in web and mobile development, but in the Data Science и DL/ML sectors (incanter, scicloj). I haven't had a cause to work within these fields yet, but I appreciated Clojure's elegance during my work with DSL.
DSL
DSL (domain specific language) is a language with a higher level of abstraction, optimized for a specific class of problems.
In Clojure DSLs are usually described with data structures, take for example Hiccup or honeysql:
Context-free grammars provide a more customizable way to define a DSL.
Let me give you a real-life example. Once I had the task of implementing an "advanced" filtering mode for the GUI similar to searchkit. This mode would extend the limited capabilities of GUI filters by using raw SQL expressions and allow using complex logical constructions.
For example, in GUI all the filter conditions could be combined solely by a conjunction (AND) or by disjunction (OR). Thus it was impossible to use both of them in the expression or re-group expressions:
<medium>condition_1 AND (condition_2 OR condition_3).<medium>
<medium>The problem was that the ElasticSearch mapping was quite specific<medium> in order to implement a flexible data schema, and ElasticSearch SQL JDBC driver wouldn't map SQL queries to the correct ElasticSearch queries.
That's why the team faced the<medium> task of parsing SQL query and transforming it into an isomorphic ElasticSearch query<medium>. It was also required to implement error highlighting and autocomplete of the query elements (e.g. reserved keywords, identifiers) based on the context.
For example, display the dropdown list with available columns or operators that can be used with the specific data types — LIKE operator for the strings and BETWEEN operator for the numeric fields.
parse SQL string into the abstract syntax tree (AST)
convert AST into the custom ElasticSearch query.
Instaparse library was used at the first step. It was designed for turning Backus-Naur form (BNF) notation for context-free grammars into an executable parser that takes string expression as an input and produces a parse tree (AST) in the Hiccup format (in fact, nested Clojure-collections).
Every node of the resulting parse tree also contains useful metadata (e.g., mapping to a line and a column of the input string), that can be used by the autocomplete implementation. Instaparse provides detailed reporting of parsed failures, which includes a line and a column of the error term, as well as a list of terms expected by the parser.
The second stage could be implemented by traversing the AST and applying pattern matching to each form.
Here is a simplified gist for parsing and executing arithmetic expressions to demonstrate this approach. The difference with the above-mentioned case is that we convert the math expression AST directly into the Clojure code and evaluate it straight away.
ClojureScript
At the moment JavaScript is the predominant technology in browsers, and one of the most widely used in mobile devices.
ClojureScript is a compiler of Clojure into JavaScript. It uses Google Closure's tools and minification compiler. The term ClojureScript is also used as the name of the language (cljs for short)
In fact, cljs retains all Clojure advantages such as data structures, functional approach, macros, etc. while offering a production-ready ecosystem and set of libraries; only the nuances of interaction with the platform differ.
Using a bundle of Clojure backend + cljs frontend facilitates development by using a single language and sharing common code. You can get seamless integration with npm packages by using shadow-cljs compiler.
Like any other technology, Clojure has its trade-offs.
Persistence structures in Clojure are less efficient than their analogs in Java. There are some tips and tricks of how to optimize performance, for example, you can use transient versions of vector, map, and set, which are more efficient, but not intended for concurrent access. Also, you can use all host-platform abilities in Clojure (i.g., data structures and Java libraries), but typical Clojure code will probably be slower than typical Java or Scala code. It's not crucial to most applications, but it can be a decisive factor when choosing a technology for a specific high-performance system or module.
Clojure beginners often mention the quite steep learning curve, but usually, it relates not to the functional approach itself, but to work out the common practices or, say, to interact with a hosting platform. <medium>Take your time, and with some practice, it should not become a problem.<medium>
Taking all that into account, I think that one of the<medium> most serious Clojure shortcomings is marketing<medium>. Clojure is a simple language with minimal syntax, a well-conceived ecosystem, and high potential. You really will be pleasantly surprised! However, its "public image", probably inherited from Lisp, is usually associated with excessive complexity and lack of standards, although things are the exact opposite. As a result, newcomer developers sometimes hesitate to invest in Clojure (Script) learning, preferring more popular alternatives, although, in fact, they could benefit more from mastering Clojure.
{{rr25-3="/custom-block-to-blog/two-page"}}
Tips for newcomers: how to learn Clojure
Three Pillars of Clojure mastery
1. <medium>Practice<medium> — such multi-purpose advice for every technology. If you're a beginner in functional programming, start with upgrading your skills on specialized resources or using collections of problems like Advent of Code or Project Euler. At the time I used Clojure for a couple of my pet projects which resulted in a strong learning boost.
2. <medium>Learn from others<medium> — ask for help from the community or consult literature оn Clojure and Lisp (there are some resources in the following section). You will certainly find some crazy (but in a good way) and inspiring things.
3. <medium>Watch Rich Hickey's talks<medium> — he reasons about Clojure features and tech solutions in clear and simple terms. It is a huge source of inspiration for clojurians.
{{rr25-4="/custom-block-to-blog/two-page"}}
Links that may come in handy
Here are some useful links that may come in handy when looking for <medium>Clojure tutorials and how-tos.<medium>
Clojurians Slack Clojure-community with content for newcomers (beginners, clojure, clojurescript) and job search (jobs, remote-jobs)
/r/ClojureClojure subReddit with announcements and discussions of libraries/lectures, etc.
Clojure Toolbox Up-to-date handbook of Clojurescript-libraries
Rich Hickey fanclub Set of video-lectures & interviews about Clojure by Rich Hickey
Brave Clojure Jobs One of the actual Clojure jobs lists for Clojure guru which you will definitely become soon (or already have become:))
— this book is intended to level up your skills by taking advantage of Clojure's powerful macro system.
Takeaway
Clojure is a <medium>general-purpose language<medium> for full-stack web & mobile development, as well as for data science, scripting & creating DSLs, etc.
Clojure is an attractive functional programming language for newcomers and experienced software developers (including Java-specialists) looking for the <medium>balance between performance and fun factor.<medium>
Being a Lisp dialect, <medium>Clojure is appreciated by geeks and talented developers<medium> who like to face challenges and overcome them in unusual, but optimal ways. But Clojure's role is more global — it was created to simplify the development, and, in my opinion, it's achieving it perfectly well.
