Clojure: how to stop worrying and fall in love

History of Clojure

Rich Hickey

Clojure was created by Rich Hickey, a geek-programmer and a cult figure among the Clojure community 🙂 Before creating Clojure, he taught C++ at New York University and developed information systems, mapping real-world processes and information to regular models. In 2005, he took a sabbatical to take time to his personal projects. After two years, the first version of Clojure appeared.

Today, Rich Hickey and Cognitect provide commercial support for Clojure.

Rationale, Reasonability

Here is a brief overview by Rich Hickey outlining the reasons for creating Clojure:

Why did I write yet another programming language? Basically, because I wanted:
- A Lisp
- for Functional Programming
- symbiotic with an established Platform
- designed for Concurrency

Rich Hickey

Creator of the Clojure programming language

You can read the detailed story in the article 'History Of Clojure'.

Clojure has much fewer new or unique approaches than you might think. Of course, it seems to be, to say the least, a pretty experimental language compared to Java. But in fact, Rich Hickey has designed language specifications based only on time-tested ideas and concepts.

Clojure is a dialect of Lisp. In turn, lambda calculus is the core of Lisp, with its first-class and high-order functions. Functions are treated as values; they can receive functions as parameters and return functions as a result.

Lisp was designed around the idea of code as data. The fundamental unit of Lisp code is the s-expression (symbolic expression):

(A (B 3) (C D) (()))

S-expression is a list composed of values and other lists. S-expressions are written in prefix notation where the first element is commonly an operator or function name, and any remaining elements are treated as arguments, for example (+ 1 2 3).

A program in Lisp is isomorphic to its abstract syntax tree (AST). This property is called homoiconicity, and it creates new opportunities for metaprogramming. In fact, it allows Lisp program code to be easily modified on the fly.

Clojure brings other new things to Lisp for better adaptation to real-world tasks. For example, it supports new data structures (maps, vectors, sets) in addition to lists. A little later, the concept of macros was introduced into the implementation of Lisp as an additional phase of macro-expansion, involving the transformation of code before interpretation. Macros significantly simplified the extension of Lisp syntax for developers.

In my opinion, one the key goals of Clojure is to popularize Lisp, primarily by leveraging the JVM platform and the support of the Clojure community.

Clojure statistics

According to the Clojure 2020 survey results, last year we witnessed the growth of Clojure usage and its community.

Clojure language is steadily attracting newcomers — 15.78% of the interviewees (393 persons out of 2491) started using it for the first time in 2020. So, learning Clojure seems to be a timely idea.

You can find more insights in our previous article about Clojure.

Three stages of learning Clojure

In the foreword of 'Clojure Applied', Russ Olsen mentions that, for most people, the process of learning Clojure unfolds in three stages.

In the initial stage, you are studying the syntax and fundamental principles: how to deal with parentheses, what is the difference between a list and a vector, etc.

The middle stage includes learning how to fit everything together, for example, how to deal with high-order functions or how to work with immutable data structures.

Finally, you enter the third stage and start diving deeply into the Clojure ecosystem.

And this is where the real fun begins!

Introduction to Clojure: getting started

Selenium Tests

I first met Clojure in 2013 while working on a Java EE project (> 300K LOC) in the telecom industry. The main web app had just been migrated to Java 7. It included try-with-resources and NIO, but still, there were no arrow functions, Java Stream API, or jshell.

It's no secret Java is considered a programming language with quite a verbose syntax. IDEs helps to deal with this verbosity with varying degrees of success, as do the new APIs and syntactic sugar in the latest Java versions (Java 15 is the actual version to date).

However, according to JetBrains 3/4 Java developers were using Java 8 in 2020. More than 80% of production applications used it, according to New Relic research.

Also, Java is not very suitable for script tasks. That's why we chose Clojure to cover the application with Selenium tests. Here are other reasons why Clojure:

• stable and familiar JVM platform
• integration with Java — the ability to reuse code and avoid duplications
• REPL and interactive testing without a need to recompile code
• simple syntax (compared with Scala or even Java), allowing QA engineers to edit test cases by themselves

Opinions differed on the last point. Some stakeholders had concerns that shifting to a functional programming paradigm would be too difficult for developers. But the development team was enthusiastic, and the management accepted the decision to try Clojure.

Clj-webdriver was chosen for developing helper functions to navigate through a website and control misc UI components. Then the work on the test cases began.

REPL benefits may not surprise anyone today, but they made a splash at the time. Test cases were written on the fly directly from the console, with no need to recompile or restart a script, and all this ran on the JVM. Unbelievable!

Here is the test case example.

Final test cases in Clojure were greatly readable; they looked familiar and... imperative! DSL covered up to ¾ cases written by QA engineers. For other cases, we involved developers. At the time, I had a minor role, which was assisting QAs with writing helper functions.

The first stage of learning Clojure is pretty simple, thanks to the minimal syntax. By the way, the fear of parentheses affecting beginners is highly exaggerated.

There are paredit plugins] for all major editors and IDEs. They help keep parentheses balanced (adding parens in pairs and avoiding accidental deletion) and support structural editing (keyboard shortcuts for s-expressions).

Structural editing can be applied to Clojure code

as well as to Hiccup-style markup.

Interacting with Java code is straightforward, but it's essential to remember a few syntactic rules and get used to the fact that function calls must comes first in a list.

For example, this Java code

can be represented in Clojure as follows.

Clojure offers wrappers for most Java libraries and modules, so you can rewrite the same code without calling Java, just by using byte-streams.

As for integration, it is worth mentioning that the JVM toolkit will work in Clojure. For example, you can use YourKit for profiling Clojure apps as well.

Intermediate phase

Collections of problems

Usually, the second stage of studying is the most difficult. I've already mentioned that switching from imperative Java style to Lisp can be challenging. Despite its minimal syntax, I was initially puzzled over the different structures. This is where a number of resources, interactive tutorials, and community support come to the rescue.

With its elegance and simplicity, Clojure is similar to the puzzle game, something like the Rubik's Cube. It can initially confuse you with its complexity, but it's a big pleasure to understand how it really works.

Such resources as project euler, 4 clojure and advent of code gave me a lot of food for thought and helped complete this level. You can find more advanced educational resources, with gamified processes and real-world tasks. But as for me, the main motivation is the interest in working with Clojure, which forced to look for answers and explore new examples.

Functional approach

The first thing you notice when creating a data model in Clojure is the lack of the familiar OOP.

In Java, we use classes to describe data and class methods to describe logic. Clojure functions don't belong to data (as methods belong to a class in Java); they process data. Namespaces, corresponding to individual files, are considered the unit of encapsulation.

Clojure allows using impure functions but it encourages a functional approach, including the use of high-order functions (such as map, reduce, filter, remove for working with collections), as well as and function composition (partial, comp, juxt).

Data-oriented approach

While in Java web development the declarative coding style is supported by Spring and IoC, in Clojure, declarativeness is partially owed to the language features (homoiconicity, immutable structures, macros), and partially to the community that actively encourages a data-oriented approach in libraries.

In the lecture «Clojure, Made Simple», Rich Hickey explains the benefits of a data-oriented approach in Clojure.

In Java, data is usually accessed through accessors/mutators or getters/setters. If you've ever worked with large POJOs, you know how tiring it can be to create lots of boilerplate for accessing your data or searching for appropriate methods to access private properties. The search for the necessary methods is facilitated by auto-completion in IDEs, and boilerplate autogeneration partially became possible with Project Lombok plugin; a similar approach (Records) was implemented not so long ago in Java 14.

Let's look at the methods of javax.servlet.http.HttpServletRequest class:

Different colors highlight interfaces for accessing the three different fields.

Different naming logic — getParameterMap vs getHeaders, methods remove and set are used for the attributes, while they are not available for parameters, etc. In addition to a cumbersome interface, we can't reuse data manipulation logic, and it's quite complex to create test stubs (remember Builder pattern).

Clojure contrasts this with the data-oriented approach, where objects are just mapped with public access to its values.

The idea is that you can apply generic sequence manipulation functions to any data, for example, to the map above, the map of headers, or any other collection.

Threading macro -> allows rewriting function calls in a more convenient way, a bit similar to the Stream API in Java.

Using a small set of data structures and adopting a functional approach significantly streamlines data manipulation, and, as a result, enhances the whole programming process and code readability.

Immutable data structures

Clojure data structures are immutable. It means that it is impossible to change the value in a collection; you can only create a new collection with a new value. At the same time, the algorithmic complexity of accessing objects doesn't change, for example, access to a vector or hash-map is O(log32(N)). A constant memory is used for storage by dividing the majority of internal data among all versions of the modified structure.

The concept of object equality takes on a new meaning — if two structures are equal, they remain equal, not just at a specific point in time.  This approach greatly facilitates working with multithreaded apps, as there is no need to use thread-safe constructs and locks (recall synchronized blocks and collections in Java).

Thread-safe references — var, atom, agent, ref — are used to store the application's state. Clojure also implements a software transactional memory (STM) system, but that is a topic for another article.

Lazy evaluations

Clojure supports lazy sequences. It means that sequence elements are not pre-evaluated and are generated as the result of a computation (also referred to as 'evaluation' or 'realization').

Lazy evaluation allows for the definition of infinite sequences. As an example, let's consider Project Euler Problem №10 — find the sum of all the primes below two million.

The definition of prime has been omitted here for brevity. In fact, primes define the entire set of prime numbers as iterate and filter are lazy functions. The primes sequence will not be evaluated until we start accessing collection values (if you input primes into the REPL, it will freeze during the evaluation).

take-while is also a lazy function, and only reduce triggers the evaluation directly when finding the sum.

Of course, you can solve the mentioned problem without using lazy infinite sequences; you can choose another approach, for example, for.

But let's agree: the notation above looks elegant and clear!

Advanced level

Web development with Clojure

After a while, I had a chance to practice web development with Clojure. So, I entered the third phase of studying and exploring the Clojure ecosystem.

I mostly work with Clojure in small teams (up to 10 people). A small team allows you to fully leverage the advantages of the language, including rapid prototyping.

Ring and rapid prototyping

‍Ring is a web development standard in Clojure; its analogs for Python and Ruby are WSGI and Rack respectively. Ring repository provides:

• specifications of request/response/middleware and a code to work with them
• basic middleware
• Jetty web server adapter

• documentation

All modern Clojure frameworks ensure compatibility with the Ring standard. This enables running apps on different platforms without applying any changes (Jetty, http-kit, Immutant, and others).

The simplicity of Ring was a revelation. The idiomatic approach around the concept of middleware based on functions is used in many programming languages and frameworks today (for example, Express JS). Drawing from my experience with Java Servlets and Spring, I compared the middleware approach to Servlet Filters and Spring Boot.

You can compare this example of a REST-service in Spring with similar code written in Clojure.

My first REST API in Clojure looked like the above example; the only thing is that I used the compojure library instead of the more advanced compojure-api. Combined with REPL, Ring significantly speeds up the app development process.

Compojure is a routing library for Ring that allows describing handlers with easy-to-read list of routes:

Hiccup templating library provides DSL for HTML:

The convenience is that in Hiccup we can manipulate the Hiccup tree just like Clojure data structures, without the need to enclose computed expressions in curly braces.

The same format is used in Reagent, a popular ClojureScript library (ReactJS analogue).

Web frameworks

For streamlined development and flexibility, the Clojure community prefers to use a custom set of popular libraries instead of classic all-in-one frameworks. Hovewer, there are many popular templates and frameworks available for web development in Clojure:

1. compojure-api
— it's a Compojure 'on steroids' for working with RESTful-services; classic compojure is enriched with support for clojure.spec and Swagger.

2. Liberator
— a library for working with RESTful-services, using Webmachine-like approach; ensures compatibility with the HTTP RFC 2616 standard.

3. yada
— like Liberator, it offers compatibility with HTTP standard. It includes Swagger, Websockets (aleph), and other services. Interesting that yada is compatible with Ring but doesn't use Ring Middleware, offering instead a data-driven approach.

4. Pedestal
— web framework by Cognitect, overall quite good (except for the documentation), but also quite opinionated (for example, it uses its own router). Pedestal boasts high optimization and compatibility with web servers (for example, the ability to deploy to AWS Lambda).

5. luminus
— project template with a specific set of libraries for working with REST services (Reitit, Swagger), Websockets (depends on the selected server), DB (HugSQL + Migratus), templating (Hiccup, Selmer), i18n (Tempura), and more.

6. Jetty (included in Ring release) and http-kit
— basic web servers for Clojure applications; support working with websockets. These are standalone libraries, meaning they will be packed in the JAR-file together with the app.

7. Immutant
— nice kit that includes Undertow web server and messaging libraries (HornetQ), together with caсhing (Infinispan) and scheduling (Quartz) ones. It can be used as a standalone library or inside a WildFly container for clustering. Supported by JBoss Community.

The current list of the categorized libraries is available in Clojure Toolbox.

Extensibility

Clojure's extensibility plays a significant role, as many popular features of the language were implemented in external libraries with macros.

For example, pattern matching was implemented in the core.match library. You can match any Clojure data structure

or extend the matching for other data types.

core.async is a library for async communications using channels (queues) and thread pool and inversion of control patterns. It's close to channels in Golang; the key difference is that in Clojure channels are implemented not in the core of the language but in a separate library.

The go block (or go-loop, which combines the go macro and the loop function) declares code that will be executed asynchronously in a separate thread and immediately passes control back.

The function <! parks a thread within the go-loop block. While receiving a message, the loop-worker function is executed; then the execution is paused, waiting for the next message.

With the blocking function >!! strings are passed to the log-chan channel; they can be outputed to the console by loop-worker.

core.async provides a fixed-size 8-threads thread pool for go blocks and is actively used in ClojureScript. There are no blocking operations in cljs implementation; it offers an idiomatic approach to solving a callback hell problem.

clojure.spec and dynamic typing

Clojure primarily owes its dynamic typing to Lisp. Rich Hickey also support using dynamic typing by default.

Discussions about typing have been going on for years, so here I describe only my personal impressions.

Indeed, with dynamic typing it's easy to shoot yourself in the foot, but on the other hand, shooting becomes much easier per se 🙂 Lack of the 'static' in Clojure webdevelopment is compensated by:

1. the ability to test separate components and the entire system on the fly in REPL

2. spec'ing data with core.spec

core.spec leverages a data-oriented approach, using Clojure data structures for data schema representing. Clojure introduces the bold idea that, in many cases, you won't need to track types from the start to the end of the module or function chain. Instead, specs can be used to check data compliance with the protocol, for example, before submitting a web form or validating inputs of a module/function.

In the example below, spec is used for request/response validation, type conversion, and swagger docs generation (for further information, see the full example).

After several years of using static typing in Java, I felt unsure when started working with Clojure. I won't lie that static typing has its indisputable advantages, such as intellisense and compiler optimizations. However, when it comes to catching errors in real-world cases, code coverage with tests and the use of clojure.spec prove effective.

Indeed, there are cases where it's reasonable to add typing, for example, for a complex algorithm or a specific module. Here the extensibility of Clojure comes to the rescue again: for example, core.typed library that implements full-fledged static typing can be added both for the entire program and for selected pieces of code.

Other options

Clojure is actively used in Data Science и DL/ML domains (incanter, scicloj). Though I haven't had an opportunity to work within these fields yet, I have already appreciated Clojure's elegance in my experience with DSL.

DSL

DSL (domain specific language) is a language with a fairly high level of abstraction, desighned to solve a narrow class of problems.

In Clojure, DSLs are usually described with data structures, take for example Hiccup or honeysql:

Context-free grammars offer a more customizable method. Let me give you a real-life example: while working on Java project, where a GUI similar to Searchkit was used for manipulating filters on ElasticSearch datasets, I was tasked with implementing an 'advanced' filtering mode. This mode would allow extending the capabilities of GUI filters by using SQL expressions and introducing complex logical constructions not supported by the GUI.

For example, all conditions in GUI filters were exclusively connected using conjunctions (AND) or disjunctions (OR), making it impossible to use both of them in the expression or re-group expressions:

condition_1 AND (condition_2 OR condition_3).

The issue was that the ElasticSearch mapping was quite specific to implement a flexible data schema, and ElasticSearch SQL JDBC driver couldn't correctly map SQL queries to the ElasticSearch ones.

Accordingly, the task was parsing SQL query and transforming it into an isomorphic ElasticSearch query; also we had to implement error highlighting and autocomplete of the query elements (e.g. reserved keywords, identifiers) based on the context.

Clojure perfectly suits those task, especially due to its homoiconicity and ease of working with collections.

The idea was to: 1. parse SQL string into an abstract syntax tree (AST) represented as a Clojure data structure, , which would then be 2. transformed into an isomorphic structure for an ElasticSearch query.

For the first step, we used Instaparse library, designed for creating parsers for context-free grammars described using BNF. It takes an expression and a BNF specification as input, and outputs an AST in hiccup format (in fact, nested Clojure collections). Each tree node also contains metadata which can be used for implementing autocompletion. In case of a parsing error, Instaparse provides a pointer to the line and column of the error, along with expected terms.

The second stage included pattern matching together with traversing an AST.

To illustrate this approach, I've prepared a a simplified gist.

In my practice, I've used a similar approach with Instaparse multiple times. Unfortunately, a detailed examination of it goes beyond the scope of this article.

ClojureScript

Today, JavaScript is a dominant technology in various domains (browsers) and one of the most widely used (mobile devices).

ClojureScript is a compiler for Clojure that targets JavaScript, using the Google Closure compiler. The term is also used as the name of the language, sometimes abbreviated as cljs.

In fact, ClojureScript brings all Clojure benefits, such as data structures, functional approach, macros, etc. It differs only in some nuances of interacting with the platoform, while offering a production-ready ecosystem and libraries.

Combining Clojure for the backend with ClojureScript for the frontend allows for code reuse, and the shadow-cljs ensures seamless integration with npm packages.

Finally, ClojureScript is used to create mobile apps with React Native.

… and on the drawbacks

Like any other technology, Clojure has its drawbacks and trade-offs.

Due to implementation specifics, persistent data structures in Clojure are less performant than their analogs in Java. Clojure provides more performant transient alternatives for data structures like vectors, maps, and sets (but they are not designed for concurrent access).

Additionally, Clojure allows leveraging the capabilities of the host platform, such as Java's data structures and libraries. However, typical Clojure code is likely to execute more slowly than typical Java or Scala code. While not critical for most applications, this could be a decisive factor when choosing a technology for specific high-performance systems or modules.

Clojure beginners often mention the relatively steep learning curve. However, it's usually associated not with the functional approach per se, but with interacting with the platform. Take your time, and with some practice, it shouldn't be a problem.

One of the most significant drawbacks of Clojure, in my opinion, is the lack of proper marketing. Clojure is a simple language with minimal syntax, a well-thought-out ecosystem, and significant potential — it can pleasantly surprise you! However, its public image, seemingly inherited from Lisp, is often linked with excessive complexity and lack of standards (although things are the exact opposite).

As a result, newcomers are often hesitant to invest time in learning Clojure(Script), even though, in practice, they tend to grasp the language faster than others 🙂

Tips for newcomers: how to learn Clojure

Three pillars of Clojure craft

1. Practice — a versatile recommendation applicable to any technology. If you're a beginner in functional programming, enhance your skills with specialized resources or problem collections like Advent of Code or Project Euler. Using Clojure for the small pet projects greatly boosted my learning and motivation.

2. Learning from others — ask for help from the community, explore literature оn Clojure and Lisp (there are some resources listed in the next section), and you will find mind-blowing and good crazy things.

3. Watching Rich Hickey's talks — he explains Clojure features and solutions in an accessible and engaging way.

Every time I watch one of his talks I feel like someone has gone in and organized my brain.

Someone from 'Rich Hickey Fanclub'

Links that may come in handy

• Clojurians
  Slack community with content for newcomers (beginners, clojure, clojurescript) and job search (jobs, remote-jobs)
• /r/Clojure
  Clojure subreddit with announcements and discussions of libraries, lectures, etc.
• Clojure Toolbox
  Up-to-date handbook of ClojureScript libraries
• Rich Hickey fanclub
  Collection of video lectures and interviews about Clojure by Rich Hickey
• Brave Clojure Jobs
  List of current Clojure vacancies

Clojure books

On clojure.org, you can find almost all the published books about Clojure. The following is my list of Clojure books I have used:

Programming Clojure

— a good introductory book to Clojure (not to be confused with Clojure Programming, another excellent reference book but already a little outdated)

Clojure Applied

— a more advanced book for developers who are already familiar with Clojure and functional programming.

Mastering Clojure Macros

— this book is intended to level up your skills by leveraging the power of of Clojure's macro system.

Takeaways

• Clojure is an interesting, actively evolving, and accessible functional programming language for developers, including Java specialists, seeking a balance between performance and the fun factor.

• Clojure is a versatile language used for full-stack web and mobile development, scripting, or working with DSL.

• As a descendant of Lisp, Clojure appeals to geeks who enjoy tackling complex problems and finding solutions. However, its role is more comprehensive — Clojure was created to simplify development, and in my opinion, it achieves this task perfectly well.
  

Initially, this article, with minor edits, was published on DOU, a community of Ukrainian developers.

If you have any questions or comments, please send them to [email protected]