Clojure Programming

From NeoWiki

Latest revision as of 11:37, 9 November 2008

Clojure is a dynamic programming language for the JVM.

Introduction

People come to Clojure from a variety of backgrounds. The purpose of this book is to help everyone get the most out of the Clojure language. It is not meant as a replacement for the documentation, found at the Clojure site. Please do not copy material from there to here!

Feel free to post your own code, written in the Clojure language, here. However, please do not post excerpts, patches or other modifications or derivations of the Clojure source itself, as the license of this Wiki is incompatible with the Clojure license. Instead, post such things to the Clojure discussion group.

Getting Started

Installation

Installation from Source

You need Subversion (svn) to download the latest source code. To compile you need the Java Development Kit (javac) and either Maven (mvn) or Ant (ant). The examples below use mvn to run the build, but ant may be used in its place.

You can download the latest code using:

   svn co https://clojure.svn.sourceforge.net/svnroot/clojure/trunk clojure

Then to compile:

   cd clojure
   mvn install

Then to start a clojure REPL:

   java -cp target/clojure-lang-1.0-SNAPSHOT.jar clojure.lang.Repl

Press Ctrl+C to get out of it. You can either use rlwrap or JLine with the Clojure REPL to get functionality, like being able to press the up arrow to retrieve the previous command. If you download JLine and copy the jar into the clojure directory, you can run:

   java -cp jline-0.9.93.jar:target/clojure-lang-1.0-SNAPSHOT.jar jline.ConsoleRunner clojure.lang.Repl

If you are on Ubuntu, you can install the JLine library like so:

   sudo apt-get install libjline-java libjline-java-doc

Run 'dpkg -L libline-java' to check the name of the library. On Ubuntu 8.0.4 you'll see something like this:

   $ dpkg -L libjline-java
   /.
   /usr/share/java
   /usr/share/java/jline-0.9.93.jar

Then run:

    java -cp /usr/share/java/jline-0.9.93.jar:target/clojure-lang-1.0-SNAPSHOT.jar jline.ConsoleRunner clojure.lang.Repl

If you want to run a script from the command line:

   echo "(println \"Hello, World\")" > hello-world.clj
   java -cp target/clojure-lang-1.0-SNAPSHOT.jar clojure.lang.Script hello-world.clj

The clojure.lang.Repl startup method also supports script arguments. This differs from clojure.lang.Script startup in that scripts are loaded in the user namespace and Clojure enters the REPL instead of exiting after the scripts are loaded.

For Linux users, here's a bash script[1] that will make using Clojure from the command line a little less verbose:

    #!/bin/bash 
    CLOJURE_DIR=/path/to/clojure
    CLOJURE_JAR=$CLOJURE_DIR/target/clojure-lang-1.0-SNAPSHOT.jar
    if [ -z "$1" ]; then 
    	java -cp $CLOJURE_DIR/jline-0.9.93.jar:$CLOJURE_JAR \
    	    jline.ConsoleRunner clojure.lang.Repl    
    else
    	java -cp $CLOJURE_JAR clojure.lang.Script $1
    fi

Put this in a file named "clj" and make it executable.

For Windows users, here's the equivalent BATCH file:

    @echo off
    set CLOJURE_DIR=path\to\clojure
    set CLOJURE_JAR=%CLOJURE_DIR%\svn\clojure\target\clojure-lang-1.0-SNAPSHOT.jar
    IF (%1)==() (
        java -cp %CLOJURE_DIR%\jline-0.9.94.jar;%CLOJURE_JAR% jline.ConsoleRunner clojure.lang.Repl
    ) ELSE (
        java -cp %CLOJURE_JAR% clojure.lang.Script %1
    )

Put this in a file named clj.bat.

After adding the location of the script to your path, you can either run a script:

   clj hello-world.clj

or invoke the REPL:

   clj

Enhancing Clojure REPL with rlwrap

To enhance the clojure REPL under Unix variants, one very useful utility is rlwrap.

It adds the following features to the Clojure interactive shell.

1. Tab Completion
2. Parenthesis matching
3. History across Clojure sessions
4. Vi or Emacs binding based on your readline .inputrc settings

As a first step build and install rlwrap. It might also be available as a part of you package repository for your GNU/Linux distribution. Ensure that rlwrap version 0.30 or above is in your path (previous versions have issues with multiline prompts).

   [cljuser:~]% rlwrap -v
   rlwrap 0.30
   [cljuser:~]%

Save the following bash script as clj and add it to your path after making it executable:

    #!/bin/bash
 
    BREAK_CHARS="(){}[],^%$#@\"\";:''|\\"
    CLOJURE_DIR=/home/cljuser/install/clojure
    CLOJURE_JAR=$CLOJURE_DIR/clojure.jar
    if [ $# -eq 0 ]; then 
         rlwrap --remember -c -b $BREAK_CHARS -f $HOME/.clj_completions \
         java -cp $CLOJURE_JAR clojure.lang.Repl
    else
         java -cp $CLOJURE_JAR clojure.lang.Script "$@"
    fi

Note that this script uses rlwrap and not JLine. The file ~/.clj_completions is the file rlwrap will use for providing tab completions.

Also, note that the file clojure.jar in the script should refer to whatever is the name of the jar built but either ant or maven. Ant builds clojure.jar. Similarly update cljuser in the script to be the username.

The following Clojure program can be used to generate this file:

    (defmacro with-out-file [pathname & body]
      `(with-open stream# (new java.io.FileWriter ~pathname)
         (binding [*out* stream#]
           ~@body)))
 
    (def completions (keys (ns-publics (find-ns 'clojure))))
    (with-out-file "clj-keys.txt" (doseq x completions (println x)))

Save the above program as clj-completions.clj. This program is run to generate a list of completions as clj-keys.txt which is copied to ~/.clj_completions:

   [cljuser:~]% clj clj-completions.clj
   [cljuser:~]% ls clj-keys.txt
   clj-keys.txt
   [cljuser:~]% cp clj-keys.txt ~/.clj_completions
   [cljuser:~]%

At this point you are good to go. Here are the settings from ~/.inputrc:

   set editing-mode vi
   tab: complete
   set completion-ignore-case on
   set blink-matching-paren on

For the Vi key bindings, the user can use % for jumping to matching parenthesis in the interactive shell. For Emacs bindings the key is ????.

Invoke the REPL as clj

   [cljuser:~]% clj
   Clojure
   user=> (de --> tab
   Desktop/      defmethod     defstruct     deref
   dec           defmulti      delay         derive
   definline     defn          delay?        descendants
   defmacro      defn-         destructure
   user=> (def --> tab
   definline  defmacro   defmethod  defmulti   defn       defn-      defstruct
   user=> (take- --> tab
   take-nth    take-while
   user=> (take-

This REPL also remembers the symbols created by you:

   [cljuser:~]% clj
   Clojure
   user=> (def foo 10)
   #'user/foo
   user=> (def foobar 20)
   #'user/foobar
   user=> (def foo-next 30)
   #'user/foo-next
   user=> (fo --> tab
   foo       foo-next  foobar    for       force
   user=> (fo

Happy REPLing!

User settings

clojure.lang.Repl will run all the files listed before it goes to the prompt. So I have my clj script updated to accept a .cljrc.clj file that has my settings.

  (set! *print-length* 50)
  (set! *print-level* 10) 

clojure.contrib

Check out the source:

   svn co https://clojure-contrib.svn.sourceforge.net/svnroot/clojure-contrib clojure-contrib 

Use ant to build, not maven (just type ant in the trunk subdirectory). Add clojure-contrib.jar from that trunk directory to the classpath in your startup script:

   java -cp $CLOJURE_JAR:$CONTRIB_JAR clojure.lang.Repl

Or skip building the clojure-contrib.jar and add clojure-contrib/src to your classpath instead.

Editors/IDEs

Emacs / inferior-lisp

After installing Clojure, there is a 'clojure-mode' emacs mode available by Lennart Staflin [[2]] (and an enhanced version by Jeffrey Chu [[3]]). Place that within your load-path. By default, M-: load-path will give you the full load path. Then, in your ~/.emacs or ~/.xemacs/init.el file, add the following, with paths changed to match your computer.

(setq inferior-lisp-program
                                        ; Path to java implementation
      (let* ((java-path "java")
                                        ; Extra command-line options
                                        ; to java.
             (java-options "")
                                        ; Base directory to Clojure.
                                        ; Change this accordingly.
             (clojure-path "~/src/clojure/")
                                        ; The character between
                                        ; elements of your classpath.
             (class-path-delimiter ";")
             (class-path (mapconcat (lambda (s) s)
                                        ; Add other paths to this list
                                        ; if you want to have other
                                        ; things in your classpath.
                                    (list (concat clojure-path "clojure.jar"))
                                    class-path-delimiter)))
        (concat java-path
                " " java-options
                " -cp " class-path
                " clojure.lang.Repl")))
 
;; Require clojure-mode to load and associate it to all .clj files.
(setq load-path (cons "insert path to folder with clojure-mode stuff" load-path))
(require 'clojure-mode)
(setq auto-mode-alist
      (cons '("\\.clj$" . clojure-mode)
            auto-mode-alist))
 
;; These are extra key defines because I kept typing them.  
;; Within clojure-mode, have Ctrl-x Ctrl-e evaluate the last 
;; expression.
;; Ctrl-c Ctrl-e is also there, because I kept typoing it.
(add-hook 'clojure-mode-hook
          '(lambda ()
             (define-key clojure-mode-map "\C-c\C-e" 'lisp-eval-last-sexp)
             (define-key clojure-mode-map "\C-x\C-e" 'lisp-eval-last-sexp)))

With that, opening a .clj file will launch clojure mode. After that, M-x run-lisp launches the inferior lisp. It may be easiest to run it in a separate frame (what the rest of us call windows, available with C-x 5 2) or in a separate buffer in the same frame (via C-x 2, switch between windows using C-x o)

M-C-x will send whatever expression the point is in to the lisp buffer, with the key defines above, C-x C-e will send the previous expression to the lisp buffer and (lisp-eval-region (point-min) (point-max)) entered at the M-: eval prompt will send the entire working buffer to the lisp buffer, useful when you've made a bunch of changes and don't want to reload them individually. M-p cycles through the repl entry history.

Emacs / Slime Integration

Clojure can be integrated with Slime (unofficial Slime git repo). Download clojure-mode and swank-clojure and put the following configuration in your .emacs file:

;; Clojure mode
(add-to-list 'load-path "/path/to/jochu-clojure-mode")
(require 'clojure-auto)
;; (require 'clojure-paredit) ; Uncomment if you use Paredit
 
;; Slime
(add-to-list 'load-path "/path/to/slime/") 
(require 'slime)
(slime-setup)
 
;; clojure swank
(setq swank-clojure-jar-path "/path/to/clojure/target/clojure-lang-1.0-SNAPSHOT.jar")
;alternatively, you can set up the clojure wrapper script and use that: 
;(setq swank-clojure-binary "/path/to/cljwrapper")
 
(add-to-list 'load-path "/path/to/jochu-swank-clojure")
(require 'swank-clojure-autoload)
 
;; is this required? I don't have this in my emacs configuration; I just execute M-x slime to start slime -- Chousuke 
(defun run-clojure ()
  "Starts clojure in Slime"
  (interactive)
  (slime 'clojure))
 
;; To use other Lisps...
;; Incidentally, you can then choose different Lisps with
;;   M-- M-x slime <tab>
(add-to-list 'slime-lisp-implementations
             '(sbcl   ("/path/to/bin/sbcl")))

Emacs tab completion

This trick was found somewhere on the internet. add after clojure configuration is done

(defun indent-or-expand (arg)
  "Either indent according to mode, or expand the word preceding
point."
  (interactive "*P")
  (if (and
       (or (bobp) (= ?w (char-syntax (char-before))))
       (or (eobp) (not (= ?w (char-syntax (char-after))))))
      (dabbrev-expand arg)
    (indent-according-to-mode)))
 
(defun my-tab-fix ()
  (local-set-key [tab] 'indent-or-expand))
 
;; add hooks for modes you want to use the tab completion for:
(add-hook 'c-mode-hook          'my-tab-fix)
(add-hook 'sh-mode-hook         'my-tab-fix)
(add-hook 'emacs-lisp-mode-hook 'my-tab-fix)
(add-hook 'clojure-mode-hook    'my-tab-fix)

Emacs / Slime / Clojure on Ubuntu - Tutorial

A detailed tutorial for installing clojure, emacs and slime on Ubuntu is available here: UbuntuSetup

Vim

Clojure Filetype Plugin

There is a set of syntax highlighting, indent and ftplugin available: VimClojure. The installation process works as usual: unzip the files into your .vim directory and make sure ftplugins are enabled. Everything else is done by Vim.

Completions

If you are using the VimClojure plugin from the previous section, Ctrl-N completions should work for you out of the box and you don't really need to bother with this section.

However, if you are using the bleeding edge Clojure from the sources, there may be some additions / changes to the API. If this is the case, you would need to generate an updated list of completions for Clojure. This can be created using the Clojure script from section Clojure_Programming#Enhancing_Clojure_REPL_with_rlwrap. Once generated this completions file may be linked into your VimClojure plugin or you could update your .vimrc. Note that the completions generator script is already available as a part of the VimClojure plugin.

Once you have the file ~/.clj_completions add the following line to your ~/.vimrc.

au Bufenter,Bufnewfile *.clj setl complete+=k~/.clj_completions

With this setting you should be able to use Ctrl+N in insert mode to see possible completions. For example, takeCtrl+N will show take-while, take-nth and take in a drop-down list.

Other Tips

[Unix Only] Vim users may find this useful. This article talks about how you can send expression from a Vim session to a running REPL.

Chimp does a similar job with immediate support for Clojure: send toplevel s-expression, send inner-most s-expression, send whole file, send visual block, switch namespaces if necessary, ... Since the basic idea is similar to the above mentioned script, you need screen. Chimp works on Unix-like systems and on Windows with Cygwin.

Tutorials and Tips

Walkthrough

Clojure for Java Programmers

Rich Hickey gave a talk entitled "An Introduction For Java Programmers". Audio and slides are available in two parts on the Clojure blip.tv channel.

Invoking Clojure from Java

The main site's Java Interop reference shows how you can call your Java code from Clojure. But because Clojure is implemented as a Java class library, you can also easily embed Clojure in your Java applications, load code, and call functions.

Repl.java and Script.java in the distribution are the canonical examples for loading code from a user or from a file. In this section we'll show how to use the same underlying machinery to load code and then manipulate it directly from Java.

Here's a script that defines a simple Clojure function.

; foo.clj
(clojure/ns user)
 
(defn foo [a b]
  (str a " " b))

And here's a Java class that loads the script, calls the foo function with some Java objects as arguments, and prints the returned object.

// Foo.java
 
import clojure.lang.RT;
import clojure.lang.Var;
 
public class Foo {
    public static void main(String[] args) throws Exception {
        // Load the Clojure script -- as a side effect this initializes the runtime.
        RT.loadResourceScript("foo.clj");
 
        // Get a reference to the foo function.
        Var foo = RT.var("user", "foo");
 
        // Call it!
        Object result = foo.invoke("Hi", "there");
        System.out.println(result);
    }
}

Compile this and run it to see the printed result:

>javac -cp clojure.jar Foo.java

>java -cp clojure.jar Foo
Hi there

>

Clojure for Scheme Programmers

When asked about using SICP to learn about Clojure, Rich had this to say:

Everyone's experience will be different, of course. Here's my 2 cents:

I don't think SICP is a book about a programming language. It's a book about programming. It uses Scheme because Scheme is in many ways an atomic programming language. Lambda calculus + TCO for loops + Continuations for control abstraction + syntactic abstraction (macros) + mutable state for when you need it. It is very small. It is sufficient.

The book really deals with the issues in programming. Modularity, abstraction, state, data structures, concurrency etc. It provides descriptions and toy implementations of generic dispatch, objects, concurrency, lazy lists, (mutable) data structures, 'tagging' etc, designed to illuminate the issues.

Clojure is not an atomic programming language. I'm too tired/old/lazy to program with atoms. Clojure provides production implementations of generic dispatch, associative maps, metadata, concurrency infrastructure, persistent data structures, lazy seqs, polymorphic libraries etc etc. Much better implementations of some of the things you would be building by following along with SICP are in Clojure already.

So the value in SICP would be in helping you understand programming concepts. If you already understand the concepts, Clojure lets you get on with writing interesting and robust programs much more quickly, IMO. And I don't think the core of Clojure is appreciably bigger than Scheme's. What do Schemers think?

I think the Lisps prior to Clojure lead you towards a good path with functional programming and lists, only to leave you high and dry when it comes to the suite of data structures you need to write real programs, such data structures, when provided, being mutable and imperative. Prior Lisps were also designed before pervasive in-process concurrency, and before the value of high-performance polymorphic dispatch (e.g. virtual functions) as library infrastructure was well understood. Their libraries have decidedly limited polymorphism.

Alas, there is no book on Clojure yet. But, to the extent Schemes go beyond the standard to provide more complete functionality (as most do), there are no books on that either. Just docs in both cases.

Learning Scheme or Common Lisp on your way to Clojure is fine. There will be specifics that don't translate (from Scheme - no TCO, false/ nil/() differences, no continuations; from CL - Lisp-1, symbol/var dichotomy). But I personally don't think SICP will help you much with Clojure. YMMV.

Clojure for Common Lisp Programmers

The table below lists some of the Common Lisp entities (functions, macros etc.) and their (somewhat) equivalent Clojure entities. Note that some of the concepts may not be an exact match. For the exact behavior of the Clojure entities it is recommended that the reader see the Clojure Reference Documentation available at the Clojure home page. Note that some of this differences may be because Clojure has its heritage in Lisp-1 and not Lisp-2.

The Clojure reference also documents common differences to lisps here.

user=> (Math/pow 10 2)
100.0

user=> (format "0x%x 0x%x 0x%x" 10 20 30)
"0xa 0x14 0x1e"

user=> `(x#)
(x__2136)

(loop [temp-one 1
       temp-two 0]
  (if (= 3 temp-two)
    temp-one
    (recur (inc temp-one) (inc temp-one)))) 
=> 3

(cond test-form1
        form1
      test-form2
        form2
      ...
      :else ;; :else or non nil or true
        form)

Clojure for Python/Ruby/Perl Programmers

A list of equivalent clojure functions for all methods in Ruby's Enumerable and Array classes.

Where did the x function go?

Sometimes builtin functions or macros need to be renamed. Here are all the old names and their most recent equivalents:

Unit Testing in Clojure

unit-test is a library to do unit testing in Clojure. The documentation on it is sparse, but it should be easy to pick for anyone that has used an xUnit framework before like JUnit.

Shebang Scripting in Clojure

This was tested on Linux only, but should work in other Un*xes.

Put this into command-line-args.clj

#^:shebang '[
exec java -cp "$HOME/src/clj/clojure/clojure.jar" clojure.lang.Script "$0" -- "$@"
]
(prn *command-line-args*)

$ chmod 755 command-line-args.clj

And then run it with parameters.

$ ~/src/clj/lab/command-line-args.clj a b c
("a" "b" "c")

The explanation for this approach is described in this mail on Clojure group.

Concepts

Basics

Numbers

Types of Numbers

Clojure supports the following numeric types:

• Integer
• Floating Point
• Ratio

Numbers in Clojure are based on java.lang.Number. BigInteger and BigDecimal are supported and hence we have arbitrary precision numbers in Clojure.

The Ratio type is described on Clojure page as

Ratio

Represents a ratio between integers. Division of integers that can't be reduced to an integer yields a ratio, i.e. 22/7 = 22/7, rather than a floating point or truncated value.

Ratios allow maintain a computation in numeric form. This can help avoid inaccuracies in long computations.

Here is a little experiment. Lets first try a computation of (1/3 * 3/1) as floating point. Later we try the same with Ratio.

   user=> (def a (/ 1.0 3.0))
   #'user/a
   user=> a
   0.3333333333333333
   user=> (def b (/ 3.0 1.0))
   #'user/b
   user=> b
   3.0
   user=> (* a b)
   1.0
   user=> (def c (* a a a a a a a a a a))
   #'user/c
   user=> c
   1.693508780843028E-5
   user=> (def d (* b b b b b b b b b b))
   #'user/d
   user=> d
   59049.0
   user=> (* c d)
   0.9999999999999996

The value of (* c d) above is 0.9999999999999996 instead of 1 which is what we want. This is due to the inaccuracies of a and b multiplying as we create c and d. You really don't want such calculations happening in your pay cheque :)

The same done with ratios below.

   user=> (def a1 (/ 1 3))
   #'user/a1
   user=> a1
   1/3
   user=> (def b1 (/ 3 1))
   #'user/b1
   user=> b1
   3
   user=> (def c (* a1 a1 a1 a1 a1 a1 a1 a1 a1 a1))
   #'user/c
   user=> x
   3.142857142857143
   user=> c
   1/59049
   user=> (def d (* b1 b1 b1 b1 b1 b1 b1 b1 b1 b1))
   #'user/d
   user=> d
   59049
   user=> (* c d)
   1

The result is 1 as we hoped for.

Number Entry Formats

Clojure supports the the usual formats for entry as shown below

   user=> 10       ; decimal
   10
   user=> 010      ; octal
   8
   user=> 0xff     ; hex
   255
   user=> 1.0e-2   ; double
   0.01
   user=> 1.0e2    ; double
   100.0

To make things easier, a radix based entry format is also supported in the form <radix>r<number>. Where radix can be any natural number between 2 and 36.

   user=> 2r1111
   15

These formats can be mixed and used.

   user=> (+ 0x1 2r1 01)
   3

Many bitwise operations are also supported by Clojure API.

   user=> (bit-and 2r1100 2r0100)
   4

Some of the others are:

• (bit-and x y)
• (bit-and-not x y)
• (bit-clear x n)
• (bit-flip x n)
• (bit-not x)
• (bit-or x y)
• (bit-set x n)
• (bit-shift-left x n)
• (bit-shift-right x n)
• (bit-test x n)
• (bit-xor x y)

Check Clojure API for the complete documentation.

Conversion Integers to Strings

One general purpose way to format any data for printing is to use java.util.Formatter

This can be done very conveniently as shown below:

   user=> (format "0x%x" (bit-and 2r1100 2r0100))
   "0x4"

Specifically for integer conversion to strings, its even easier to use java.lang.Integer.

   user=> (. Integer toBinaryString 10)
   "1010"
   user=> (. Integer toHexString 10)
   "a"
   user=> (. Integer toOctalString 10)
   "12"
   user=>

The above examples deal with standard base conversions. For a more general purpose conversion we can do the following:

   user=> (. Integer toString 10 2)
   "1010"

Where 10 is the number to be converted and 2 is the radix.

. (dot) in the above examples is a special form used for accessing Java as explained in the Clojure Reference (Java Interop).

Converting Strings to Integers

For converting strings to integers, we can again use java.lang.Integer. This is shown below.

   user=> (. Integer parseInt "A" 16)      ; hex
   10
   user=> (. Integer parseInt "1010" 2)    ; bin
   10
   user=> (. Integer parseInt "10" 8)      ; oct
   8
   user=> (. Integer parseInt "8")         ; dec
   8

The above sections give an overview of the integer-to-string and string-to-integer formatting. There is a very rich set of well documented functions available in the Java libraries (too rich to document here). These functions can easily be used to meet varied needs.

Structures

Structures in Clojure are a little different from those in languages like Java or C++. They are also different from structures in Common Lisp (even though we have a defstruct in Clojure).

In Clojure, structures are a special case of maps and are explained in the data structures section in the reference.

The idea is that multiple instance of the structures will need to access their field values using the field names which are basically the map keys. This is fast and convenient, especially because Clojure automatically defined the keys as accessors for the structure instances.

Following are the important functions dealing with structures:

• defstruct
• create-struct
• struct
• struct-map

For the full API refer to data structures section in Clojure reference.

Structures are created using defstruct which is a macro wrapping the function create-struct which actually creates the struct. defstruct creates the structure using create-struct and binds it to the structure name supplied to defstruct.

The object returned by create-struct is what is called the structure basis. This is not a structure instance but contains information of what the structure instances should look like. New instances are created using struct or struct-map.

The structure field names of type keyword or symbols are automatically usable as functions to access fields of the structure. This is possible as structures are maps and this feature is supported by maps. This is not possible for other types of field names such as strings or numbers. It is quite common to use keywords for field names for structures due to the above reason. Also, Clojure optimises structures to share base key information.

The following REPL interaction shows sample usage.

user=> (defstruct employee :name :id)
#'user/employee

user=> (struct employee "Mr. X" 10)
{:name "Mr. X", :id 10}

user=> (struct-map employee :id 20 :name "Mr. Y")
{:name "Mr. Y", :id 20}

user=> (def a (struct-map employee :id 20 :name "Mr. Y"))
#'user/a

user=> (def b (struct employee "Mr. X" 10))
#'user/b

user=> (:name a) ; observe that :name is an accessor
"Mr. Y"

user=> (:id b)   ; same with :id
10

user=> (b :id)
10

user=> (b :name)
"Mr. X"

Clojure also supports the accessor function that can be used to get accessor functions for fields to allow easy access. This is important when field names are of types other than keyword or symbols. This is seen in the interaction below.

user=> (def e-str (struct employee "John" 123))
#'user/e-str

user=> e-str
{"name" "John", "id" 123}

user=> ("name" e-str) ; FAIL: string not an accessor
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IFn
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IFn
        at user.eval__2537.invoke(Unknown Source)
        at clojure.lang.Compiler.eval(Compiler.java:3847)
        at clojure.lang.Repl.main(Repl.java:75)

user=> (def e-name (accessor employee "name"))  ; bind accessor to e-name
#'user/e-name

user=> (e-name e-str) ; use accessor
"John"

As structures are maps, new fields can be added to structure instances using assoc. dissoc can be used to remove these instance specific keys. Note however that struct base keys cannot be removed.

user=> b
{:name "Mr. X", :id 10}

user=> (def b1 (assoc b :function :engineer))
#'user/b1
user=> b1
{:name "Mr. X", :id 10, :function :engineer}

user=> (def b2 (dissoc b1 :function)) ; this works as :function is instance
#'user/b2                             ; specific
user=> b2
{:name "Mr. X", :id 10}

user=> (dissoc b2 :name)  ; this fails. base keys cannot be dissociated
java.lang.Exception: Can't remove struct key
 ............

assoc can also be used to "update" a structure.

user=> a
{:name "Mr. Y", :id 20}

user=> (assoc a :name "New Name")
{:name "New Name", :id 20}

user=> a                   ; note that 'a' is immutable and did not change
{:name "Mr. Y", :id 20}

user=> (def a1 (assoc a :name "Another New Name")) ; bind to a1
#'user/a1

user=> a1
{:name "Another New Name", :id 20}

Observe that like other sequences in Clojure, structures are also immutable, hence, simply doing assoc above does not change a. Hence we rebind it to a1. While it is possible to rebind the new value back to a, this is not considered good style.

Exception Handling

Clojure supports Java based Exceptions. This may need some getting used to for Common Lisp users who are used to the Common Lisp Condition System.

Clojure does not support a condition system and is not expected to be supported anytime soon as per this message. That said, the more common exception system which is adopted by Clojure is well suited for most programming needs.

If you are new to exception handling, the Java Tutorial on Exceptions is a good place to learn about them.

In Clojure, exceptions can be handled using the following functions:

• (try expr* catch-clause* finally-clause?)
  • catch-clause -> (catch classname name expr*)
  • finally-clause -> (finally expr*)
• (throw expr)

Two types of exceptions you may want to handle in Clojure are:

• Clojure Exception: These are exception generated by Clojure or the underlying Java engine
• User Defined Exception: These are exceptions which you might create for your applications

Clojure Exceptions

Below is a simple interaction at the REPL that throws an exception:

user=> (/ 1 0)
java.lang.ArithmeticException: Divide by zero
java.lang.ArithmeticException: Divide by zero
        at clojure.lang.Numbers.divide(Numbers.java:142)
        at user.eval__2127.invoke(Unknown Source)
        at clojure.lang.Compiler.eval(Compiler.java:3847)
        at clojure.lang.Repl.main(Repl.java:75)

In the above case we see a java.lang.ArithmeticException being thrown. This is a runtime exception exception which is thrown by the underlying JVM. The long message can sometimes be intimidating for new users but the trick is to simply look at the exception (java.lang.ArithmeticException: Divide by zero) and not bother with the rest of the trace.

Similar exceptions may be thrown by the compiler at the REPL.

user=> (def xx yy)
java.lang.Exception: Unable to resolve symbol: yy in this context
clojure.lang.Compiler$CompilerException: NO_SOURCE_FILE:4: Unable to resolve symbol: yy in this context
        at clojure.lang.Compiler.analyze(Compiler.java:3669)
        at clojure.lang.Compiler.access$200(Compiler.java:37)
        at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:335)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:3814)
        at clojure.lang.Compiler.analyze(Compiler.java:3654)
        at clojure.lang.Compiler.analyze(Compiler.java:3627)
        at clojure.lang.Compiler.eval(Compiler.java:3851)
        at clojure.lang.Repl.main(Repl.java:75)

In the above case, the compiler does not find the binding for yy and hence it throws the exception. If your program is correct (i.e. in this case yy is defined (def yy 10)) , you won't see any compile time exceptions.

The following interaction shows how runtime exceptions like ArithmeticException can be handled.

user=> (try (/ 1 0)
            (catch Exception e (prn "in catch"))
            (finally (prn "in finally")))
"in catch"
"in finally"
nil
user=>

The syntax for the try block is (try expr* catch-clause* finally-clause?).

As can be seen, its quite easy to handle exceptions in Clojure. One thing to note is that (catch Exception e ...) is a catch all for exceptions as Exception is a superclass of all exceptions. It is also possible for catch specific exceptions which is generally a good idea.

In the example below, we specifically catch ArithmeticException.

user=> (try (/ 1 0) (catch ArithmeticException e (prn "in catch")) (finally (prn "in finally")))
"in catch"
"in finally"
nil

When we use some other exception type in the catch block, we find that the ArithmeticException is not caught and is seen by the REPL.

user=> (try (/ 1 0) (catch IllegalArgumentException e (prn "in catch")) (finally (prn "in finally")))
"in finally"
java.lang.ArithmeticException: Divide by zero
java.lang.ArithmeticException: Divide by zero
        at clojure.lang.Numbers.divide(Numbers.java:142)
        at user.eval__2138.invoke(Unknown Source)
        at clojure.lang.Compiler.eval(Compiler.java:3847)
        at clojure.lang.Repl.main(Repl.java:75)
user=>

User Defined Exceptions

As mentioned previously, all exceptions in Clojure need to be a subclass of java.lang.Exception (or generally speaking - java.lang.Throwable which is the superclass for Exception). This means that even when you want to define your own exceptions in Clojure, you need to derive it from Exception.

Don't worry, thats easier than it sounds :)

Clojure API provides a function gen-and-load-class which can be used to extend java.lang.Exception for user defined exceptions. gen-and-load-class generates and immediately loads the bytecode for the specified class.

Now, rather than talking too much, lets quickly look at code.

(gen-and-load-class 'user.UserException :extends Exception)
 
(defn user-exception-test []
  (try
    (throw (new user.UserException "msg: user exception was here!!"))
    (catch user.UserException e
      (prn "caught exception" e))
    (finally (prn "finally clause invoked!!!"))))

Here we are creating a new class 'user.UserException that extends java.lang.Exception. We create an instance of user.UserException using the special form (new Classname-symbol args*). This is then thrown.

Sometimes you may come across code like (user.UserException. "msg: user exception was here!!"). This is just another way to say new. Note the . (dot) after the user.UserException. This does exactly the same thing.

Here is the interaction:

user=> (load-file "except.clj")
#'user/user-exception-test

user=> (user-exception-test)
"caught exception" user.UserException: msg: user exception was here!!
"finally clause invoked!!!"
nil
user=>

So here we have, both the catch and the finally clauses being invoked. Thats all there is to it.

With the support for Java Interop, its possible for the user to create exceptions in Java and catch them in Clojure but just doing it in Clojure is much more convenient.

Mutation Facilities

Employee Record Manipulation

Data structures and sequences in Clojure are immutable as seen in the examples presented in Clojure_Programming#Structures. It is suggested that the reader go through this section first.

While immutable data has its advantages, any project of reasonable size will require the programmer to maintain some sort of state. Managing state in a language with immutable sequences and data structures is a frequent source of confusion for people used to programming languages that allow mutation of data.

A good essay on the Clojure approach is [http://clojure.org/state Values and Change - Clojure's approach to Identity and State], written by Rich Hickey.

It may be useful to watch Clojure Concurrency screen cast as some of those concepts are used in this section. Specifically refs and transactions.

In this section we create a simple employee record set and provide functions to:

• Add an employee
• Delete employee by name
• Change employee role by name

The example is purposely kept simple as the intent is to show the state and mutation facilities rather than provide full functionality.

Lets dive into the code.

(alias 'set 'clojure.set)   ; use set/fn-name rather than clojure.set/fn-name
 
(defstruct employee
           :name :id :role) ; == (def employee (create-struct :name :id ..))
 
(def employee-records (ref #{}))
 
;;;===================================
;;; Private Functions: No Side-effects
;;;===================================
 
(defn- update-role [n r recs]
  (let [rec    (first (set/select #(= (:name %) n) recs))
        others (set/select #(not (= (:name %) n)) recs)]
    (set/union (set [(assoc rec :role r)]) others)))
 
(defn- delete-by-name [n recs]
  (set/select #(not (= (:name %) n)) recs))
 
;;;=============================================
;;; Public Function: Update Ref employee-records
;;;=============================================
(defn update-employee-role [n r]
  "update the role for employee named n to the new role r"
  (dosync 
    (ref-set employee-records (update-role n r @employee-records))))
 
(defn delete-employee-by-name [n]
  "delete employee with name n"
  (dosync
    (ref-set employee-records
             (delete-by-name n @employee-records))))
 
(defn add-employee [e]
  "add new employee e to employee-records"
  (dosync (commute employee-records conj e)))
 
;;;=========================
;;; initialize employee data
;;;=========================
(add-employee (struct employee "Jack" 0 :Engineer))
(add-employee (struct employee "Jill" 1 :Finance))
(add-employee (struct-map employee :name "Hill" :id 2 :role :Stand))

In the first few lines we define the employee structure. The interesting definition after that is employee-records.

(def employee-records (ref #{}))

In Clojure refs allow mutation of a storage location with a transaction.

user=> (def x (ref [1 2 3]))
#'user/x
user=> x
clojure.lang.Ref@128594c
user=> @x
[1 2 3]
user=> (deref x)
[1 2 3]
user=>

Next we define private functions update-role and delete-by-name using defn- (note the minus '-' at the end). Observe that these are pure functions without any side-effects.

update-role takes the employee name n, the new role r and a table of employee records recs. As sequences are immutable, this function returns a new table of records with the employee role updated appropriately. delete-by-name also behaves in a similar manner by returning a new table of employees after deleting the relevant employee record.

For an explanation of the set API see Clojure API reference.

We still haven't looked at how state is maintained. This is done by the public functions in the listing update-employee-role, delete-employee-by-name and add-employee.

These functions delegate the job of record processing to the private functions. The important things to note are the use of the following functions:

• ref-set sets the value of a ref.
• dosync is mandatory as refs can only be updated in a transaction and dosync sets up the transaction.
• commute updates the in-transaction value of a ref.

For a detailed explanation of these functions see the refs section in API reference.

The add-employee function is quite trivial and hence not broken up into private and public function.

The source listing initializes the records with sample data towards the end.

Below is the interaction for this program.

user=> (load-file "employee.clj")
#{{:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Finance}}

user=> @employee-records
#{{:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Finance}}

user=> (add-employee (struct employee "James" 3 :Bond))
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Finance}}
user=> @employee-records
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Finance}}

user=> (update-employee-role "Jill" :Sr.Finance)
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Sr.Finance}}
user=> @employee-records
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Hill", :id 2, :role :Stand} {:name "Jill", :id 1, :role :Sr.Finance}}

user=> (delete-employee-by-name "Hill")
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Jill", :id 1, :role :Sr.Finance}}
user=> @employee-records
#{{:name "James", :id 3, :role :Bond} {:name "Jack", :id 0, :role :Engineer} {:name "Jill", :id 1, :role :Sr.Finance}}

Two things to note about the program:

• Using refs and transactions makes the program inherently thread safe. If we want to extend the program for a multi-threaded environment (using Clojure agents) it will scale with minimal change.
• Keeping the pure functionality separate from the public function that manages state, it is easier to ensure that the functionality is correct as pure functions are easier to test.

Functional Programming

Anonymous Functions

Clojure supports anonymous functions using fn or the shorter reader macro #(..). The #(..) is convenient due to its conciseness but is somewhat limited as #(..) form cannot be nested.

Below are some examples using both forms:

user=> ((fn [x] (* x x)) 3)
9

user=> (map #(list %1 (inc %2)) [1 2 3] [1 2 3])
((1 2) (2 3) (3 4))

user=> (map (fn [x y] (list x (inc y))) [1 2 3] [1 2 3])
((1 2) (2 3) (3 4))

user=> (map #(list % (inc %)) [1 2 3])
((1 2) (2 3) (3 4))

user=> (map (fn [x] (list x (inc x))) [1 2 3])
((1 2) (2 3) (3 4))

user=> (#(apply str %&) "Hello")
"Hello"

user=> (#(apply str %&) "Hello" ", " "World!")
"Hello, World!"

Note that in #(..) form, %N is used for arguments (1 based) and %& for the rest argument. % is a synonym for %1.

Lazy Evaluation of Sequences

This section tries to walk through some code to give a better feel for the lazy evaluation of sequences by Clojure and how that might be useful. We also measure memory and time to understand whats happening better.

Consider a scenario where we want to do a big-computation (1 second each) on records in a list with a billion items. Typically we may not need all the billion items processed (e.g. we may need only a filtered subset).

Lets define a little utility function free-mem to help us monitor memory usage and another function big-computation that takes 1 second to do its job.

(defn free-mem [] (.. Runtime (getRuntime) (freeMemory)))
 
(defn big-computation [x] (. Thread sleep 1000) (* 10 x))

In the functions above we use java.lang.Runtime and java.lang.Thread for getting free memory and supporting sleep.

We will also be using the built in function time to measure our performance.

Here is a simple usage at REPL:

user=> (defn free-mem [] (.. Runtime (getRuntime) (freeMemory)))
#'user/free-mem

user=> (defn big-computation [x] (. Thread sleep 1000) (* 10 x))
#'user/big-computation

user=> (time (big-computation 1))
"Elapsed time: 1000.339953 msecs"
10

Now we define a list of 1 billion numbers called nums.

user=> (time (def nums (range 1000000000)))
"Elapsed time: 0.166994 msecs"
#'user/nums

Note that it takes Clojure only 0.17 ms to create a list of 1 billion numbers. This is because the list is not really created. The user just has a promise from Clojure that the appropriate number from this list will be returned when asked for.

Now, lets say, we want to apply big-computation to x from 10000 to 10005 from this list.

This is the code for it:

;; The comments below should be read in the numbered order
;; to better understand this code.
 
(time                              ; [7] time the transaction
  (def v                           ; [6] save vector as v
    (apply vector                  ; [5] turn the list into a vector
           (map big-computation    ; [4] process each item for 1 second
                (take 5            ; [3] take first 5 from filtered items
                      (filter      ; [2] filter items 10000 to 10010
                        (fn [x] (and (> x 10000) (< x 10010)))
                        nums)))))) ; [1] nums = 1 billion items

Putting this code at the REPL, this is what we get:

user=> (free-mem)
2598000
user=> (time (def v (apply vector (map big-computation (take 5 (filter (fn [x] (and (> x 10000) (< x 10010))) nums))))))
"Elapsed time: 5036.234311 msecs"
#'user/v
user=> (free-mem)
2728800

The comments in the code block indicate the working of this code. It took us ~5 seconds to execute this. Here are some points to note:

• It did not take us 10000 seconds to filter out item number 10000 to 10010 from the list
• It did not take us 10 seconds to get first 5 items from the list of 10 filtered list
• Overall, it took the computation only 5 seconds which is basically the computation time.
• The amount of free memory is virtually the same even though we now have the promise of a billion records for processing. (It actually seems to have gone up a bit due to garbage collection)

Now if we access v it takes negligible time.

user=> (time (seq v))
"Elapsed time: 0.042045 msecs"
(100010 100020 100030 100040 100050)
user=>

Another point to note is that a lazy sequence does not mean that the computation is done every time; once the computation is done, it gets cached.

Try the following:

user=> (time (def comps (map big-computation nums)))
"Elapsed time: 0.113564 msecs"
#'user/comps

user=> (defn t5 [] (take 5 comps))
#'user/t5

user=> (time (doall (t5)))
"Elapsed time: 5010.49418 msecs"
(0 10 20 30 40)

user=> (time (doall (t5)))
"Elapsed time: 0.096104 msecs"
(0 10 20 30 40)

user=>

In the first step we map big-computation to a billion nums. Then we define a function t5 that takes 5 computations from comps. Observe that the first time t5 takes 5 seconds and after that it takes neglegible time. This is because once the calculation is done, the results are cached for later use. Since the result of t5 is also lazy, doall is needed to force it to be eagerly evaluated before time returns to the REPL.

Lazy data structures can offer significant advantage assuming that the program is designed to leverage that. Designing a program for lazy sequences and infinite data structures is a paradigm shift from eagerly just doing the computation in languages like C and Java vs giving a promise of a computation.

This section is based on a this mail in the Clojure group.

Infinite Data Source

As Clojure supports lazy evaluation of sequences, it is possible to have infinite data sources in Clojure. Here is an example:

user=> (def nums (iterate inc 0))
#'user/nums
user=> (take 5 nums)
(0 1 2 3 4)
user=> (take 10 nums)
(0 1 2 3 4 5 6 7 8 9)
user=>

Here we see two functions that are used for create an infinite list of numbers starting from 0. As Clojure supports lazy sequences, only the required items are generated and taken of the head of this list. In the above case, if you were to type out (iterate inc 0) directly at the prompt, the [http://clojure.org/reader reader] would continue getting the next number forever and you would need to terminate the process.

(iterate f x) is a function that continuously applies f to the result of the previous application of f to x. Meaning, the result is ...(f(f(f(f .....(f(f(f x))))).... (iterate inc 0) first gives 0 as the result, then (inc 0) => 1, then (inc (inc 0)) => 2 and so on.

(take n coll) basically removes n items from the collection. There are many variation of this theme:

• (take n coll)
• (take-nth n coll)
• (take-while pred coll)
• (drop n coll)
• (drop-while pred coll)

The reader is encouraged to look at the Clojure Sequence API for details.

List Comprehension

List Comprehensions are the constructs offered by a language that make it easy to create new lists from old ones. As simple as it sounds, it is a very powerful concept. Clojure has good support for List comprehensions.

Lets say we want a set of all x + 1 for all x divisible by 4 with x starting from 0.

Here is one way to do it in Clojure:

user=> (def nums (iterate inc 0))
#'user/nums
user=> (def s (for [x nums :when (zero? (rem x 4))] (inc x)))
#'user/s
user=> (take 5 s)
(1 5 9 13 17)
user=>

nums is the infinite list of numbers that we saw in the previous section. We need to (def s ...) for the set as we are creating an infinite source of numbers. Running it directly at the prompt will make the reader suck out numbers from this source indefinitely.

The key construct here is the for macro. Here the expression [x nums ... says that x comes out of nums one at a time. The next clause .. :when (zero? (rem x 4)) .. basically says that x should be pulled out only if it meets this criteria. Once this x is out, inc is applied to it. Binding all this to s gives us an infinite set. Hence, the (take 5 s) and the expected result that we see.

Another way to achieve the same result is to use map and filter.

user=> (def s (map inc (filter (fn [x] (zero? (rem x 4))) nums)))
#'user/s
user=> (take 5 s)
(1 5 9 13 17)
user=>

Here we create a predicate (fn [x] (zero? (rem x 4))) and pull out xs from nums only if this predicate is satisfied. This is done by filter. Note that as Clojure is lazy, what filter gives is only a promise of giving the next number that satisfies the predicate. It does not (and cannot in this particular case) evaluate the entire list. One we have this stream of xs, its simplay a matter of mapping inc to it (map inc ....

The choice between List Comprehension i.e. for and map/filter is largely a matter of user preference. There is no major advantage of one over the other.

Lisp

Sequence Functions

(first coll)

Gets the first element of a sequence. Returns nil for an empty sequence or nil.

user=> (first (list 1 2 3 4))
1
user=> (first (list))
nil
user=> (first nil)
nil
user=> (first (drop 3 (list 1 2 3 4)))
4

(rest coll)

Gets everything except the first element of a sequence. Returns nil for an empty sequence or nil.

user=> (rest (list 1 2 3 4))
(2 3 4)
user=> (rest (list))
nil
user=> (rest nil)
nil
user=> (rest (take 3 (list 1 2 3 4)))
(2 3)

(map f colls*)

Applies f lazily to each item in the sequences, returning a lazy sequence of the return values of f.

Because the supplied function always returns true, these both return a sequence of true, repeated ten times.

user=> (map (fn [x] true) (range 10))
(true true true true true true true true true true)
user=> (map (constantly true) (range 10)) 
(true true true true true true true true true true)

These two functions both multiply their argument by 2, so (map ...) returns a sequence where every item in the original is doubled.

user=> (map (fn [x] (* 2 x)) (range 10))
(0 2 4 6 8 10 12 14 16 18)
user=> (map (partial * 2) (range 10))
(0 2 4 6 8 10 12 14 16 18)

(map ...) may take as many sequences as you supply to it (though it requires at least one sequence), but the function argument must accept as many arguments as there are sequences.

Thus, these two functions give the sequences multiplied together:

user=> (map (fn [a b] (* a b)) (range 10) (range 10))
(0 1 4 9 16 25 36 49 64 81)
user=> (map * (range 10) (range 10))
(0 1 4 9 16 25 36 49 64 81)

But the first one will only take two sequences as arguments, whereas the second one will take as many as are supplied.

user=> (map (fn [a b] (* a b)) (range 10) (range 10) (range 10))
java.lang.IllegalArgumentException: Wrong number of args passed
user=> (map * (range 10) (range 10) (range 10))
(0 1 8 27 64 125 216 343 512 729)

(map ...) will stop evaluating as soon as it reaches the end of any supplied sequence, so in all three of these cases, (map ...) stops evaluating at 5 items (the length of the shortest sequence,) despite the second and third giving it sequences that are longer than 5 items (in the third example, the longer sequence is of infinite length.)

Each of these takes a a sequence made up solely of the number 2 and a sequence of the numbers (0 1 2 3 4) and multiplies them together.

user=> (map * (replicate 5 2) (range 5))
(0 2 4 6 8)
user=> (map * (replicate 10 2) (range 5))
(0 2 4 6 8)
user=> (map * (repeat 2) (range 5))
(0 2 4 6 8)

(every? pred coll)

Returns true if pred is true for every item in a sequence. False otherwise. pred, in this case, is a function taking a single argument and returning true or false.

As this function returns true always, (every? ...) evaluates to true. Note that these two functions say the same thing.

user=> (every? (fn [x] true) (range 10))
true
user=> (every? (constantly true) (range 10))
true

(pos? x) returns true when its argument is greater than zero. Since (range 10) gives a sequence of numbers from 0 to 9 and (range 1 10) gives a sequence of numbers from 1 to 10, (pos? x) returns false once for the first sequence and never for the second.

user=> (every? pos? (range 10))
false
user=> (every? pos? (range 1 10))
true

This function returns true when its argument is an even number. Since the range between 1 and 10 and the sequence (1 3 5 7 9) contain odd numbers, (every? ...) returns false.

As the sequence (2 4 6 8 10) contains only even numbers, (every? ...) returns true.

user=> (every? (fn [x] (== 0 (rem x 2))) (range 1 10))
false
user=> (every? (fn [x] (== 0 (rem x 2))) (range 1 10 2))
false
user=> (every? (fn [x] (== 0 (rem x 2))) (range 2 10 2))
true

If I had a need, elsewhere, to check if a number were even, I might, instead, write the following, making (even? num) an actual function before passing it as an argument to (every? ...)

user=> (defn even? [num] (== 0 (rem num 2)))
#<Var: user/even?>
user=> (every? even? (range 1 10 2))
false
user=> (every? even? (range 2 10 2))
true

Complementary function: (not-every? pred coll)

Returns the complementary value to (every? pred coll). False if pred is true for all items in the sequence, true if otherwise.

user=> (not-every? pos? (range 10))
true
user=> (not-every? pos? (range 1 10))
false

Looping and Iterating

Three different ways to loop from 1 to 20, increment by 2, printing the loop index each time (from mailing list discussion):

;; Version 1
(loop [i 1]
  (when (< i 20)
    (println i)
    (recur (+ 2 i))))
 
;; Version 2
(dorun (for [i (range 1 20 2)]
         (println i)))
 
;; Version 3
(doseq i (range 1 20 2)
  (println i))

Mutual Recursion

Mutual recursion is tricky but possible in Clojure. The form of (defn ...) allows the body of a function to refer to itself or previously existing names only. However, Clojure does allow dynamic redefinition of function bindings, in the following way:

;;; Mutual recursion example
 
; forward declaration
(def even?)
 
; define odd in terms of 0 or even
(defn odd? [n]
  (if (zero? n)
      false
      (even? (dec n))))
 
;define even? in terms of 0 or odd
(defn even? [n]
  (if (zero? n)
      true
      (odd? (dec n))))
 
; Is 3 even or odd?
(even? 3)
 
user=> (even? 3)
false

Mutual recursion is not possible in internal functions defined with let. To declare a set of private recursive functions, you can use the above technique with defn- instead of defn, which will generate private definitions.

However one can emulate mutual recursive functions with loop and recur.

(use 'clojure.contrib.fcase)
 
(defmacro multi-loop
  [vars & clauses]
  (let [loop-var  (gensym "multiloop__")
        kickstart (first clauses)
        loop-vars (into [loop-var kickstart] vars)]
    `(loop ~loop-vars
       (case ~loop-var
          ~@clauses))))
 
(defn even?
  [n]
  (multi-loop [n n]
    :even (if (zero? n)
            true
            (recur :odd (dec n)))
    :odd  (if (zero? n)
            false
            (recur :even (dec n)))))

Collection Abstractions

Concurrency

Macros

Being a Lisp, Clojure also supports a very powerful facility: Macros. So what are macros? One can think of macros as functions, which are called at compile time. They take data structures and transform them to other data structures, which are handed off again to the compiler for further processing. This is a very powerful tool, but also easy to misuse (or even abuse). So let us dive right into the matter and learn with an example.

The first question, you should always ask, before starting to write a macro: Is it really necessary? Most of the "macros" could as well be written as function. This is almost always preferable over a macro. A rough guideline when a macro is really necessary, can be the question "Do we have to control the evaluation of the arguments?" Famous examples for this case are let and and.

As example we will implement a simple version of the defvar macro from clojure.contrib.def. Obviously we need a macro, since you must not evaluate the name of the variable we want to define. So what do we need? A symbol denoting the name of the variable, a value assigned to it and a docstring describing the value. And how should it look like?

(defvar some-var some-value "a doc string")

To translate this into a macro, we have to think about, what we want to do. How would it look like, if we did it manually?

(def #^{:doc "a doc string"} some-var some-value)

Having this representation is actually already half the work. And actually many macros are born from the observation of repetition. So, now our macro. We use the ` (syntax-quote) to turn our code above straight into a macro.

(defmacro defvar
  [name value docstring]
  `(def ~(with-meta name {:doc docstring}) ~value))

So the macro is basically a function of three parameters. The compiler calls the macro at compile time and passes it three arguments: the name, the value and the docstring. Since we have to return something, which the compiler can further process. In this case, this is a list , the symbol def, a map with the key :doc and the corresponding docstring, the symbol denoting the variable's name and finally the value. We could do this for example in the following way.

(list 'def (with-meta name {:doc docstring}) value)

However compared to the ` form above, this is more verbose and harder to read. Further more you have to pay attention whether to use ' or ` for quoting in the list. Using ' to quote will resolve the quoted var in the namespace where the macro is used. Using ` however will resolve the quoted var in the namespace, where the macro was defined. Since the latter is almost always what you want, using the ` is generally the better way.

However now that we quoted the whole form with `, how to we get the arguments in there?

`(def (with-meta name {:doc docstring}) value)

This is wrong, because now the arguments are quoted. The quoted form doesn't know, that eg. name is an argument and not part of the expansion. We can check the expansion with macroexpand or macroexpand-1.

user=> (macroexpand-1 `(def (with-meta name {:doc docstring}) value))
(def (clojure/with-meta clojure/name {:doc user/docstring}) user/value)

So what happens here? Since all the symbols are quoted with a ` they are resolved in the defining namespace, in this case user. (Exercise: compare to (macroexpand-1 '(def (with-meta name {:doc docstring}) value))) name and with-meta are already defined in the clojure namespace, which is refered in the user namespace. Hence they are resolved as clojure/name resp. clojure/with-meta. value is not defined, hence it is resolved to the defining namespace, user in this case. Analog for docstring. def is a special case since it is a special form. A simple def always denotes the "usual" def. A var named def must be refered to as namespace/def. Hence it is left alone by the `.

We see, we have to tell the ` form, what is an argument and what is part of the macro expansion. This is exactly done, with ~. Anything preceded with a ~ is "unquoted" and replaced by its value. So when we have a look at the definition of defvar, we see, that value is unquoted and hence replace by its value. Similar for the with-meta call. So one can also unquote a function call in a quoted form.

Let's check, what we got so far.

user=> (macroexpand-1 '(defvar foo :bar "a doc string"))
(def foo :bar)
user=> (defvar foo :bar "a doc string")
#=(var user/foo)
user=> (doc foo)
-------------------------
user/foo
nil
  a doc string
nil

So everything works as expected. Although the expansion of the macro shows no sign of the docstring, we see, that with-meta did its job.

The cunning reader may have already noticed, that there is another unquote reader macro: ~@. To demonstrate its use, we have a look at another example: a simple apply macro.

(defmacro simple-apply
  [f args]
  `(~f ~@args)))

To put it in a simple picture: ~@ takes the following thing unquotes it and "removes" the parentheses, which means "thing" should be a seq. Using the above definition (simple-apply + [1 2 3]) expands to (+ 1 2 3). Would we have used the ~, the macro would expand to (+ [1 2 3]).

A caveat of macros however is the source of its power: the arguments are not evaluated! So compare the following example.

(defn set-title-and-show
  [frame title]
  (.setTitle frame title)
  (.setVisible frame true)
  frame)
 
(defmacro set-title-and-show
  [frame title]
  `(do
    (.setTitle ~frame ~title)
    (.setVisible ~frame true)
    ~frame))
 
(set-title-and-show (new JFrame) "Hello")

The macro is not only a bad idea because the function does the same job, it's also completely broken. Image you call the above function as shown. What happens? The (new JFrame) gets evaluated and returns a frame. This is passed to the function and everything works as expected. Now imagine this function would have been defined as a macro as shown above. What happens? The (new JFrame) is NOT evaluated. So the macro actually gets the form passed. So the macro expands to a form containing three calls to (new JFrame). So the macro sets the title on one frame, shows another and returns a third, completely fresh frame.

To remedy this, we have to introduce a temporary variable to hold the intermediate result. This happens implicitely in the function call.

`(let [~'frame ~frame]
   (.setTitle ~'frame ~title)
   (.setVisible ~'frame)
   ~'frame)

We defined a let with a variable frame. Note that ~' prevents the resolution of the symbol. (Check with macroexpand). Now everything works out.... But wait. We are some dirty little buggers, eh? This is a so-called unhygienic macro. It hijacks the meaning of frame. And this is almost always not what you want! Consider the following code using our macro.

(let [frame "Hello"]
  (set-title-and-show (new JFrame) frame))

Check the expansion with macroexpand. The problematic line is (.setTitle frame frame). Suddenly the meaning of the user's frame is different. This effect can be used, but only in case it is clearly documented and well thought out, eg. this in Clojure's proxy. Now one can try to come up with ever more spiffy names or one uses gensym.

(let [xframe (gensym)]
  `(let [~xframe ~frame]
     (.setTitle ~xframe ~title)
     (.setVisible ~xframe true)
     ~xframe))

gensym generates a symbol which is guaranteed not to be used by the user's code. Since this is a common scheme, Clojure also provides a short: frame#. The trailing # works as an implicit gensym. However this short-cut has it's limitations. Eg. it is not possible to do something like the following.

`(~(do-something-with foo#) ~((fn [] `(foo#))))

The first form has the foo# in an unquoted expression. This is not possible. The second actually defines a second foo# since it is used in a different quoted form. In such situations one has to use gensym.

A often useful pattern for macros is the split between the macro and a driver function doing the actual work. Let's see with an example. Suppose we encounter the following pattern.

(try
 (let [f (new FileInputStream fname)]
   (do-something-with f))
 (finally
   (.close f)))

We do this to ensure, that the file is always closed. No matter whether an Exception is thrown or not. Such a scheme is likely to repeat in the code quite a few times. So why not turn it into a macro?

(defmacro with-open-file
 [f fname & body]
 `(try
    (let [~f (new FileInputStream ~fname)]
      ~@body)
    (finally
      (.close ~f))))
 
(with-open-file f fname
 (do-something-with f))

Now we saved quite some typing, no? However is the macro really necessary? The whole thing could have been also implemented as a function.

(defn with-open-file*
 [fname thunk]
 (try
   (let [f (new FileInputStream fname)]
     (thunk f))
   (finally
     (.close f))))
 
(with-open-file* fname (fn [f] (do-something-with f)))

However, we need to package up the body in a closure, which is called inside with-open-file*, because otherwise the body gets evaluated outside the function. But wait! Now the bells should ring! We can turn that part into a macro!

(defmacro with-open-file
 [f fname & body]
 `(with-open-file* ~fname (fn [~f] ~@body)))

(Usage as with the first macro.)

The macro together with the *-driver is actually the optimal solution. We did as much as possible in a function and only went back to the macro where absolutely necessary. But still we maintained a nice form of usage.

To wrap this chapter up, a short list of the important aspects of macro development:

• Can the same be done using a function? Then use a function!
• macroexpand and macroexpand-1 are your friends!
• Know the difference between ' and `!
• Use syntax-quote: `(...)
• Know what ~' and '~ do. Think very hard before hijacking a variable.
• Remember gensym and the # notation for generating macro-internal variables.
• Know when # does not work.
• Think very deeply before using eval. Re-think again and then don't do it.

Libraries

The lib package from clojure.contrib is now integrated into clojure. It is easy to define libraries which can be loaded by other scripts. Suppose we have an awesome add1 function which we want to provide to other developers. So what do we need? First we settle on a namespace, eg. example.ourlib. Now we have to create a file in the classpath with the filename "example/ourlib/ourlib.clj". The contents are pretty straight forward.

(clojure/ns example.ourlib)
 
(defn add1
  [x]
  (add x 1))

All we have to do now is to use the functionality of clojure/ns. Suppose we have another file, where we want to use our function. clojure/ns let's us specify our requirements in a lot of ways. The simplest is :require

(clojure/ns example.otherns
  (:require example.ourlib))
 
(defn check-size
  [x]
  (if (too-small x)
    (example.ourlib/add1 x)
    x))

But what if we need the add1 function several times? We have to type always the namespace in front. We could add a (refer 'example.ourlib) as we do with the Clojure namespace. But we can have this easier. Just use :uses instead of :require! :uses loads the library as :require does and immediately refers to the namespace.

So now we have already two small libraries which are maybe used in a third program.

(clojure/ns example.thirdns
  (:require example.ourlib)
  (:require example.otherns))

Again we can save some typing here. Similar to import we can factor out the common prefix of our libraries' namespaces.

(clojure/ns example.thirdns
  (:require (example ourlib otherns)))

Of course ourlib contains 738 more functions, not only the shown above. We don't really want to use due to the namespace combustion, but we also don't want to type the namespace all the time. So the first thing we do is an alias. But wait! You guessed it: clojure/ns helps us again.

(clojure/ns example.otherns
  (:require (example [ourlib :as ol])))

The :as takes care of the aliasing and now we can refer to our add1 function as ol/add1!

Up to now it is already quite nice. But if we think a bit about our source code organisation, we might end up with the insight, that 739 functions in one single file is maybe not the best idea to keep the overview. So we decide the do some refactoring. We create a file "example/ourlib/add1.clj" and put our function there. To avoid, that the user has to load now 500 files instead of one, we modify the "example/ourlib/ourlib.clj" file as follows.

(clojure/ns example.ourlib)
(clojure/load-resources "add1.clj")

So the user still loads the "global" ourlib.clj file, which takes care of loading the rest.

For more information see the docstring of require - (doc require).

Examples

Norvig's Spelling Corrector

See How to Write a Spelling Corrector

(defn words [text] (re-seq #"[a-z]+" (. text (toLowerCase))))
 
(defn train [features]
  (reduce (fn [model f] (assoc model f (inc (get model f 1)))) 
          {} features))
 
(def *nwords* (train (words (slurp "big.txt"))))
 
(defn edits1 [word]
  (let [alphabet "abcdefghijklmnopqrstuvwxyz", n (count word)]
    (distinct (concat
      (for [i (range n)] (str (subs word 0 i) (subs word (inc i))))
      (for [i (range (dec n))]
        (str (subs word 0 i) (nth word (inc i)) (nth word i) (subs word (+ 2 i))))
      (for [i (range n) c alphabet] (str (subs word 0 i) c (subs word (inc i))))
      (for [i (range (inc n)) c alphabet] (str (subs word 0 i) c (subs word i)))))))
 
(defn known [words nwords] (for [w words :when (nwords w)]  w))
 
(defn known-edits2 [word nwords] 
  (for [e1 (edits1 word) e2 (edits1 e1) :when (nwords e2)]  e2))
 
(defn correct [word nwords]
  (let [candidates (or (known [word] nwords) (known (edits1 word) nwords) 
                       (known-edits2 word nwords) [word])]
    (apply max-key #(get nwords % 1) candidates)))
 
(comment    
(correct "misstake" *nwords*)
(correct "speling" *nwords*)
)

Note: training with "big.txt" may require the jvm to be started with a heap bigger than its default. You can increase the heap size either by starting java with the "-server" option or setting the maximum heap size explicitly: "java -Xmx128m ..."

Simple REPL on a Socket

Note: This uses features currently available only in the SVN version of Clojure, not yet in release.

(import '(java.net ServerSocket Socket SocketException)
        '(java.io InputStreamReader OutputStreamWriter)
        '(clojure.lang LineNumberingPushbackReader))
 
(defn on-thread [f]
  (doto (new Thread f) (start)))
 
(defn create-server 
  "creates and returns a server socket on port, will pass the client
  socket to accept-socket on connection" 
  [accept-socket port]
    (let [ss (new ServerSocket port)]
      (on-thread #(when-not (. ss (isClosed))
                    (try (accept-socket (. ss (accept)))
                         (catch SocketException e))
                    (recur)))
      ss))
 
(defn repl
  "runs a repl on ins and outs until eof"
  [ins outs]
    (binding [*ns* (create-ns 'user)
              *warn-on-reflection* false
              *out* (new OutputStreamWriter outs)]
      (let [eof (new Object)
            r (new LineNumberingPushbackReader (new InputStreamReader ins))]
        (loop [e (read r false eof)]
          (when-not (= e eof)
            (prn (eval e))
            (flush)
            (recur (read r false eof)))))))
 
(defn socket-repl 
  "starts a repl thread on the iostreams of supplied socket"
  [s] (on-thread #(repl (. s (getInputStream)) (. s (getOutputStream)))))
 
(def server (create-server socket-repl 13579))
(def client (new Socket "localhost" 13579))
 
(def rdr (new LineNumberingPushbackReader 
              (new InputStreamReader (. client (getInputStream)))))
(def wtr (new OutputStreamWriter (. client (getOutputStream))))
 
(binding [*out* wtr]
  (prn '(+ 1 2 3))
  (flush)
  (read rdr))
 
(. server (close))
(. client (close))

Simple GUI : Temperature Converter

Simple example GUI application (from discussion group, updated for 20080329 release):

(import '(javax.swing JFrame JLabel JTextField JButton)
        '(java.awt.event ActionListener)
        '(java.awt GridLayout))
 
(let [frame (new JFrame "Celsius Converter")
      temp-text (new JTextField)
      celsius-label (new JLabel "Celsius")
      convert-button (new JButton "Convert")
      fahrenheit-label (new JLabel "Fahrenheit")]
    (. convert-button
        (addActionListener
           (proxy [ActionListener] []
                (actionPerformed [evt]
                    (let [c (. Double (parseDouble (. temp-text (getText))))]
                      (. fahrenheit-label
                         (setText (str (+ 32 (* 1.8 c)) " Fahrenheit"))))))))
    (doto frame 
                ;(setDefaultCloseOperation (. JFrame EXIT_ON_CLOSE)) ;uncomment this line to quit app on frame close
                (setLayout (new GridLayout 2 2 3 3))
                (add temp-text)
                (add celsius-label)
                (add convert-button)
                (add fahrenheit-label)
                (setSize 300 80)
                (setVisible true)))

Lazy Fibonacci

A function which lazily produces Fibonacci numbers:

(defn fib
  ([] (concat [0 1] (fib 0 1)))
  ([a b] (lazy-cons (+ a b) (fib b (+ a b)))))
 
user> (take 20 (fib))
(0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181)

Rich gives a better solution. Name a sequence that is seeded with [0 1] and forego the function so that instead of starting over with each call, you get a "cached infinite seq, lazily extended."

(def fib-seq
  (concat
   [0 1]
   ((fn rfib [a b]
        (lazy-cons (+ a b) (rfib b (+ a b)))) 0 1)))
 
user> (take 20 fib-seq)
(0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181)

Slight variant on Rich's that gets rid of the initial concat, while being (I think) clearer to understand :

(def fib-seq 
  ((fn rfib [a b] 
       (lazy-cons a (rfib b (+ a b)))) 
    0 1))
 
user> (take 20 fib-seq)
(0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181)

Finally, there might be a problem in the 2 precedent versions : they create fibonacci lazy-sequences that are bound to top level vars. And as such they are not garbage collected, and if used to compute a very long sequence, will grok a lot of heap. It could be smarter to define fib-seq as a no-arg function that will return a lazy-seq on demand. Then the lazy seq could be put by the caller in the appropriate scope (hopefully not the top level scope) :

(defn fib-seq []
  ((fn rfib [a b] 
       (lazy-cons a (rfib b (+ a b)))) 
    0 1))
 
user> (take 20 (fib-seq))
(0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181)

Talking to Excel

Interfacing to other programs via COM can be easily done in Java using the JACOB library. However coping with all the low-level interface when talking to an application is quite tedious. Especially since the interface is designed for C not for Java, not to mention Clojure. So we will try to improve the situation a little bit using the power of Clojure.

Basic Interface

Let's start with a simple example. We start Excel, make the instance visible. Then we shutdown the whole system again.

  (import '(com.jacob.com Dispatch
                          ComThread))
 
  (def xl (new Dispatch "Excel.Application"))
  (. Dispatch put xl "Visible" true)
  (. Dispatch call xl "Quit")
  (. ComThread Release)

The ugliness of the interface should be immediately visible. We have to call a C interface through Java. So let's wrap the "Visible" call.

  (defn excel-application-visible
    ([xl]
     ; No argument? Return the current value.
     (. Dispatch get xl "Visible"))
    ([xl v]
     ; Argument given? Set it as new value.
     (. Dispatch put xl "Visible" v)))
 
  (excel-application-visible xl true)

This is much better. We have hidden away the low-level interface behind a simple Clojure function call. Now we have to wrap all the properties and function calls into such functions. But wait! Up to now, this was nothing special. Just a function call. Clojure can save us some work here.

Functions of First Class

Defining a few functions to wrap the interface quickly shows a lot of repetition. Take the previous example: we pass-in the Excel instance and maybe a value and call the underlying interface accordingly. But this is always the same task. So we can put this functionality into a driver function and use partial evaluation to create the wrapper function.

  (defn property-driver
    ([prop xl]
     (. Dispatch get xl prop))
    ([prop xl v]
     (. Dispatch put xl prop v)))
 
  (def excel-application-visible (partial property-driver "Visible"))
 
  (excel-application-visible xl true)

So what happens here? We define a new function which is also parametrized by the property name. Then we use the built-in partial to create a specialized function, which is specialized to act on the "Visible" property. So partial has a function as return value.

Macros

Now we have a set of wrappers to hide away the low-level interface. But what about the last instruction in the first example: (. ComThread Release)? Well, this is needed to clean up the COM side and has to be called in each thread, which used COM. This can get especially tedious when faced with exceptions. To get rid of the low-level handling, we use another part of Clojure's power: macros!

  (defn with-excel*
    [thunk]
    (let [worker (fn []
                   (let [xl (excel-application-new)]
                     (try
                       (thunk xl)
                       (finally
                         (excel-quit xl)
                         (. ComThread Release)))))
          t      (new Thread worker)]
      (. t start)
      (. t join)))
 
  (defmacro with-excel
    [varname & body]
    `(with-excel* (fn [~varname] ~@body)))

First we define again a driver function, which does the heavy lifting. It takes a function, which it calls in newly created thread. The function is handed over the Excel instance. Everything is wrapped into a try to ensure proper cleanup even in case the function throws an exception.

Of course it is not our intention to always define a new function manually, when we want to call this wrapper. Hence we define a macro, which does this for us. It takes a symbol which makes the Excel instance available to the body. The body is wrapped into an anonymous function, which closes the body over its bindings and packages everything up for our wrapper.

Wrap-Up

Rewriting with this modifications our initial example gives us this nice and concise code.

  (with-excel xl
    (excel-application-visible xl true))

Of course this example isn't really useful, but it clearly shows the improvement over the previous pure Java code. Also the power lies in the part we don't see: even if we were throwing an exception in the with-excel everything would be cleaned up.

Further experimenting is left to the user. Things which were left out are eg. a macro constructing the wrappers, or a function handling Variant types of the COM interface. So experiment and have fun!

genclass and finalize

The gen-class functions provide the most general and powerful mechanism in Clojure for defining new Java classes. If you need to implement an existing interface you can probably use the simpler proxy function instead, but if you need to define new Java instance methods you may need to use gen-and-load-class or gen-and-save-class.

By way of a quick introduction, here is an example of a new class named MyClass in a package named foo. This class has two methods: mymethod, a standard Java finalize:

(gen-and-load-class 'foo.MyClass
   :methods [['mymethod [String] String]
             ['finalize [] Void/TYPE]])

At this point the class exists and is loaded into the JVM, but isn't useful yet as the methods do not have definitions. Define them like this:

(clojure/ns foo)
 
(defn MyClass-mymethod [this m]
  (str "Your arg was: " m))
 
(defn MyClass-finalize [this]
  (println "Finalizing " this))

This class can now be used like any Java class in Clojure:

(println (.mymethod (new foo.MyClass) "foo"))

That example creates a new instance, calls mymethod, and prints the return value. The finalize method will probably not be called yet, but an explicit call to the garbage collector may trigger it:

(System/gc)

Links to more Examples

• Sending Emails [4]
• Chapter about music database from "Practical Common Lisp" translated to clojure [5]
• Simple SparkLine Generator in Clojure [6]
• Using Google Charts from Clojure [7]
• Context Free Art [8]
• Socket Programming Walkthrough [9]
• Webjure: Simple Web Programming for Clojure [10]
• Compojure: Another web framework for Clojure [11]
• OpenGL Gears demo ported from JOGL demos [12]
• "Practical Common Lisp" translated into Clojure [13]

Reference

See the website: clojure.org