Building APIs with Clojure –
Building APIs with Clojure –
0 0
clojure-api-slides
Presentation on building APIs with ClojureOn Github peter / clojure-api-slides
Building APIs with Clojure
Created by Peter Marklund / @peter_marklund
What is Clojure?
-
Programming language for the JVM, JavaScript et al
-
Lisp, dynamically typed, functional with emphasis on pure functions, data-oriented
-
Released in 2007
Why Clojure?
- Functional programming
- Dynamic, concise, high programmer productivity
- Performance on par with Java
- Leverages the JVM and has direct access to Java libraries
- Supported on both the client and the server
- Syntactic extensibility via macros
- Concurrency support
Philosophy
-
Simplicity is a prerequisite for reliability
-
The complexity of our systems dominate our ability to be productive in the long run
-
Programming languages differ not in what they make possible but in what they make practical and idiomatic
-
Number of bugs is the number of lines of code to the power of 1.5
Data Orientation
- Programming is data processing
- Emphasize data over code
- Objects are ok for process but a bad fit for information
- Data is just simple immutable information
- Access data via a large set of generic functions
Death through Specificity
By choosing data instead of custom objects we get:
- A standard way to inspect/compare/modify data
- Thread safety
- Less code through code reuse and generic functions
Lisp Syntax
- Code is data
- No operators, just functions, macros, and a handful of special forms (if, def, fn, let etc.)
- Everything is an expression
- Side effects are allowed
- The reader takes code (text) and produces in memory data structures to be evaluated/compiled
- No precedence rules are needed. All functions evaluate left to right and inside out.
Three Ways to Define a Function
(defn greeting [name]
(str "Hello " name))
(def greeting (fn [name]
(str "Hello " name)))
(def greeting #(str "Hello " %))
functions can be named or anonymous. Anonymous functions
can be written in two ways. All function definitions are equivalent.Calling a Function
(greeting "Joe")
; => "Hello Joe"
(greeting)
; => ArityException
functions can be named or anonymous. Anonymous functions
can be written in two ways. All function definitions are equivalent.Multi-Arity Functions
(defn greeting
([name] (str "Hello " name))
([] (greeting "World")))
(greeting)
; => "Hello World"
(greeting "Joe")
; => "Hello Joe"
Variadic Functions and Apply
(defn complement [f]
(fn [& args]
(not (apply f args))))
(def not-nil? (complement nil?))
(not-nil? "foobar")
; => true
This is also an example of a higher order function that returns a function.Let
(let [a 1
b 2]
(+ a b))
; => 3
(defn foo [c]
(let [a 1
b 2]
(+ a b c)))
(foo 3)
; => 6
Global vs Local Scope
(def thing "global")
(let [thing "local"]
thing)
; => "local"
thing
; => "global"
Let Example
(defn authenticate [email password]
(if (and (= email "open")
(= password "sesame"))
"token"
nil))
(defn login [request]
(let [email (get-in request [:params :email])
password (get-in request [:params :password])
token (authenticate email password)]
(if token
{:status 200 :body {:token token}}
{:status 401})))
(login {:params {:email "open" :password "sesame"}})
; => {:status 200 :body {:token "token"}}
Atomic Data Types
123 ; java.lang.Long
1.23 ; java.lang.Double
"foobar" ; java.lang.String
true, false ; java.lang.Boolean
nil ; java null
#"fo.*bar" ; java.util.regex.Pattern
my-variable ; clojure.lang.Symbol
:my-keyword ; clojure.lang.Keyword
these are the usual Java types java.lang.Long/Double/String/BooleanImmutable Data Structures
'(1 "foobar" 3); clojure.lang.PersistentList
[5 nil 3] ; clojure.lang.PersistentVector
{:a 1 :b 2} ; clojure.lang.PersistentArrayMap
#{:foo :bar} ; clojure.lang.PersistentHashSet
Lists are singly linked and add efficiently at the front.
Vectors are functions that take an index and return a value.
Maps are functions that take a key and return a value.
All data structures can hold heterogeneous data and can be nested.
Lists and vectors have a sequential shape. Hash maps and sets
are associative. Vectors are also associative on the index and
work with assoc.Data as Functions
([5 nil 3] 2) ; => 3
(nth [5 nil 3] 1 :default) ; => nil
({:a 1 :b 2} :b) ; => 2
(:b {:a 1 :b 2}) ; => 2
(get {:a 1 :b nil} :b :default) ; => nil
(#{:foo :bar} :bar) ; => :bar
(contains? #{:foo :bar} :bar)
; => true
For
(for [color [:red :blue] animal [:mouse :duck]]
(str (name color) " " (name animal)))
; => ("red mouse" "red duck" "blue mouse" "blue duck")
(for [n (range 10) :when (even? n)]
n)
; => (0 2 4 6 8)
Seq: first, rest, cons
(defn my-map [f coll]
(if-let [s (seq coll)]
(cons (f (first coll))
(my-map f (rest coll)))))
(my-map #(* % 2) [1 2 3 4])
Stack Overflow
(defn my-map [f coll]
(if-let [s (seq coll)]
(cons (f (first coll))
(my-map f (rest coll)))))
(my-map #(* % 2) (range 100000))
; => StackOverflowError
Seq: recur
(defn my-map [f coll]
(let [map-fn (fn [f coll acc]
(if-let [s (seq coll)]
(recur f (rest coll)
(conj acc (f (first coll))))
acc))]
(map-fn f coll [])))
(my-map #(* % 2) (range 100000))
; => [0 2 4 6 8 10 ...]
Recur
(defn every?
"Returns true if (pred x) is true for every x in coll"
[pred coll]
(cond
(nil? (seq coll)) true
(pred (first coll)) (recur pred (next coll))
:else false))
Loop and Recur
(defn zipmap
"Returns a map with the keys mapped to vals"
[keys vals]
(loop [map {}
ks (seq keys)
vs (seq vals)]
(if (and ks vs)
(recur (assoc map (first ks) (first vs))
(next ks)
(next vs))
map)))
Notice the use of first and next which is typical in recursive code.
One reason to use seq here is that an empty seq is nil and thus false in a boolean context.
Also notice the use of a documentation string for functions.Sequence Functions
map/filter/remove/reduce
conj/cons
first/second/last/next
not-empty
every?/not-any?/some
count
take/drop
reverse
partition/flatten
interleave/interpose
group-by
sort-by
distinct
doseq
...
Sequence to Map Transformations
(def keys [:a :b])
(def values [1 2])
(into {} (map vector keys values))
(apply hash-map (interleave keys values))
(zipmap keys values)
; => {:a 1 :b 2}
(reduce (fn [m [k v]] (assoc m k v))
{}
(partition 2 [:a 1 :b 2]))
; => {:a 1 :b 2}
CSV Example
(defn csv-map [lines]
(map #(zipmap (first lines) %)
(rest lines)))
(csv-map [["Name" "Age"] ["Sven" 9] ["Anna" 6]])
; => ({"Name" "Sven", "Age" 9} {"Name" "Anna", "Age" 6})
Map Transformations
(assoc {:a 1} :b 2) ; => {:a 1, :b 2}
(dissoc {:a 1 :b 2} :b) ; => {:a 1}
(get-in {:a {:b 1}} [:a :b]) ; => 1
(update-in {:a {:b 1}} [:a :b] inc) ; => {:a {:b 2}}
(update-in {:a} [:a :b] (fnil inc 0)) ; => {:a {:b 0}}
(select-keys {:a 1 :b 2} [:b]) ; => {:b 2}
(clojure.set/rename-keys {:a 1} {:a :b}) ; => {:b 1}
(merge {:a {:b 1}} {:a {:c 2}}) ; => {:a {:c 2}}
(assoc [:foo :bar] 3 :baz) ; => IndexOutOfBoundsException
Transforming Map Values
(defn map-values [f dict]
(zipmap (keys m) (map f (vals m))))
(defn map-values [f m]
(reduce (fn [r [k v]] (assoc r k (f v)))
{}
m))
(map-values (partial * 2) {:a 1 :b 2})
; => {:a 2, :b 4}
Deep Map Merge
(defn deep-merge
"Recursively merges maps"
[& vals]
(if (every? map? vals)
(apply merge-with deep-merge vals)
(last vals)))
(deep-merge {:a {:b 1}} {:a {:c 2}})
; => {:a {:b 1 :c 2}}
Set
(clojure.set/union #{:a :b} #{:b :c})
; => #{:c :b :a}
(clojure.set/intersection #{:a :b} #{:b :c})
; => #{:b}
(clojure.set/difference #{:a :b} #{:b :c})
; => #{:a}
Strings
(re-matches #"^foo.*" "foobar")
; => "foobar"
(re-seq #"\w+" "Hello World")
; => ("Hello" "World")
(str/split "A fine day it is" #"\W+")
(str/replace "foobar foobar" #"(foo)(bar)" "$2$1")
; => "barfoo barfoo"
(str "Hello" " " "World")
; => "Hello World"
(format "Hello there, %s" "bob")
; "Hello there, bob"
; lower-case, blank?, starts-with?/ends-with?, includes?
Equality
Value semantics for all data types and structures
(= "foobar" "foobar")
; => true
(= 1 1.0)
; => false
(== 1 1.0)
; => true
(= '(1 2) [1 2])
; => true
(= {:a 1 :b 2} {:a 1 :b 2})
; => true
Truth
Only nil and false are falsey, everything else is truthy
(defn truthy? [value]
(if value true false))
(truthy? 0)
; => true
(truthy? "")
; => true
(truthy? [])
; => true
(truthy? nil)
; => false
The not-empty function
is useful if you need empty data to evaluate to boolean false.Control Flow
(if true "it is true" "it is false")
; => "it is true"
(if (or (and "foo" "bar") "baz") "true" "false")
; => "true"
(def n 5)
(cond
(< n 0) "is negative"
(= n 0) "is zero"
(> n 0) "is positive")
; => "is positive"
there is also when and case. Note that and, or etc. are variadic in arity.
Sometimes you need to use do to wrap multiple statements for example in an if clause.Nil
(:foo nil)
; => nil
(get nil :foo :default)
; => :default
(first nil)
; => nil
(next nil)
; => nil
(inc nil)
; => NullPointerException
Defining a Namespace
; File: app/util.clj
(ns app.util)
(def pi 3.14)
(defn area [radius]
(* pi (Math/pow radius 2)))
Using a Namespace
; File: app/web.clj
(ns app.web
(require [app.util :as util]))
util/pi
; => 3.14
(util/area 2)
; => 12.56
Introspection
(class 1.23)
; => java.lang.Double
(ancestors (class 1.23))
; => #{java.io.Serializable java.lang.Comparable ...}
(instance? Double 1.23)
; => true
(map #(% 1.23) [number? float? integer? zero?
string? keyword? map? vector? nil?])
; => (true true false false false false false false false)
(clojure.reflect/reflect 1)
; => detailed info on class, ancestors, methods etc.
Destructuring
(defn map-invert [map]
(reduce (fn [m [k v]] (assoc m v k))
{}
map))
(map-invert {:a 1 :b 2})
; => {1 :a, 2 :b}
(def data {:a 1 :b 2})
(let [{:keys [a b c] :or {c 3}} data]
(println a b c))
; => 1 2 3
; => nil
defrecord
(defrecord User [name
email
admin])
(def joe (->User "Joe" "joe@example.com" false))
(def peter (map->User {
:name "Peter"
:email "peter@example.com"
:admin true}))
(class peter) ; => user.User
defrecord with Type Hints
(defrecord User [^String name
^String email
^Boolean admin])
Type Hints and Performance
(defn len [x]
(.length x))
(defn len2 [^String x]
(.length x))
(time (reduce + (map len (repeat 1000000 "asdf"))))
; => "Elapsed time: 3007.198 msecs"
(time (reduce + (map len2 (repeat 1000000 "asdf"))))
; => "Elapsed time: 308.045 msecs"
Structural typing
Example from TypeScript:
interface Named {
name: string;
}
class Person {
name: string;
}
let p: Named;
p = new Person(); // OK, because of structural typing
See the TypeScript documentation
for more details.defrecord with schema
(require '[schema.core :as s])
(s/defrecord Ingredient
[name :- s/Str
quantity :- s/Int
unit :- s/Keyword])
(s/defrecord Recipe
[name :- s/Str
ingredients :- [Ingredient]])
Schema Validation
(require '[schema.core :as s])
(def flour (map->Ingredient {:name "Flour"}))
(def pancakes (map->Recipe {
:name "Pancakes"
:ingredients [flour]
}))
(s/validate Recipe pancakes)
; => ExceptionInfo Value does not match schema...
defn with schema
(s/defn add-ingredients :- Recipe
[recipe :- Recipe & ingredients :- [Ingredient]]
(update-in recipe [:ingredients] into ingredients))
defprotocol
(defprotocol Lifecycle
(start [component]
"Synchronous, returns updated version of component")
(stop [component]
"Synchronous, returns updated version of component"))
defrecord with protocols
(defrecord Application [database config]
Lifecycle
(start [component]
(println "Starting Application..."))
(stop [component]
(println "Stopping Application...")))
You can use extend-protocol to extend basic types with protocols.
You can check if an object satisfies a protocol with satisfies?extend Declarations
(defprotocol Foo
(foo [this]))
(defprotocol Bar
(bar [this]))
(extend java.lang.Number
Bar
{:bar (fn [this] 42)})
(extend java.lang.String
Foo
{:foo (fn [this] "foo")}
Bar
{:bar (fn [this] "forty two")})
extend Usage
(satisfies? Foo "foobar") ; => true
(foo "foobar") ; => "foo"
(bar "foobar") ; => "forty two"
(bar 123) ; => 42
(foo 123) ; => IllegalArgumentException No implementation of method
Thread First Macro
(require '[clojure.string :as str])
(def channels ["TV4" "TV4 totalt"])
(defn channel-key [channel]
(-> channel
(str/lower-case)
(str/replace #" " "_")
(keyword)))
(map channel-key channels) ; => [:tv4 :tv4_totalt]
For a discussion around handling errors in a function chain, see
"Good Enough" error handling in ClojureThread Last Macro
(def divisible? #(or (zero? (mod % 3)) (zero? (mod % 5))))
(->>
(range 1000)
(filter divisible?)
(reduce +)
println)
(reduce + (filter divisible? (range 1000)))
Euler problem 1 - the sum of all numbers under 1,000 that are divisible by either 3 or 5Concurrency - Atom
(def counter (atom 0))
(swap! counter inc)
(println @counter)
; => 1
(swap! counter inc)
(println @counter)
; => 0
Allows thread safe access to shared data.
Example usage: memoize function.
The available reference types in Clojure are
Refs, Atoms, Agents, and Vars.Atom Example
(def counter (atom 0))
(defn print-inc [value]
(println (str "thread " (.getId (Thread/currentThread))
" value " (inc value)))
(inc value))
(let [n 5]
(future (dotimes [_ n] (swap! counter print-inc)))
(future (dotimes [_ n] (swap! counter print-inc)))
(future (dotimes [_ n] (swap! counter print-inc))))
@counter
; => 15
Concurrency - Ref
(def foo (ref 0))
(def bar (ref 1))
(alter foo inc)
; => IllegalStateException No transaction running
(dosync
(alter foo inc)
(alter bar dec))
@foo ; => 1
@bar ; => 0
Concurrency - Agent
(def foo (agent 0))
(send foo inc)
@foo ; => 1
(send foo #(/ % 0))
(send foo inc)
; => ArithmeticException Divide by zero
Concurrency - Agent, Take 2
(defn error-handler [agent e]
(println "Agent" agent " threw error " e))
(def foo (agent 0 :error-handler error-handler))
(send foo inc)
@foo ; => 1
(send foo #(/ % 0))
; => Agent ... threw error ...
(send foo inc)
@foo ; => 2
Concurrency - Channels
(require '[clojure.core.async :refer [chan >!! <:!!]])
(def foo (chan 10)) ; => 10 buffered channel
(>!! foo 1)
(<!! foo)
; => 1
(<!! foo)
; blocks...
Concurrency - Channels, Take 2
(require '[clojure.core.async :refer [chan go alts! >!]])
(def foo (chan))
(def bar (chan))
(go
(loop []
(let [v (alts! [foo bar])]
(println "Received " v))
(recur)))
(go (>! foo 1))
(go (>! bar 2))
Infinite/Lazy Sequences
(defn new-counter []
(let [count (atom 0)]
#(swap! count inc)))
(def counter (new-counter))
(take 10 (repeatedly counter))
; => (1 2 3 4 5 6 7 8 9 10)
(take 10 (iterate inc 0))
; => (0 1 2 3 4 5 6 7 8 9)
(take 10 (repeatedly rand))
; => (0.9000394153557845 0.6840742826831324 ...)
Macros
(defmacro unless [arg & body]
`(if (not ~arg)
(do ~@body)))
(unless false (println "body executing"))
; => body executing
(macroexpand '(unless false (println "body executing")))
; => (if (clojure.core/not false)
; (do (println "body executing")))
A nice walk through of macros is available at
Learn X in Y minutes and at
Clojure for the Brave and True.Index of Chars in Java
// From Apache Commons Lang, http://commons.apache.org/lang/
public static int indexOfAny(String str, char[] searchChars) {
if (isEmpty(str) || ArrayUtils.isEmpty(searchChars)) {
return -1;
}
for (int i = 0; i < str.length(); i++) {
char ch = str.charAt(i);
for (int j = 0; j < searchChars.length; j++) {
if (searchChars[j] == ch) {
return i;
}
}
}
return -1;
}
Index of Chars in Clojure 1
(defn indexed [coll] (map-indexed vector coll))
(defn index-filter [pred coll]
(for [[idx elt] (indexed coll) :when (pred elt)] idx))
(defn index-of-any [pred coll]
(first (index-filter pred coll)))
(index-of-any #{\b \y} "zzabyycdxx")
; => 3
Index of Chars in Clojure 2
(defn index-filter [pred coll]
(let [index first
value second]
(->> (map-indexed vector coll)
(filter (comp pred value))
(map index))))
(first (index-filter #{\b \y} "zzabyycdxx"))
; => 3
Working with Files
(spit "/tmp/foobar.txt" "here is some contents")
(slurp "/tmp/foobar.txt")
Java Interop
(.toUpperCase "fred") ; => "FRED"
(System/getenv "USER") ; => "peter.marklund"
(Thread/sleep (rand-int 1000))
Math/PI ; => 3.141592...
(doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2))
; => {"a" 1, "b" 2}
(import [java.util Date])
(Date.)
; => #inst "2016-04-14T14:38:36.486-00:00"
Dynamic Function Dispatch
(defn greet [name]
(str "Hello " name))
(defn load-var [ns-name var-name]
(require (symbol ns-name))
(ns-resolve (find-ns (symbol ns-name))
(symbol var-name)))
((load-var (ns-name *ns*) "greet") "Joe")
; => "Hello Joe"
Web Development with Ring
- Ring is the standard web application library for Clojure
- It's light weight and has a simple API
- It comes with useful middleware for code reloading, JSON and parameter handling, authentication etc.
- It's easily deployed to Heroku with the Jetty adapter
Ring Handlers
Handlers are pure functions that take an HTTP request map and return an HTTP response map
(defn home [request]
{:status 200
:headers {"Content-Type" "text/html"}
:body "Hello World"})
Request Map Keys
:uri
:headers
:query-string
:request-method
:protocol
:body
...
Ring Middleware
Middleware are handlers that wrap other handlers.
(defn wrap-response-time [handler]
(fn [req]
(time (handler req))))
Middleware for JSON APIs
(-> handler
(wrap-keyword-params)
(wrap-json-params {})
(wrap-params {})
(wrap-json-response {:pretty true}))
These middleware together conviently put query params and JSON
bodies in a params map in the request. The wrap-json-response
middleware will serialize a data structure in the response body
to JSON. In development you also want the wrap-reload middleware
for reloading code.Ring Example
(ns app.web
(:require [ring.adapter.jetty :as jetty]))
(defn app [request]
{:status 200
:headers {"Content-Type" "text/html"}
:body "Welcome to my Clojure API"})
(defn port []
(Integer. (or (System/getenv "PORT") 5000)))
(defn -main []
(jetty/run-jetty app {:port (port) :join? false}))
Calling REST APIs
(require '[clj-http.client :as client])
(client/get "https://api-endpoint" {
:query-params {} :as :json})
(client/post "https://api-endpoint" {
:form-params {:foo "bar"}
:content-type :json
:oauth-token "token"})))
Parallell API Calls
(defn invoke-api-1 []
(Thread/sleep 1000)
"api-1-data")
(defn invoke-api-2 []
(Thread/sleep 1000)
"api-2-data")
(time (println (pcalls invoke-api-1 invoke-api-2)))
; => (api-data-1 api-data-2)
; => "Elapsed time: 1004.511257 msecs"
Named Parallell API Calls
(def apis {
:api1 invoke-api-1
:api2 invoke-api-2})
(defn pcalls-map [map]
(zipmap (keys map)
(apply pcalls (vals map))))
(time (println (pcalls-map apis)))
; => {:api1 api-data-1 :api2 api-data-2}
; => "Elapsed time: 1005.052411 msecs"
JSON
(require '[cheshire.core :as json])
(json/generate-string {:foo "bar"})
; => {"foo":"bar"}
(json/parse-string "{\"foo\":\"bar\"}" true)
; => {:foo "bar"}
API Boilerplate
There is a bare bones API example and a more extensive CMS REST API on Github.
Resources
Building APIs with Clojure
Created by Peter Marklund / @peter_marklund