The video is up for a talk I gave at QCon 2014 in which I spoke about the benefits of using immutable data to build a frontend and my experience working on Prismatic’s ClojureScript web application.

Watch on InfoQ

Brief summary: Mutable state is ubiquitous in UI programming and contributes to large amounts of additional complexity and bugs. Functional reactive data-flows can greatly simplify rendering logic and state management. ClojureScript’s immutable data structures pair nicely with React.js to build UIs that are declarative and performant.

This function uses the twitter-text-js library to count the length of a tweet after link shortening by calling twttr.txt.getTweetLength(). Our misbehaving ClojureScript function looked something like:

1
2
3
4

(defn tweet-length [text]
  (if ^boolean (.-twttr js/window)
    (.. js/window -twttr -txt (getTweetLength text))
    (count text)))

Pretty straight forward: it checks to see if the external script has been before calling a function that it provides or falls back counting the number of characters.

Let’s see what the advanced compiled code looks like:

1
2
3

function(a) {
  return window.de ? window.de.txt.getTweetLength(a) : M(a)
}

Spot the issue? The optimizer munged window.twttr to window.de even though the extern was configured.

As you may have guessed by now, externs are not recognized when referenced through window. It makes sense, but may not be immediately obvious, especially when debugging. After all, this function only broke after advanced compilation was used.

The fixed function:

1
2
3
4

(defn tweet-length [text]
  (if (exists? js/twttr)
    (.. js/twttr -txt (getTweetLength text))
    (count text)))

Note the use of cljs.core/exists?, to test whether a variable exists. Simply testing js/twttr as boolean when the script hasn’t been evaluated would throw a ReferenceError.

The advanced compiled result looks like:

1
2
3

function(a) {
  return "undefined" !== twttr ? twttr.txt.getTweetLength(a) : M(a)
}

Lessons learned: reference externs directly from js/ and, more generally, avoid js/window unless you’re accessing one of its actual properties.

Using the advanced compiler can bring tons of little nuances like this. They’re often things you may not consider when writing code and thinking about its correctness.

In the end, a rather simple lesson learned, but hopefully it’s helpful to others.

Let’s see how you might use the Push State API directly to accomplish this:

1
2
3
4
5
6
7
8
9
10

(declare scroll-top) ;; returns current scroll position

(defn set-scroll-top! []
  (let [state (or (.-state js/history) #js {})]
    (aset state "scroll-top" (scroll-top))
    (.replaceState js/history state )))

(defn get-scroll-top []
  (when-let [state (.-state js/history)]
    (aget state "scroll-top")))

This works, but what if we could interface with history.state as if it were Clojure data? More specifically, stateful Clojure data, like an atom.

It might look something like this:

1
2
3
4
5
6
7
8
9

(require 'history)

(declare scroll-top)

(defn set-scroll-top! []
  (swap! history/state assoc :scroll-top (scroll-top)))

(defn get-scroll-top []
  (:scroll-top @history/state))

And in fact, this can be done by anonymously implementing a few protocols.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

(ns history)

(def state
  (let [clj-state #(js->clj (.-state js/history) :keywordize-keys true)]
    (reify
      IDeref
      (-deref [_]
        (clj-state))
      IReset
      (-reset! [_ v]
        (.replaceState js/history (clj->js v) (.-title js/document)))
      ISwap
      (-swap! [s f]
        (-reset! s (f (clj-state))))
      (-swap! [s f x]
        (-reset! s (f (clj-state) x)))
      (-swap! [s f x y]
        (-reset! s (f (clj-state) x y)))
      (-swap! [s f x y more]
        (-reset! s (apply f (clj-state) x y more))))))

You can checkout a more complete wrapper for js/history here: https://gist.github.com/loganlinn/930c043331c52cb73a98

The Venue

The event was held at the George Washington Masonic Memorial, a landmark structure in Alexandria, Virginia.

Being a VA native, I have seen this building several times while visiting the area, but never been inside. Even if I had, I don’t think I’d imagine that it would be a good spot to host a programming conference, but it was.

There was a single stage with theater seating — the acoustics were great and it seemed to be exactly the right size. Not being able to bring food & drinks (i.e., caffeine) into the theater might have been the only inconvenience, but totally understandable given the significance of the building.

The Talks

Day 1

Stu Halloway – data.fresssian

Stu kicked off the conference by introducing a new data encoding library, data.fressian (pronounced ‘data freshen’), that’s being used in Datomic to efficiently transport data across the wire. It has similar properties to edn, like being language agnostic, self-describing, and extensible, but in a binary format. The library’s designers prioritized effectiveness before efficiency to avoid common idiosyncrasies found in existing (JVM) serialization libraries.

Checkout the clojure implementation and the format’s wiki for more information.

Update: video

Mike Anderson – Enter the Matrix

Mike presented the core.matrix API for matrix and vector operations. At its core, the library is a set of protocols that abstract both fundamental (required) and composite (optional) matrix operations. This allows any of the existing matrix libraries for the JVM to be leveraged. By providing a standard API to build applications and leaving the underlying matrix engine pluggable, you can optimize for your particular use case. Since this is all protocol-based, you could conceivably mix & match them.

core.matrix currently ships with protocol extensions for vectorz-clj, Clatrix and NDArray.

Update: video

Aria Haghighi – Prismatic’s Schema

I was looking forward to this talk about Prismatic’s schema library, as I had seen their introductory blog post and started playing with it on my flight to the conference. Schema, as its name suggests, is a way to describe and validate the shape of your data.

Aria made the point that while it’s certainly possible to infer the shape of data from the source code or convey with a docstrings, it’s time consuming and imprecise even in the most trivial of situations. Schemas provide a clear and concise documentation that is non-intrusive like static type systems can be. He reported noticeable productivity gains at Prismatic from using schema and simple code organizational patterns.

Aside from documentation and validation, he mentioned that schemas can be used for other interesting things, like code generation. Prismatic has been using schema to generate ObjectiveC and Java code for their mobile iOS and Android applications, which they plan to open source in the future.

Update: video

Chris Houser & Jonathan Claggett – Illuminated Macros

Clojure has powerful macro facilities, but they can be hard to debug when something goes wrong. There aren’t many ways for a macro’s author to provide fine-grained error messages or suggested fixes. And aside from macroexpand, macros can be opaque to inspect and understand.

Chouser presented properties of macros and ideas that we can learn from (good parts of) Racket’s macros. And Jonathan demoed seqex, a library for “s-expression regexes.”

He gave examples by describing the syntax for the let macro that provides helpful feedback to users and even generates a colorized, complete syntax specification. It would be really cool to see some of these patterns become popular in Clojure.

Update: video

Nada Amin & William Byrd – From Greek to Clojure!

A talk that was equally entertaining as it was thought-provoking; Nada and Will gave tips on how to approach technical research papers with some live coding using core.logic.

Update: video

Rich Hickey – Harmonikit

Rich took a break from his normal keynote this year to present a “fun” project that he’s been working on. Harmonikit is a musical synthesis system built with core.async, overtone, Supercollider, Lemur and a keyboard controller that I didn’t catch the name of.

As always, Rich did an exemplary job of dissecting & describing the problem domain, choosing the distinct problems he wants to address and modeling a system to do it. This system focused on realistically synthesizing acoustics of harmonic (e.g., stringed, wind) instruments.

I won’t do this project & presentation justice if I continue attempting to describe it, so I’ll spare you (and myself) and let you check out the recording when it is released.

Update: video

Day 2

Carin Meier – Learning to Talk to Machines with Speech Acts

Carin is a great presenter and story teller. She told a story illustrating how computer science can learn from philosophy and vice versa. She did an awesome demo of a flying drone using babar and clj-drone, and of course, now I want a drone.

Update: video

Timothy Baldridge – core.async

Tim skipped talking about the rationale of core.async, which is well documented elsewhere, and jumped right into the usage and mechanics of the library that has recently taken the Clojure & ClojureScript communities by storm. I’ve been using this library in a side-project, yet learned several new things like the size of the threadpool.

Update: video

Michael O’Keefe – Predicting Vehicle Usage with Clojure

Michael is a mechanical engineer by trade who has written several vehicle efficiency simulators in the past, with his most recent being in Clojure. He had helpful graphics of the simulation model and totally awesome original illustrations. It seemed that Clojure’s immutable values and ease of building dataflows via functional composition were significant advantages of this type of system.

Interestingly, he was also the only presenter to use LightTable editor during a demo (AFAIK), which was very easy to follow (one presenter used vim, and the rest used emacs)

Update: video

Tim Ewald – Clojure: Programming with Hand Tools

This was one of the most memorable talks I’ve been to. This wasn’t a Clojure or even programming specific talk. Instead, Tim shed some light on the craft of woodworking, particularly with hand-tools rather than than power-tools. Here are a few of the observations he made that I can remember:

• The less you do, the less you know. Automating a complex process isn’t a bad thing in and of itself, but it can build reliance on machines to do a job; humans begin to lose the skills and appreciation for the task at hand.
• Your tools change the way you perceive the world. This is related to the “if all you have is a hammer, everything begins to look like a nail” lesson. Confirmation bias leads to bad decisions that don’t appear bad on the surface.
• Simpler tools give you a broader perspective. You are closer to your trade when your tools allow you (humans) to create unique items, leaving automation to handle the tasks that never change. Regardless of how archaic hand-tools may seem, it can be impossible to match the level of precision that they provide.

Looking forward to watching the recording of this talk.

Update: video

Day 3

Thomas Kristensen – Propagators in Clojure

Thomas presented propaganda, a library for declarative logic programming with propagators. It provides similar functionality as core.logic, but benefits from a more transparent processing model and has efficiency benefits for numerical problems.

Update: video

Fogus – Zeder: Production Rule Systems in Clojure

Zeder is a production rule system (think OSP5), that Fogus has been building. The syntax was very similar to Datomic’s and looks very powerful. He doesn’t appear to have released the source for it yet, but I did find a gist with an example.

Update: video

Russ Olsen – To the Moon!

Russ gave a vivid recollection of America’s race to the moon in the 60’s and the little and big lessons we can learn from it today. The talk’s theme was centered around one of JFK’s quotes, “We choose to go to the moon, not because it’s easy, but because it’s hard.” As a Millennial, I’ve heard and seen this story in popular culture countless times, but he covered some amazing context & details that most leave out.

Russ is a great speaker and the story was inspirational. Definitely watch this talk when the video is released.

Update: video

Last but Not Least

There were a handful of talks not mentioned here, including some lightning talks. One of which was from Alan Dipert on gherkin, aka bash killer. I also got a kick out of talks by Steve Miner and Amit Rathore.

Update: video

Final Words

Huge thanks to conference organizers, Alex Miller and Lynn Grogan, for your hard work and leadership. I am immensely grateful.

And, of course, thanks to all of the speakers, as well as the sponsors, for making it happen. For me, this has been a real testament to the authenticity of the Clojure community.

Lastly, I wanted to give a big thank you to my employer, Huddler, for sponsoring me to attend this conference (and letting me work in Clojure).

Until next year!

Here’s a my implementation that I thought was decent enough to share :)

Normally, the algorithm finds the shortest path to all other nodes in graph, but you can optionally pass a specific destination node. The algorithm will exit early once destination has been visited.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

(def ^:private inf Double/POSITIVE_INFINITY)

(defn update-costs
  "Returns costs updated with any shorter paths found to curr's unvisisted
  neighbors by using curr's shortest path"
  [g costs unvisited curr]
  (let [curr-cost (get costs curr)]
    (reduce-kv
      (fn [c nbr nbr-cost]
        (if (unvisited nbr)
          (update-in c [nbr] min (+ curr-cost nbr-cost))
          c))
      costs
      (get g curr))))

(defn dijkstra
  "Returns a map of nodes to minimum cost from src using Dijkstra algorithm.
  Graph is a map of nodes to map of neighboring nodes and associated cost.
  Optionally, specify destination node to return once cost is known"
  ([g src]
    (dijkstra g src nil))
  ([g src dst]
    (loop [costs (assoc (zipmap (keys g) (repeat inf)) src 0)
           curr src
           unvisited (disj (apply hash-set (keys g)) src)]
      (cond
       (= curr dst)
       (select-keys costs [dst])

       (or (empty? unvisited) (= inf (get costs curr)))
       costs

       :else
       (let [next-costs (update-costs g costs unvisited curr)
             next-node (apply min-key next-costs unvisited)]
         (recur next-costs next-node (disj unvisited next-node)))))))

It could be used like like so:

1
2
3
4
5
6
7
8
9
10
11

(def demo-graph {:red    {:green 10, :blue   5, :orange 8},
                 :green  {:red 10,   :blue   3},
                 :blue   {:green 3,  :red    5, :purple 7},
                 :purple {:blue 7,   :orange 2},
                 :orange {:purple 2, :red    2}})

(dijkstra demo-graph :red)
;; => {:green 8, :blue 5, :purple 10, :red 0, :orange 8}

(dijkstra demo-graph :red :purple)
;; => {:purple 10}

There are two popular workarounds that people have proposed:

1. Make URLs unique by adding query parameter, such as a timestamp, to the URL.
2. Set the header, Cache-Control: no-cache, in the response

I noticed that some of my POST requests were not getting cached even though I had neither of these workarounds in place. After some poking around, I’ve discovered that having must-revalidate or no-store in the Cache-Control response header also works.

So, before you go adding no-cache to all of your POST responses, check to see if must-revalidate or no-store is already getting set.

Update:

I’ve published a simple Sinatra app that helped me test this: https://github.com/loganlinn/ajax-post-cache-test

I recently came across HackerRank, a site with programming puzzles, and decided to try one out. The site’s still in a closed beta, so the only challenge that’s currently available is the “SpaceX” challenges. The premise is as follows:

Emily is an astronaut applying to be a part of the SpaceX project. To get the job, she must interview with a series of researchers, following this interview pattern:

She asks a factual question first, which the scientist answers in an encrypted string. The scientist immediately responds with a different string that she has to decrypt with the same key as the first. The encryption algorithm is the same for all questions.

Here’s the first challenge:

Emily: Who is the author of the book Three O’Clock Dinner?

Scientist: Mrvhsklqh Slqfnqhb

Scientist: I work for “hljkw kxqguhg dqg iliwb-wzr” hours a year. How many hours do I work?

The first 10,000 challenges follow this pattern. When I first saw this output, I noticed that the capitalization and non-alphabetic characters seemed to be preserved. I also noticed that word boundries seemed to be preserved, too. Could it be a simple substitution cipher?

Since I’ve been playing with Clojure recently, I fired up a REPL to see if I could solve the cipher.

Solving the Cipher

Googling the “Three O’Clock Dinner” tells me that Josephine Pinckney is the author. So, I started comparing the difference between the ASCII values of the coresponding characters…

1
2
3
4
5
6

user=> (- (int \J) (int \M))
-3
user=> (- (int \o) (int \r))
-3
user=> (- (int \s) (int \v))
-3

So everything looks to be shifted down 3 characters. Let’s apply this to the encrypted answer:

1
2
3
4

user=> (reduce str
         (map #(char (+ % -3))
              (map int "hljkw kxqguhg dqg iliwb-wzr")))
"eight^]hundred^]and^]fift_*two"

We can pick out the words to see that the answer is 852, but clearly, substracting 3 for all characters isn’t going to produce the correct result. This is because only alphabetic characters are shifted. Also, the ‘y’ in ‘fifty’ seems to be missing. That’s becuase the cipher wraps letters; the encrypted character, ‘b’, should be shifted, like ‘a’ –> ‘z’ –> ‘x’ –> ‘y’. I whipped up the following function to shift a single character:

1
2
3
4
5
6
7
8

user=> (defn alpha-rotate [c shift]
         "Circularly shifts an alphabetic character"
         (let [between #(and (>= %1 (int %2)) (<= %1 (int %3)))
               alpha-wrap (fn [c shift offset]
                            (+ (mod (- (+ c shift) offset) 26) offset))]
           (if (between c \A \Z) (alpha-wrap c shift (int \A))
             (if (between c \a \z) (alpha-wrap c shift (int \a)) c))))
#'user/alpha-rotate

Dropping in this function, we get:

1
2
3
4

user=> (reduce str
         (map #(char (alpha-rotate % -3))
              (map int "hljkw kxqguhg dqg iliwb-wzr")))
"eight hundred and fifty-two"

Much better!

To speed things up, I wrote another function to take the encrypted question, it’s answer, and the encrypted number to figure out the shift width, and shift the encrypted number.

1
2
3
4
5
6

user=> (solve-sample "Mrvhsklqh Slqfnqhb" "Josephine Pinckney" "hljkw kxqguhg dqg iliwb-wzr")
"eight hundred and fifty-two"
user=> (solve-sample "Fssj Ufwwnxm" "Anne Parrish" "tsj ymtzxfsi fsi ymwjj")
"one thousand and three"
user=> (solve-sample "Whuhth Jpaf" "Panama City" "upul aovbzhuk, zlclu obukylk huk mpmaf-vul")
"nine thousand, seven hundred and fifty-one"

I did these three just to verify that things were working.

I quickly realized that obtaining the shift width is the tricky part of these problems. For the first challenge, it was -3, but turns out to be different between challenges. But more importantly, I had to make a Google search for each challenge… Clearly, not most efficient way to solve this

Let’s leverage the fact that all of the answers are really just a number that’s been spelled-out. There’s a pretty small set of words that can be in each answer. If we cosider each word individually, we can reduce the set of spelled-out numbers by a good amount just by the word’s length (eg. “zlclu” must be “three”, “seven”, “eight”, “fifty”, or “sixty”). Lastly, we igore words that don’t have a consistent shift width our encrypted word.

If we do this for each encrypted word and find a common shift width, we can decypher the spelled-out number.

Extracting the Value

Since a challenges is answered by providing the numeric value of the spelled-out number, we need to be able to parse the strings that we have decrypted.

After a quick search, I didn’t find an existing Clojure library for this, so I took a stab at it myself. It turned out to be a relatively simple set of functions to solve this.

1
2
3
4

hackerrank.numberparse=> (parse "eight thousand, four hundred and fifty-three")
8453
hackerrank.numberparse=> (parse "four million and two")
4000002

Check out the source here

Shipping it

With the ability to solve these challenges progamatically, the only thing remaining steps are requesting challenges and submitting answers.

I used dakrone’s clj-http and cheshire libraries to make HTTP requests and parse JSON.

You can check out the full source of my solution on GitHub.

Although this wasn’t a particularly difficult programming puzzle to solve, it was a fun exercise to practice my Clojure skills.

Thanks to the @HackerRank guys for putting up the puzzles!

Still trying to feed my Ruby appetite, I decided to use Sinatra to make a super simple app that would serve my static Markdown files. But somewhere in between graduating and moving across the country, I lost track of that project and its source code. I also learned about Jekyll, which did everything I wanted my app to do and more. I got discouraged again.

Today I learned about Octopress, a Sinatra based framework that utilizes Jekyll. Described as an, “obsessively designed framework”, I was delighted to find that it’s almost exactly what I wanted to build myself. And since it’s a Rack app, it plays nice with pow, another tool I’ve been looking for a reason to use. So, shout out to Brandon Mathis and the Octopress contributors for making a great tool!

[1] They’ll also tell you that I have attention deficit disorder when it comes to learning and using new programming languages.

You can check out the project is at http://gitcrunch.com.

Update: This is now offline. I will try to resurrect the project sometime.