Building Dynamic Form Pages in Clojure

Opening Remarks:

This article does not describe how to create a basic HTML form, nor does it go into extraneous detail of how to set up the database and connect it to the form. I am assuming that you have built a form in your favorite programming language. I am also assuming you have some ability to mentally parse Lisp (Clojure is a Lisp), so I won't go into the specific syntax of Lisp except where it is necessary.

While it would be helpful if you know how to set up a basic project in Leiningen and install dependencies, this knowledge isn't strictly necessary. I'll be sure to point out the code areas where it may get confusing.

The code is from my very first Clojure project, which I built about 3 months prior to this writing. This project was built using Noir, Korma, and Hiccup. Noir has been deprecated [1], so it is time to move the code-base over to Compojure, but I want to write this article before I get started with that. I believe it is perfectly acceptable to use Noir in this article since the abstractions it creates makes certain ideas more obvious and intuitive. I'll leave it up to the reader to discover how to use Compojure.

Since Solo Resume [2] is currently a prototype, there is no user-based routing or cookies active on the site, thus the code in this article does not include these protocols.

I make no claims that the following is representative of the “Clojure Way,” nor do I claim to be an expert programmer. I do claim that I built the code and followed the thought-process described here, and I do claim that I would like people who are interested in Clojure to see something “practical” in Clojure, and I hope the real experts can raise this idea to the levels it deserves to be. If I receive enough suggestions to fix items in this article, I will create a follow-up article that includes those suggested fixes.

Why Clojure?

I understand some people are interested in learning why I feel Clojure is better than language X. I'll leave it up to the reader's interest to discover this question. A simple query with his or her favorite search engine will lead to many articles expounding upon the supposed superiority of Clojure, but our interested reader will find a dearth of articles discussing how to do anything significant with Clojure. Instead of discussing why, the goal of this article is to show the how, and I leave it up to the reader to ponder the benefits of using Clojure in lieu of his or her favorite language.

The Effect:

We start up with a blank form field

Fill in some fields

Press enter then Presto! The entered information is above the input fields

The Code

It may be helpful to have the entire code opened in a new tab and follow along since the code described here is quite large. You can view it here.

To be official, I'll start up with the namespace boilerplate:

(ns soloResume.views.emppages
  (:require 
   [soloResume.views.common :as common]
   [noir.response :as resp]
   [soloResume.models.datapass :as db]
   [noir.session :as sesh])
  (:use [noir.core] 
        [korma.core]
        [hiccup.page-helpers]
        [hiccup.form-helpers]))

The above is self-explanatory if you are familiar with Clojure. The first line defines a new name-space with the ns macro The :require key makes calls to other namespaces within the code base and imports them. The :as simply means that I am giving certain namespaces aliases. The convention, as I understand it, calls for using :as keywords in conjunction with :require, and to use the :use keyword when there are no aliases.

The notion of namespaces are no different than Python. If the above was in Python, it may look like this:

import common as common ## namespace I created with alias 
import noir ## library namespace
from pageHelpers import hiccup ## library namespace with alias 

You may have noticed that the images shown display “Employer 3.” Solo Resume currently has 5 employer pages. Originally, there was only 3 employers with 3 “duty” fields. My goal at the beginning was to generalize the code enough so that I can easily add pages and form fields without having to copy / paste code everywhere. I could have taken this concept much further, but here I only created the pages so they reflect their namespaces. I think this strikes an okay balance between over-abstracting and usablity. In this case, I am in the emppages namespace which holds the form operations on the employer pages.

The first thing I did was create a mapping of the page numbers to a string. This will make more sense in a minute, but mainly this code sets up basic routing capabilities:

(def emp-page-num {:one "one" :two "two" :three "three" :four "four" :five "five"})

I also wanted to create a mapping from the routes to the appropriate database calls and assign it a value that I can use in my input tags. The template for the data-input map is defined as such:


(def data-input 
  {"page-number" [[:alias-for-value db/mapping-to-database-table :table-column]
                    [:alias-for-value db/mapping-to-database-table :table-column]]})

where db is the alias for the datapass namespace. Inside of datapass is all of my Korma definitions. I'll describe exactly what is happening in a minute.

My code base does have five keys so that each key is one page, but I feel that showing all five pages will make the map too long for this article, so I shortened this map to represent two pages. You can imagine that it extends all the way to “five” keys:

(def data-input {"one" [[:suser-emp-name db/jobhunteremployers :empname] 
                        [:suser-emp-city db/jobhunteremployers :city]
                        [:suser-emp-state-and-country db/jobhunteremployers :stateandcountry]
                        [:suser-emp-position db/jobhunteremployers :position]
                        [:suser-emp-start-month db/jobhunteremployers :startmonth]
                        [:suser-emp-start-year db/jobhunteremployers :startyear]
                        [:suser-emp-end-month db/jobhunteremployers :endmonth]
                        [:suser-emp-end-year db/jobhunteremployers :endyear]
                        [:suser-emp-start-date db/jobhunteremployers :startdate]
                        [:suser-emp-end-date db/jobhunteremployers :enddate]
                        
                        [:suser-emp-duty1 db/jobhunteremployers :duty1]
                        [:suser-emp-duty2 db/jobhunteremployers :duty2]
                        [:suser-emp-duty3 db/jobhunteremployers :duty3]
                        [:suser-emp-duty4 db/jobhunteremployers :duty4]
                        [:suser-emp-duty5 db/jobhunteremployers :duty5]]
                 
                 "two" [[:suser-emp-name db/jobhunteremployers :empname] 
                        [:suser-emp-city db/jobhunteremployers :city]
                        [:suser-emp-state-and-country db/jobhunteremployers :stateandcountry]
                        [:suser-emp-position db/jobhunteremployers :position]
                        [:suser-emp-start-month db/jobhunteremployers :startmonth]
                        [:suser-emp-start-year db/jobhunteremployers :startyear]
                        [:suser-emp-end-month db/jobhunteremployers :endmonth]
                        [:suser-emp-end-year db/jobhunteremployers :endyear]
                        [:suser-emp-start-date db/jobhunteremployers :startdate]
                        [:suser-emp-end-date db/jobhunteremployers :enddate]
                        
                        [:suser-emp-duty1 db/jobhunteremployers :duty1]
                        [:suser-emp-duty2 db/jobhunteremployers :duty2]
                        [:suser-emp-duty3 db/jobhunteremployers :duty3]
                        [:suser-emp-duty4 db/jobhunteremployers :duty4]
                        [:suser-emp-duty5 db/jobhunteremployers :duty5]]}) 

Next, I wanted to pull out the data in the above map. This next piece of code takes considerable staring to understand and I do have to give credit where credit is due [3]:

(defn data-build [data-key suser page-num]
  (loop [output {}
         items (seq (data-input data-key))]
    (if items
      (recur
       (let [[kw db fieldname] (first items)]
         (assoc output kw 
                (select db 
                        (fields fieldname) 
                        (where {:username suser :employernumber page-num})))) 
       (next items))
      output)))

defn is a noir abstraction over (defn []), and follows the same generally format of

(defn function-name [arg-one arg-two & args])

Loop creates a new local map called output and returns the data from the database as a new mapping. It is probably easier to show the output of (data-build) via the repl:

$ lein repl

=> (use 'soloResume.views.emppages)
nil

I added some data to a fake profile called “user1.” I would now like to pass arguments into (data-build) so we can explore what is happening. Here is the function call in plain English:

(data-build "page-number" "user-name" employer-number-as-int)

Looking back into the repl:

=> (data-build "one" "user1" 1)
"{:suser-emp-city [{:city "My City"}], :suser-emp-start-month [{:startmonth "
January"}], :suser-emp-start-year [{:startyear "2000"}], :suser-emp-end-month
 [{:endmonth "January"}], :suser-emp-duty5 [{:duty5 "My Duty Five"}], :suser
-emp-duty4 [{:duty4 "My Duty Four"}], :suser-emp-duty3 [{:duty3 "My Duty Thre
e"}], :suser-emp-duty2 [{:duty2 "My Duty Two"}], :suser-emp-position [{:posit
ion "My Position"}], :suser-emp-end-year [{:endyear "2010"}], :suser-emp-sta
te-and-country [{:stateandcountry "My State"}], :suser-emp-duty1 [{:duty1 "My
 Duty One"}], :suser-emp-name [{:empname "My Employer"}], :suser-emp-start-da
te [{:startdate nil}], :suser-emp-end-date [{:enddate nil}]}"

Now, I'm going to create a new value called information:

=> (def information (data-build "one" "user1" 1))
#'soloResume.server/information

=> information
"{:suser-emp-city [{:city "My City"}], :suser-emp-start-month [{:startmonth "
January"}], :suser-emp-start-year [{:startyear "2000"}], :suser-emp-end-month
 [{:endmonth "January"}], :suser-emp-duty5 [{:duty5 "My Duty Five"}], :suser
-emp-duty4 [{:duty4 "My Duty Four"}], :suser-emp-duty3 [{:duty3 "My Duty Thre
e"}], :suser-emp-duty2 [{:duty2 "My Duty Two"}], :suser-emp-position [{:posit
ion "My Position"}], :suser-emp-end-year [{:endyear "2010"}], :suser-emp-sta
te-and-country [{:stateandcountry "My State"}], :suser-emp-duty1 [{:duty1 "My
 Duty One"}], :suser-emp-name [{:empname "My Employer"}], :suser-emp-start-da
te [{:startdate nil}], :suser-emp-end-date [{:enddate nil}]}"

In order to access the values of information, I have to do some destructuring. Destructuring is pretty difficult to explain, and it's sort of like math in that you can't understand it until you really dive in and do it. It's a little strange at first, but once you understand it, it's very easy.

Let's start by looking at one small part of information:

{:suser-emp-city [{:city "My City"}]}

And to destructure, I am using a let-binding, which has the following structure:

(let [value item-to-destructure]
  (function value))

In order to destructure properly, we have to start by reversing the map:

{[{"My City" :city}] :suser-emp-city}

We can generalize this because we'd always want to pull out the string in all similar queries:

{[{val :city}] :suser-emp-city}

Then fill in the blanks in the let-binding:

(let [{[{val :city}] :suser-emp-city} information]
   (format "%s" val))

And entering this into the repl gives:

=> (let [{[{val :city}] :suser-emp-city} information]
(format "%s" val))
"null"

This happens because the data-build build function returns the map as a string even though I didn't apply a string to the map. So, I have to interpret the string with a load-string function:

=> (let [{[{val :city}] :suser-emp-city} (load-string information)]
  (format "%s" val))
"My City"

And now I have the correct result.

Now that we understand the output of the data-build function, let's move on to the next helper function:

(defn form-builder [pid pname data-key data-table information field-key common-key]
  [pid pname	
   [:i (let [{[{val data-key}] data-table} information]
         (format "%s" val))]
   [:br]
   (if (= common-key "")
     (text-field field-key)
     (common/menu-build field-key common-key))])

And skip this for now, because I have to explain the logic of how the page function works first.

Noir uses an abstraction on defroutes called (defpage). I'm sort of playing with the truth on that one, but for the intents of this exercise, I think this statement is close enough to make sense.

Here is the entire function:

(defpage "/employer-:emp-page-num" {:keys [emp-page-num]}
  (common/res-heads
   (let [suser (sesh/get :uname)]
     (let [information (load-string (data-build emp-page-num suser
                       (common/to-integer emp-page-num)))]
       [:div.span6.offset1
        (form-to [:post (str "/employer-"emp-page-num)]
                 [:h3 (str "Employment History: Employer " (common/to-numeral emp-page-num))]
                 (form-builder :p "Employer Name: " :empname
                               :suser-emp-name information "empname" "")
                 (form-builder :p "City: " :city :suser-emp-city information "empcity" "")
                 (form-builder :p#statePar "State (and Country): "
                               :stateandcountry :suser-emp-state-and-country
                               information "empstate" "")
                 (form-builder :p "Position: " :position :suser-emp-position
                               information "emppos" "")
                 [:br][:p [:hr]][:br]
                 (form-builder :p "Start Month: " :startmonth
                               :suser-emp-start-month information "startmonth" :month)
                 (form-builder :p#startYear "Start Year: " :startyear
                               :suser-emp-start-year information "startyear" :year)
                 (form-builder :p "End Month: " :endmonth
                               :suser-emp-end-month information "endmonth" :month)
                 (form-builder :p#endYear "End Year: " :endyear
                               :suser-emp-end-year information "endyear" :year)
                 [:br][:p [:hr]][:br]
                 (form-builder :p "Duty One: " :duty1 :suser-emp-duty1
                               information "dutyone" "")
                 (form-builder :p "Duty Two: " :duty2 :suser-emp-duty2
                               information "dutytwo" "")
                 (form-builder :p "Duty Three: " :duty3 :suser-emp-duty3
                               information "dutythree" "")
                 (form-builder :p "Duty Four: " :duty4 :suser-emp-duty4
                               information "dutyfour" "")
                 (form-builder :p "Duty Five: " :duty5 :suser-emp-duty5
                               information "dutyfive" "")
                 [:input.buttons {:type "submit" :value "Update"}])]))
   [:div.blankDiv [:p]]))

Following the earlier pattern, I want to break the above down a little bit so that there isn't so much clutter and you can follow my logic a better:

(defpage "/employer-:emp-page-num" {:keys [emp-page-num]}
  (common/res-heads
   (let [suser (sesh/get :uname)]
     (let [information (load-string (data-build emp-page-num suser
                       (common/to-integer emp-page-num)))]
       [:div.span6.offset1
        (form-to [:post (str "/employer-"emp-page-num)]
                 [:h3 (str "Employment History: Employer " (common/to-numeral emp-page-num))]
                 (form-builder :p "Employer Name: " :empname
                               :suser-emp-name information "empname" "")
                 (form-builder :p "City: " :city :suser-emp-city information "empcity" "")
                 (form-builder :p#statePar "State (and Country): "
                               :stateandcountry :suser-emp-state-and-country
[:input.buttons {:type "submit" :value "Update"}])]))
[:div.blankDiv [:p]]))

And now we can go through this line-by-line:

(defpage "/employer-:emp-page-num" {:keys [emp-page-num]}

The key to the page-number is the call to the emp-page-num value at the top of this namespace. The employer-page is dynamically generated and I am using this property to pass a string into the argument. Here is the emp-page-num map again:

(def emp-page-num {:one "one" :two "two" :three "three" :four "four" :five "five"})

and now you can see how it fits into (defpage [])

This next line is pretty simple: I am calling the common namespace and pulling out the res-heads value. The res-heads value holds the basic layouts of this and other pages.

(common/res-heads

This let-binding calls the session variable :uname and places it into a value called suser. Note: Yes, :uname is a mutating variable.

 (let [suser (sesh/get :uname)]

And this let-binding takes the data-build map and binds it to and information value. This data-build is the exact same function I was playing with in the repl earlier, except that this is NOT in destructured form because it calls the map generated fro m data-build.

(let [information (load-string (data-build emp-page-num suser (common/to-integer emp-page-num)))]

This next part is classic Hiccup. Hiccup has several advantages to you if you aren't married to the idea of not touching a designer's HTML:

       [:div.span6.offset1
        (form-to [:post (str "/employer-"emp-page-num)]
                 [:h3 (str "Employment History: Employer " (common/to-numeral emp-page-num))]

Hiccup is basically an HTML-preprocessor written in Clojure, which comes along with many of the features of Clojure.

The basic syntax of a hiccup value is:

[:html-tag "Text Entered"]

;; THUS... 

[:p "Hello World"]

produces:

Hello World

For those of you who like using snippets, Hiccup is quite similar to snippets. If you are using paredit, you'd use 4 keystrokes to produce [:p ""]

To nest elements:

[:body 
 [:div 
  [:p ""]]]

There are two way to add classes to the tags:

[:div.myClass]

is equivalent to

[:div {:class "myClass"}]

You can add ids like so:

[:div#myId]

is equivalent to

[:div {:id "myClass"}]

And for those who like OOCSS:

[div.classOne.classTwo.classThree#myId]

Let's look at the above snippet again:


[:div.span6.offset1
 (form-to [:post (str "/employer-"emp-page-num)]
          [:h3 (str "Employment History: Employer " (common/to-numeral emp-page-num))]

And you can start to see what is happening with the hiccup (and see that I used Twitter Bootstrap). Hiccup also offers some short-cut snippets as well. Above you can see

 (form-to [:post (str "/employer-"emp-page-num)].....)

which is short-hand for writing out a form with post parameters. In the above, you can see that I used the routing parameters to generate a dynamic post-method. Clojure doesn't have string concatenation in the Python sense where you'd simply “add” two string together. Clojure requires (str) to concatenate like I have above.

The next line once again exploits the routing to generate the pages. (common/to-numeral) simply calls (to-numeral) in my common namespace. (to-numeral) simply takes a string as a parameters and returns a string representation of the appropriate integer.

[:h3 (str "Employment History: Employer " (common/to-numeral emp-page-num))]

Now, I'd like to explore Hiccup a little bit more. Back the repl:

=> (use 'hiccup.core)
nil
=> [:p "hello"]
[:p "hello"]

The above is all pretty obvious. but this next part may not be as obvious. I create a new value called “val” and bind “:p” and pass it in as the first parameter to the Hiccup vector:

=> (def val :p)
#'soloResume.server/val
=> val
:p
=> [val "hello"]
[:p "hello"]

And of course, it's not limited to “:p”:

=> (def n2 :div)
#'soloResume.server/n2
=> [n2 "a div?"]
[:div "a div?"]

Earlier, I skipped past (form-builder), so now that I worked over some Hiccup, it is a good time to open the repl and see what is happening:

=> (use 'soloResume.views.emppages)
nil
=> form-builder
#
=> (form-builder :p "Employer Name: " :empname :suser-emp-name
information "empname" )
java.lang.Exception: Unable to resolve symbol: information in this context (NO_S
OURCE_FILE:12)

Oops. Just to get it working, I will pass a string in information:

=>=> (form-builder :p "Employer Name: " :empname :suser-emp-name
"my City" "empname" "")
"<p>Employer Name: <i>null</i><br /><input id="empname" name="empname" type=
"text" /></p>"
"<p>Employer Name:   <i>null</i>
    <br />
    <input id="empname" name="empname" type="text" />
</p>"

So far so good. The only problem so far is that the above will show “null” next to the Employer name, and we don't want that. Recall from earlier that I defined information as a map, so lets go through form-builder to see how it all breaks down.

(defn form-builder [pid pname data-key data-table information field-key common-key]
  [pid pname	
   [:i (let [{[{val data-key}] data-table} information]
         (format "%s" val))]
   [:br]
   (if (= common-key )
     (text-field field-key)
     (common/menu-build field-key common-key))])

We already saw how the argument work. Remember that information takes in a raw mapping to Korma, and we already saw destructuring, which is exactly what is happening inside of the the let-binding. All that part does is sees if a value exists in the map and if so, it will print out whatever is in writing in italic, else, it will display nothing.

The (if) section allows me to either return a text-field:

<input = 'type' >

or return a drop-down list. (menu-build) is in my common namespace. It takes in an argument for what kind of item to grab and and it looks like this:

(def menu-list {:year ["-""2013""2012""2011""2010""2009""2008""2007"
                                             "2006""2005""2004""2003""2002""2001""2000""1999"
                                             "1998""1997""1996""1995""1994""1993""1992""1991"
                                             "1990""1989""1988""1987""1986""1985""1984""1983"
                                             "1982""1981""1980""1979""1978""1977""1976""1975"
                                             "1974""1973""1972""1971""1970""1969""1968""1967"
                                             "1966""1965""1964""1963""1962""1961""1960""1959"
                                             "1958""1957""1956""1955""1954""1953""1952""1951"
                                             "1950""1949""1948""1947""1946""1945""1944""1943"
                                             "1942""1941""1940""1939""1938""1937""1936""1935"
                                             "1934""1933""1932""1931""1930""1929""1928""1927"
                                             "1926""1925""1924""1923""1922""1921""1920""1919"
                                             "1918""1917""1916""1915""1914""1913"]
                :month ["-""January" "February" "March" "April" "May" 
                        "June" "July" "August" "September" "October" 
                        "November" "December"]
                :state ["-""AL""AK""AZ""AR""CA""CO""CT""DE""FL"
                        "GA""HI""ID""IL""IN""IA""KS""KY""LA""ME"
                        "MD""MA""MI""MN""MS""MO""MT""NE""NV""NH""NJ"
                        "NM""NY""NC""ND""OH""OK""OR""PA""RI""SC""SD""TN"
                        "TX""UT""VT""VA""WA""WV""WI""WY"]
                :yn ["-""yes""no"]})

(defn menu-build [menu-name param]
  (drop-down menu-name (menu-list param)))

(drop-down) is another easy abstraction offered by Hiccup.

Post Page

Finally, I created the post-page. Once again, I'm chopping off a lot of it because it gets repetative to look at, but basically, it uses the same idea I did in the previous (defpage), where I routed in the :emp-page-num. Basically, it figures out if any of the fields have information and if it does, it pushes the information into the database, and if not, it, well, doesn't do anything, really. I feel like there could have been a better way of doing this, but here is the code:


(defpage [:post "/employer-:emp-page-num"] {:keys [emp-page-num empname empcity empstate]}
  (let [suser (sesh/get :uname)]
    (if (not (= empname ))
      (update db/jobhunteremployers 
              (set-fields {:empname empname})
              (where {:username suser :employernumber (common/to-integer emp-page-num)})))
    (if (not (= empcity ))
      (update db/jobhunteremployers 
              (set-fields {:city empcity})
              (where {:username suser :employernumber (common/to-integer emp-page-num)})))
    (if (not (= empstate clojure.core$_@75536174))
      (update db/jobhunteremployers 
              (set-fields {:stateandcountry empstate})
              (where {:username suser :employernumber (common/to-integer emp-page-num)})))
    (resp/redirect (str "/employer-"emp-page-num))))

The Database and Korma

Korma is a popular DSL used to make queries easier to use and make them more Clojure-like. You may have noticed that there are areas in the code-base that look like common select clauses. One example:


(if (not (= empname ))
  (update db/jobhunteremployers 
          (set-fields {:empname empname})
          (where {:username suser :employernumber (common/to-integer emp-page-num)})))

In the datapass namespace (aliased as db), I have a list of (entities). For example, the entity that correlates to what you see above is:

(defentity jobhunteremployers)

I've noticed from code discussion forums that many people are under the impression that you have to create full-out functions with Korma to use it, and this is not the case at all. In order to connect to the table, you only have to create a (defentity) to the name of the table you want to operate on.

With that said, the above piece of code should be rather intuitive, but I should take a bit of time to explain exactly what is happening. I'll start by first opening up PgAdmin (I'm using PostgreSQL, but this shouldn't be important):

And you see that I created the table so that each user has 5 tuples in the table, where each tuple has one employer number mapped to it. This allowed me to use the page routes to update the database without having to copy/paste this function over and over.

Now (update db/ jobhunteremployers) should make sense, so let's take a look at the update clause:


(set-fields {:empname empname})
(where {:username suser :employernumber (common/to-integer emp-page-num)})

The :empname is the name of the column I'd like to update, In this case, I am inserting the value of empname into the column. The same thing happens with the (where) function. I am updating where the column name = suser (the user name) and where employer-number = page-number, which is once again converted to an integer via the function inside of the common namespace.

Final Thoughts

You may be wondering why I went through the trouble of creating some of the functions seen in this article, and in particular, the data-build function. The reason isn't because I wanted to explore all the cool things Clojure can do. Solo Resume features many different pages that generate dynamic content, much of it to a larger level than you see here. So, this one abstraction and variations of it are used all over the place.

Clojure, I'll admit, looks very intimidating to look at, but it really isn't that difficult to use. The entire code-base in Solo Resume is around 1500 LOC. The job hunter view has 10 fill-in pages and one extra page that generates a resume. The employer view has a few pages for viewing, searching, and saving resumes. Despite not knowing how to program Clojure at all, I manged to build this site alone in about 50 to 75 hours over the course of one month, and that includes screwing up, building the database 2 times, and learning how to use the language, so despite the intimidating look of the language, it is far from intractable. Need further proof? When I built Solo Resume, I had little more than a year-and-a-half of programming under my belt.

Links: