My Bookshelf: Create a Project Skeleton for Clojure Web Application

In the last post, I discussed my Clojure learning strategy in general and introduced my new hobby project My Bookshelf.

This is the very first post in the My Bookshelf series in which I’ll describe how to create a skeleton of a simple Clojure web application based on Luminus template.

Getting Started

When I presented My Clojure Learning Strategy, I mentioned the importance of practical experience and learning just enough to get started. This is exactly what Step 7 of 10-step learning process is about.

As a preparation, I read the first chapter in Web Development with Clojure, 2nd ed. This was more than enough to get ready for creating a new project for My Bookshelf.

Tools

To be able to generate the project skeleton with Luminus template you need Java JDK and Leiningen.

I installed Leiningen via brew:

brew install leiningen

You’ll also need to install a database of your choice. I chose PostgreSQL:

# install PostgreSQL
brew install postgresql
# Run PostgreSQL as a service whenever the computer is started.
brew services start postgresql

After that, you’re ready to go.

Generate a Project Skeleton

Luminus is a handy template providing an opinionated selection of high-quality Clojure libraries. It’s really useful for beginners. It’ll save you plenty of time and a terrible headache which you’d get trying to explore all available Clojure web development libraries.

Having that said, it’s trivial to create a new project:

lein new luminus my-bookshelf +postgres

We’ve just generated simple web app using the +postgres switch which instructs Luminus to add PostgreSQL dependencies and generate db.core namespace.

You can, of course, select another database if you want. You’ll find the list of many possible options in Application Profiles documentation.

Set Up a Database

To be able to connect to the database you need to perform an initial setup:

  • create a database user for your application
  • create a database itself

Both these steps are trivial with PostgreSQL. Run following commands in a shell:

# creates a new user 'bookworm' which will be used by our application to connect to the DB
createuser --createdb bookworm

# create 2 databases - one for development, another one for testing
createdb -O bookworm my_bookshelf_dev
createdb -O bookworm my_bookshelf_test

DB config

Database connection info is stored in profiles.clj in project root directory. This file is automatically generated by luminus template and contains database url for dev and test profiles:

{:profiles/dev  {:env {:database-url "jdbc:postgresql://localhost/my_bookshelf_dev?user=bookworm&password=password"}}
 :profiles/test {:env {:database-url "jdbc:postgresql://localhost/my_bookshelf_test?user=bookworm&password=password"}}}

The database url can always be overriden via DATABASE_URL environment variable.

Migrate the Database

Luminus uses Migratus library for handling database migrations.

To migrate the database just run the following command:

lein migratus migrate

Migratus will scan the resources/migrations directory and look for the .sql files which have to follow naming convention ^(\d+)-([^\.]+)\.(up|down)\.sql

In our case, it’ll find the 20170218124009-add-books-table.up.sql file and execute the SQL found there:

CREATE TABLE books
(id VARCHAR(20) PRIMARY KEY,
 title VARCHAR(30),
 author VARCHAR(60),
 issued DATE,
 status VARCHAR(30));

Note: If you leave the password in profiles.clj blank (jdbc:postgresql://localhost/my_bookshelf_dev?user=bookworm&password=password”) you’ll get an obscure error when trying to migrate the database:

2017-03-07 09:37:12,682 [main] ERROR migratus.database - Error creating DB connection for postgresql://localhost/my_bookshelf_dev?user=bookworm&password=
java.lang.IllegalArgumentException: Vector arg to map conj must be a pair
        at clojure.lang.ATransientMap.conj(ATransientMap.java:37) ~[clojure-1.8.0.jar:na]
...
        at clojure.java.jdbc$parse_properties_uri.invokeStatic(jdbc.clj:178) ~[na:na]
...
        at migratus.database.Database.connect(database.clj:338) [na:na]
...
2017-03-07 09:37:12,683 [main] INFO  migratus.core - Ending migrations
Exception in thread "main" java.lang.NullPointerException, compiling:(/private/var/folders/hn/tgwyrdmj1tb5pmmbdkd1g_qc0000gn/T/form-init7084456921097733207.clj:1:125)
...
Caused by: java.lang.NullPointerException
        at migratus.database$connect_STAR_.invokeStatic(database.clj:223)
        at migratus.database$connect_STAR_.invoke(database.clj:217)
        at migratus.database.Database.connect(database.clj:338)
        ...

So make sure that some password is set.

Run the Application

Now we are ready to run our application skeleton. Go to the terminal and execute lein run.

This will compile the application and start an embedded web server listening on port 3000.

Configure a Different Port For a Web Application

Since many Clojure web applications use port 3000 by default, it can be useful to change the port to another one if you want to avoid conflicts.

For instance, I use the port 3003. You can change this setting in config.edn file.

Add a Simple Welcome page

As a first step, we just want to modify the main page to show a title of our shiny new web application.

To make it easy, we’ll use Hiccup to generate HTML on a server:

(defn render-home [books]
  (hiccup.core/html
    [:body [:h1 "My Bookshelf"]]))

Notice, that we ignore the input argument books for now. We just want to show a static text.

Now run the application (if you haven’t run it already) and open the main page in a browser:

Conclusion

That’s it!

We managed to generate a skeleton for a new Clojure web application, set up a PostgreSQL database and display our first primitive HTML page rendered by Hiccup.

In the next post, we’ll add a basic functionality like showing the list of books to the user.

Source code

You can find complete source code for My Bookshelf in master branch. Beware that this is the latest version and it might be confusing to use it while reading this article.

The version corresponding to this post is tagged (git tag) as 01-skeleton and you can find it in Releases tab on GitHub.

 

My (Clojure) learning strategy

In this post, I’ll discuss the general learning strategy and show an example how to apply it to Learning the Clojure programming language.
I was heavily inspired by 10-step learning process from John Sonmez which I first encountered in his book Soft Skills: The software developer’s life manual and the process outlined in the book Novice to Expert: 6 Steps to Learn Anything, Increase Your Knowledge, and Master New Skills by Steve Scott.

The past

I used to approach learning new things without a strategy.
When I found a cool piece of technology that I’d like to know more about, I  spent a weekend by reading about it. I might have tried to play with some code a little, but mostly only by copying exercises from the tutorial into console/editor. Once the weekend was over, I just ran out of time or got distracted by some other cool technology and never got to the former one again. I might even have forgotten that I read about that thing at all.
I burnt thousands of hours without actually learning too much. I thought that I could rush through the tons of information and actually learn it.
It doesn’t work that way – not for me.

A better approach

If you really want to learn something you need consistency. You have to accept that it will be boring. You must prepare yourself for the hard work – there are no shortcuts.
Having that said, what are my tips for learning Clojure or any other topic?

Focus on a single learning project at a time

  • Learning new complex topic is already hard, and you’ll make it a lot harder if you try to learn many different things at once.
  • Especially, if you have a full-time job try to focus on a single topic. This approach will increase your chance to finish it in a foreseeable future.
  • Trying to learn many new things at once will probably result in frustration, and you will end up failing at all of them.

Create learning plan

  • To succeed you need to start with a clear S.M.A.R.T (Specific. Measurable. Achievable. Relevant. Time-boxed.) end goal in your mind.
    • An example of a bad goal: “I want to learn Clojure and ClojureScript”
    • An example of a better goal: “I’ll learn enough of Clojure, ClojureScript and web development libraries so that I can build the first version of a non-trivial Clojure web application with REST API and simple UI within three months”
      • Here I’m playing with an idea to create My Bookshelf application for simple book management (I love books).
  • Another key point to realize is that you don’t need to achieve mastery in everything you want to learn. Try to always identify the desired level from 1-10 (1 – very basic overview; 10 – mastery).
  • More details later in section 10-step learning process

Execute the plan on a daily basis

  • A consistent, deliberate practice is essential for success
  • Reserve at least 0.5 h each day for mastering new skill
    • The minimal duration can vary, but I’d rather recommend spending at least 1 hour working on your project each day
  • Work on each step in iterations until you finish all steps – one step at a time
  • Two tasks daily rule
    • one repeating task
      • You can practice Clojure programming skills via Exercism.io, 4Clojure, or Clojure Koans.
      • You can also decide to spend time by reading a relevant book and trying out the examples
    • one task from a project list
      • Pick one task from your project and try to complete it.
      • For example: “Create a project structure for My Bookshelf application. Create git repository and push the skeleton.”
  • Focus on practical experience rather than just cramming a huge amount of information from books and the internet into (and out of) your head
    • Create a learning project to practice everything you want to learn

Example of 10-step learning process

What follows, is my implementation of 10-step learning process outlined in Soft Skills: The software developer’s life manual by John Sonmez.
The example demonstrated here is for My Bookshelf Clojure web application:

Step 1: Get the big Picture – 50,000 ft view

  • I’ve already written some Clojure/ClojureScript code in the past and read lots of articles and Living Clojure book.
  • Following are my observations:
    • I know what I want to do: Create a web application for book management.
    • I know what other options I have (GoodReads, Libib, etc.) but none of these options seems to be suitable for my needs.
    • I know a little bit of Clojure/ClojureScript so I can get started.

Step 2: Determine the scope of what you want to learn

  • I want to build a real application that I can use for my own book management.
  • I’ve identified various features – this will be an ongoing project.
    • I even have Trello board for tasks related to this project
  • As a first milestone, I want to implement MVP using some libraries mentioned in Web Development with Clojure book.
  • I want to embrace REPL driven development combined with TDD – I’ll write a lot of tests.

Step 3: Define how the success will look like

  • My Bookshelf is up and running (at least on my local computer) and I’m able to add new books to bookshelf database, update or delete existing book, list all books, show book detail.
    • MUST-HAVE: basic attributes like “Title”, “Author”, “Year”, and “Status” (to distinguish books which I want to read/buy and those which I’ve already read/bought)
    • COULD-HAVE: further attributes can be added if useful for practical usage
    • NICE-TO-HAVE: the ability to search books by keywords on Amazon, show the results and be able to select the book which I want to add.
  • Book database is persisted across different runs of the application.
  • There’s at least primitive UI which can show me a list of books that I’ve added.
  • I have an automated acceptance test checking the supported features.

Step 4: Find resources

Step 5: Create a learning plan

  1. DONE. Define learning strategy & resources
  2. Read the first parts of Clojure applied and Web Development with Clojure, 2nd ed. to know how to start
  3. Create application skeleton – client and server part -> Display hello world
  4. Create the first draft of application domain; in-memory store
  5. Write some tests
  6. Primitive read-only UI showing the book list
  7. Persist entities to survive application restart
  8. UI for adding a new book
  9. UI for updating & deleting existing books
  10. Redesign domain/data model based on recent experience
    • Use Clojure.spec?
  11. Add possibility to store details (notes) about the book
  12. Book tags
  13. Add basic search
  14. Track/watch books
  15. Iterate adding new features; evaluating different options/libraries/frameworks

Step 6: Filter the resources

Most important resources from step 4. Find resources:

Step 7: Learn enough to get started

One example could be Reading the first chapter in Web Development with Clojure, 2nd ed. and then generating new project structure from leiningen template.

Step 8: Play around

For example: Examine the application skeleton generated in the previous step and play with the application in REPL. Try to execute some DB queries and make sure you are familiar with each important part of the project structure.

Step 9: Learn enough to do something useful

The result of this step should be adding a new feature to your project, e.g. Implementation of “Add a New Book” feature.

Step 10: Teach

Teach what you learned to others. This is a great way to make sure you have a solid understanding of the material you’ve learned. The other persons can be your colleagues, friends, local meetup group, readers of your blog, or even your spouse.

6 steps

While outlining the 10-step learning process and executing it on a daily basis, it can help you to think about 6 steps which Steve Scott mentions in his book Novice to Expert: 6 Steps to Learn Anything, Increase Your Knowledge, and Master New Skills.
This book deserves its own blog post, so I’ll just very briefly summarize the 6 steps:
  1.  Identify your learning style
    • Different people tend to favor different learning styles – social, solitary, visual, aural, verbal, physical, logical.
    • Try to identify 2-3 learning styles which suit you best and embrace them.
  2. Pick a skill you’d like to master
    • Choose a single skill – identify #1 goal you’d like to accomplish.
    • Focus on one skill at a time.
    • Pick your desired level of expertise.
  3. Build the learning habit
    • The quickest path to skill mastery is to focus on this area completely.
    • Focus on daily training.
    • Practice just-in-time learning – research information related to the next step, ignore everything else.
  4. Take actionable notes
    • Check out Cornell Method. It’s a very useful way to capture, understand and review a learning material.
  5. Build a project around a skill
    • In order to learn something, you need to practice it.
    • Create a real-world project for the skill you want to master and apply the things you’ve learned in the project.
  6. Deliberate practice
    • Deliberate practice is the most important thing; talent is usually of minor importance.

Final thoughts

I’ve provided you with an example how to create a learning strategy for Clojure project. I’ll definitely write more blog posts about my My Bookshelf learning project in the future.
For more on How to discover your learning style, Pickup the skill you’d like to master, Building the learning habit, Taking actionable notes, Building a project around a skill and Deliberate practice, you can read the Novice to Expert book or just wait until I write a blog post about it :).
Now, go ahead and pick a new skill you’d like to learn. Then create your learning plan and start executing it.