Play with MongoDB

For the normal data storage in most of todays applications a relational DB is not the optimal solution for several reasons. There are many great alternatives now on the market. I like MongoDb because it is not that far away from the concepts of a relational DB, it is easy to set up and has proven to be stable and reliable.

Thus I clearly want a MongoDb as persistence in my Play 2! project. Goal is

  • to have a good abstraction. Not too much magic but not too much boilerplate either
  • to have it running locally and in the cloud (I’ll go for Heroku here)

I found Salat a pretty good abstraction. It got it running quickly locally. But when I tried to get it running on Heroku it became a little awkward. At Heroku – as on every other PaaS I had a look at – you get a URL for the MongoDb you added to your app, normally provided as system property. Same holds if rent a DB in the cloud at a “pure” cloud DB provider. But there are no means (that I know of course) to easily let Salat read and use the URL.

So I wrote a MongoDao that does the following things:

  • it reads a MongoURL defined in the application.conf via the mongo.uri property.

In my template application I provided an example that reads the environment variable MONGOLAB_URI which is the system variable on a Heroku app with an installed MongoLab instance. If no such variable is set, the property is left empty. This lets the MongoDao take the standard parameters for a local MongoDb installation.

  • a collection method that returns a collection as required by the SalatDao, so you can easily define the SalatDao in your implementation
  • if you want to differentiate between different local DBs you can overwrite the defaultDbName val in you class.

This is pretty much it. If any one of you has good ideas to make the abstraction more convenient, clean, elegant I’d be happy to read your comments.

Advertisements

Basic authentication with Play! 2

I recently had the problem that I wanted to secure a page preview we developed with basic authentication. Easy thing I thought. But it wasn’t. I was used to the use of frameworks/containers like tomcat, which provide easy means to do such things. With Play! though you don’t have such things. You’ll have to do it yourself.

The good thing is that you’ll understand what is actually going on:

The basic stuff happening is:

  1. Request for a certain page is sent to the server
  2. On the server the header with the name “Authorization” is looked up
  3. If it exists the contained string is parsed. It’s in the format “name:pwd”. Then name and pwd are checked.
  4. If either the header does not exist or nam/pwd are incorrect an unauthorized 401 response with a header of name “WWW-Authenticate” and a content that must contain the word basic is sent. This lets the browser know what actually to do.

Thats pretty much it. With Play! 1 this was pretty easy as you had direct access to the response as you might be used to in several MVC frameworks. With Play! 2 this is not so obvious although also really easy:

You can add a header to the response by using the method withHeaders. A good example can be found here.

At the time I needed it, I was unable to find an example in the net. Now as I wanted to finish this post and made another quick query and immediately found the example of Guillaume Bort. As he develops the play framework I decided to just add the link here without any code.

Getting Play 2 with Scalate to run on heroku

After having everything in place I thought it’d be a great idea to deploy the small app in the cloud. After some research I was really excited about Heroku. In my eyes a brilliant idea and realization of PaaS.
After the decision for Heroku it took me only a couple of minutes, till I was ready to go,  with the quite  good tutorial of the heroku guys. The push worked flawlessly. But when I wanted to admire my first app in the cloud, I got an error message on the page. Shouldn’t be too easy anyway, right?
The logs told me the following:

2012-04-18T06:00:44+00:00 app[web.1]: ! @6a4hnb7bj - Internal server error, for request [GET /] ->
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:834) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]:
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.util.IOUtil$.writeBinaryFile(IOUtil.scala:111) ~[scalate-util-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.actor.Actor$class.apply(Actor.scala:290) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.layout.DefaultLayoutStrategy.org$fusesource$scalate$layout$DefaultLayoutStrategy$$tryLayout(DefaultLayoutStrategy.scala:77) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: [error] o.f.s.l.DefaultLayoutStrategy - Unhandled: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]: play.core.ActionInvoker$$anonfun$receive$1$$anon$1: Execution exception [[TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)]]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.liftedTree1$1(TemplateEngine.scala:411) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:63) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:475) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker.apply(Invoker.scala:61) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: Caused by: java.io.FileNotFoundException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]:
2012-04-18T06:00:44+00:00 app[web.1]: Caused by: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.actor.ActorCell.invoke(ActorCell.scala:617) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.(FileOutputStream.java:160) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:82) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:405) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: [error] application -
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:179) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.open(Native Method) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:747) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.(FileOutputStream.java:209) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]

It took me a while to find out what the problem was. The scalate integrating code contained paths like “/app/views” which worked perfectly on my machine but wasn’t on heroku. Deleting the first slash solved the problem.

While investigating the problem I thought that the problem might come from the not yet precompiled Scamls. So I enabled precompilation for Scalate. This can be done as follows:

Add the xsbt scalate generator to the plugins.sbt of your project:

libraryDependencies  "com.mojolly.scalate" %% "xsbt-scalate-generator" % (v + "-0.1.6"))

Additionally a build.sbt must be added to the home directory of the project with the following content:

import com.mojolly.scalate.ScalatePlugin._

seq(scalateSettings:_*)

scalateTemplateDirectory in Compile <<= (baseDirectory) { _ / "app/views" }

//scalateImports ++= Seq(
//  "import OAuth2Imports._",
//  "import model._"
//)
//
//scalateBindings ++= Seq(
//  Binding("flash", "scala.collection.Map[String, Any]", defaultValue = "Map.empty"),
//  Binding("session", "org.scalatra.Session"),
//  Binding("sessionOption", "Option[org.scalatra.Session]"))

The commented parts are bindings and “auto-imports” that might become handy if you need em. If you don’t use the e.g. oauth libraries, you’ll of course get compilation errors as the imports are added to the templates. I’ve left some examples in the file as a reminder that and how it is possible to use these features.

As precompilation is something you should have enabled in production anyway, I left it enabled after finding out why the app on heroku didn’t run.

Coming up

So far all entered data is “persisted” to memory. To change this I’ll use MongoDB, as this is an excellent alternative for a classic SQL.

Play 2.0 with Scala and Scaml, Part 3: The web stuff

First of all I’d like to reference to an excellent series of posts I found that cover similar stuff. It can be found here and covers a lot more topics than I did here.
In this part I’m going to show how to actually create the little blogging app I intend to write. I canceled my original intention of using LESS when I found the wonderful Bootstrap framework made by the Twitter guys. My intention was integrating technologies in Play 2 I’d like to have at hand when developing a web application, but LESS is integral part of it anyway, so I decided to play a little with Bootstrap. It’s amazingly easy to create at least considerably good looking pages, you’ll see.
If you have the need to use LESS, what makes definitly sense for bigger projects, you can do that as well as described here.

Creating some HTML

First thing I’ve done was creating some basic webpage to show the content initiated in the Global object. The nicely concise Scaml snippet could look like this:

-@ val posts: List[models.helwich.Post]
!!!5
%html
    %title My kewl blogging application
    %body
        -for(post <- posts)
            %div
                %h2 post.title
                %textarea(readonly="readonly") post.content

Together with an Action for the index of the page as shown in the Snippet below, the content is shown correctly as plain HTML.

def index = Action {
    Ok( Scalate("main-blog.scaml").render('posts -> Post.findAll()))
}

But plain HTML is not the design we expect for a modern web application. To get some sexyness regarding design and layout we must add some CSS. As I’ve mentioned in the beginning I found Bootstrap to be the perfect solution to create a good looking website not being a web-designer. To make it happen just download the Bootstrap framework here and copy the CSS files in your /public/stylesheets folder of your app.
To get a consistent look in the whole application, you should define a default.scaml in /views/layouts. This is the Scalate way and you can use this feature exactly as described in the very good documentation. I’ve put all the CSS, JavaScript loading as well as the html, title and the body tag in the default.scaml.
To apply the styles on our page you must add some classes and ids on the HTML elements. The resulting Scamls look like this.

default.scaml:

-@ var body:String
-@ var title: String = "The new Blog"

!!!5
%link(rel="stylesheet" type="text/css" href="/assets/stylesheets/bootstrap.min.css")
:css
    body {
        padding-top: 50px;
        padding-bottom: 40px;
    }
%link(rel="stylesheet" type="text/css" href="/assets/stylesheets/bootstrap-responsive.min.css")

%html
    %title #{title}
    %body
        -unescape(body)

-#load js at the end to have the page displayed first thus quicker
%script(type="text/javascript" src="assets/javascripts/jquery-1.7.1.min.js")

I haven’t found a way to get the Play convenience methods for accessing Assets working in the Scamls. If anyone knows how to do this, please let me know.

main-blog.scaml:

%div.navbar.navbar-fixed-top
    %div.navbar-inner
        %div.container
            %a.brand Here comes the Blog
%div.container#blogarea
    -for(post <- posts)
        %div.row.well
            %h2 post.title
            %textarea.input-block-level(readonly="readonly") post.content

In Scaml/Haml, CSS as well as in CoffeeScript (as you’ll see later) the . is used to define classes and the # is used to define IDs. This consistent syntax in all web related languages is really nice, I think. The used classes here like .navbar.navbar-fixed-top, input-block-level, row, well and so forth are references to the styles defined by Bootstrap.
Here I defined the standard layout with a fixed navigation bar at the top, where only a short header “Here comes the blog” is shown. Below that the blogs are shown in rows where the content is in a read-only textarea with the applied style input-block-level.

Creating some AJAX magic

My next step was to get some AJAX magic happen. The good thing is that Play 2.0 comes fully equipped with jQuery and CoffeeScript. If you’re experienced with these frameworks you can start coding immediately. I wasn’t, so I wanted to start slowly and in small steps. The first task easy enough for was “load the content in the background via an GET request using jQuery”.

main.coffee

$.get "/listposts", (data) ->
    $.each data, (index, post) ->
      $("#blogarea").append (
          "<div class=\"row well\">" +
            "<h2>" + post.title + "</h2>" +
            "<textarea readonly=\"readonly\" class=\"input-block-level\">" + post.content + "</textarea>" +
          "</div>"
      )

This CoffeeScript file sends a get request /listposts and adds for each returned line a html-snippet to the first element with the ID blogarea. If you want to use it like this, you have to delete the lines in the main-blog.scaml where the blogs are added of course.
Furthermore there are two things to do. First add a correspoding route to the routes file of the application like this:

GET     /listposts                  controllers.Application.listPosts

Second add the listPosts method to the Application object. Its important to add generate .. as(JSON) here as this JSON is what jQuery understands.

  def listPosts = Action {
    Ok( generate(Post.findAll())).as(JSON)
  }

Thats it. But just loading data in the background for this kind of page doesn’t really make sense, does it?

Adding posts via AJAX

But if posts could be added it would be really nice to avoid the complete page reload. To accomplish there are three things to do:

  • add a form for to enter title and content
  • create a post request to send data from the client to the server
  • clear the old posts and reload the actual data from the server

To create a small example of how to modularize Scamls, I planed creating a form that could be reused somewhere else. Scalate offers different ways of structuring and reusing code. The most simple and straight forward is the render mechanism. You can create a file with some Scaml code and use -render(“yourfilename.scaml”) anywhere in a Scaml to render that file there.
In the given example this looks like this.

enter-blog-form.scaml:

%div.row.well
    %form#blog-form(name="blog" action="")
        %label(for="title") Title
        %input.input-block-level(type="text" name="title")
        %label(for="content") Content
        %textarea.input-block-level(name="content")
        %input#reset.hide(type="reset")
        %button#button.btn Send

And the main.scaml changes to:

%div.navbar.navbar-fixed-top
    %div.navbar-inner
        %div.container
            %a.brand Here comes the Blog
%div.container
    -render("enter-blog-form.scaml")
%div.container#blogarea

Hiding a button like I used to know it didn’t work here. That is why the reset button has no attribute hidden but uses the hide class of Bootstrap instead.
In some examples I found there were inputs used for the buttons. When the .btn class of Bootstrap is applied to a html element more or less everything will look like a button. I use the button tag for the send button here anyway. This has the advantage that we can use the tab key to get the focus on it.
There is not action defined for this form. This is because the post will be performed in the background via AJAX like this:

main.coffee:

readAllPosts =  ->
  $("#blogarea").empty()
  $.get "/listposts", (data) ->
    $.each data, (index, post) ->
      $("#blogarea").append (
          "<div class=\"row well\">" +
            "<h2>" + post.title + "</h2>" +
            "<textarea readonly=\"readonly\" class=\"input-block-level\">" + post.content + "</textarea>" +
          "</div>"
      )

$(document).ready ->
  readAllPosts.apply()

  $('#button').click ->
    dataString = $("#blog-form").serialize()
    $.ajax(
      type: 'POST'
      url: '/submit-new-blog'
      data: dataString
      success: ->
        $("#reset").click()
        readAllPosts.apply()
      error: (jqXHR, textStatus, errorThrown) ->
        alert(errorThrown)
    )
    false

So, what happens here? First thing to mention is the clearing of the blog data in the first function readAllPosts. This is needed as we want to reload all posts when a new one is added. Otherwise not only the new post would be added, but the complete set of posts would be added to the already shown ones with ugly duplications.
Then there is the $(document).ready stuff. This is the jQuery place for functions that shall be activated on certain events on objects of the document. I also added the initialization here – loading all posts from the server. After that I used jQuery to define a function for the element of the document with the ID button. If the button is clicked the function will be executed.
This jQuery function serializes the content of the form and makes an AJAX post to /submit-new-blog delivering the dataString to the server. In case of a success, the form is reseted and the posts are reloaded by executing the readAllPosts function. In case of an error it shows it to you. Quite important is to return false here. Otherwise the form will be submitted normally, thus there will be a normal request with the content of the form in the query. Very ugly.

Conclusion
This is it. We have a small application with some AJAX magic, with amazingly few lines of code, but very clear in its intention. No hidden side effects as you tend to have ’em in most component based frameworks. I am very enthused about Play 2 in conjunction with Anorm, Scaml and CoffeeScript. It is very promising regarding concisness, maintainability, development speed and quality of the solutions. I hope beeing able to develop some real worl stuff soon.

You can get the complete source code here. For more stuff on Play 2 and related technologies I’d like to hint to the series of blogs of Matt Raible again.

Play 2.0 with Scala and Scaml, Part 2: Setup example data at startup and integrating Scalate

In this part I’ll show how to set up the web part. Especially setting up Scalate for the use of Scaml needs some extra effort as it is not part of the Play core. I’ll first show how to create some example data so we have something to see on the website later on. As a prerequisite for the integration of Scalate I’ll shortly describe how to add the needed dependency and get it all correctly setup in IntelliJ. Then I’ll show how to get Scalate with Scaml running within Play.

Setting up a global object for creation of example data

To get some example data in my (still in memory!!!) database, you’ll need some means to get something running exactly once per server startup. In Play this can be done in the global object. The global object is per convention the object with the name Global. If you want an Object named differently to be the global object, you can configure it in the application.config (look for the global object).

Here is how Global might look to set up some very simple data.

object Global extends GlobalSettings {

  override def onStart(app: play.api.Application) {
    val u1 = User.create(new User("u1", "pwd", "name1"))
    val u2 = User.create(new User("u2", "pwd", "name2"))

    Post.create( new Post("title1", new DateTime().toDate, "content number 1", u1))
    Post.create( new Post("title2", new DateTime().toDate, "content number 2", u2))
  }
}

One thing I am still wondering about is that when having the server run, the DB in the test cases also contains the entities set up on startup. If anyone knows why and how to avoid, please let me know.

Adding Scalate to the project and getting IntelliJ working

Adding the Scalate dependency to my project puzzled me a while, so I’ll describe a solution here hoping I can save some time of yours. I am using IntelliJ and for me it’s really important to have the IDE as working aid. The basic workflow for adding dependencies when using IntelliJ is as follows:

  1. Add the dependency to Build.scala
  2. Enter reload in the console
  3. Enter idea in the console

It’s important that there are no compilation errors when adding the dependency. So if you add code for which you need a certain new dependency you’ll have to comment this code out until you have added the dependency successfully.

So to add Scalate just add “org.fusesource.scalate” % “scalate-core” % “1.5.3” to appDependencies in your Build.scala and proceed as described above.

Getting Scalate to work

I have set up the Scalate templating engine in my company recently and am astonished how clean and DRY scaml is. For me as non-weby as well as for our webbies it became a pleasure to delve into our new mobile-page-code written in scaml. Structure and intention really pops out.

As mentioned above, there is no Scalate plugin for 2.0. After some research I found this 1.x example to integrate Scalate in Play. It’s more or less ready to go. You have to change the package for play.Play to play.api.Play and change the call in the controller to something like

Ok( Scalate("first.scaml").render('posts -> Post.findAll())).as(HTML)

But this sucks right? I don’t want to add as(HTML) when I trigger the rendering. But without the as(HTML) the content type is set to text and thus not rendered by the browser. To change this we have to define a Writable – that encodes the content – and a ContentType. Both are defined as implicits and need a type on which they work. I’ve defined a simple ScalateContent that wraps the rendered output and enables the correct application of the implicits.

Scalate format

I have changed the original code so that the format used can be defined in the application.config via scalate.format. Right now this is only important for the default layout, as I explicitly state the format of the template to render in the controller as you can see above. I personally prefer it this way as it seems more explicit what exactly is meant. You can easily extend the apply method if you prefer calls like Scalate(“first”).render.

Default layout

Scalate uses default templates for stuff that layouts all pages. To enable this I added the default layout strategy. The default template can now be used as documented. In the Play project the path is app/views/layouts/default.{format}.

So after all this the code for the Scalate object looked like this:

package controllers

import play.api._
import http.{Writeable, ContentTypeOf, ContentTypes}
import mvc.Codec
import play.api.Play.current
import org.fusesource.scalate.layout.DefaultLayoutStrategy

object Scalate {

  import org.fusesource.scalate._
  import org.fusesource.scalate.util._

  var format = Play.configuration.getString("scalate.format") match {
    case Some(configuredFormat) => configuredFormat
    case _ => "test"
  }

  lazy val scalateEngine = {
    val engine = new TemplateEngine
    engine.resourceLoader = new FileResourceLoader(Some(Play.getFile("/app/views")))
    engine.layoutStrategy = new DefaultLayoutStrategy(engine, Play.getFile("/app/views/layouts/default." + format).getAbsolutePath)
    engine.classpath = Play.getFile("/tmp/classes").getAbsolutePath
    engine.workingDirectory = Play.getFile("tmp")
    engine.combinedClassPath = true
    engine.classLoader = Play.classloader
    engine
  }

  def apply(template: String) = Template(template)

  case class Template(name: String) {

    def render(args: (Symbol, Any)*) = {
      ScalateContent{
        scalateEngine.layout(name, args.map {
          case (k, v) => k.name -> v
        } toMap)
      }
    }

  }

  case class ScalateContent(val cont: String)

  implicit def writeableOf_ScalateContent(implicit codec: Codec): Writeable[ScalateContent] = {
    Writeable[ScalateContent](scalate => codec.encode(scalate.cont))
  }

  implicit def contentTypeOf_ScalateContent(implicit codec: Codec): ContentTypeOf[ScalateContent] = {
    ContentTypeOf[ScalateContent](Some(ContentTypes.HTML))
  }

}

Next time

From here on you should be able to use Scalate in Play 2.0, making your web development more fun and of course more efficient. Next time I’ll show how to set up the pages with Scaml, adding some sexieness with Less and some client side logic with CoffeeScript.

Play 2.0 with Scala and Scaml, Part1: Setup of test infrastructure, model and persistence with Anorm

In this series I’ll try to provide some help on topics I had problems with when I implemented my first Play 2.0 project. My goal is it to get a small version of the blogging engine of the 1.0 tutorial. I will not provide a complete tutorial, but only stuff that might be helpfull setting up the project.

Hint for all beginners with the 2.0 RC: One source of knowledge I was missing in the beginning is the 2.0 wiki, which is often enough quite useful.

Setting up IntelliJ

I setup IntelliJ as described in here. Everything worked except the server startup. To have it started you have to use a different main class: play.core.server.NettyServer. Server startup works like this, but the application is not loaded correctly. As this has no priority for me, I didn’t spend much time trying to solve this. If anyone knows what the problem is, please let me know.

The first model

In the tutorial three classes were defined: User, Post and Comment. From my perspective Post and User should be enough to cover all the basic topics and problems, so I’ll skip the comment. The definition of User and Post in scala is pretty easy. The Pk herein is needed as there is foreign key relationship between the tables as you’ll see later. It’s an optional parameters and thus most users will hardly ever notice that it’s there.

case class User(email: String, password: String, fullname: String, isAdmin: Boolean = false, id:Pk[Int] = NotAssigned)

case class Post(title: String,  posted: Date, content: String, author: User)

Evolutions

To persist the entities we’ll need some persistence. I will use the standard h2 in memory db. If the data is read and written mainly in whole entities by a PK, like here,  I’d normally prefer a NoSQL-DB like Mongo or Neo4J. But for the sake of simplicity I’ll stick with the standard DB here.

Play is using the concept of Evolutions for DBs. The basic idea of evolutions is to describe what must be executed to perform the evolution (Ups) and what must be done to undo the evolution (Downs). This is probably not the best and golden way, but a pretty good one especially for smaller projects. For a good wrap up about problems and solutions of DB changes have a look here.

For Post and User the first evolution might look like this and has to be defined in [PROJECT_HOME]/conf/evolutions/default/1.sql:

# --- First database schema
# --- !Ups
CREATE TABLE myuser (
id                            INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
email                         VARCHAR(255) NOT NULL,
password                      VARCHAR(255) NOT NULL,
fullname                      VARCHAR(255) ,
isAdmin                       BOOLEAN NOT NULL
);
CREATE TABLE posts(
title                         VARCHAR(255) NOT NULL,
posted                        DATE NOT NULL,
content                       VARCHAR(2550) NOT NULL,
authorId                      INT NOT NULL,
FOREIGN KEY                   (authorId) REFERENCES myuser(id)
);

# --- !Downs
DROP TABLE IF EXISTS myuser;
DROP TABLE IF EXISTS posts;

Implementing the persistence test driven with Anorm

In the standard tutorial the test mode is described, where one can open the @test url and see all tests. I haven’t found this in 2.0 yet. Right now I don’t bother too much, as I’ll run the tests from within the IDE. Anyway: If anyone knows how to get the web based tests running again, gimme a note.

I like to write test first. The implementation of integration tests for the persistence differs from version 1. Thus I’ll explain it in a little more detail. My first test for the persistence looked like this:


"The user class" should {
  "be persisted" in {
    running(FakeApplication()) {
      User.create(User("email", "pwd", null))
      User.findAll() must have size 1
    }
  }
}

I made some mistakes in the setup and needed quite a time to figure out what the problem was. One thing that really bothered me was, that the log of the test didn’t tell what the problem was. It just stated that the table MYUSER doesn’t exist. In my case there were 2 problems. First of all there was no evolution applied as I named the directory wrong. Second there was a syntax error in my SQL script. One of the real valuable benefits of test should be to get a hint where an error is located. To achieve this I wrote another test, that explicitly tested the application of the evolutions:

class DBEvolutionsTest extends Specification {

  "Evolutions" should {
    "be applied without errors" in {
      evolutionFor("default")
      running(FakeApplication()) {
        DB.withConnection {
          implicit connection =>
            SQL("select count(1) from myuser").execute()
            SQL("select count(1) from posts").execute()
        }
      }
      success
    }
  }
}

What happens here? First we tell the testing Helpers class that it should apply the evolutions for the DB default. The connection for this DB is defined in the application.conf (db.default.driver etc.) and the evolution as already described in the 1.sql, where the db name default is defined in the path. Then a fake application is created and a simple SQL is executed to check, that the table exists.

Written like this, logs and error messages have the verbosity I prefer. You can now see in your test log how the evolution is applied. If there is a syntax error it’ll be shown. If you haven’t put the evolution at the right place (as I did) you will at least see, that the evolution is not applied.

You might have noticed that I have one nullable column in my schema definition. Normally I’m not a great friend of null values and try to avoid them wherever possible. In this case I’d definitely make it not nullable too. But to have examples of all the “standard” cases, I thought it would be a good idea to have at least one nullable column in the example.

I’ll skip the rest of the tests as they are pretty straight forward from here on. The resulting User object used for persistence looked like this in the end. I used the SqlParser of Anorm to create a parser that could be used in the data access methods later on. As you can see here, the nullable column is PITA again. You’ll have to access it via an option and have to provide a value in case there is none given. Otherwise there will be an UnexpectedNullableFound exception be thrown. To be “consistent” I’ve choosen null to be set in User again .. argh.

The Pk of a new entity is set after insertion of a new User via SELECT SCOPE_IDENTITY(). I’m not sure if this is the way to go and if it is thread safe, but it works for this little example at least. The Pk definition is needed for the persistence of the posts again, as we have a foreign key relationship between the two.

object User  {
  val parser = {
      get[Pk[Int]]("id") ~
      get[String]("email") ~
      get[String]("password")~
      get[Option[String]]("fullname")~
      get[Boolean]("isAdmin") map {
      case pk ~ mail ~ name ~ fullname ~ isAdmin => User(mail, name, fullname.getOrElse(null), isAdmin, pk)
    }
  }

  def findBy(id: Pk[Int]): User = {
    DB.withConnection {
      implicit connection =>
        SQL("select * from myuser where id = {id}").on("id" -> id.get).using(parser).single()
    }
  }

  def findAll(): Seq[User] = {
    DB.withConnection {
      implicit connection =>
        SQL("select * from myuser").as(User.parser *)
    }
  }

  def create(user: User): User = {
    DB.withConnection {
      implicit connection =>
        SQL("insert into myuser(email, password, fullname, isadmin) values ({email}, {password}, {fullname}, {isAdmin});").on(
          'email -> user.email,
          'password -> user.password,
          'fullname -> user.fullname,
          'isAdmin -> user.isAdmin
        ).executeUpdate()
        val id = SQL("SELECT SCOPE_IDENTITY()")().collect {
          case Row(id: Int) => id
        }.head
        return User(user.email, user.password, user.fullname, user.isAdmin, new Id(id))
    }
  }
}

In the following definition of the post object you can see the why the Pk of the user is needed after creation.

object Post{
  val parser = {
      get[String]("title") ~
      get[Date]("posted") ~
      get[String]("content")~
      get[Pk[Int]]("authorId") map {
      case title ~ posted ~ content ~ author => Post(title,  posted, content, User.findBy(author))
    }
  }

  def findAll() = {
    DB.withConnection {
      implicit connection =>
        SQL("select * from posts").as(parser *)
    }
  }

  def create(post: Post): Unit = {
    DB.withConnection {
      implicit connection =>
        SQL("insert into posts(title, posted, content, authorId) values ({title}, {posted}, {content}, {authorId})").on(
          'title -> post.title,
          'posted -> post.posted,
          'content -> post.content,
          'authorId -> post.author.id
        ).executeUpdate()
    }
  }
}

Conclusion

Setup of the project and integration in the IDE worked like a charm. Same holds for setting up the test infrastructure if you have found where its documented. The docs will be made more visible when 2.0 is officially released, so no problem. I like the verbosity of the test helpers. It becomes really obvious what is happening, without making the tests too chatty. I really love the basic idea about the Anorm framework for database access. It’s clear and concise and again it becomes clear in the code what is happening. I’m not sure what I think about the lack of the Magic trait. Having to write words like password 7 times is a major PITA for me. I’ definitely love seeing it again in one of the upcoming versions.