Play with MongoDB

For the normal data storage in most of todays applications a relational DB is not the optimal solution for several reasons. There are many great alternatives now on the market. I like MongoDb because it is not that far away from the concepts of a relational DB, it is easy to set up and has proven to be stable and reliable.

Thus I clearly want a MongoDb as persistence in my Play 2! project. Goal is

  • to have a good abstraction. Not too much magic but not too much boilerplate either
  • to have it running locally and in the cloud (I’ll go for Heroku here)

I found Salat a pretty good abstraction. It got it running quickly locally. But when I tried to get it running on Heroku it became a little awkward. At Heroku – as on every other PaaS I had a look at – you get a URL for the MongoDb you added to your app, normally provided as system property. Same holds if rent a DB in the cloud at a “pure” cloud DB provider. But there are no means (that I know of course) to easily let Salat read and use the URL.

So I wrote a MongoDao that does the following things:

  • it reads a MongoURL defined in the application.conf via the mongo.uri property.

In my template application I provided an example that reads the environment variable MONGOLAB_URI which is the system variable on a Heroku app with an installed MongoLab instance. If no such variable is set, the property is left empty. This lets the MongoDao take the standard parameters for a local MongoDb installation.

  • a collection method that returns a collection as required by the SalatDao, so you can easily define the SalatDao in your implementation
  • if you want to differentiate between different local DBs you can overwrite the defaultDbName val in you class.

This is pretty much it. If any one of you has good ideas to make the abstraction more convenient, clean, elegant I’d be happy to read your comments.

Using imports in LESS

I’m playing around with bootstrap and am using the LESS sources of bootstrap in my project. When I tried to run the code I encountered the following error statement:

EcmaError: TypeError: Cannot call method “split” of undefined (less-1.2.1.js#347)

What happened was that I tried to import a file that imports files and this is obviously not allowed in LESS.

Basic authentication with Play! 2

I recently had the problem that I wanted to secure a page preview we developed with basic authentication. Easy thing I thought. But it wasn’t. I was used to the use of frameworks/containers like tomcat, which provide easy means to do such things. With Play! though you don’t have such things. You’ll have to do it yourself.

The good thing is that you’ll understand what is actually going on:

The basic stuff happening is:

  1. Request for a certain page is sent to the server
  2. On the server the header with the name “Authorization” is looked up
  3. If it exists the contained string is parsed. It’s in the format “name:pwd”. Then name and pwd are checked.
  4. If either the header does not exist or nam/pwd are incorrect an unauthorized 401 response with a header of name “WWW-Authenticate” and a content that must contain the word basic is sent. This lets the browser know what actually to do.

Thats pretty much it. With Play! 1 this was pretty easy as you had direct access to the response as you might be used to in several MVC frameworks. With Play! 2 this is not so obvious although also really easy:

You can add a header to the response by using the method withHeaders. A good example can be found here.

At the time I needed it, I was unable to find an example in the net. Now as I wanted to finish this post and made another quick query and immediately found the example of Guillaume Bort. As he develops the play framework I decided to just add the link here without any code.

Getting Play 2 with Scalate to run on heroku

After having everything in place I thought it’d be a great idea to deploy the small app in the cloud. After some research I was really excited about Heroku. In my eyes a brilliant idea and realization of PaaS.
After the decision for Heroku it took me only a couple of minutes, till I was ready to go,  with the quite  good tutorial of the heroku guys. The push worked flawlessly. But when I wanted to admire my first app in the cloud, I got an error message on the page. Shouldn’t be too easy anyway, right?
The logs told me the following:

2012-04-18T06:00:44+00:00 app[web.1]: ! @6a4hnb7bj - Internal server error, for request [GET /] ->
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:834) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]:
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.util.IOUtil$.writeBinaryFile(IOUtil.scala:111) ~[scalate-util-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.actor.Actor$class.apply(Actor.scala:290) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.layout.DefaultLayoutStrategy.org$fusesource$scalate$layout$DefaultLayoutStrategy$$tryLayout(DefaultLayoutStrategy.scala:77) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: [error] o.f.s.l.DefaultLayoutStrategy - Unhandled: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]: play.core.ActionInvoker$$anonfun$receive$1$$anon$1: Execution exception [[TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)]]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.liftedTree1$1(TemplateEngine.scala:411) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:63) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:475) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker.apply(Invoker.scala:61) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: Caused by: java.io.FileNotFoundException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]:
2012-04-18T06:00:44+00:00 app[web.1]: Caused by: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.scaml.scala (No such file or directory)
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.actor.ActorCell.invoke(ActorCell.scala:617) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.(FileOutputStream.java:160) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:82) [play_2.9.1-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:405) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: [error] application -
2012-04-18T06:00:44+00:00 app[web.1]: 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:179) [akka-actor-2.0.jar:2.0]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.open(Native Method) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:747) ~[scalate-core-1.5.3.jar:1.5.3]
2012-04-18T06:00:44+00:00 app[web.1]: 	at java.io.FileOutputStream.(FileOutputStream.java:209) ~[na:1.6.0_20]
2012-04-18T06:00:44+00:00 app[web.1]: 	at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]

It took me a while to find out what the problem was. The scalate integrating code contained paths like “/app/views” which worked perfectly on my machine but wasn’t on heroku. Deleting the first slash solved the problem.

While investigating the problem I thought that the problem might come from the not yet precompiled Scamls. So I enabled precompilation for Scalate. This can be done as follows:

Add the xsbt scalate generator to the plugins.sbt of your project:

libraryDependencies  "com.mojolly.scalate" %% "xsbt-scalate-generator" % (v + "-0.1.6"))

Additionally a build.sbt must be added to the home directory of the project with the following content:

import com.mojolly.scalate.ScalatePlugin._

seq(scalateSettings:_*)

scalateTemplateDirectory in Compile <<= (baseDirectory) { _ / "app/views" }

//scalateImports ++= Seq(
//  "import OAuth2Imports._",
//  "import model._"
//)
//
//scalateBindings ++= Seq(
//  Binding("flash", "scala.collection.Map[String, Any]", defaultValue = "Map.empty"),
//  Binding("session", "org.scalatra.Session"),
//  Binding("sessionOption", "Option[org.scalatra.Session]"))

The commented parts are bindings and “auto-imports” that might become handy if you need em. If you don’t use the e.g. oauth libraries, you’ll of course get compilation errors as the imports are added to the templates. I’ve left some examples in the file as a reminder that and how it is possible to use these features.

As precompilation is something you should have enabled in production anyway, I left it enabled after finding out why the app on heroku didn’t run.

Coming up

So far all entered data is “persisted” to memory. To change this I’ll use MongoDB, as this is an excellent alternative for a classic SQL.

Play 2.0 with Scala and Scaml, Part 3: The web stuff

First of all I’d like to reference to an excellent series of posts I found that cover similar stuff. It can be found here and covers a lot more topics than I did here.
In this part I’m going to show how to actually create the little blogging app I intend to write. I canceled my original intention of using LESS when I found the wonderful Bootstrap framework made by the Twitter guys. My intention was integrating technologies in Play 2 I’d like to have at hand when developing a web application, but LESS is integral part of it anyway, so I decided to play a little with Bootstrap. It’s amazingly easy to create at least considerably good looking pages, you’ll see.
If you have the need to use LESS, what makes definitly sense for bigger projects, you can do that as well as described here.

Creating some HTML

First thing I’ve done was creating some basic webpage to show the content initiated in the Global object. The nicely concise Scaml snippet could look like this:

-@ val posts: List[models.helwich.Post]
!!!5
%html
    %title My kewl blogging application
    %body
        -for(post <- posts)
            %div
                %h2 post.title
                %textarea(readonly="readonly") post.content

Together with an Action for the index of the page as shown in the Snippet below, the content is shown correctly as plain HTML.

def index = Action {
    Ok( Scalate("main-blog.scaml").render('posts -> Post.findAll()))
}

But plain HTML is not the design we expect for a modern web application. To get some sexyness regarding design and layout we must add some CSS. As I’ve mentioned in the beginning I found Bootstrap to be the perfect solution to create a good looking website not being a web-designer. To make it happen just download the Bootstrap framework here and copy the CSS files in your /public/stylesheets folder of your app.
To get a consistent look in the whole application, you should define a default.scaml in /views/layouts. This is the Scalate way and you can use this feature exactly as described in the very good documentation. I’ve put all the CSS, JavaScript loading as well as the html, title and the body tag in the default.scaml.
To apply the styles on our page you must add some classes and ids on the HTML elements. The resulting Scamls look like this.

default.scaml:

-@ var body:String
-@ var title: String = "The new Blog"

!!!5
%link(rel="stylesheet" type="text/css" href="/assets/stylesheets/bootstrap.min.css")
:css
    body {
        padding-top: 50px;
        padding-bottom: 40px;
    }
%link(rel="stylesheet" type="text/css" href="/assets/stylesheets/bootstrap-responsive.min.css")

%html
    %title #{title}
    %body
        -unescape(body)

-#load js at the end to have the page displayed first thus quicker
%script(type="text/javascript" src="assets/javascripts/jquery-1.7.1.min.js")

I haven’t found a way to get the Play convenience methods for accessing Assets working in the Scamls. If anyone knows how to do this, please let me know.

main-blog.scaml:

%div.navbar.navbar-fixed-top
    %div.navbar-inner
        %div.container
            %a.brand Here comes the Blog
%div.container#blogarea
    -for(post <- posts)
        %div.row.well
            %h2 post.title
            %textarea.input-block-level(readonly="readonly") post.content

In Scaml/Haml, CSS as well as in CoffeeScript (as you’ll see later) the . is used to define classes and the # is used to define IDs. This consistent syntax in all web related languages is really nice, I think. The used classes here like .navbar.navbar-fixed-top, input-block-level, row, well and so forth are references to the styles defined by Bootstrap.
Here I defined the standard layout with a fixed navigation bar at the top, where only a short header “Here comes the blog” is shown. Below that the blogs are shown in rows where the content is in a read-only textarea with the applied style input-block-level.

Creating some AJAX magic

My next step was to get some AJAX magic happen. The good thing is that Play 2.0 comes fully equipped with jQuery and CoffeeScript. If you’re experienced with these frameworks you can start coding immediately. I wasn’t, so I wanted to start slowly and in small steps. The first task easy enough for was “load the content in the background via an GET request using jQuery”.

main.coffee

$.get "/listposts", (data) ->
    $.each data, (index, post) ->
      $("#blogarea").append (
          "<div class=\"row well\">" +
            "<h2>" + post.title + "</h2>" +
            "<textarea readonly=\"readonly\" class=\"input-block-level\">" + post.content + "</textarea>" +
          "</div>"
      )

This CoffeeScript file sends a get request /listposts and adds for each returned line a html-snippet to the first element with the ID blogarea. If you want to use it like this, you have to delete the lines in the main-blog.scaml where the blogs are added of course.
Furthermore there are two things to do. First add a correspoding route to the routes file of the application like this:

GET     /listposts                  controllers.Application.listPosts

Second add the listPosts method to the Application object. Its important to add generate .. as(JSON) here as this JSON is what jQuery understands.

  def listPosts = Action {
    Ok( generate(Post.findAll())).as(JSON)
  }

Thats it. But just loading data in the background for this kind of page doesn’t really make sense, does it?

Adding posts via AJAX

But if posts could be added it would be really nice to avoid the complete page reload. To accomplish there are three things to do:

  • add a form for to enter title and content
  • create a post request to send data from the client to the server
  • clear the old posts and reload the actual data from the server

To create a small example of how to modularize Scamls, I planed creating a form that could be reused somewhere else. Scalate offers different ways of structuring and reusing code. The most simple and straight forward is the render mechanism. You can create a file with some Scaml code and use -render(“yourfilename.scaml”) anywhere in a Scaml to render that file there.
In the given example this looks like this.

enter-blog-form.scaml:

%div.row.well
    %form#blog-form(name="blog" action="")
        %label(for="title") Title
        %input.input-block-level(type="text" name="title")
        %label(for="content") Content
        %textarea.input-block-level(name="content")
        %input#reset.hide(type="reset")
        %button#button.btn Send

And the main.scaml changes to:

%div.navbar.navbar-fixed-top
    %div.navbar-inner
        %div.container
            %a.brand Here comes the Blog
%div.container
    -render("enter-blog-form.scaml")
%div.container#blogarea

Hiding a button like I used to know it didn’t work here. That is why the reset button has no attribute hidden but uses the hide class of Bootstrap instead.
In some examples I found there were inputs used for the buttons. When the .btn class of Bootstrap is applied to a html element more or less everything will look like a button. I use the button tag for the send button here anyway. This has the advantage that we can use the tab key to get the focus on it.
There is not action defined for this form. This is because the post will be performed in the background via AJAX like this:

main.coffee:

readAllPosts =  ->
  $("#blogarea").empty()
  $.get "/listposts", (data) ->
    $.each data, (index, post) ->
      $("#blogarea").append (
          "<div class=\"row well\">" +
            "<h2>" + post.title + "</h2>" +
            "<textarea readonly=\"readonly\" class=\"input-block-level\">" + post.content + "</textarea>" +
          "</div>"
      )

$(document).ready ->
  readAllPosts.apply()

  $('#button').click ->
    dataString = $("#blog-form").serialize()
    $.ajax(
      type: 'POST'
      url: '/submit-new-blog'
      data: dataString
      success: ->
        $("#reset").click()
        readAllPosts.apply()
      error: (jqXHR, textStatus, errorThrown) ->
        alert(errorThrown)
    )
    false

So, what happens here? First thing to mention is the clearing of the blog data in the first function readAllPosts. This is needed as we want to reload all posts when a new one is added. Otherwise not only the new post would be added, but the complete set of posts would be added to the already shown ones with ugly duplications.
Then there is the $(document).ready stuff. This is the jQuery place for functions that shall be activated on certain events on objects of the document. I also added the initialization here – loading all posts from the server. After that I used jQuery to define a function for the element of the document with the ID button. If the button is clicked the function will be executed.
This jQuery function serializes the content of the form and makes an AJAX post to /submit-new-blog delivering the dataString to the server. In case of a success, the form is reseted and the posts are reloaded by executing the readAllPosts function. In case of an error it shows it to you. Quite important is to return false here. Otherwise the form will be submitted normally, thus there will be a normal request with the content of the form in the query. Very ugly.

Conclusion
This is it. We have a small application with some AJAX magic, with amazingly few lines of code, but very clear in its intention. No hidden side effects as you tend to have ‘em in most component based frameworks. I am very enthused about Play 2 in conjunction with Anorm, Scaml and CoffeeScript. It is very promising regarding concisness, maintainability, development speed and quality of the solutions. I hope beeing able to develop some real worl stuff soon.

You can get the complete source code here. For more stuff on Play 2 and related technologies I’d like to hint to the series of blogs of Matt Raible again.

Play 2.0 with Scala and Scaml, Part 2: Setup example data at startup and integrating Scalate

In this part I’ll show how to set up the web part. Especially setting up Scalate for the use of Scaml needs some extra effort as it is not part of the Play core. I’ll first show how to create some example data so we have something to see on the website later on. As a prerequisite for the integration of Scalate I’ll shortly describe how to add the needed dependency and get it all correctly setup in IntelliJ. Then I’ll show how to get Scalate with Scaml running within Play.

Setting up a global object for creation of example data

To get some example data in my (still in memory!!!) database, you’ll need some means to get something running exactly once per server startup. In Play this can be done in the global object. The global object is per convention the object with the name Global. If you want an Object named differently to be the global object, you can configure it in the application.config (look for the global object).

Here is how Global might look to set up some very simple data.

object Global extends GlobalSettings {

  override def onStart(app: play.api.Application) {
    val u1 = User.create(new User("u1", "pwd", "name1"))
    val u2 = User.create(new User("u2", "pwd", "name2"))

    Post.create( new Post("title1", new DateTime().toDate, "content number 1", u1))
    Post.create( new Post("title2", new DateTime().toDate, "content number 2", u2))
  }
}

One thing I am still wondering about is that when having the server run, the DB in the test cases also contains the entities set up on startup. If anyone knows why and how to avoid, please let me know.

Adding Scalate to the project and getting IntelliJ working

Adding the Scalate dependency to my project puzzled me a while, so I’ll describe a solution here hoping I can save some time of yours. I am using IntelliJ and for me it’s really important to have the IDE as working aid. The basic workflow for adding dependencies when using IntelliJ is as follows:

  1. Add the dependency to Build.scala
  2. Enter reload in the console
  3. Enter idea in the console

It’s important that there are no compilation errors when adding the dependency. So if you add code for which you need a certain new dependency you’ll have to comment this code out until you have added the dependency successfully.

So to add Scalate just add “org.fusesource.scalate” % “scalate-core” % “1.5.3” to appDependencies in your Build.scala and proceed as described above.

Getting Scalate to work

I have set up the Scalate templating engine in my company recently and am astonished how clean and DRY scaml is. For me as non-weby as well as for our webbies it became a pleasure to delve into our new mobile-page-code written in scaml. Structure and intention really pops out.

As mentioned above, there is no Scalate plugin for 2.0. After some research I found this 1.x example to integrate Scalate in Play. It’s more or less ready to go. You have to change the package for play.Play to play.api.Play and change the call in the controller to something like

Ok( Scalate("first.scaml").render('posts -> Post.findAll())).as(HTML)

But this sucks right? I don’t want to add as(HTML) when I trigger the rendering. But without the as(HTML) the content type is set to text and thus not rendered by the browser. To change this we have to define a Writable – that encodes the content – and a ContentType. Both are defined as implicits and need a type on which they work. I’ve defined a simple ScalateContent that wraps the rendered output and enables the correct application of the implicits.

Scalate format

I have changed the original code so that the format used can be defined in the application.config via scalate.format. Right now this is only important for the default layout, as I explicitly state the format of the template to render in the controller as you can see above. I personally prefer it this way as it seems more explicit what exactly is meant. You can easily extend the apply method if you prefer calls like Scalate(“first”).render.

Default layout

Scalate uses default templates for stuff that layouts all pages. To enable this I added the default layout strategy. The default template can now be used as documented. In the Play project the path is app/views/layouts/default.{format}.

So after all this the code for the Scalate object looked like this:

package controllers

import play.api._
import http.{Writeable, ContentTypeOf, ContentTypes}
import mvc.Codec
import play.api.Play.current
import org.fusesource.scalate.layout.DefaultLayoutStrategy

object Scalate {

  import org.fusesource.scalate._
  import org.fusesource.scalate.util._

  var format = Play.configuration.getString("scalate.format") match {
    case Some(configuredFormat) => configuredFormat
    case _ => "test"
  }

  lazy val scalateEngine = {
    val engine = new TemplateEngine
    engine.resourceLoader = new FileResourceLoader(Some(Play.getFile("/app/views")))
    engine.layoutStrategy = new DefaultLayoutStrategy(engine, Play.getFile("/app/views/layouts/default." + format).getAbsolutePath)
    engine.classpath = Play.getFile("/tmp/classes").getAbsolutePath
    engine.workingDirectory = Play.getFile("tmp")
    engine.combinedClassPath = true
    engine.classLoader = Play.classloader
    engine
  }

  def apply(template: String) = Template(template)

  case class Template(name: String) {

    def render(args: (Symbol, Any)*) = {
      ScalateContent{
        scalateEngine.layout(name, args.map {
          case (k, v) => k.name -> v
        } toMap)
      }
    }

  }

  case class ScalateContent(val cont: String)

  implicit def writeableOf_ScalateContent(implicit codec: Codec): Writeable[ScalateContent] = {
    Writeable[ScalateContent](scalate => codec.encode(scalate.cont))
  }

  implicit def contentTypeOf_ScalateContent(implicit codec: Codec): ContentTypeOf[ScalateContent] = {
    ContentTypeOf[ScalateContent](Some(ContentTypes.HTML))
  }

}

Next time

From here on you should be able to use Scalate in Play 2.0, making your web development more fun and of course more efficient. Next time I’ll show how to set up the pages with Scaml, adding some sexieness with Less and some client side logic with CoffeeScript.