Web/Literature/Practical web programming in Haskell

This page is under construction. Feel free to help out. If you make substantial edits, please add your name to the authors list at the bottom of this page, so that you can be credited if this is ever published in another medium.

Introduction

This tutorial aims to get you started with writing web applications in Haskell. We describe a relatively light-weight approach to Haskell web programming which uses a CGI library and an XHTML combinator library.

We think that while the approach we describe here is not as sophisticated or innovative as some other approaches, it is simple, portable and easy to understand if you are already familiar with web programming in other languages.

The tutorial starts with preliminaries such as how to install the necessary software and how to compile and run your web applications. We then show a number of working small example programs which introduce the basic features of the CGI and XHtml libraries. We then move on to how to use monad transformers to add application specific functionality such as sessions to the CGI monad, and how to create database-driven web applications. We also present FastCGI, and an approach to using dynamically loaded Haskell code.

Other approaches

[Web Authoring System Haskell (WASH) [1]. Domain-specific embedded language. Type-safe forms handling. Threads continuation through client. This gives good back-button and session splitting properties.

Haskell Application Server (HAppS) [2]. Complete system including web server in one program. Uses XSLT for output.

Haskell Server Pages (HSP) [3]. Uses preprocessor to make XML tags into Haskell expressions. Dynamic compilation.

Assumed knowledge

This tutorial is not meant as an introduction to Haskell or web programming. We will assume that you have some familiarity with the following concepts:

Haskell

This tutorial is not meant as a first introduction to Haskell. If you want to learn about Haskell in general, have a look at the lists of books and tutorials. You may want to start with Haskell in 5 steps.

(X)HTML

HTML (HyperText Markup Language) is the "the lingua franca for publishing hypertext on the World Wide Web. The XHtml library which we use in this tutorial produces XHTML 1.0, which is HTML 4.0 formulated as XML.

The combinators in the XHtml library do not make much sense unless you understand at least some parts of HTML.

CGI

CGI (Common Gateway Interface) programs are programs which run on the web server. They are given input which comes from the user's browser, and their output is given to the browser.

To really understand how the CGI library works, you probably need to know a thing or two about CGI. The authoritative resource on CGI is the CGI specification.

Required software

Haskell compiler

GHC, the Glasgow Haskell Compiler, is the Haskell implementation that we will use in this tutorial. However, any Haskell implementation that supports Haskell98 and multi-parameter type classes should work.

Libraries: xhtml and cgi

If your Haskell implementation does not come with the xhtml and cgi packages, download them from HackageDB.

Web server

You need to have access to a web server on which you can run CGI programs. The most convenient way to do this when learning and developing is to run a web server on your development machine. If you run the programs on some other machine you need to make sure that you compile your programs so that they can run on that machine. This normally means that the machines must to have the same architecture and run the same operating system.

Deploying statically linked applications

Linking your applications statically by giving the flags -static -optl-static to GHC will avoid problems with missing libraries on the web server.

For example, this simple program,

    import Database.SQLite
    main = print "hey, test this"

when compiled as $ ghc A.hs --make is dynamically linked against:

    $ ldd A
    A:
        Start            End              Type Open Ref GrpRef Name
        0000000000000000 0000000000000000 exe  1    0   0      A
        0000000041a85000 0000000041ee5000 rlib 0    1   0      /usr/local/lib/libsqlite3.so.9.0
        0000000049b04000 0000000049f1d000 rlib 0    1   0      /usr/lib/libm.so.2.3
        0000000042213000 000000004264f000 rlib 0    1   0      /usr/local/lib/libgmp.so.7.0
        0000000047d0e000 00000000481e0000 rlib 0    1   0      /usr/lib/libc.so.42.0
        0000000047900000 0000000047900000 rtld 0    1   0      /usr/libexec/ld.so

Now, we can just pass some linker flags through to statically link this lot,

    $ ghc A.hs --make -optl-static -no-recomp
    $ ldd A
    ldd: A: not a dynamic executable
    $ file A
    A: ELF 64-bit LSB executable, AMD64, version 1, for OpenBSD, statically linked, not stripped

You could also use the Haskell Web Server.

Caveats:

• The -static flag in GHC 6.8.2 does not link the libraries in the correct order, resulting in a link failure (which you can hack around if you have to by shuffling -lpthread after -lrt in the gargantuan linker invocation). This problem should disappear with GHC 6.8.3.
• Sometimes you will need to add extra-libraries fields to various libraries' .cabal files. This manifests as missing symbols. Note that many linkers are sensitive to the order of the -l arguments, so the order of libraries in this field matters.

Compiling and running web applications

Use GHC to produce a binary executable called prog.cgi from the Haskell source code file prog.hs:

ghc --make -package cgi -package xhtml -o prog.cgi prog.hs

Put the compiled program in the cgi-bin directory, or give it the extension .cgi, depending on the configuration of the web server.

Linking your applications statically by giving the flags -static -optl-static to GHC will avoid problems with missing libraries on the web server.

To run the compiled program, visit the URL of the CGI program with your web browser.

Simple examples

Hello World

Here is a very simple example which just outputs some static HTML. The type signatures in this code are optional. We show them here for clarity, but omit them in some later examples.

import Network.CGI
import Text.XHtml

page :: Html 
page = body << h1 << "Hello World!"

cgiMain :: CGI CGIResult
cgiMain = output $ renderHtml page

main :: IO ()
main = runCGI $ handleErrors cgiMain

The page function constructs an HTML document which consists of a body containing a single header element which contains the text "Hello World". The CGI-action cgiMain renders the HTML document as a string, and produces that string as output. The main function runs cgiMain, using the normal CGI protocol for input and output. It also uses handleErrors to output an error page in case |cgiMain| throws an exception.

Fans of one-liners may like this version better (handleErrors has been omitted since this simple program will not throw any exceptions):

import Text.XHtml
import Network.CGI

main = runCGI $ output $ renderHtml $ body << h1 << "Hello World!"

These are some of the important functions used in this example:

-- creates a string containing the HTML document.
renderHtml :: Html -> String

-- outputs a string as the body of the HTTP response.
output :: String -> CGI CGIResult

-- Catches any exception thrown by the given CGI action, returns an 
-- error page with a 500 Internal Server Error, showing the exception 
-- information, and logs the error.
handleErrors :: CGI CGIResult -> CGI CGIResult

-- Runs a CGI action which produces a CGIResult, using the CGI protocol
-- to get the inputs and send the outputs.
runCGI :: CGI CGIResult -> IO ()

HTML combinators

See also [4].

Html is the type of HTML fragments. It comes from the Text.XHtml module. There are functions for all XHTML 1.0 elements. Some examples:

• header, body
• h1, h2, ...
• thediv
• p
• image

The << operator is used for nesting HTML.

+++ concatenates HTML.

Attributes are added to tags using the ! operator.

The function renderHtml (FIXME: explain variants) produces a string containing the document.

Getting user input

This program shows a form which asks the user for her name. When the form is submitted, the program greets the user by name.

import Network.CGI
import Text.XHtml

inputForm = form << [paragraph << ("My name is " +++ textfield "name"),
                     submit "" "Submit"]

greet n = paragraph << ("Hello " ++ n ++ "!")

page t b = header << thetitle << t +++ body << b

cgiMain = do mn <- getInput "name"
             let x = maybe inputForm greet mn
             output $ renderHtml $ page "Input example" x

main = runCGI $ handleErrors cgiMain

-- Get the value of an input variable, for example from a form. 
-- If the variable has multiple values, the first one is returned.
getInput :: String -> CGI (Maybe String)

Cookies

import Network.CGI
import Text.XHtml

import Control.Monad (liftM)
import Data.Maybe (fromMaybe)

hello :: Int -> Html
hello 0 = h1 << "Welcome!"
          +++ p << "This is the first time I see you."
hello c = h1 << "Welcome back!"
          +++ p << ("I have seen you " ++ show c ++ " times before.")

page :: String -> Html -> Html
page t b = header << thetitle << t +++ body << b

cgiMain :: CGI CGIResult
cgiMain = do c <- liftM (fromMaybe 0) $ readCookie "mycookie"
             setCookie (newCookie "mycookie" (show (c+1)))
             output $ renderHtml $ page "Cookie example" $ hello c

main :: IO ()
main = runCGI $ handleErrors cgiMain

Here we use newCookie, setCookie and readCookie to store and retrieve a counter cookie in the browser. If you want to get the string value of a cookie, use getCookie instead of readCookie.

File uploads

FIXME: use a safer example

-- Accepts file uploads and saves the files in the given directory.
-- WARNING: this script is a SECURITY RISK and only for 
-- demo purposes. Do not put it on a public web server.

import Network.CGI
import Text.XHtml

import qualified Data.ByteString.Lazy as BS

import Control.Monad (liftM)
import Data.Maybe (fromJust)

uploadDir = "../upload"

fileForm = form ! [method "post", enctype "multipart/form-data"]
             << [afile "file", submit "" "Upload"]
saveFile n =
    do cont <- liftM fromJust $ getInputFPS "file"
       let f = uploadDir ++ "/" ++ basename n
       liftIO $ BS.writeFile f cont
       return $ paragraph << ("Saved as " +++ anchor ! [href f] << f +++ ".")

page t b = header << thetitle << t +++ body << b

basename = reverse . takeWhile (`notElem` "/\\") . reverse

cgiMain = 
    do mn <- getInputFilename "file"
       h <- maybe (return fileForm) saveFile mn
       output $ renderHtml $ page "Upload example" h

main = runCGI $ handleErrors cgiMain

We first output a file upload form, which should use the HTTP POST method, and the multipart/form-data content type. Here we seen an example of the use of HTML attributes, added with the ! operator.

For efficiency reasons, we use Data.ByteString.Lazy to represent the file contents. getInputFPS gets the value of an input variable as a lazy ByteString.

Error handling

handleErrors catches all exceptions and outputs a default error page with some information about the exception. You can write you own exception handler if you want to do something else when an exception is thrown. It can be useful to set the response code, e.g. 404.

Returning non-HTML

Of course we do not have to output HTML. Use setHeader to set the value of the Content-type header, and you can output whatever string you like.

In this example we return an image:

import Network.CGI
import System.IO
import qualified Data.ByteString.Lazy as B

main = do
	b <- img "./img/test.jpg" -- read the image
	runCGI $ handleErrors (cgiMain b)

cgiMain :: B.ByteString -> CGI CGIResult
cgiMain p = do
        -- we need to set the appropriate content-type
	setHeader "Content-type" "image/jpg"
	outputFPS p

img :: FilePath -> IO B.ByteString
img p = do 
	handle <- openFile p ReadMode
	B.hGetContents handle

Examples: RSS

Setting response headers

You can use the setHeader function to set arbitrary HTTP response headers. You can also set the response code, as seen above.

Example: output raw file data (with last-modified)

Going further

This section explores some of possibilities beyond the basic web application programming.

Extending the CGI monad with monad transformers

At this point, you should be able to create many useful CGI scripts. As your scripts get more ambitious, however, you may find yourself needing to pass "global" parameters to your CGI actions (e.g. database connections, session information.) Rather than explicitly passing these values around, you can extend the CGI monad to do this work for you.

The Network.CGI.Monad module defines a CGI monad transformer, allowing us to build a new monad that does everything the CGI monad does -- and more!

For example, let's define a new CGI monad that provides a database connection (in this example, we use the Database.HSQL.PostgreSQL module for our database.) Since it will be used by the CGI application, I'll call the new monad "App".

Should this not compile for you, you need to enable some extensions:

{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE FlexibleInstances #-}

After importing the appropriate modules, we define a new type, AppT that is made up two monad transformers, CGIT and ReaderT. The CGIT monad "wraps" the base monad "m". The CGIT monad, in turn, is wrapped by the ReaderT monad, which contains, in its environment, the database Connection.

AppT takes two type parameters. The first is the base monad that the monad transformers are modifying. Usually this will be the IO monad. The second type is the data type that an action in the monad will return.

import Control.Monad.Reader
import Network.CGI
import Network.CGI.Monad
import Database.HSQL.PostgreSQL

newtype AppT m a = App (ReaderT Connection (CGIT m) a)
   deriving (Monad, MonadIO, MonadReader Connection)

Like CGI, we make a type synonym that defines the most common use of this new monad.

type App a = AppT IO a

We're not quite finished defining App yet. In order to be used like the CGI monad, App needs to be an instance of the MonadCGI class. This class defines two functions that we must support.

instance MonadCGI (AppT IO) where
    cgiAddHeader n v = App $ lift $ cgiAddHeader n v
    cgiGet x = App $ lift $ cgiGet x

So now we have an App monad that gives us all the functionality of CGI, but also carries around a database connection. The last step is to define the function that creates the monad so we can run actions inside it.

import Control.Exception (bracket)
import System.IO (stdin, stdout)

runApp :: App CGIResult -> IO ()
runApp (App a) =
    bracket (connect "host" "db" "user" "password")
            disconnect
            (\c -> do { env <- getCGIVars
                      ; hRunCGI env stdin stdout (runCGIT (runReaderT a c))
                      ; return () } )

(either fill in your account/password information, or change runApp to accept the paramters as function arguments.) The function uses bracket so that the database connection gets released properly when the monad ends or if an exception is thrown.

Templating

There are times when you absolutely do not want to embed (X)HTML in Haskell. You can separate the code and the presentation (the Holy Grail of erm, web development). The code will be, well, Haskell, and the presentation will be buried inside templates. This might not be the case: fortunately, there's a very nice templating engine available, called HStringTemplate.

You can get an introduction to HStringTemplate over here.

FastCGI

FastCGI is a standard for CGI-like programs that are not restarted for every request. This reduces the overhead involved in handling each request, and reduces the servers response time for each request. The overhead involved in starting a new process for each request can also include the need to set up new DB connections every time. With FastCGI, DB connections can be reused.

Install FastCGI. Get a web server which can run FastCGI programs. Import Network.FastCGI. Use runFastCGI.

See also a tutorial by Paul R Brown: Wiring Haskell Into a FastCGI Web Server

Take a look at lightweight, minimalistic FastCGI-based web frameworks: HVAC (Haskell view and controller) and Kibro.

SCGI

SCGI is a simpler alternative to FastCGI for writing CGI-like programs in persistent processes, external to the web server. SCGI is less featureful than FastCGI, but has the advantage that it does not require an external library.

Install SCGI, import Network.SCGI, and use runSCGI. Everything else is then done inside a CGI monad as above.

URL rewriting

Dynamic loading

Database-driven web-applications

Database connectivity

See Takusen and HDBC. If you would like to write queries in Haskell (and not SQL), see also HaskellDB, which integrates with HDBC.

Persistent DB connections with FastCGI

FastCGI aren't restarted for each request, only the runFastCGI part is re-run. Everything (handles, datastructures etc.) you do outside of that loop will be persistent. However you need to handle errors yourself, because you're operating outside of handleErrors.

Web services

Web frameworks

Turbinado, an early stab at Ruby On Rails.

Existing applications

HackageDB web interface.

Real World HAppS: The Cabalized, Self-Demoing HAppS Tutorial.

Gitit, a wiki written using Git, HAppS and Pandoc.

Hope.

Authors: Björn Bringert Authors: Don Stewart Authors: Artyom Shalkhakov