Difference between revisions of "IO inside"

From HaskellWiki
Jump to: navigation, search
Line 809: Line 809:
   
   
=== Example: emulating OOP with record type ===
+
=== Example: emulating OOP with record types ===
   
 
Let's implement the classical OOP example: drawing figures. There are
 
Let's implement the classical OOP example: drawing figures. There are
 
figures of different types: circles, rectangles and so on. The task is
 
figures of different types: circles, rectangles and so on. The task is
to create heterogeneous figures list. All figures in this list should
+
to create a heterogeneous list of figures. All figures in this list should
 
support the same set of operations: draw, move and so on. We will
 
support the same set of operations: draw, move and so on. We will
represent these operations as IO procedures. Instead of class let's
+
represent these operations as IO procedures. Instead of a "class" let's
 
define a structure containing implementations of all the procedures
 
define a structure containing implementations of all the procedures
 
required:
 
required:
Line 828: Line 828:
   
   
Constructor of each figures' type should just return a Figure record:
+
The constructor of each figure's type should just return a Figure record:
   
 
<haskell>
 
<haskell>
Line 840: Line 840:
   
 
We will "draw" figures by just printing their current parameters.
 
We will "draw" figures by just printing their current parameters.
Let's start with simplified implementation of 'circle' and 'rectangle'
+
Let's start with a simplified implementation of the 'circle' and 'rectangle'
 
constructors, without actual 'move' support:
 
constructors, without actual 'move' support:
   
Line 854: Line 854:
   
   
As you see, they just returns fixed 'draw' procedure that prints
+
As you see, each constructor just returns a fixed 'draw' procedure that prints
parameters with which this concrete figure was created. Let's go to
+
parameters with which the concrete figure was created. Let's test it:
test it:
 
   
 
<haskell>
 
<haskell>
Line 870: Line 870:
   
   
Now let's go to define "full-featured" figures that can be really
+
Now let's define "full-featured" figures that can actually be
 
moved around. In order to achieve this, we should provide each figure
 
moved around. In order to achieve this, we should provide each figure
with mutable variables what holds it's current screen location. Their
+
with mutable variables that holds the figure's current screen location. Their
type will be "IORef Point". These variables should be created in figure
+
types will be "IORef Point". These variables should be created in the figure
 
constructor and manipulated in IO procedures (closures) enclosed in
 
constructor and manipulated in IO procedures (closures) enclosed in
Figure record:
+
the Figure record:
   
 
<haskell>
 
<haskell>
Line 919: Line 919:
   
   
Let's pay attention that we are not limited to include only IO actions
+
It's important to realize that we are not limited to including only IO actions
in the record that simulates C++/Java interface. The record can include
+
in a record that simulates a C++/Java-style interface. The record can include
 
values, IORefs, pure functions - in short, any type of data. For
 
values, IORefs, pure functions - in short, any type of data. For
 
example, we can easily add to Figure interface fields for area and
 
example, we can easily add to Figure interface fields for area and

Revision as of 03:08, 4 July 2006

Haskell I/O has always been a source of confusion and surprises for new Haskellers. While simple I/O code in Haskell looks very similar to its equivalents in imperative languages, attempts to write somewhat more complex code often result in a total mess. This is because Haskell I/O is really very different internally. Haskell is a pure language and even the I/O system can't break this purity.

The following text is an attempt to explain the details of Haskell I/O implementations. This explanation should help you eventually master all the smart I/O tricks. Moreover, I've added a detailed explanation of various traps you might encounter along the way. After reading this text, you will receive a "Master of Haskell I/O" degree that is equal to a Bachelor in Computer Science and Mathematics, simultaneously :)

If you are new to Haskell I/O you may prefer to start by reading the Introduction to IO page.


Haskell is a pure language

Haskell is a pure language, which means that the result of any function call is fully determined by its arguments. Pseudo-functions like rand() or getchar() in C which return different results on each call, are simply impossible to write in Haskell. Moreover, Haskell functions can't have side effects, which means that they can't effect any changes to the "real world", like changing files, writing to the screen, printing, sending data over the network, and so on. These two restrictions together mean that any function call can be omitted, repeated, or replaced by the result of a previous call with the same parameters, and the language guarantees that all these rearrangements will not change the program result!

Let's compare this to C: optimizing C compilers try to guess which functions have no side effects and don't depend on global mutable variables. If this guess is wrong, an optimization can change the program's semantics! To avoid this kind of disaster, C optimizers are conservative in their guesses or require hints from the programmer about the purity of functions.

Compared to an optimizing C compiler, a Haskell compiler is a set of pure mathematical transformations. This results in much better high-level optimization facilities. Moreover, pure mathematical computations can be much more easily divided into several threads that may be executed in parallel, which is increasingly important in these days of multi-core CPUs. Finally, pure computations are less error-prone and easier to verify, which adds to Haskell's robustness and to the speed of program development using Haskell.

This purity creates other problems: how we can do I/O or work with stateful algorithms and side effects in a pure language? This question has had many different solutions proposed in 18 years of Haskell development, though a solution based on monads is now the standard.


What is a monad?

What is a monad? It's something from mathematical category theory, which I don't know anymore :) In order to understand how monads are used to solve the problem of I/O and side effects, you don't need to know it. It's enough to just know elementary mathematics, like I do :)

Let's imagine that we want to implement in Haskell the well-known 'getchar' function. What type should it have? Let's try:

getchar :: Char

get2chars = [getchar,getchar]

What will we get with 'getchar' having just the 'Char' type? You can see all the possible problems in the definition of 'get2chars':

  1. because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "excessive" calls to 'getchar' and use one returned value two times
  2. even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two chars in the order in which they were read, or in the opposite order? Nothing in the definition of 'get2chars' answers this question.

How can these problems be solved, from the programmer's viewpoint? Let's introduce a fake parameter of 'getchar' to make each call "different" from the compiler's point of view:

getchar :: Int -> Char

get2chars = [getchar 1, getchar 2]

Right away, this solves the first problem mentioned above - now the compiler will make two calls because it sees them as having different parameters. The whole 'get2chars' function should also have a fake parameter, otherwise we will have the same problem calling it:

getchar   :: Int -> Char
get2chars :: Int -> String

get2chars _ = [getchar 1, getchar 2]


Now we need to give the compiler some clue to determine which function it should call first. The Haskell language doesn't provide any way to express order of evaluation... except for data dependencies! How about adding an artificial data dependency which prevents evaluation of the second 'getchar' before the first one? In order to achieve this, we will return from 'getchar' an additional fake result that will be used as a parameter for the next 'getchar' call:

getchar :: Int -> (Char, Int)

get2chars _ = [a,b]  where (a,i) = getchar 1
                           (b,_) = getchar i

So far so good - now we can guarantee that 'a' is read before 'b' because reading 'b' needs the value ('i') that is returned by reading 'a'!

We've added a fake parameter to 'get2chars' but the problem is that the Haskell compiler is too smart! It can believe that the external 'getchar' function is really dependent on its parameter but for 'get2chars' it will see that we're just cheating and throw it away! How can we fix this? How about passing this fake parameter to the 'getchar' function?! In this case the compiler can't guess that it is really unused :)

get2chars i0 = [a,b]  where (a,i1) = getchar i0
                            (b,i2) = getchar i1


And more - 'get2chars' has all the same purity problems as the 'getchar' function. If you need to call it two times, you need a way to describe the order of these calls. Look at:

get4chars = [get2chars 1, get2chars 2]  -- order of 'get2chars' calls isn't defined

We already know how to deal with these problems - 'get2chars' should also return some fake value that can be used to order calls:

get2chars :: Int -> (String, Int)

get4chars i0 = (a++b)  where (a,i1) = get2chars i0
                             (b,i2) = get2chars i1


But what's the fake value 'get2chars' should return? If we use some integer constant, the too-smart Haskell compiler will guess that we're cheating again :) What about returning the value returned by 'getchar'? See:

get2chars :: Int -> (String, Int)
get2chars i0 = ([a,b], i2)  where (a,i1) = getchar i0
                                  (b,i2) = getchar i1

Believe it or not, but we've just constructed the whole "monadic" Haskell I/O system.


Welcome to the RealWorld, baby :)

The 'main' Haskell function has the type:

main :: RealWorld -> ((), RealWorld)

where 'RealWorld' is a fake type used instead of our Int. It's something like the baton passed in a relay race. When 'main' calls some IO function, it passes the "RealWorld" it received as a parameter. All IO functions have similar types involving RealWorld as a parameter and result. To be exact, "IO" is a type synonym defined in the following way:

type IO a  =  RealWorld -> (a, RealWorld)

So, 'main' just has type "IO ()", 'getChar' has type "IO Char" and so on. Let's look at 'main' calling 'getChar' two times:

getChar :: RealWorld -> (Char, RealWorld)

main :: RealWorld -> ((), RealWorld)
main world0 = let (a, world1) = getChar world0
                  (b, world2) = getChar world1
              in ((), world2)


Look at this closely: 'main' passes to first 'getChar' the "world" it received. This 'getChar' returns some new value of type RealWorld, that is used in the next call. Finally, 'main' returns the "world" it got from the second 'getChar'.

  1. Is it possible here to omit any call of 'getChar' if the Char it read is not used? No, because we need to return the "world" that is the result of the second 'getChar' and this in turn requires the "world" returned from the first 'getChar'.
  2. Is it possible to reorder the 'getChar' calls? No: the second 'getChar' can't be called before the first one because it uses the "world" returned from the first call.
  3. Is it possible to duplicate calls? In Haskell semantics - yes, but real compilers never duplicate work in such simple cases (otherwise, the programs generated will not have any speed guarantees).


As we already said, RealWorld values are used like a baton which gets passed between all routines called by 'main' in strict order. Inside each routine called, RealWorld values are used in the same way. Overall, in order to "compute" the world to be returned from 'main', we should perform each IO procedure that is called from 'main', directly or indirectly. This means that each procedure inserted in the chain will be performed just at the moment (relative to the other IO actions) when we intended it to be called. Let's consider the following program:

main = do a <- ask "What is your name?"
          b <- ask "How old are you?"
          return ()

ask s = do putStr s
           readLn

Now you have enough knowledge to rewrite it in a low-level way and check that each operation that should be performed will really be performed with the arguments it should have and in the order we expect.


But what about conditional execution? No problem. Let's define the well-known 'when' operation:

when :: Bool -> IO () -> IO ()
when condition action world =
    if condition
      then action world
      else ((), world)

As you can see, we can easily include or exclude from the execution chain IO procedures (actions) depending on the data values. If 'condition' will be False on the call of 'when', 'action' will never be called because real Haskell compilers, again, never call functions whose results are not required to calculate the final result (i.e., here, the final "world" value of 'main').

Loops and more complex control structures can be implemented in the same way. Try it as an exercise!


Finally you may want to know how much this passing of RealWorld values around the program costs. It's free! These fake values exist solely for the compiler while it analyzes and optimizes the code, but when it gets to assembly code generation, it "suddenly" realize that this type is like "()", so all these parameters and result values can be omitted from the final generated code. Isn't it beautiful? :)


'>>=' and 'do' notation

All beginners (including me :)) start by thinking that 'do' is some magic statement that executes IO actions. That's wrong - 'do' is just syntactic sugar that simplifies the writing of procedures that use IO. 'do' notation eventually gets translated to statements passing "world" values around like we've manually written above and is used to simplify the gluing of several IO actions together. You don't need to use 'do' for just one statement:

  main = do putStr "Hello!"

is desugared to:

  main = putStr "Hello!"

But nevertheless it's considered Good Style to use 'do' even for one statement because it simplifies adding new statements in the future.


Let's examine how to desugar a 'do' with multiple statements in the following example:

main = do putStr "What is your name?"
          putStr "How old are you?"
          putStr "Nice day!"

The 'do' statement here just joins several IO actions that should be performed sequentially. It's translated to sequential applications of the so-called "binding operator", namely '>>':

main = (putStr "What is your name?")
       >> ( (putStr "How old are you?")
            >> (putStr "Nice day!")
          )

This binding operator just combines two IO actions, executing them sequentially by passing the "world" between them:

(>>) :: IO a -> IO b -> IO b
(action1 >> action2) world0 =
   let (a, world1) = action1 world0
       (b, world2) = action2 world1
   in (b, world2)

If defining operators this way looks strange to you, read this definition as follows:

action1 >> action2 = action
  where
    action world0 = let (a, world1) = action1 world0
                        (b, world2) = action2 world1
                    in (b, world2)

Now you can substitute the definition of '>>' at the places of its usage and check that program constructed by the 'do' desugaring is actually the same as we could write by manually manipulating "world" values.


A more complex example involves the binding of variables using "<-":

main = do a <- readLn
          print a

This code is desugared into:

main = readLn
       >>= (\a -> print a)

As you should remember, the '>>' binding operator silently ignores the value of its first action and returns as an overall result just the result of its second action. On the other hand, '>>=' allows us to use the value of its first action - it's passed as an additional parameter to the second one! Look at the definition:

(>>=) :: IO a -> (a->IO b) -> IO b
(action1 >>= action2) world0 =
   let (a, world1) = action1 world0
       (b, world2) = action2 a world1
   in (b, world2)

First, what does the type of the second "action", namely "a->IO b", mean? By substituting the "IO" definition, we get "a -> RealWorld -> (b, RealWorld)". This means that second action actually has two parameters - the type 'a' actually used inside it, and the value of type RealWorld used for sequencing of IO actions. That's always the case - any IO procedure has one more parameter compared to what you see in its type signature. This parameter is hidden inside the definition of the type alias "IO".

Second, you can use these '>>' and '>>=' operations to simplify your program. For example, in the code above we don't need to introduce the variable, because the result of 'readLn' can be send directly to 'print':

main = readLn >>= print


And third - as you see, the notation:

 do x <- action1
    action2

where 'action1' has type "IO a" and 'action2' has type "IO b", translates into:

 action1 >>= (\x -> action2)

where the second argument of '>>=' has the type "a->IO b". It's the way the "<-" binding is processed - it just becomes a parameter of subsequent operations represented as one large IO action. Look at the next example:

main = do putStr "What is your name?"
          a <- readLn
          putStr "How old are you?"
          b <- readLn
          print (a,b)

This code is desugared into:

main = putStr "What is your name?"
       >> readLn
       >>= \a -> putStr "How old are you?"
       >> readLn
       >>= \b -> print (a,b)

I omitted parentheses here; both '>>' and '>>=' operations are left-associative, which means that the 'a' and 'b' bindings introduced here are valid for all remaining actions. As an exercise, add the parentheses yourself and translate this procedure into the low-level code that explicitly passes "world" values. I think it should be enough to help you finally realize how the 'do' translation and binding operators work.


Oh, no! I forgot the third monadic operator - 'return'. It just combines its two parameters - the value passed and "world":

return :: a -> IO a
return a world0  =  (a, world0)

How about translating a simple example of 'return' usage? Say,

main = do a <- readLn
          return (a*2)


Programmers with an imperative languages background often think that 'return' in Haskell, as in other languages, immediately returns from the IO procedure. As you can see in its definition (and even just from its type!), such an assumption is totally wrong. The only purpose of using 'return' is to "lift" some value (of type 'a') into the result of a whole action (of type "IO a") and therefore it should generally be used only as the last executed statement of some IO sequence. For example try to translate the following procedure into the corresponding low-level code:

main = do a <- readLn
          when (a>=0) $ do
              return ()
          print "a is negative"

and you will realize that the 'print' statement is executed anyway. If you need to escape from the middle of an IO procedure, you can use the 'if' statement:

main = do a <- readLn
          if (a>=0)
            then return ()
            else print "a is negative"

Moreover, Haskell layout rules allow us to use the following layout:

main = do a <- readLn
          if (a>=0) then return ()
            else do
          print "a is negative"
          ...

that may be useful for escaping from the middle of a longish 'do' statement.


Last exercise: implement a function 'liftM' that lifts operations on plain values to the operations on monadic ones. Its type signature:

liftM :: (a->b) -> (IO a -> IO b)

If it's too hard for you, start with the following high-level definition and rewrite it in low-level fashion:

liftM f action = do x <- action
                    return (f x)


Mutable data (references, arrays, hash tables...)

As you should know, all names in Haskell are bound to one fixed value. This greatly simplifies understanding of algorithms and optimization of code, but is inappropriate in some cases. As we all know, there are plenty of algorithms that are simpler to implement in terms of updatable variables, arrays and so on. This means that the value associated with a variable, for example, can be different at different execution points, so reading its value can't be considered as a pure function. Imagine, for example, the following code:

main = do let a0 = readVariable varA
              _  = writeVariable varA 1
              a1 = readVariable varA
          print (a0,a1)

Does this look strange? First, the two calls to 'readVariable' look the same, so the compiler can just reuse the value returned by the first call. Second, the result of the 'writeVariable' call isn't used so the compiler can (and will!) omit this call completely. To finish the picture, these three calls may be rearranged to any order because they appear to be independent of each other. What is the solution? You already know this - use IO actions! IO actions guarantee us that:

  1. execution order will be retained
  2. each action will have to be executed
  3. the result of the "same" action (such as "readVariable varA") will not be reused

So, the code above really should be written as:

main = do varA <- newIORef 0  -- Create and initialize a new variable
          a0 <- readIORef varA
          writeIORef varA 1
          a1 <- readIORef varA
          print (a0,a1)

Here, 'varA' has the type "IORef Int" which means "a variable (reference) in the IO monad holding a value of type Int". newIORef creates a new variable (reference) and returns it, and then read/write actions use this reference. The value returned by the "readIORef varA" action may depend not only on the variable involved but also on the moment of performing this operation so it can return different values on each call.

Arrays, hash tables and any other _mutable_ data structures are defined in the same way - for each of them, there's an operation that creates new "mutable values" and returns a reference to it. Then special read and write operations in the IO monad are used. The following code shows an example of using mutable arrays:

 import Data.Array.IO
 main = do arr <- newArray (1,10) 37 :: IO (IOArray Int Int)
           a <- readArray arr 1
           writeArray arr 1 64
           b <- readArray arr 1
           print (a,b)

Here, an array of 10 elements with 37 as the initial value at each location is created. After reading the value of the first element (index 1) to 'a' this element's value is changed to 64 and then read again to 'b'. As you can see by executing this code, 'a' will be set to 37 and 'b' to 64.


Other state-dependent operations are also often implemented as IO actions. For example, a random number generator should return a different value on each call. It looks natural to give it a type involving IO:

rand :: IO Int

Moreover, when you import C routines you should be careful - if this routine is impure, i.e. its result depends on something in the "real world" (file system, memory contents...), internal state and so on, you should give it an IO type. Otherwise, the compiler can "optimize" repetitive calls of this procedure with the same parameters! :)

For example, we can write a non-IO type for:

foreign import ccall
   sin :: Double -> Double

because the result of 'sin' depends only on its argument, but

foreign import ccall
   tell :: Int -> IO Int

If you will declare 'tell' as a pure function (without IO) then you may get the same position on each call! :)

IO actions as values

By this point you should understand why it's impossible to use IO actions inside non-IO (pure) procedures. Such procedures just don't get a "baton"; they don't know any "world" value to pass to an IO action. The RealWorld type is an abstract datatype, so pure functions also can't construct RealWorld values by themselves, and it's a strict type, so 'undefined' also can't be used. So, the prohibition of using IO actions inside pure procedures is just a type system trick as it usually is in Haskell :)

But while pure code can't _execute_ IO actions, it can work with them as with any other functional values - they can be stored in data structures, passed as parameters and returned as results, collected in lists, and partially applied. But an IO action will remain a functional value because we can't apply it to the last argument - of type RealWorld.

In order to _execute_ the IO action we need to apply it to some RealWorld value that can be done only inside some IO procedure, in its "actions chain". And real execution of this action will take place only when this procedure is called as part of the process of "calculating the final value of world" for 'main'. Look at this example:

main world0 = let get2chars = getChar >> getChar
                  ((), world1) = putStr "Press two keys" world0
                  (answer, world2) = get2chars world1
              in ((), world2)

Here we first bind a value to 'get2chars' and then write a binding involving 'putStr'. But what's the execution order? It's not defined by the order of writing bindings, it's defined by the order of processing "world" values! You can arbitrarily reorder the binding statements - in any case execution order will be defined by dependence on passing "world" values. Let's see how this 'main' looks in the 'do' notation:

main = do let get2chars = getChar >> getChar
          putStr "Press two keys"
          get2chars
          return ()

As you can see, the 'let' binding that isn't included in the IO chain, is translated just to the 'let' statement inside the 'do' sequence. And as you now should understand, placement of this 'let' doesn't have any impact on the evaluation order, which is defined by the order of passing "world" values that is, in turn, defined by the order of the ordinal (non-let) statements inside 'do'!

Moreover, IO actions like 'get2chars' can't be executed directly because they are functions with a RealWorld parameter. To execute them, we need to supply the RealWorld parameter, i.e. insert them in the 'main' chain, placing them in some 'do' sequence executed from 'main'. Until that is done, they will remain like any function, in partially evaluated form. And we can work with IO actions as with any other functions - bind them to names (like above), save them to data structures, pass them as function parameters and return them as results - and they will not be performed until you give them this magic RealWorld parameter!


Example: a list of IO actions

Let's try defining a list of IO actions:

ioActions :: [IO ()]
ioActions = [(print "Hello!"),
             (putStr "just kidding"),
             (getChar >> return ())
            ]

I used additional parentheses around each action, although they are not really required. If you still can't believe that these actions will not be executed immediately, just recall the real type of this list:

ioActions :: [RealWorld -> ((), RealWorld)]

Well, now we want to execute some of these actions. No problem, just insert them into the 'main' chain:

main = do head ioActions
          ioActions !! 1
          last ioActions

Looks strange, right? :) Really, any IO action that you write in the 'do' statement (or use as a parameter for the '>>'/'>>=' operators) is an expression returning a result of type 'IO a' for some type 'a'. Typically, you use some function that has the type 'x -> y -> ... -> IO a' and provide all the x, y, etc. parameters. But you're not limited to this standard scenario - don't forget that Haskell is a functional language and you're free to compute the functional value required (recall that 'IO a' is really a function type) in any possible way. Here we just extracted several functions from the list - no problem. This functional value can also be constructed on-the-fly, as we've done in the previous example - that's also OK. Want to see this functional value passed as the parameter? Just look at the 'when' definition. Hey, we can sell, buy and rent these IO actions just like we can with any other functional values! For example, let's define a function that executes all the IO actions in the list:

sequence_ :: [IO a] -> IO ()
sequence_ [] = return ()
sequence_ (x:xs) = do x
                      sequence_ xs

No black magic - we just extract IO actions from the list and insert them into a chain of IO operations that should be performed to "compute the final world value" of the entire 'sequence_' call.

With the help of 'sequence_', we can rewrite our last 'main' as:

main = sequence_ ioActions


Haskell's ability to work with IO actions as with any other (functional and non-functional) values allows us to define control structures of arbitrary complexity. Try, for example, to define a control structure that repeats an action until it returns the 'False' result:

while :: IO Bool -> IO ()
while action = ???


Example: returning an IO action as a result

How about returning an IO action as the function result? Well, we've done this each time we've defined an IO procedure - they all return IO actions that need a RealWorld value to be performed. While we usually just execute them as part of a higher-level IO procedure, it's also possible to just collect them without actual execution:

main = do let a = sequence ioActions
              b = when True getChar
              c = getChar >> getChar
          putStr "'let' statements are not executed!"

These assigned IO procedures can be used as parameters to other procedures, or written to global variables, or processed in some other way, or just executed later, as we did in the example with 'get2chars'.

But how about returning procedure a parameterized IO action from an IO procedure? Let's define a procedure that returns the i'th byte from a file represented as a Handle:

readi h i = do hSeek h i AbsoluteSeek
               hGetChar h

So far so good. But how about a procedure that returns the i'th byte of a file with a given name without reopening it each time?

readfilei :: String -> IO (Integer -> IO Char)
readfilei name = do h <- openFile name ReadMode
                    return (readi h)

As you can see, it's an IO procedure that opens a file and returns... another IO procedure that will read the specified byte. But we can go further and include the 'readi' body into 'readfilei':

readfilei name = do h <- openFile name ReadMode
                    let readi h i = do hSeek h i AbsoluteSeek
                                       hGetChar h
                    return (readi h)

Good? It may be better. But why do we add 'h' as a parameter to 'readi' if it can be obtained from the environment where 'readi' is now defined? A shorter version would be:

readfilei name = do h <- openFile name ReadMode
                    let readi i = do hSeek h i AbsoluteSeek
                                     hGetChar h
                    return readi

What have we done here? We've build a parameterized IO action involving local names inside 'readfilei' and returned it as the result. Now it can be used in the following way:

main = do myfile <- readfilei "test"
          a <- myfile 0
          b <- myfile 1
          print (a,b)


Such usage of IO actions is very typical for Haskell programs - you just construct one or more IO actions that you need, with or without parameters, possibly involving the parameters that your "constructor" received, and return them to the caller. Then these IO actions can be used in the rest of the program without any knowledge about your internal implementation strategy. Actually, this is used to partially emulate the OOP (or more precisely, the ADT) programming paradigm.


Example: a memory allocator generator

As an example, one of my program's modules is the memory suballocator. It receives the address and size of a large memory block and returns two procedures - one to allocate a subblock of a given size and the other to free the allocated subblock:

memoryAllocator :: Ptr a -> Int -> IO (Int -> IO (Ptr b),
                                       Ptr c -> IO ())

memoryAllocator buf size = do ......
                              let alloc size = do ...
                                                  ...
                                  free ptr = do ...
                                                ...
                              return (alloc, free)

How this is implemented? 'alloc' and 'free' work with references created inside this procedure. Because the creation of these references is a part of the 'memoryAllocator' IO actions chain, a new independent set of references will be created for each memory block for which 'memoryAllocator' is called:

memoryAllocator buf size = do start <- newIORef buf
                              end <- newIORef (buf `plusPtr` size)
                              ...

These two references (we will implement a very simple memory allocator) are read and written in the 'alloc' and 'free' definitions:

      let alloc size = do addr <- readIORef start
                          writeIORef start (addr `plusPtr` size)
                          return addr
                          
      let free ptr = do writeIORef start ptr

What we've defined here is just a pair of closures that use state available at the moment of their definition. As you can see, it's as easy as in any other functional language, despite Haskell's lack of direct support for impure functions.

The following example uses procedures, returned by memoryAllocator, to simultaneously allocate/free blocks in two independent memory buffers:

main = do buf1 <- mallocBytes (2^16)
          buf2 <- mallocBytes (2^20)
          (alloc1, free1) <- memoryAllocator buf1 (2^16)
          (alloc2, free2) <- memoryAllocator buf2 (2^20)
          ptr11 <- alloc1 100
          ptr21 <- alloc2 1000
          free1 ptr11
          free2 ptr21
          ptr12 <- alloc1 100
          ptr22 <- alloc2 1000


Example: emulating OOP with record types

Let's implement the classical OOP example: drawing figures. There are figures of different types: circles, rectangles and so on. The task is to create a heterogeneous list of figures. All figures in this list should support the same set of operations: draw, move and so on. We will represent these operations as IO procedures. Instead of a "class" let's define a structure containing implementations of all the procedures required:

data Figure = Figure { draw :: IO (),
                       move :: Displacement -> IO ()
                     }

type Displacement = (Int, Int)  -- horizontal and vertical displacement in points


The constructor of each figure's type should just return a Figure record:

circle    :: Point -> Radius -> IO Figure
rectangle :: Point -> Point -> IO Figure

type Point = (Int, Int)  -- point coordinates
type Radius = Int        -- circle radius in points


We will "draw" figures by just printing their current parameters. Let's start with a simplified implementation of the 'circle' and 'rectangle' constructors, without actual 'move' support:

circle center radius = do
    let description = "  Circle at "++show center++" with radius "++show radius
    return $ Figure { draw = putStrLn description}

rectangle from to = do
    let description = "  Rectangle "++show from++"-"++show to)
    return $ Figure { draw = putStrLn description}


As you see, each constructor just returns a fixed 'draw' procedure that prints parameters with which the concrete figure was created. Let's test it:

drawAll :: [Figure] -> IO ()
drawAll figures = do putStrLn "Drawing figures:"
                     mapM_ draw figures

main = do figures <- sequence [circle (10,10) 5,
                               circle (20,20) 3,
                               rectangle (10,10) (20,20),
                               rectangle (15,15) (40,40)]
          drawAll figures


Now let's define "full-featured" figures that can actually be moved around. In order to achieve this, we should provide each figure with mutable variables that holds the figure's current screen location. Their types will be "IORef Point". These variables should be created in the figure constructor and manipulated in IO procedures (closures) enclosed in the Figure record:

circle center radius = do
    centerVar <- newIORef center
    
    let drawF = do center <- readIORef centerVar
                   putStrLn ("  Circle at "++show center
                             ++" with radius "++show radius)
                   
    let moveF (addX,addY) = do (x,y) <- readIORef centerVar
                               writeIORef centerVar (x+addX, y+addY)
                               
    return $ Figure { draw=drawF, move=moveF }

    
rectangle from to = do
    fromVar <- newIORef from
    toVar   <- newIORef to

    let drawF = do from <- readIORef fromVar
                   to   <- readIORef toVar
                   putStrLn ("  Rectangle "++show from++"-"++show to)
                   
    let moveF (addX,addY) = do (fromX,fromY) <- readIORef fromVar
                               (toX,toY)     <- readIORef toVar
                               writeIORef fromVar (fromX+addX, fromY+addY)
                               writeIORef toVar   (toX+addX, toY+addY)

    return $ Figure { draw=drawF, move=moveF }


Now we can add moving figures around to our test:

main = do figures <- sequence [circle (10,10) 5,
                               rectangle (10,10) (20,20)]
          drawAll figures
          mapM_ (\fig -> move fig (10,10)) figures
          drawAll figures


It's important to realize that we are not limited to including only IO actions in a record that simulates a C++/Java-style interface. The record can include values, IORefs, pure functions - in short, any type of data. For example, we can easily add to Figure interface fields for area and origin:

data Figure = Figure { draw :: IO (),
                       move :: Displacement -> IO (),
                       area :: Double,
                       origin :: IORef Point
                     }


unsafePerformIO and unsafeInterleaveIO

Programmers with imperative background often still looks for a ways to execute IO actions inside the pure procedures. But that this means? Imagine that you try to write procedure that reads contents of file with given name:

readContents :: Filename -> String

Defining it as pure function will simplify the code that use it, i agree. But this creates troubles for the compiler:

  1. This call is not inserted in sequence of "world transformations", so compiler don't get a hint - at what exact moment you want to execute this action. For example, if file contents is one at the program start and another at the end - what contents you want to see? Moment of "consumption" of this value don't make strong guarantees for execution order, because Haskell see all the functions as pure and fell free to reorder their execution as needed.
  2. Attempts to read contents of file with the same name can be factorized despite the fact that file (or current directory) can be changed between calls. Again, Haskell looks at all the functions as pure ones and feel free to omit excessive calls with the same parameters.

So, implementing functions that interacts with Real World as pure ones considered as a Bad Behavior. Good boys never do it ;)


Nevertheless, there are (semi-official) ways to use IO actions inside of pure functions. As you should remember this is prohibited by requiring "baton" to call IO action. Pure function don't have the baton, but there is special procedure, that procures this baton from nowhere, uses it to call IO action and then throws resulting "world" away! A little low-level magic :) This very special procedure is:

unsafePerformIO :: IO a -> a

Let's look at it's (possible) definition:

unsafePerformIO :: (RealWorld -> (a,RealWorld)) -> a
unsafePerformIO action = let (a,world1) = action createNewWorld
                         in a

where 'createNewWorld' is internal function producing new value of RealWorld type.

Using unsafePerformIO, you can easily write pure functions that does I/O inside. But don't do this without real need, and remember to follow this rule: compiler don't know that you are cheating, it still consider each non-IO function as pure one. Therefore, all the usual optimization rules can (and will!) be applied to it's execution. So you must ensure that:

  1. Result of each call depends only on it's arguments
  2. You don't rely on side-effects of this function, which may be not executed if it's results are not used


Let's investigate this problem deeper. Function evaluation in Haskell are ruled by value's necessity - computed only the values that really required to calculate final result. But that this means according to 'main' function? To "calculate final world's" value, it's required to perform all the intermediate IO actions that included in 'main' chain. By using 'unsafePerformIO' we call IO actions outside of this chain. What can guarantee that they will be run? Nothing. The only case when they will be run is if that is required to compute overall function result (that in turn should be required to perform some action in 'main' chain). Here we return to the Haskell-natural evaluation-on-value-need. Now you should clearly see the difference:

- IO action inside IO procedure guaranteed to execute as long as it is inside 'main' chain - even when it's result is not used. You directly specify order of action's execution inside IO procedure. Data dependencies are simulated via "world" values.

- IO action inside 'unsafePerformIO' will be performed only if result of this operation is really used. Evaluation order is not guaranteed and you should not rely on it (except when you sure about data dependency).


I should also say that inside 'unsafePerformIO' call you can organize small internal chain of IO actions with help of the same binding operators and/or 'do' sugar:

one = unsafePerformIO $ do var <- newIORef 0
                           writeIORef var 1
                           readIORef var

and in this case ALL the operations in this chain will be performed as long as 'unsafePerformIO' result will be demanded. To ensure this, the actual 'unsafePerformIO' implementation evaluates "world" returned by the 'action':

unsafePerformIO action = let (a,world1) = action createNewWorld
                         in (world1 `seq` a)

('seq' operation strictly evaluates it's first argument before returning the value of second one)


But there is even more strange operation - 'unsafeInterleaveIO' that gets "official baton", makes it's piratical copy, and then run's "illegal" relay-race in parallel with main one! I can't further say about it's behavior without grief and indignation, it's not surprise that this operation is widely used in such software-piratical countries as Russia and China! ;) Don't even ask me - i will say nothing about this dirty trick i using permanently ;)


fixIO and 'mdo'

ST monad

Q monad

Welcome to the machine: the actual GHC implementation

A little disclaimer: after all, I should say that I don't describe here what a monad is (I don't even know it myself) and my explanation shows only one _possible_ way to implement them in Haskell. For example, the hbc Haskell compiler implements monads via continuations. I also haven't said anything about exception handling, which is a natural part of the "monad" concept. You can read the "All About Monads" guide to learn more about these topics.

But there is good news: first, the monad understanding you've acquired will work with any implementation. You just can't work with RealWorld values directly.

Second, the IO monad implementation described here is really used in the GHC, yhc/nhc (Hugs/jhc, too?) compilers. Here is the actual IO definition from the GHC sources:

newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #))

It uses the "State# RealWorld" type instead of our RealWorld, it uses the "(# #)" strict tuple for optimization, and it adds an IO data constructor around the type. Nevertheless, there are no significant changes from the standpoint of our explanation. Knowing the principle of "chaining" IO actions via fake "state of world" values, you can now easily understand and write low-level implementations of GHC I/O operations.


The Yhc/nhc98 implementation

data World = World
newtype IO a = IO (World -> Either IOError a)

This implementation makes the "World" disappear somewhat, and returns Either a result 'a', or if an error occurs then 'IOError'. The lack of the World on the right hand side of the function can only be done because the compiler knows special things about the IO type, and will not overoptimise it.


Further reading

This tutorial is largely based on the Simon Peyton Jones' paper Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell. I hope that my tutorial improves his original explanation of the Haskell I/O system and brings it closer to the point of view of beginning Haskell programmers. But if you need to learn about concurrency, exceptions and FFI in Haskell/GHC, the original paper is the best source of information.

You can find more information about concurrency, FFI, STM and exceptions at the GHC/Concurrency#Starting points page.

The Arrays page contains exhaustive explanations about using mutable arrays.

Look also at the Books and tutorials#Using Monads page, which contains tutorials and papers really describing these mysterious monads :)

Do you have more questions? Ask in the haskell-cafe mailing list.