Difference between revisions of "IO inside"

From HaskellWiki
Jump to: navigation, search
(Introduction to the ST monad)
m (use in-scope variable)
 
(39 intermediate revisions by 14 users not shown)
Line 1: Line 1:
Haskell I/O has always been a source of confusion and surprises for new Haskellers. While simple I/O code in Haskell looks very similar to its equivalents in imperative languages, attempts to write somewhat more complex code often result in a total mess. This is because Haskell I/O is really very different internally. Haskell is a pure language and even the I/O system can't break this purity.
+
Haskell I/O can be a source of confusion and surprises for new Haskellers - if that's you, a good place to start is the [[Introduction to IO]] which can help you learn the basics (e.g. the syntax of I/O expressions) before continuing on.
   
The following text is an attempt to explain the details of Haskell I/O implementations. This explanation should help you eventually master all the smart I/O tricks. Moreover, I've added a detailed explanation of various traps you might encounter along the way. After reading this text, you will receive a "Master of Haskell I/O" degree that is equal to a Bachelor in Computer Science and Mathematics, simultaneously :)
 
  +
----
   
If you are new to Haskell I/O you may prefer to start by reading the [[Introduction to IO]] page.
 
  +
While simple I/O code in Haskell looks very similar to its equivalents in imperative languages, attempts to write somewhat more complex code often result in a total
  +
mess. This is because Haskell I/O is really very different in how it actually works.
   
  +
The following text is an attempt to explain the details of Haskell I/O implementations. This explanation should help you eventually learn all the smart I/O tips.
  +
Moreover, I've added a detailed explanation of various traps you might encounter along the way. After reading this text, you will be well on your way towards mastering I/O in Haskell.
   
 
== Haskell is a pure language ==
 
== Haskell is a pure language ==
   
Haskell is a pure language, which means that the result of any function call is fully determined by its arguments. Pseudo-functions like rand() or getchar() in C, which return different results on each call, are simply impossible to write in Haskell. Moreover, Haskell functions can't have side effects, which means that they can't effect any changes to the "real world", like changing files, writing to the screen, printing, sending data over the network, and so on. These two restrictions together mean that any function call can be replaced by the result of a previous call with the same parameters, and the language '''guarantees''' that all these rearrangements will not change the program result!
+
Haskell is a pure language and even the I/O system can't break this purity. Being pure means that the result of any function call is fully determined by its arguments. Procedural entities like <code>rand()</code> or <code>getchar()</code> in C, which return different results on each call, are simply impossible to write in Haskell. Moreover, Haskell functions can't have side effects, which means that they can't effect any changes to the "real world", like changing files, writing to the screen, printing, sending data over the network, and so on. These two restrictions together mean that any function call can be replaced by the result of a previous call with the same parameters, and the language ''guarantees'' that all these rearrangements will not change the program result!
   
 
Let's compare this to C: optimizing C compilers try to guess which functions have no side effects and don't depend on mutable global variables. If this guess is wrong, an optimization can change the program's semantics! To avoid this kind of disaster, C optimizers are conservative in their guesses or require hints from the programmer about the purity of functions.
 
Let's compare this to C: optimizing C compilers try to guess which functions have no side effects and don't depend on mutable global variables. If this guess is wrong, an optimization can change the program's semantics! To avoid this kind of disaster, C optimizers are conservative in their guesses or require hints from the programmer about the purity of functions.
Line 14: Line 17:
 
Compared to an optimizing C compiler, a Haskell compiler is a set of pure mathematical transformations. This results in much better high-level optimization facilities. Moreover, pure mathematical computations can be much more easily divided into several threads that may be executed in parallel, which is increasingly important in these days of multi-core CPUs. Finally, pure computations are less error-prone and easier to verify, which adds to Haskell's robustness and to the speed of program development using Haskell.
 
Compared to an optimizing C compiler, a Haskell compiler is a set of pure mathematical transformations. This results in much better high-level optimization facilities. Moreover, pure mathematical computations can be much more easily divided into several threads that may be executed in parallel, which is increasingly important in these days of multi-core CPUs. Finally, pure computations are less error-prone and easier to verify, which adds to Haskell's robustness and to the speed of program development using Haskell.
   
Haskell purity allows compiler to call only functions whose results
 
  +
Haskell's purity allows the compiler to call only functions whose results are really required to calculate the final value of a top-level function (e.g. <code>main</code>) - this is called lazy evaluation. It's a great thing for pure mathematical computations, but how about I/O actions? A function like
are really required to calculate final value of high-level function
 
(i.e., main) - this is called lazy evaluation. It's great thing for
 
pure mathematical computations, but how about I/O actions? Function
 
like (<hask>putStrLn "Press any key to begin formatting"</hask>) can't return any
 
meaningful result value, so how can we ensure that compiler will not
 
omit or reorder its execution? And in general: how we can work with
 
stateful algorithms and side effects in an entirely lazy language?
 
This question has had many different solutions proposed in 18 years of
 
Haskell development (see [[History of Haskell]]), though a solution based on [[monad]]s is now
 
the standard.
 
   
== What is a monad? ==
 
  +
<haskell>
  +
putStrLn "Press any key to begin formatting"
  +
</haskell>
   
What is a [[monad]]? It's something from mathematical category theory, which I
 
  +
can't return any meaningful result value, so how can we ensure that the compiler will not omit or reorder its execution? And in general: How we can work with stateful algorithms and side effects in an entirely lazy language? This question has had many different solutions proposed while Haskell was developed (see [[History of Haskell]]), with one solution eventually making its way into the current standard.
don't know anymore :) In order to understand how monads are used to
 
solve the problem of I/O and side effects, you don't need to know it. It's
 
enough to just know elementary mathematics, like I do :)
 
   
Let's imagine that we want to implement in Haskell the well-known
 
  +
== I/O in Haskell, simplified ==
'getchar' function. What type should it have? Let's try:
 
  +
  +
Let's imagine that we want to implement the well-known <code>getchar</code> I/O operation in Haskell. What type should it have? Let's try:
   
 
<haskell>
 
<haskell>
 
getchar :: Char
 
getchar :: Char
   
get2chars = [getchar,getchar]
+
get2chars = [getchar, getchar]
 
</haskell>
 
</haskell>
   
What will we get with 'getchar' having just the 'Char' type? You can see
+
What will we get with <code>getchar</code> having just the <code>Char</code> type? You can see one problem in the definition of <code>get2chars</code> immediately:
all the possible problems in the definition of 'get2chars':
 
   
# Because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "excessive" calls to 'getchar' and use one returned value twice.
+
* because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "unnecessary" calls to <code>getchar</code> and use one returned value twice:
# Even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two chars in the order in which they were read, or in the opposite order? Nothing in the definition of 'get2chars' answers this question.
+
  +
  +
<haskell>
  +
get2chars = let x = getchar in [x, x] -- this should be a legitimate optimisation!
  +
</haskell>
   
How can these problems be solved, from the programmer's viewpoint?
 
  +
How can this problem be solved from the programmer's perspective? Let's introduce a fake parameter of <code>getchar</code> to make each call "different" from the compiler's point of view:
Let's introduce a fake parameter of 'getchar' to make each call
 
"different" from the compiler's point of view:
 
   
 
<haskell>
 
<haskell>
Line 57: Line 48:
 
</haskell>
 
</haskell>
   
Right away, this solves the first problem mentioned above - now the
+
Right away, this solves the first problem mentioned above - now the compiler will make two calls because it sees that the calls have different parameters. But there's another problem:
compiler will make two calls because it sees them as having different
+
parameters. The whole 'get2chars' function should also have a
+
* even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two characters in the order in which they were read, or in the opposite order? Nothing in the definition of <code>get2chars</code> answers this question.
fake parameter, otherwise we will have the same problem calling it:
+
  +
We need to give the compiler some clue to determine which function it should call first. The Haskell language doesn't provide any way to specify the sequence needed to evaluate <code>getchar 1</code> and <code>getchar 2</code> - except for data dependencies! How about adding an artificial data dependency which prevents evaluation of the second <code>getchar</code> before the first one? In order to achieve this, we will return an additional fake result from <code>getchar</code> that will be used as a parameter for the next <code>getchar</code> call:
   
 
<haskell>
 
<haskell>
getchar :: Int -> Char
+
getchar :: Int -> (Char, Int)
get2chars :: Int -> String
 
   
get2chars _ = [getchar 1, getchar 2]
+
get2chars _ = [a, b] where (a, i) = getchar 1
  +
(b, _) = getchar i
 
</haskell>
 
</haskell>
   
  +
So far so good - now we can guarantee that <code>a</code> is read before <code>b</code> because reading <code>b</code> needs the value (<code>i</code>) that is returned by reading <code>a</code>!
   
Now we need to give the compiler some clue to determine which function it
 
  +
We've added a fake parameter to <code>get2chars</code> but the problem is that the Haskell compiler is too smart! It can believe that the external <code>getchar</code> function is really dependent on its parameter but for <code>get2chars</code> it will see that we're just cheating because we throw it away! Therefore it won't feel obliged to execute the calls in the order we want.
should call first. The Haskell language doesn't provide any way to express
 
  +
order of evaluation... except for data dependencies! How about adding an
 
  +
How can we fix this? How about passing this fake parameter to the <code>getchar</code> function? In this case the compiler can't guess that it is really unused.
artificial data dependency which prevents evaluation of the second
 
'getchar' before the first one? In order to achieve this, we will
 
return an additional fake result from 'getchar' that will be used as a
 
parameter for the next 'getchar' call:
 
   
 
<haskell>
 
<haskell>
getchar :: Int -> (Char, Int)
 
  +
get2chars i0 = [a, b] where (a, i1) = getchar i0
  +
(b, i2) = getchar i1
  +
</haskell>
   
get2chars _ = [a,b] where (a,i) = getchar 1
 
  +
Furthermore, <code>get2chars</code> has the same purity problems as the <code>getchar</code> function. If you need to call it two times, you need a way to describe the order of these calls. Consider this:
(b,_) = getchar i
 
  +
  +
<haskell>
  +
get4chars = [get2chars 1, get2chars 2] -- order of calls to 'get2chars' isn't defined
 
</haskell>
 
</haskell>
   
So far so good - now we can guarantee that 'a' is read before 'b'
 
  +
We already know how to deal with this problem: <code>get2chars</code> should also return some fake value that can be used to order calls:
because reading 'b' needs the value ('i') that is returned by reading 'a'!
 
  +
  +
<haskell>
  +
get2chars :: Int -> (String, Int)
  +
  +
get4chars i0 = (a++b) where (a, i1) = get2chars i0
  +
(b, i2) = get2chars i1
  +
</haskell>
   
We've added a fake parameter to 'get2chars' but the problem is that the
 
  +
But what should the fake return value of <code>get2chars</code> be? If we use some integer constant, the excessively smart Haskell compiler will guess that we're cheating again. What about returning the value returned by <code>getchar</code>? See:
Haskell compiler is too smart! It can believe that the external 'getchar'
 
function is really dependent on its parameter but for 'get2chars' it
 
will see that we're just cheating because we throw it away! Therefore it won't feel obliged to execute the calls in the order we want. How can we fix this? How about passing this fake parameter to the 'getchar' function?! In this case
 
the compiler can't guess that it is really unused :)
 
   
 
<haskell>
 
<haskell>
get2chars i0 = [a,b] where (a,i1) = getchar i0
+
get2chars :: Int -> (String, Int)
(b,i2) = getchar i1
+
get2chars i0 = ([a, b], i2) where (a, i1) = getchar i0
  +
(b, i2) = getchar i1
 
</haskell>
 
</haskell>
   
  +
While that does work, it's error-prone:
   
And more - 'get2chars' has all the same purity problems as the 'getchar'
 
  +
<haskell>
function. If you need to call it two times, you need a way to describe
 
  +
get2chars :: Int -> (String, Int)
the order of these calls. Look at:
 
  +
get2chars i0 = ([a, b], i2) where (a, i1) = getchar i2 -- this might take a while...
  +
(b, i2) = getchar i1
  +
</haskell>
  +
  +
Using individual <code>let</code>-bindings is an improvement:
   
 
<haskell>
 
<haskell>
get4chars = [get2chars 1, get2chars 2] -- order of 'get2chars' calls isn't defined
 
  +
get2chars :: Int -> (String, Int)
  +
get2chars i0 = let (a, i1) = getchar i2 in -- error: i2 is undefined!
  +
let (b, i2) = getchar i1 in
  +
([a, b], i2)
 
</haskell>
 
</haskell>
   
We already know how to deal with these problems - 'get2chars' should
 
  +
but only a minor one:
also return some fake value that can be used to order calls:
 
   
 
<haskell>
 
<haskell>
 
get2chars :: Int -> (String, Int)
 
get2chars :: Int -> (String, Int)
  +
get2chars i0 = let (a, i1) = getchar i0 in
  +
let (b, i2) = getchar i2 in -- here we go again...
  +
([a, b], i2)
  +
</haskell>
   
get4chars i0 = (a++b) where (a,i1) = get2chars i0
 
  +
So how in Haskell shall we prevent such mistakes from happening? With a [[monad]]!
(b,i2) = get2chars i1
 
  +
  +
=== What is a monad? ===
  +
  +
But what is a monad? For Haskell, it's a three-way partnership between:
  +
* a type: <code>M a</code>
  +
* an operator <code>unit(M) :: a -> M a</code>
  +
* an operator <code>bind(M) :: M a -> (a -> M b) -> M b</code>
  +
  +
where <code>unit(M)</code> and <code>bind(M)</code> satisify the [[monad laws]].
  +
  +
This would translate literally into Haskell as:
  +
  +
<haskell>
  +
class Monad m where
  +
unit :: a -> m a
  +
bind :: m a -> (a -> m b) -> m b
 
</haskell>
 
</haskell>
   
  +
For now, we'll just define <code>unit</code> and <code>bind</code> directly - no type classes.
   
But what's the fake value 'get2chars' should return? If we use some integer constant, the excessively-smart Haskell compiler will guess that we're cheating again :) What about returning the value returned by 'getchar'? See:
 
  +
So how does something so <strike>vague</strike> abstract help us with I/O? Because this abstraction allows us to hide the manipulation of all those fake values - the ones we've been using to maintain the correct sequence of evaluation. We just need a suitable type:
   
 
<haskell>
 
<haskell>
get2chars :: Int -> (String, Int)
+
type IO' a = Int -> (a, Int)
get2chars i0 = ([a,b], i2) where (a,i1) = getchar i0
 
(b,i2) = getchar i1
 
 
</haskell>
 
</haskell>
   
Believe it or not, but we've just constructed the whole "monadic"
 
  +
and appropriate defintions for <code>unit</code> and <code>bind</code>:
Haskell I/O system.
 
  +
  +
<haskell>
  +
unit :: a -> IO' a
  +
unit x = \i0 -> (x, i0)
  +
  +
bind :: IO' a -> (a -> IO' b) -> IO' b
  +
bind m k = \i0 -> let (x, i1) = m i0 in
  +
let (y, i2) = k x i1 in
  +
(y, i2)
  +
</haskell>
  +
  +
Now for some extra changes to <code>getchar</code> and <code>get2chars</code>:
  +
  +
<haskell>
  +
getchar :: IO' Char {- = Int -> (Char, Int) -}
  +
  +
get2chars :: IO' String {- = Int -> (String, Int) -}
  +
  +
get2chars = \i0 -> let (a, i1) = getchar i0 in
  +
let (b, i2) = getchar i1 in
  +
let r = [a, b] in
  +
(r, i2)
  +
</haskell>
  +
  +
before we use <code>unit</code> and <code>bind</code>:
  +
  +
<haskell>
  +
getchar :: IO' Char
  +
  +
get2chars :: IO' String
  +
get2chars = getchar `bind` \a ->
  +
getchar `bind` \b ->
  +
unit [a, b]
  +
</haskell>
  +
  +
We no longer have to mess <strike>up</strike> with those fake values directly! We just need to be sure that all the operations on I/O actions like <code>unit</code> and <code>bind</code> use them correctly. We can then make <code>IO'</code>, <code>unit</code>, <code>bind</code> and (in this example) <code>getchar</code> into an ''abstract data type'' and just use those abstract I/O operations instead -
  +
only the Haskell implementation (e.g. compilers like ghc or jhc) needs to know how I/O actions actually work.
  +
  +
So there you have it - a miniature monadic I/O system in Haskell!
  +
  +
== Running with the <code>RealWorld</code> ==
   
== Welcome to the RealWorld, baby :) ==
 
  +
Warning: The following story about I/O is incorrect in that it cannot actually explain some important aspects of I/O (including interaction and concurrency). However, some people find it useful to begin developing an understanding.
   
The 'main' Haskell function has the type:
+
The <code>main</code> Haskell function has the type:
   
 
<haskell>
 
<haskell>
Line 135: Line 196:
 
</haskell>
 
</haskell>
   
where 'RealWorld' is a fake type used instead of our Int. It's something
+
where <code>RealWorld</code> is a fake type used instead of our Int. It's something
like the baton passed in a relay race. When 'main' calls some IO function,
+
like the baton passed in a relay race. When <code>main</code> calls some I/O action,
it passes the "RealWorld" it received as a parameter. All IO functions have
+
it passes the <code>RealWorld</code> it received as a parameter. All I/O actions have
similar types involving RealWorld as a parameter and result. To be
+
similar types involving <code>RealWorld</code> as a parameter and result. To be
exact, "IO" is a type synonym defined in the following way:
+
exact, <code>IO</code> is a type synonym defined in the following way:
   
 
<haskell>
 
<haskell>
Line 145: Line 206:
 
</haskell>
 
</haskell>
   
So, 'main' just has type "IO ()", 'getChar' has type "IO Char" and so
+
So, <code>main</code> just has type <code>IO ()</code>, <code>getChar</code> has type <code>IO Char</code> and so
on. You can think of the type "IO Char" as meaning "take the current RealWorld, do something to it, and return a Char and a (possibly changed) RealWorld". Let's look at 'main' calling 'getChar' two times:
+
on. You can think of the type <code>IO Char</code> as meaning "take the current <code>RealWorld</code>, do something to it, and return a <code>Char</code> and a (possibly changed) <code>RealWorld</code>". Let's look at <code>main</code> calling <code>getChar</code> two times:
   
 
<haskell>
 
<haskell>
Line 157: Line 218:
 
</haskell>
 
</haskell>
   
  +
Look at this closely: <code>main</code> passes the "world" it received to the first <code>getChar</code>. This <code>getChar</code> returns some new value of type <code>RealWorld</code>
  +
that gets used in the next call. Finally, <code>main</code> returns the "world" it got
  +
from the second <code>getChar</code>.
   
Look at this closely: 'main' passes the "world" it received to the first 'getChar'. This 'getChar' returns some new value of type RealWorld
 
  +
* Is it possible here to omit any call of <code>getChar</code> if the <code>Char</code> it read is not used? No: we need to return the "world" that is the result of the second <code>getChar</code> and this in turn requires the "world" returned from the first <code>getChar</code>.
that gets used in the next call. Finally, 'main' returns the "world" it got
 
from the second 'getChar'.
 
   
# Is it possible here to omit any call of 'getChar' if the Char it read is not used? No, because we need to return the "world" that is the result of the second 'getChar' and this in turn requires the "world" returned from the first 'getChar'.
+
* Is it possible to reorder the <code>getChar</code> calls? No: the second <code>getChar</code> can't be called before the first one because it uses the "world" returned from the first call.
# Is it possible to reorder the 'getChar' calls? No: the second 'getChar' can't be called before the first one because it uses the "world" returned from the first call.
 
# Is it possible to duplicate calls? In Haskell semantics - yes, but real compilers never duplicate work in such simple cases (otherwise, the programs generated will not have any speed guarantees).
 
   
  +
* Is it possible to duplicate calls? In Haskell semantics - yes, but real compilers never duplicate work in such simple cases (otherwise, the programs generated will not have any speed guarantees).
   
As we already said, RealWorld values are used like a baton which gets passed
+
As we already said, <code>RealWorld</code> values are used like a baton which gets passed
between all routines called by 'main' in strict order. Inside each
+
between all routines called by <code>main</code> in strict order. Inside each
routine called, RealWorld values are used in the same way. Overall, in
+
routine called, <code>RealWorld</code> values are used in the same way. Overall, in
order to "compute" the world to be returned from 'main', we should perform
+
order to "compute" the world to be returned from <code>main</code>, we should perform
each IO procedure that is called from 'main', directly or indirectly.
+
each I/O action that is called from <code>main</code>, directly or indirectly.
This means that each procedure inserted in the chain will be performed
+
This means that each action inserted in the chain will be performed
just at the moment (relative to the other IO actions) when we intended it
+
just at the moment (relative to the other I/O actions) when we intended it
 
to be called. Let's consider the following program:
 
to be called. Let's consider the following program:
   
Line 186: Line 249:
 
check that each operation that should be performed will really be
 
check that each operation that should be performed will really be
 
performed with the arguments it should have and in the order we expect.
 
performed with the arguments it should have and in the order we expect.
 
   
 
But what about conditional execution? No problem. Let's define the
 
But what about conditional execution? No problem. Let's define the
well-known 'when' operation:
+
well-known <code>when</code> operation:
   
 
<haskell>
 
<haskell>
Line 200: Line 262:
   
 
As you can see, we can easily include or exclude from the execution chain
 
As you can see, we can easily include or exclude from the execution chain
IO procedures (actions) depending on the data values. If 'condition'
+
I/O actions depending on the data values. If <code>condition</code>
will be False on the call of 'when', 'action' will never be called because
+
will be <code>False</code> on the call of <code>when</code>, <code>action</code> will never be called because real Haskell compilers, again, never call functions whose results
real Haskell compilers, again, never call functions whose results
+
are not required to calculate the final result (''i.e.'' here, the final "world" value of <code>main</code>).
are not required to calculate the final result (''i.e.'', here, the final "world" value of 'main').
 
   
 
Loops and more complex control structures can be implemented in
 
Loops and more complex control structures can be implemented in
 
the same way. Try it as an exercise!
 
the same way. Try it as an exercise!
   
  +
Finally, you may want to know how much passing these <code>RealWorld</code>
  +
values around the program costs. It's free! These fake values exist solely for the compiler while it analyzes and optimizes the code, but when it gets to assembly code generation, it notices that this type is like <code>()</code>, so
  +
all these parameters and result values can be omitted from the final generated code - they're not needed any more!
   
Finally, you may want to know how much passing these RealWorld
 
  +
== <code>(>>=)</code> and <code>do</code> notation ==
values around the program costs. It's free! These fake values exist solely for the compiler while it analyzes and optimizes the code, but when it gets to assembly code generation, it "suddenly" realize that this type is like "()", so
 
all these parameters and result values can be omitted from the final generated code. Isn't it beautiful? :)
 
   
  +
All beginners (including me) start by thinking that <code>do</code> is some
  +
super-awesome statement that executes I/O actions. That's wrong - <code>do</code> is just
  +
syntactic sugar that simplifies the writing of definitions that use I/O (and also other monads, but that's beyond the scope of this tutorial). <code>do</code> notation eventually gets translated to
  +
a series of I/O actions passing "world" values around like we've manually written above.
  +
This simplifies the gluing of several I/O actions together.
   
 
  +
You don't need to use <code>do</code> for just one action; for example,
== '>>=' and 'do' notation ==
 
 
All beginners (including me :)) start by thinking that 'do' is some
 
magic statement that executes IO actions. That's wrong - 'do' is just
 
syntactic sugar that simplifies the writing of procedures that use IO (and also other monads, but that's beyond the scope of this tutorial). 'do' notation eventually gets translated to statements passing "world" values around like we've manually written above and is used to simplify the gluing of several
 
IO actions together. You don't need to use 'do' for just one statement; for instance,
 
   
 
<haskell>
 
<haskell>
Line 231: Line 293:
 
</haskell>
 
</haskell>
   
But nevertheless it's considered Good Style to use 'do' even for one statement
 
  +
Let's examine how to desugar a <code>do</code>-expression with multiple actions in the
because it simplifies adding new statements in the future.
 
 
 
Let's examine how to desugar a 'do' with multiple statements in the
 
 
following example:
 
following example:
   
Line 244: Line 302:
 
</haskell>
 
</haskell>
   
The 'do' statement here just joins several IO actions that should be
+
The <code>do</code>-expression here just joins several I/O actions that should be
 
performed sequentially. It's translated to sequential applications
 
performed sequentially. It's translated to sequential applications
of one of the so-called "binding operators", namely '>>':
+
of one of the so-called "binding operators", namely <code>(>>)</code>:
   
 
<haskell>
 
<haskell>
Line 255: Line 313:
 
</haskell>
 
</haskell>
   
This binding operator just combines two IO actions, executing them
+
This binding operator just combines two I/O actions, executing them
 
sequentially by passing the "world" between them:
 
sequentially by passing the "world" between them:
   
Line 277: Line 335:
 
</haskell>
 
</haskell>
   
Now you can substitute the definition of '>>' at the places of its usage
+
Now you can substitute the definition of <code>(>>)</code> at the places of its usage
and check that program constructed by the 'do' desugaring is actually the
+
and check that program constructed by the <code>do</code> desugaring is actually the
 
same as we could write by manually manipulating "world" values.
 
same as we could write by manually manipulating "world" values.
   
 
  +
A more complex example involves the binding of variables using <code><-</code>:
A more complex example involves the binding of variables using "<-":
 
   
 
<haskell>
 
<haskell>
Line 296: Line 353:
 
</haskell>
 
</haskell>
   
As you should remember, the '>>' binding operator silently ignores
 
  +
where <code>(>>=)</code> corresponds to the <code>bind</code> operation in our miniature I/O system.
  +
  +
As you should remember, the <code>(>>)</code> binding operator silently ignores
 
the value of its first action and returns as an overall result
 
the value of its first action and returns as an overall result
the result of its second action only. On the other hand, the '>>=' binding operator (note the extra '=' at the end) allows us to use the result of its first action - it gets passed as an additional parameter to the second one! Look at the definition:
+
the result of its second action only. On the other hand, the <code>(>>=)</code> binding operator (note the extra <code>=</code> at the end) allows us to use the result of its first action - it gets passed as an additional parameter to the second one! Look at the definition:
   
 
<haskell>
 
<haskell>
 
(>>=) :: IO a -> (a -> IO b) -> IO b
 
(>>=) :: IO a -> (a -> IO b) -> IO b
(action1 >>= action2) world0 =
+
(action >>= reaction) world0 =
let (a, world1) = action1 world0
+
let (a, world1) = action world0
(b, world2) = action2 a world1
+
(b, world2) = reaction a world1
 
in (b, world2)
 
in (b, world2)
 
</haskell>
 
</haskell>
   
First, what does the type of the second "action" (more precisely, a function which returns an IO action), namely "a -> IO b", mean? By
 
  +
* What does the type of <code>reaction</code> - namely <code>a -> IO b</code> - mean? By substituting the <code>IO</code> definition, we get <code>a -> RealWorld -> (b, RealWorld)</code>. This means that <code>reaction</code> actually has two parameters - the type <code>a</code> actually used inside it, and the value of type <code>RealWorld</code> used for sequencing of I/O actions. That's always the case - any I/O definition has one more parameter compared to what you see in its type signature. This parameter is hidden inside the definition of the type synonym <code>IO</code>:
substituting the "IO" definition, we get "a -> RealWorld -> (b, RealWorld)".
 
This means that second action actually has two parameters
 
- the type 'a' actually used inside it, and the value of type RealWorld used for sequencing of IO actions. That's always the case - any IO procedure has one
 
more parameter compared to what you see in its type signature. This
 
parameter is hidden inside the definition of the type alias "IO".
 
   
Second, you can use these '>>' and '>>=' operations to simplify your
 
  +
<haskell>
program. For example, in the code above we don't need to introduce the
 
  +
type IO a = RealWorld -> (a, RealWorld)
variable, because the result of 'readLn' can be send directly to 'print':
 
  +
</haskell>
  +
  +
* You can use these <code>(>>)</code> and <code>(>>=)</code> operations to simplify your program. For example, in the code above we don't need to introduce the variable, because the result of <code>readLn</code> can be send directly to <code>print</code>:
   
 
<haskell>
 
<haskell>
Line 323: Line 379:
 
</haskell>
 
</haskell>
   
 
  +
As you see, the notation:
And third - as you see, the notation:
 
   
 
<haskell>
 
<haskell>
Line 331: Line 386:
 
</haskell>
 
</haskell>
   
where 'action1' has type "IO a" and 'action2' has type "IO b",
+
where <code>action1</code> has type <code>IO a</code> and <code>action2</code> has type <code>IO b</code>,
 
translates into:
 
translates into:
   
Line 338: Line 393:
 
</haskell>
 
</haskell>
   
where the second argument of '>>=' has the type "a -> IO b". It's the way
+
where the second argument of <code>(>>=)</code> has the type <code>a -> IO b</code>. It's the way
the '<-' binding is processed - the name on the left-hand side of '<-' just becomes a parameter of subsequent operations represented as one large IO action. Note also that if 'action1' has type "IO a" then 'x' will just have type "a"; you can think of the effect of '<-' as "unpacking" the IO value of 'action1' into 'x'. Note also that '<-' is not a true operator; it's pure syntax, just like 'do' itself. Its meaning results only from the way it gets desugared.
+
the <code><-</code> binding is processed - the name on the left-hand side of <code><-</code> just becomes a parameter of subsequent operations represented as one large I/O action. Note also that if <code>action1</code> has type <code>IO a</code> then <code>x</code> will just have type <code>a</code>; you can think of the effect of <code><-</code> as "unpacking" the I/O value of <code>action1</code> into <code>x</code>. Note also that <code><-</code> is not a true operator; it's pure syntax, just like <code>do</code> itself. Its meaning results only from the way it gets desugared.
   
 
Look at the next example:
 
Look at the next example:
Line 361: Line 416:
 
</haskell>
 
</haskell>
   
I omitted the parentheses here; both the '>>' and the '>>=' operators are
+
I omitted the parentheses here; both the <code>(>>)</code> and the <code>(>>=)</code> operators are
left-associative, but lambda-bindings always stretches as far to the right as possible, which means that the 'a' and 'b' bindings introduced
+
left-associative, but lambda-bindings always stretches as far to the right as possible, which means that the <code>a</code> and <code>b</code> bindings introduced
 
here are valid for all remaining actions. As an exercise, add the
 
here are valid for all remaining actions. As an exercise, add the
parentheses yourself and translate this procedure into the low-level
+
parentheses yourself and translate this definition into the low-level
code that explicitly passes "world" values. I think it should be enough to help you finally realize how the 'do' translation and binding operators work.
+
code that explicitly passes "world" values. I think it should be enough to help you finally realize how the <code>do</code> translation and binding operators work.
   
 
  +
Oh, no! I forgot the third monadic operator - <code>return</code>, which corresponds to <code>unit</code> in our miniature I/O system. It just combines its two parameters - the value passed and "world":
Oh, no! I forgot the third monadic operator - 'return'. It just
 
combines its two parameters - the value passed and "world":
 
   
 
<haskell>
 
<haskell>
Line 376: Line 429:
 
</haskell>
 
</haskell>
   
How about translating a simple example of 'return' usage? Say,
+
How about translating a simple example of <code>return</code> usage? Say,
   
 
<haskell>
 
<haskell>
Line 382: Line 435:
 
return (a*2)
 
return (a*2)
 
</haskell>
 
</haskell>
 
   
 
Programmers with an imperative language background often think that
 
Programmers with an imperative language background often think that
'return' in Haskell, as in other languages, immediately returns from
+
<code>return</code> in Haskell, as in other languages, immediately returns from
the IO procedure. As you can see in its definition (and even just from its
+
the I/O definition. As you can see in its definition (and even just from its
 
type!), such an assumption is totally wrong. The only purpose of using
 
type!), such an assumption is totally wrong. The only purpose of using
'return' is to "lift" some value (of type 'a') into the result of
+
<code>return</code> is to "lift" some value (of type <code>a</code>) into the result of
a whole action (of type "IO a") and therefore it should generally be used only as the last executed statement of some IO sequence. For example try to
+
a whole action (of type <code>IO a</code>) and therefore it should generally
translate the following procedure into the corresponding low-level code:
+
be used only as the last executed action of some I/O sequence. For example try to
  +
translate the following definition into the corresponding low-level code:
   
 
<haskell>
 
<haskell>
Line 399: Line 451:
 
</haskell>
 
</haskell>
   
and you will realize that the 'print' statement is executed even for non-negative values of 'a'. If you need to escape from the middle of an IO procedure, you can use the 'if' statement:
+
and you will realize that the <code>print</code> call is executed even for non-negative values of <code>a</code>. If you need to escape from the middle of an I/O definition, you can use an <code>if</code> expression:
   
 
<haskell>
 
<haskell>
Line 418: Line 470:
 
</haskell>
 
</haskell>
   
that may be useful for escaping from the middle of a longish 'do' statement.
+
that may be useful for escaping from the middle of a longish <code>do</code>-expression.
   
 
  +
Last exercise: implement a function <code>liftM</code> that lifts operations on
Last exercise: implement a function 'liftM' that lifts operations on
 
 
plain values to the operations on monadic ones. Its type signature:
 
plain values to the operations on monadic ones. Its type signature:
   
Line 435: Line 486:
 
return (f x)
 
return (f x)
 
</haskell>
 
</haskell>
 
 
   
 
== Mutable data (references, arrays, hash tables...) ==
 
== Mutable data (references, arrays, hash tables...) ==
Line 453: Line 502:
 
</haskell>
 
</haskell>
   
Does this look strange? First, the two calls to 'readVariable' look the same, so the compiler can just reuse the value returned by the first call. Second,
 
  +
Does this look strange?
the result of the 'writeVariable' call isn't used so the compiler can (and will!) omit this call completely. To complete the picture, these three calls may be rearranged in any order because they appear to be independent of each
 
  +
# The two calls to <code>readVariable</code> look the same, so the compiler can just reuse the value returned by the first call.
other. This is obviously not what was intended. What's the solution? You already know this - use IO actions! Using IO actions guarantees that:
 
  +
# The result of the <code>writeVariable</code> call isn't used so the compiler can (and will!) omit this call completely.
  +
# These three calls may be rearranged in any order because they appear to be independent of each other.
   
# the execution order will be retained as written
 
  +
This is obviously not what was intended. What's the solution? You already know this - use I/O actions! Doing that guarantees:
  +
  +
# the result of the "same" action (such as <code>readVariable varA</code>) will not be reused
 
# each action will have to be executed
 
# each action will have to be executed
# the result of the "same" action (such as "readVariable varA") will not be reused
 
  +
# the execution order will be retained as written
   
 
So, the code above really should be written as:
 
So, the code above really should be written as:
Line 472: Line 524:
 
</haskell>
 
</haskell>
   
Here, 'varA' has the type "IORef Int" which means "a variable (reference) in
+
Here, <code>varA</code> has the type <code>IORef Int</code> which means "a variable (reference) in
the IO monad holding a value of type Int". newIORef creates a new variable
+
the I/O monad holding a value of type <code>Int</code>". <code>newIORef</code> creates a new variable
 
(reference) and returns it, and then read/write actions use this
 
(reference) and returns it, and then read/write actions use this
reference. The value returned by the "readIORef varA" action depends not
+
reference. The value returned by the <code>readIORef varA</code> action depends not
 
only on the variable involved but also on the moment this operation is performed so it can return different values on each call.
 
only on the variable involved but also on the moment this operation is performed so it can return different values on each call.
   
 
Arrays, hash tables and any other _mutable_ data structures are
 
Arrays, hash tables and any other _mutable_ data structures are
defined in the same way - for each of them, there's an operation that creates new "mutable values" and returns a reference to it. Then special read and write
+
defined in the same way - for each of them, there's an operation that creates new "mutable values" and returns a reference to it. Then value-specific read and write
operations in the IO monad are used. The following code shows an example
+
operations in the I/O monad are used. The following code shows an example
 
using mutable arrays:
 
using mutable arrays:
   
Line 492: Line 544:
 
</haskell>
 
</haskell>
   
Here, an array of 10 elements with 37 as the initial value at each location is created. After reading the value of the first element (index 1) into 'a' this element's value is changed to 64 and then read again into 'b'. As you can see by executing this code, 'a' will be set to 37 and 'b' to 64.
+
Here, an array of 10 elements with 37 as the initial value at each location is created. After reading the value of the first element (index 1) into <code>a</code> this element's value is changed to 64 and then read again into <code>b</code>. As you can see by executing this code, <code>a</code> will be set to 37 and <code>b</code> to 64.
 
 
   
Other state-dependent operations are also often implemented as IO
+
Other state-dependent operations are also often implemented with I/O
 
actions. For example, a random number generator should return a different
 
actions. For example, a random number generator should return a different
value on each call. It looks natural to give it a type involving IO:
+
value on each call. It looks natural to give it a type involving <code>IO</code>:
   
 
<haskell>
 
<haskell>
Line 505: Line 557:
 
routine is impure, i.e. its result depends on something in the "real
 
routine is impure, i.e. its result depends on something in the "real
 
world" (file system, memory contents...), internal state and so on,
 
world" (file system, memory contents...), internal state and so on,
you should give it an IO type. Otherwise, the compiler can
+
you should give it an <code>IO</code> type. Otherwise, the compiler can
"optimize" repetitive calls of this procedure with the same parameters! :)
+
"optimize" repetitive calls to the definition with the same parameters!
   
For example, we can write a non-IO type for:
+
For example, we can write a non-<code>IO</code> type for:
   
 
<haskell>
 
<haskell>
Line 515: Line 567:
 
</haskell>
 
</haskell>
   
because the result of 'sin' depends only on its argument, but
+
because the result of <code>sin</code> depends only on its argument, but
   
 
<haskell>
 
<haskell>
Line 522: Line 574:
 
</haskell>
 
</haskell>
   
If you will declare 'tell' as a pure function (without IO) then you may
+
If you will declare <code>tell</code> as a pure function (without <code>IO</code>) then you may
get the same position on each call! :)
+
get the same position on each call!
  +
  +
=== Encapsulated mutable data: ST ===
  +
  +
If you're going to be doing things like sending text to a screen or reading data from a scanner, <code>IO</code> is the type to start with - you can then
  +
customise or add existing or new I/O operations as you see fit. But what if that shiny-new (or classic) algorithm you're working on really only needs
  +
mutable state - then having to drag that <code>IO</code> type from <code>main</code> all the way down to wherever you're implementing the algorithm can
  +
get quite annoying.
  +
  +
Fortunately there is a better way! One that remains totally pure and yet allows the use of references, arrays, and so on - and it's done using, you guessed it, Haskell's versatile type system (and one extension). It is the <code>ST</code> type, and it too is monadic!
  +
  +
So what's the big difference between the <code>ST</code> and <code>IO</code> types? In one word - <code>runST</code>:
  +
<haskell>
  +
runST :: (forall s . ST s a) -> a
  +
</haskell>
  +
  +
Yes - it has a very unusual type. But that type allows you to run your stateful computation ''as if it was a pure definition!''
  +
  +
The <code>s</code> type variable in <code>ST</code> is the type of the local state. Moreover, all the fun mutable stuff available for <code>ST</code> is
  +
quantified over <code>s</code>:
  +
<haskell>
  +
newSTRef :: a -> ST s (STRef s a)
  +
newArray_ :: Ix i => (i, i) -> ST s (STArray s i e)
  +
</haskell>
  +
  +
So why does <code>runST</code> have such a funky type? Let's see what would happen if we wrote
  +
<haskell>
  +
makeSTRef :: a -> STRef s a
  +
makeSTRef a = runST (newSTRef a)
  +
</haskell>
  +
This fails, because <code>newSTRef a</code> doesn't work for all state types <code>s</code> - it only works for the <code>s</code> from the return type <code>STRef s a</code>.
  +
  +
This is all sort of wacky, but the result is that you can only run an <code>ST</code> computation where the output type is functionally pure, and makes no references
  +
to the internal mutable state of the computation. In exchange for that, there's no access to I/O operations like writing to the console - only references, arrays, and
  +
such that come in handy for pure computations.
  +
  +
Important note - the state type doesn't actually mean anything. We never have a value of type <code>s</code>, for instance. It's just a way of getting the type system
  +
to do the work of ensuring purity is preserved.
  +
  +
On the inside, <code>runST</code> runs a computation with a baton similar to <code>RealWorld</code> for the <code>IO</code> type.
  +
Once the computation has completed <code>runST</code> separates the resulting value from the final baton. This value is then returned by <code>runST</code>.
  +
The internal implementations are so similar there's there's a function:
  +
  +
<haskell>
  +
stToIO :: ST RealWorld a -> IO a
  +
</haskell>
  +
  +
The difference is that <code>ST</code> uses the type system to forbid unsafe behavior like extracting mutable objects from their safe <code>ST</code> wrapping, but allowing purely functional outputs to be performed with all the handy access to mutable references and arrays.
  +
  +
For example, here's a particularly convoluted way to compute the integer that comes after zero:
  +
  +
<haskell>
  +
oneST :: ST s Integer -- note that this works correctly for any s
  +
oneST = do var <- newSTRef 0
  +
modifySTRef var (+1)
  +
readSTRef var
  +
  +
one :: Int
  +
one = runST oneST
  +
</haskell>
   
== IO actions as values ==
+
== I/O actions as values ==
   
By this point you should understand why it's impossible to use IO
+
By this point you should understand why it's impossible to use I/O
actions inside non-IO (pure) procedures. Such procedures just don't
+
actions inside non-I/O (pure) functions. Such functions just don't
get a "baton"; they don't know any "world" value to pass to an IO action.
+
get a "baton"; they don't know any "world" value to pass to an I/O action.
The RealWorld type is an abstract datatype, so pure functions also can't construct RealWorld values by themselves, and it's a strict type, so 'undefined' also can't be used. So, the prohibition of using IO actions inside pure procedures is just a type system trick (as it usually is in Haskell :)).
+
The <code>RealWorld</code> type is an abstract datatype, so pure functions
  +
also can't construct <code>RealWorld</code> values by themselves, and it's
  +
a strict type, so <code>undefined</code> also can't be used. So, the
  +
prohibition of using I/O actions inside pure functions is maintained by the
  +
type system (as it usually is in Haskell).
   
But while pure code can't _execute_ IO actions, it can work with them
+
But while pure code can't ''execute'' I/O actions, it can work with them
 
as with any other functional values - they can be stored in data
 
as with any other functional values - they can be stored in data
 
structures, passed as parameters, returned as results, collected in
 
structures, passed as parameters, returned as results, collected in
lists, and partially applied. But an IO action will remain a
+
lists, and partially applied. But an I/O action will remain a
 
functional value because we can't apply it to the last argument - of
 
functional value because we can't apply it to the last argument - of
type RealWorld.
+
type <code>RealWorld</code>.
   
In order to _execute_ the IO action we need to apply it to some
+
In order to ''execute'' the I/O action we need to apply it to some
RealWorld value. That can be done only inside some IO procedure,
+
<code>RealWorld</code> value. That can be done only inside other I/O actions,
in its "actions chain". And real execution of this action will take
+
in their "actions chains". And real execution of this action will take
place only when this procedure is called as part of the process of
+
place only when this action is called as part of the process of
"calculating the final value of world" for 'main'. Look at this example:
+
"calculating the final value of world" for <code>main</code>. Look at this example:
   
 
<haskell>
 
<haskell>
Line 552: Line 604:
 
</haskell>
 
</haskell>
 
 
Here we first bind a value to 'get2chars' and then write a binding
+
Here we first bind a value to <code>get2chars</code> and then write a binding
involving 'putStr'. But what's the execution order? It's not defined
+
involving <code>putStr</code>. But what's the execution order? It's not defined
by the order of the 'let' bindings, it's defined by the order of processing
+
by the order of the <code>let</code> bindings, it's defined by the order of processing
"world" values! You can arbitrarily reorder the binding statements - the execution order will be defined by the data dependency with respect to the
+
"world" values! You can arbitrarily reorder those local bindings - the execution order will be defined by the data dependency with respect to the
"world" values that get passed around. Let's see what this 'main' looks like in the 'do' notation:
+
"world" values that get passed around. Let's see what this <code>main</code> looks like in the <code>do</code> notation:
   
 
<haskell>
 
<haskell>
Line 565: Line 617:
 
</haskell>
 
</haskell>
   
As you can see, we've eliminated two of the 'let' bindings and left only the one defining 'get2chars'. The non-'let' statements are executed in the exact order in which they're written, because they pass the "world" value from statement to statement as we described above. Thus, this version of the function is much easier to understand because we don't have to mentally figure out the data dependency of the "world" value.
+
As you can see, we've eliminated two of the <code>let</code> bindings and left only the one defining <code>get2chars</code>. The non-<code>let</code> actions are executed in the exact order in which they're written, because they pass the "world" value from action to action as we described above. Thus, this version of the function is much easier to understand because we don't have to mentally figure out the data dependency of the "world" value.
   
Moreover, IO actions like 'get2chars' can't be executed directly
+
Moreover, I/O actions like <code>get2chars</code> can't be executed directly
because they are functions with a RealWorld parameter. To execute them,
+
because they are functions with a <code>RealWorld</code> parameter. To execute them,
we need to supply the RealWorld parameter, i.e. insert them in the 'main'
+
we need to supply the <code>RealWorld</code> parameter, i.e. insert them in the <code>main</code>
chain, placing them in some 'do' sequence executed from 'main' (either directly in the 'main' function, or indirectly in an IO function called from 'main'). Until that's done, they will remain like any function, in partially
+
chain, placing them in some <code>do</code> sequence executed from <code>main</code> (either directly in the <code>main</code> function, or indirectly in an
evaluated form. And we can work with IO actions as with any other
+
I/O function called from <code>main</code>). Until that's done, they will remain like any function, in partially
  +
evaluated form. And we can work with I/O actions as with any other
 
functions - bind them to names (as we did above), save them in data
 
functions - bind them to names (as we did above), save them in data
 
structures, pass them as function parameters and return them as results - and
 
structures, pass them as function parameters and return them as results - and
they won't be performed until you give them the magic RealWorld
+
they won't be performed until you give them that inaugural <code>RealWorld</code>
 
parameter!
 
parameter!
   
  +
=== Example: a list of I/O actions ===
   
 
  +
Let's try defining a list of I/O actions:
=== Example: a list of IO actions ===
 
 
Let's try defining a list of IO actions:
 
   
 
<haskell>
 
<haskell>
Line 598: Line 648:
   
 
Well, now we want to execute some of these actions. No problem, just
 
Well, now we want to execute some of these actions. No problem, just
insert them into the 'main' chain:
+
insert them into the <code>main</code> chain:
   
 
<haskell>
 
<haskell>
Line 606: Line 656:
 
</haskell>
 
</haskell>
   
Looks strange, right? :) Really, any IO action that you write in a 'do'
+
Looks strange, right? Really, any I/O action that you write in a <code>do</code>-expression (or use as a parameter for the <code>(>>)</code>/<code>(>>=)</code> operators) is an expression
statement (or use as a parameter for the '>>'/'>>=' operators) is an expression
+
returning a result of type <code>IO a</code> for some type <code>a</code>. Typically, you use some function that has the type <code>x -> y -> ... -> IO a</code> and provide all the x, y, etc. parameters. But you're not limited to this standard scenario -
returning a result of type 'IO a' for some type 'a'. Typically, you use some function that has the type 'x -> y -> ... -> IO a' and provide all the x, y, etc. parameters. But you're not limited to this standard scenario -
 
 
don't forget that Haskell is a functional language and you're free to
 
don't forget that Haskell is a functional language and you're free to
compute the functional value required (recall that "IO a" is really a function
+
compute the functional value required (recall that <code>IO a</code> is really a function
 
type) in any possible way. Here we just extracted several functions
 
type) in any possible way. Here we just extracted several functions
 
from the list - no problem. This functional value can also be
 
from the list - no problem. This functional value can also be
 
constructed on-the-fly, as we've done in the previous example - that's also
 
constructed on-the-fly, as we've done in the previous example - that's also
 
OK. Want to see this functional value passed as a parameter?
 
OK. Want to see this functional value passed as a parameter?
Just look at the definition of 'when'. Hey, we can buy, sell, and rent
+
Just look at the definition of <code>when</code>. Hey, we can buy, sell, and rent
these IO actions just like we can with any other functional values! For example, let's define a function that executes all the IO actions in the list:
+
these I/O actions just like we can with any other functional values! For example,
  +
let's define a function that executes all the I/O actions in the list:
   
 
<haskell>
 
<haskell>
Line 624: Line 674:
 
</haskell>
 
</haskell>
   
No black magic - we just extract IO actions from the list and insert
+
No mirrors or smoke - we just extract I/O actions from the list and insert
them into a chain of IO operations that should be performed one after another (in the same order that they occurred in the list) to "compute the final world value" of the entire 'sequence_' call.
+
them into a chain of I/O operations that should be performed one after another (in the same order that they occurred in the list) to "compute the final world value" of the entire <code>sequence_</code> call.
   
With the help of 'sequence_', we can rewrite our last 'main' function as:
+
With the help of <code>sequence_</code>, we can rewrite our last <code>main</code> function as:
   
 
<haskell>
 
<haskell>
Line 633: Line 683:
 
</haskell>
 
</haskell>
   
 
  +
Haskell's ability to work with I/O actions as with any other
Haskell's ability to work with IO actions as with any other
 
 
(functional and non-functional) values allows us to define control
 
(functional and non-functional) values allows us to define control
 
structures of arbitrary complexity. Try, for example, to define a control
 
structures of arbitrary complexity. Try, for example, to define a control
structure that repeats an action until it returns the 'False' result:
+
structure that repeats an action until it returns the <code>False</code> result:
   
 
<haskell>
 
<haskell>
Line 646: Line 695:
 
Most programming languages don't allow you to define control structures at all, and those that do often require you to use a macro-expansion system. In Haskell, control structures are just trivial functions anyone can write.
 
Most programming languages don't allow you to define control structures at all, and those that do often require you to use a macro-expansion system. In Haskell, control structures are just trivial functions anyone can write.
   
  +
=== Example: returning an I/O action as a result ===
   
=== Example: returning an IO action as a result ===
+
How about returning an I/O action as the result of a function? Well, we've done
+
this for each I/O definition - they all return I/O actions
How about returning an IO action as the result of a function? Well, we've done
+
that need a <code>RealWorld</code> value to be performed. While we usually just
this each time we've defined an IO procedure - they all return IO actions
+
execute them as part of a higher-level I/O definition, it's also
that need a RealWorld value to be performed. While we usually just
 
execute them as part of a higher-level IO procedure, it's also
 
 
possible to just collect them without actual execution:
 
possible to just collect them without actual execution:
   
Line 657: Line 707:
 
b = when True getChar
 
b = when True getChar
 
c = getChar >> getChar
 
c = getChar >> getChar
putStr "These 'let' statements are not executed!"
+
putStr "These let-bindings are not executed!"
 
</haskell>
 
</haskell>
   
These assigned IO procedures can be used as parameters to other
+
These assigned I/O actions can be used as parameters to other
procedures, or written to global variables, or processed in some other
+
definitions, or written to global variables, or processed in some other
way, or just executed later, as we did in the example with 'get2chars'.
+
way, or just executed later, as we did in the example with <code>get2chars</code>.
   
But how about returning a parameterized IO action from an IO procedure? Let's define a procedure that returns the i'th byte from a file represented as a Handle:
+
But how about returning a parameterized I/O action from an I/O definition?
  +
Here's a definition that returns the i'th byte from a file represented as a Handle:
   
 
<haskell>
 
<haskell>
readi h i = do hSeek h i AbsoluteSeek
+
readi h i = do hSeek h AbsoluteSeek i
 
hGetChar h
 
hGetChar h
 
</haskell>
 
</haskell>
   
So far so good. But how about a procedure that returns the i'th byte of a file
+
So far so good. But how about a definition that returns the i'th byte of a file
 
with a given name without reopening it each time?
 
with a given name without reopening it each time?
   
Line 680: Line 730:
 
</haskell>
 
</haskell>
   
As you can see, it's an IO procedure that opens a file and returns...
+
As you can see, it's an I/O definition that opens a file and returns...an
another IO procedure that will read the specified byte. But we can go
+
I/O action that will read the specified byte. But we can go
further and include the 'readi' body in 'readfilei':
+
further and include the <code>readi</code> body in <code>readfilei</code>:
   
 
<haskell>
 
<haskell>
 
readfilei name = do h <- openFile name ReadMode
 
readfilei name = do h <- openFile name ReadMode
let readi h i = do hSeek h i AbsoluteSeek
+
let readi h i = do hSeek h AbsoluteSeek i
 
hGetChar h
 
hGetChar h
 
return (readi h)
 
return (readi h)
 
</haskell>
 
</haskell>
   
That's a little better. But why do we add 'h' as a parameter to 'readi' if it can be obtained from the environment where 'readi' is now defined? An even shorter version is this:
+
That's a little better. But why do we add <code>h</code> as a parameter to <code>readi</code> if it can be obtained from the environment where <code>readi</code> is now defined? An even shorter version is this:
   
 
<haskell>
 
<haskell>
 
readfilei name = do h <- openFile name ReadMode
 
readfilei name = do h <- openFile name ReadMode
let readi i = do hSeek h i AbsoluteSeek
+
let readi i = do hSeek h AbsoluteSeek i
 
hGetChar h
 
hGetChar h
 
return readi
 
return readi
 
</haskell>
 
</haskell>
   
What have we done here? We've build a parameterized IO action involving local
+
What have we done here? We've build a parameterized I/O action involving local
names inside 'readfilei' and returned it as the result. Now it can be
+
names inside <code>readfilei</code> and returned it as the result. Now it can be
 
used in the following way:
 
used in the following way:
   
Line 711: Line 761:
 
</haskell>
 
</haskell>
   
 
  +
This way of using I/O actions is very typical for Haskell programs - you
This way of using IO actions is very typical for Haskell programs - you
 
  +
just construct one or more I/O actions that you need,
just construct one or more IO actions that you need,
 
 
with or without parameters, possibly involving the parameters that your
 
with or without parameters, possibly involving the parameters that your
"constructor" received, and return them to the caller. Then these IO actions
+
"constructor" received, and return them to the caller. Then these I/O actions
 
can be used in the rest of the program without any knowledge about your
 
can be used in the rest of the program without any knowledge about your
 
internal implementation strategy. One thing this can be used for is to
 
internal implementation strategy. One thing this can be used for is to
 
partially emulate the OOP (or more precisely, the ADT) programming paradigm.
 
partially emulate the OOP (or more precisely, the ADT) programming paradigm.
 
   
 
=== Example: a memory allocator generator ===
 
=== Example: a memory allocator generator ===
   
As an example, one of my programs has a module which is a memory suballocator. It receives the address and size of a large memory block and returns two
+
As an example, one of my programs has a module which is a memory suballocator.
procedures - one to allocate a subblock of a given size and the other to
+
It receives the address and size of a large memory block and returns two
free the allocated subblock:
+
specialised I/O operations - one to allocate a subblock of a given size
  +
and the other to free the allocated subblock:
   
 
<haskell>
 
<haskell>
Line 739: Line 787:
 
</haskell>
 
</haskell>
   
How this is implemented? 'alloc' and 'free' work with references
+
How this is implemented? <code>alloc</code> and <code>free</code> work with references
created inside the memoryAllocator procedure. Because the creation of these references is a part of the memoryAllocator IO actions chain, a new independent set of references will be created for each memory block for which
+
created inside the <code>memoryAllocator</code> definition. Because the creation of these references is a part of the
memoryAllocator is called:
+
<code>memoryAllocator</code> I/O-action chain, a new independent set of references will be created for each memory block for which
  +
<code>memoryAllocator</code> is called:
   
 
<haskell>
 
<haskell>
Line 749: Line 797:
 
</haskell>
 
</haskell>
   
These two references are read and written in the 'alloc' and 'free' definitions (we'll implement a very simple memory allocator for this example):
+
These two references are read and written in the <code>alloc</code> and <code>free</code> definitions (we'll implement a very simple memory allocator for this example):
   
 
<haskell>
 
<haskell>
Line 765: Line 813:
 
of direct support for impure functions.
 
of direct support for impure functions.
 
 
The following example uses procedures, returned by memoryAllocator, to
+
The following example uses the operations returned by <code>memoryAllocator</code>, to
 
simultaneously allocate/free blocks in two independent memory buffers:
 
simultaneously allocate/free blocks in two independent memory buffers:
   
Line 780: Line 828:
 
ptr22 <- alloc2 1000
 
ptr22 <- alloc2 1000
 
</haskell>
 
</haskell>
 
 
   
 
=== Example: emulating OOP with record types ===
 
=== Example: emulating OOP with record types ===
Line 789: Line 835:
 
to create a heterogeneous list of figures. All figures in this list should
 
to create a heterogeneous list of figures. All figures in this list should
 
support the same set of operations: draw, move and so on. We will
 
support the same set of operations: draw, move and so on. We will
represent these operations as IO procedures. Instead of a "class" let's
+
define these operations using I/O actions. Instead of a "class" let's
define a structure containing implementations of all the procedures
+
define a structure containing implementations of all the operations
 
required:
 
required:
   
Line 801: Line 847:
 
</haskell>
 
</haskell>
   
 
  +
The constructor of each figure's type should just return a <code>Figure</code> record:
The constructor of each figure's type should just return a Figure record:
 
   
 
<haskell>
 
<haskell>
Line 811: Line 856:
 
type Radius = Int -- circle radius in points
 
type Radius = Int -- circle radius in points
 
</haskell>
 
</haskell>
 
   
 
We will "draw" figures by just printing their current parameters.
 
We will "draw" figures by just printing their current parameters.
Let's start with a simplified implementation of the 'circle' and 'rectangle'
+
Let's start with a simplified implementation of the <code>circle</code> and <code>rectangle</code>
constructors, without actual 'move' support:
+
constructors, without actual <code>move</code> support:
   
 
<haskell>
 
<haskell>
Line 827: Line 871:
 
</haskell>
 
</haskell>
   
 
  +
As you see, each constructor just returns a fixed <code>draw</code> operation that prints
As you see, each constructor just returns a fixed 'draw' procedure that prints
 
 
parameters with which the concrete figure was created. Let's test it:
 
parameters with which the concrete figure was created. Let's test it:
   
Line 847: Line 890:
 
moved around. In order to achieve this, we should provide each figure
 
moved around. In order to achieve this, we should provide each figure
 
with a mutable variable that holds each figure's current screen location. The
 
with a mutable variable that holds each figure's current screen location. The
type of this variable will be "IORef Point". This variable should be created in the figure constructor and manipulated in IO procedures (closures) enclosed in
+
type of this variable will be <code>IORef Point</code>. This variable should
the Figure record:
+
be created in the figure constructor and manipulated in I/O operations (closures) enclosed in
  +
the <code>Figure</code> record:
   
 
<haskell>
 
<haskell>
Line 862: Line 905:
 
 
 
return $ Figure { draw=drawF, move=moveF }
 
return $ Figure { draw=drawF, move=moveF }
 
  +
 
 
rectangle from to = do
 
rectangle from to = do
 
fromVar <- newIORef from
 
fromVar <- newIORef from
Line 879: Line 921:
 
return $ Figure { draw=drawF, move=moveF }
 
return $ Figure { draw=drawF, move=moveF }
 
</haskell>
 
</haskell>
 
   
 
Now we can test the code which moves figures around:
 
Now we can test the code which moves figures around:
Line 891: Line 932:
 
</haskell>
 
</haskell>
   
 
  +
It's important to realize that we are not limited to including only I/O actions
It's important to realize that we are not limited to including only IO actions
 
  +
in a record that's intended to simulate a C++/Java-style interface. The record can also include values, <code>IORef</code>s, pure functions - in short, any type of data. For example, we can easily add to the <code>Figure</code> interface fields for area and origin:
in a record that's intended to simulate a C++/Java-style interface. The record can also include values, IORefs, pure functions - in short, any type of data. For example, we can easily add to the Figure interface fields for area and origin:
 
   
 
<haskell>
 
<haskell>
Line 903: Line 943:
 
</haskell>
 
</haskell>
   
  +
== Exception handling (under development) ==
   
  +
Although Haskell provides a set of exception raising/handling features comparable to those in popular OOP languages (C++, Java, C#), this part of the language receives much less attention. This is for two reasons:
   
== Exception handling (under development) ==
 
  +
* you just don't need to worry as much about them - most of the time it just works "behind the scenes".
   
Although Haskell provides set of exception rasising/handling features comparable to those in popular OOP languages (C++, Java, C#), this part of language receives much less attention than there. First reason is that you just don't need to pay attention - most times it just works "behind the scene". Second reason is that Haskell, being lacked OOP inheritance, doesn't allow to easily subclass exception types, therefore limiting flexibility of exception handling.
+
* Haskell, lacking OOP-style inheritance, doesn't allow the programmer to easily subclass exception types, therefore limiting the flexibility of exception handling.
   
First, Haskell RTS raise more exceptions than traditional languages - pattern match failures, calls with invalid arguments (such as '''head []''') and computations whose results depend on special values '''undefined''' and '''error "...."''' all raise their own exceptions:
+
The Haskell RTS raises more exceptions than traditional languages - pattern match failures, calls with invalid arguments (such as <code>head []</code>) and computations whose results depend on special values <code>undefined</code> and <code>error "...."</code> all raise their own exceptions:
   
example 1:
+
* example 1:
 
<haskell>
 
<haskell>
 
main = print (f 2)
 
main = print (f 2)
Line 919: Line 961:
 
</haskell>
 
</haskell>
   
example 2:
+
* example 2:
 
<haskell>
 
<haskell>
 
main = print (head [])
 
main = print (head [])
 
</haskell>
 
</haskell>
   
example 3:
+
* example 3:
 
<haskell>
 
<haskell>
 
main = print (1 + (error "Value that wasn't initialized or cannot be computed"))
 
main = print (1 + (error "Value that wasn't initialized or cannot be computed"))
 
</haskell>
 
</haskell>
   
This allows to write programs in much more error-prone way.
+
This allows the writing of programs in a much more error-prone way.
   
 
== Interfacing with C/C++ and foreign libraries (under development) ==
 
== Interfacing with C/C++ and foreign libraries (under development) ==
   
While Haskell is great at algorithm development, speed isn't its best side. We can combine best of both worlds, though, by writing speed-critical parts of program in C and rest in Haskell. We just need a way to call C functions from Haskell and vice versa, and to marshal data between two worlds.
+
While Haskell is great at algorithm development, speed isn't its best side. We can combine the best of both worlds, though, by writing speed-critical parts of program in C and the rest in Haskell. We just need a way to call C functions from Haskell and vice versa, and to marshal data between both worlds.
   
We also need to interact with C world for using Windows/Linux APIs, linking to various libraries and DLLs. Even interfacing with other languages requires to go through C world as "common denominator". Appendix [6] to Haskell'98 standard provides complete description of interfacing with C.
+
We also need to interact with the C world for using Windows/Linux APIs, linking to various libraries and DLLs. Even interfacing with other languages often requires going through C world as a "common denominator". [https://www.haskell.org/onlinereport/haskell2010/haskellch8.html Chapter 8 of the Haskell 2010 report] provides a complete description of interfacing with C.
   
We will learn FFI via series of examples. These examples includes C/C++ code, so they need C/C++ compilers to be installed, the same will be true if you need to include code written in C/C++ in your program (C/C++ compilers are not required when you need just to link with existing libraries providing APIs with C calling convention). On Unix (and MacOS?) systems system-wide default C/C++ compiler typically used by GHC installation. On Windows, no default compilers exist, so GHC typically shipped with C compiler, and you may find on download page GHC distribution with bundled C and C++ compilers. Alternatively, you may find and install gcc/mingw32 version compatible with your GHC installation.
+
We will learn FFI via a series of examples. These examples include C/C++ code, so they need C/C++ compilers to be installed, the same will be true if you need to include code written in C/C++ in your program (C/C++ compilers are not required when you just need to link with existing libraries providing APIs with C calling convention). On Unix (and Mac OS?) systems, the system-wide default C/C++ compiler is typically used by GHC installation. On Windows, no default compilers exist, so GHC is typically shipped with a C compiler, and you may find on the download page a GHC distribution bundled with C and C++ compilers. Alternatively, you may find and install a GCC/MinGW version compatible with your GHC installation.
   
If you need to make your C/C++ code as fast as possible, you may compile your code by Intel compilers instead of gcc. However, these compilers are not free, moreover on Windows code compiled by Intel compilers may be interact with GHC-compiled code only if one of them is put into DLLs (due to RTS incompatibility) [not checked! please correct if i'm wrong].
+
If you need to make your C/C++ code as fast as possible, you may compile your code by Intel compilers instead of GCC. However, these compilers are not free, moreover on Windows, code compiled by Intel compilers may not interact correctly with GHC-compiled code, unless one of them is put into DLLs (due to object file incompatibility).
   
 
[http://www.haskell.org/haskellwiki/Applications_and_libraries/Interfacing_other_languages More links]:
 
[http://www.haskell.org/haskellwiki/Applications_and_libraries/Interfacing_other_languages More links]:
Line 946: Line 988:
 
:A lightweight tool for implementing access to C libraries from Haskell.
 
:A lightweight tool for implementing access to C libraries from Haskell.
   
;[http://hsffig.sourceforge.net HSFFIG]
 
  +
;[[HSFFIG]]
:Haskell FFI Binding Modules Generator (HSFFIG) is a tool that takes a C library include file (.h) and generates Haskell Foreign Functions Interface import declarations for items (functions, structures, etc.) the header defines.
+
:Haskell FFI Binding Modules Generator (HSFFIG) is a tool that takes a C library header (".h") and generates Haskell Foreign Functions Interface import declarations for items (functions, structures, etc.) the header defines.
   
 
;[http://quux.org/devel/missingpy MissingPy]
 
;[http://quux.org/devel/missingpy MissingPy]
:MissingPy is really two libraries in one. At its lowest level, MissingPy is a library designed to make it easy to call into Python from Haskell. It provides full support for interpreting arbitrary Python code, interfacing with a good part of the Python/C API, and handling Python objects. It also provides tools for converting between Python objects and their Haskell equivalents. Memory management is handled for you, and Python exceptions get mapped to Haskell Dynamic exceptions. At a higher level, MissingPy contains Haskell interfaces to some Python modules.
+
:MissingPy is really two libraries in one. At its lowest level, MissingPy is a library designed to make it easy to call into Python from Haskell. It provides full support for interpreting arbitrary Python code, interfacing with a good part of the Python/C API, and handling Python objects. It also provides tools for converting between Python objects and their Haskell equivalents. Memory management is handled for you, and Python exceptions get mapped to Haskell <code>Dynamic</code> exceptions. At a higher level, MissingPy contains Haskell interfaces to some Python modules.
   
[[HsLua Haskell interface to Lua scripting language]]
 
  +
;[[HsLua]]
  +
:A Haskell interface to the Lua scripting language
   
 
=== Calling functions ===
 
=== Calling functions ===
   
First, we will learn how to call C functions from Haskell and Haskell functions from C. The first example consists of three files:
+
We begin by learning how to call C functions from Haskell and Haskell functions from C. The first example consists of three files:
   
main.hs:
+
''main.hs:''
 
<haskell>
 
<haskell>
 
{-# LANGUAGE ForeignFunctionInterface #-}
 
{-# LANGUAGE ForeignFunctionInterface #-}
Line 974: Line 1,017:
 
</haskell>
 
</haskell>
   
evil.c:
+
''vile.c:''
 
<haskell>
 
<haskell>
 
#include <stdio.h>
 
#include <stdio.h>
Line 986: Line 1,029:
 
</haskell>
 
</haskell>
   
prototypes.h:
+
''prototypes.h:''
 
<haskell>
 
<haskell>
 
extern void c_function (void);
 
extern void c_function (void);
Line 993: Line 1,036:
   
 
It may be compiled and linked in one step by ghc:
 
It may be compiled and linked in one step by ghc:
ghc --make main.hs evil.c
+
ghc --make main.hs vile.c
   
Or, you may compile C module(s) separately and link in .o files (it may preferable if you use make and don't want to recompile unchanged sources - ghc's --make option provides smart recompilation only for .hs files):
+
Or, you may compile C module(s) separately and link in ".o" files (this may be preferable if you use <code>make</code> and don't want to recompile unchanged sources; ghc's <code>--make</code> option provides smart recompilation only for ".hs" files):
ghc -c evil.c
+
ghc -c vile.c
ghc --make main.hs evil.o
+
ghc --make main.hs vile.o
   
You may use gcc/g++ directly to compile your C/C++ files but i recommend to do linking via ghc because it adds a lots of libraries required for execution of Haskell code. For the same reasons, even if your main routine is written in C/C++, i recommend you to call it from Haskell function main - otherwise you'll have to explicitly init/shutdown the GHC RTS (run-time system).
+
You may use gcc/g++ directly to compile your C/C++ files but I recommend to do linking via ghc because it adds a lot of libraries required for execution of Haskell code. For the same reason, even if your <code>main</code> routine is written in C/C++, I recommend calling it from the Haskell function <code>main</code> - otherwise you'll have to explicitly init/shutdown the GHC RTS (run-time system).
   
We use "foreign import" specification to import foreign routines into our Haskell world, and "foreign export" to export Haskell routines into external world. Note that import statement creates new Haskell symbol (from external one), while export statement uses Haskell symbol previously defined. Techically speaking, both types of statements creates wrappers that converts naming and calling conventions from C to Haskell world or vice versa.
+
We use the <code>foreign import</code> declaration to import foreign routines into our Haskell world, and <code>foreign export</code> to export Haskell routines into the external world. Note that <code>import</code> creates a new Haskell symbol (from the external one), while <code>export</code> uses a Haskell symbol previously defined. Technically speaking, both types of declarations create a wrapper that converts the names and calling conventions from C to Haskell or vice versa.
   
=== All about "foreign" statement ===
+
=== All about the <code>foreign</code> declaration ===
   
"ccall" specifier in foreign statements means use of C (not C++ !) calling convention. This means that if you want to write external function in C++ (instead of C) you should add '''export "C"''' specification to its declaration - otherwise you'll get linking error. Let's rewrite out first example to use C++ instead of C:
+
The <code>ccall</code> specifier in foreign declarations means the use of the C (not C++ !) calling convention. This means that if you want to write the external function in C++ (instead of C) you should add <code>export "C"</code> specification to its declaration - otherwise you'll get linking errors. Let's rewrite our first example to use C++ instead of C:
   
prototypes.h:
+
''prototypes.h:''
 
<haskell>
 
<haskell>
 
#ifdef __cplusplus
 
#ifdef __cplusplus
Line 1,023: Line 1,066:
 
Compile it via:
 
Compile it via:
   
ghc --make main.hs evil.cpp
+
ghc --make main.hs vile.cpp
   
where evil.cpp is just renamed evil.c from the first example. Note that new prototypes.h is written in the manner that allows to compile it both as C and C++ code. When it's included from evil.cpp, it's compiled as C++ code. When GHC compiles main.hs via C compiler (enabled by -fvia-C option), it also includes prototypes.h but compiles it in C mode. It's why you need to specify .h files in "foreign" declarations - depending on Haskell compiler you use, these files may be included to check consistency of C and Haskell declarations.
+
where "vile.cpp" is just a renamed copy of "vile.c" from the first example. Note that the new "prototypes.h" is written to allow compiling it both as C and C++ code. When it's included from "vile.cpp", it's compiled as C++ code. When GHC compiles "main.hs" via the C compiler (enabled by the <code>-fvia-C</code> option), it also includes "prototypes.h" but compiles it in C mode. It's why you need to specify ".h" files in <code>foreign</code> declarations - depending on which Haskell compiler you use, these files may be included to check consistency of C and Haskell declarations.
   
Quoted part of foreign statement may also be used to import or export function under another name:
+
The quoted part of the foreign declaration may also be used to import or export a function under another name - for example,
   
 
<haskell>
 
<haskell>
Line 1,037: Line 1,080:
 
</haskell>
 
</haskell>
   
specifies that C function called CFunction will become known as Haskell function c_function, while Haskell function haskell_function will be known in C world as HaskellFunction. It's required when C name doesn't conform to Haskell naming requirements.
+
specifies that the C function called <code>CFunction</code> will become known as the Haskell function <code>c_function</code>, while the Haskell function <code>haskell_function</code> will be known in the C world as <code>HaskellFunction</code>. It's required when the C name doesn't conform to Haskell naming requirements.
   
Although Haskell FFI standard tells about many other call type conventions in addition to ccall - cplusplus, jvm, net - current Haskell implementations support only ccall and stdcall. Later, also known as Pascal calling convention, used to interface with WinAPI:
+
Although the Haskell FFI standard tells about many other calling conventions in addition to <code>ccall</code> (e.g. <code>cplusplus</code>, <code>jvm</code>, <code>net</code>) current Haskell implementations support only <code>ccall</code> and <code>stdcall</code>. The latter, also called the "Pascal" calling convention, is used to interface with WinAPI:
   
 
<haskell>
 
<haskell>
Line 1,046: Line 1,089:
 
</haskell>
 
</haskell>
   
And finally about safe/unsafe specifier: C function imported with "unsafe" keyword is called directly and Haskell runtime is stopped while C function is executed (when there are several OS threads executing Haskell program, only current OS thread is delayed). This call doesn't allowed to recursively enter into Haskell world by calling any Haskell function - the Haskell RTS is just not prepared to such event. OTOH, unsafe calls are as quick as calls in C world. It's ideal for "momentary" calls that quickly returns back to the caller.
+
And finally, about the <code>safe</code>/<code>unsafe</code> specifier: a C function imported with the <code>unsafe</code> keyword is called directly and the Haskell runtime is stopped while the C function is executed (when there are several OS threads executing the Haskell program, only the current OS thread is delayed). This call doesn't allow recursively entering into the Haskell world by calling any Haskell function - the Haskell RTS is just not prepared for such an event. However, <code>unsafe</code> calls are as quick as calls in the C world. It's ideal for "momentary" calls that quickly return back to the caller.
   
When "safe" is specified, C function called in safe environment - Haskell execution context is saved, so it's possible to call back to Haskell and, if C call executed too much time, other OS thread may be started to execute Haskell code (of course, in threads other that one called C code). This has its own price, though - around 1000 CPU ticks per call.
+
When <code>safe</code> is specified, the C function is called in a safe environment - the Haskell execution context is saved, so it's possible to call back to Haskell and, if the C call takes a long time, another OS thread may be started to execute Haskell code (of course, in threads other than the one that called the C code). This has its own price, though - around 1000 CPU ticks per call.
   
You can read more about interaction between FFI calls and Haskell concurrency in [7].
+
You can read more about interaction between FFI calls and Haskell concurrency in [[#readmore|[7]]].
   
 
=== Marshalling simple types ===
 
=== Marshalling simple types ===
   
Calling by itself is relatively easy, the real problem of interfacing languages with different data models is passing data between them. There is no even guarantee that Haskell Int is the same type as C int, Haskell Double is the same as C double and so on. While on *some* platforms they are the same and you can write throw-away programs relying on these, portability goal requires you to declare imported and exported functions using special types described in FFI standard which are guaranteed to correspond to C types. These are:
+
Calling by itself is relatively easy; the real problem of interfacing languages with different data models is passing data between them. In this case, there is no guarantee that Haskell's <code>Int</code> is represented in memory the same way as C's <code>int</code>, nor Haskell's <code>Double</code> the same as C's <code>double</code> and so on. While on ''some'' platforms they are the same and you can write throw-away programs relying on these, the goal of portability requires you to declare imported and exported functions using special types described in the FFI standard, which are guaranteed to correspond to C types. These are:
   
 
<haskell>
 
<haskell>
Line 1,070: Line 1,113:
 
</haskell>
 
</haskell>
   
Note that pure C functions (whose results are depend only on their arguments) are imported without IO in their return type. "const" C specifiers doesn't reflect in Haskell types, so appropriate compiler checks are not performed.
+
Note that pure C functions (those whose results depend only on their arguments) are imported without <code>IO</code> in their return type. The <code>const</code> specifier in C is not reflected in Haskell types, so appropriate compiler checks are not performed. <!-- What would these be? -->
   
All these numeric types are instances of the same classes as their Haskell cousins (Ord, Num, Show and so on), so you may perform calculations on these data directly. Alternatively, you may convert them to native Haskell types. It's very typical to write simple wrappers around imported and exported functions just to provide interfaces having native Haskell types:
+
All these numeric types are instances of the same classes as their Haskell cousins (<code>Ord</code>, <code>Num</code>, <code>Show</code> and so on), so you may perform calculations on these data directly. Alternatively, you may convert them to native Haskell types. It's very typical to write simple wrappers around imported and exported functions just to provide interfaces having native Haskell types:
   
 
<haskell>
 
<haskell>
Line 1,103: Line 1,146:
 
=== Marshalling composite types ===
 
=== Marshalling composite types ===
   
C array may be manipulated in Haskell as [http://haskell.org/haskellwiki/Arrays#StorableArray_.28module_Data.Array.Storable.29 StorableArray].
+
A C array may be manipulated in Haskell as [http://haskell.org/haskellwiki/Arrays#StorableArray_.28module_Data.Array.Storable.29 StorableArray].
   
There is no built-in support for marshalling C structures and using C constants in Haskell. These are implemented in c2hs preprocessor, though.
+
There is no built-in support for marshalling C structures and using C constants in Haskell. These are implemented in the c2hs preprocessor, though.
   
Binary marshalling (serializing) of data structures of any complexity is implemented in library Binary.
+
Binary marshalling (serializing) of data structures of any complexity is implemented in the library module "Binary".
   
 
=== Dynamic calls ===
 
=== Dynamic calls ===
   
 
=== DLLs ===
 
=== DLLs ===
''because i don't have experience of using DLLs, can someone write into this section? ultimately, we need to consider the following tasks:''
+
''because i don't have experience of using DLLs, can someone write into this section? Ultimately, we need to consider the following tasks:''
* using DLLs of 3rd-party libraries (such as ziplib)
+
* using DLLs of 3rd-party libraries (such as ''ziplib'')
* putting your own C code into DLL to use in Haskell
+
* putting your own C code into a DLL to use in Haskell
* putting Haskell code into DLL which may be called from C code
+
* putting Haskell code into a DLL which may be called from C code
   
== Dark side of IO monad ==
+
== '''The dark side of the I/O monad''' ==
=== unsafePerformIO ===
 
   
Programmers coming from an imperative language background often look for a way to execute IO actions inside a pure procedure. But what does this mean?
 
  +
Unless you are a systems developer, postgraduate CS student, or have alternate (and eminent!) verificable qualifications you should have '''no need whatsoever''' for this section - [https://stackoverflow.com/questions/9449239/unsafeperformio-in-threaded-applications-does-not-work here] is just one tiny example of what can go wrong if you don't know what you are doing. Look for other solutions!
Imagine that you're trying to write a procedure that reads the contents of a file with a given name, and you try to write it as a pure (non-IO) function:
 
  +
  +
=== '''unsafePerformIO''' ===
  +
Do you remember that initial attempt to define <code>getchar</code>?
  +
  +
<haskell>
  +
getchar :: Char
  +
  +
get2chars = [getchar, getchar]
  +
</haskell>
  +
  +
Let's also recall the problems arising from this ''faux''-definition:
  +
  +
# Because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "unnecessary" calls to <code>getchar</code> and use one returned value twice;
  +
# Even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two characters in the order in which they were read, or in the opposite order? Nothing in the definition of <code>get2chars</code> answers this question.
  +
  +
Despite these problems, programmers coming from an imperative language background often look for a way to do this - disguise one or more I/O actions as a pure definition. Having seen procedural entities similar in appearance to:
  +
  +
<haskell>
  +
void putchar(char c);
  +
</haskell>
  +
  +
the thought of just writing:
  +
  +
<haskell>
  +
putchar :: Char -> ()
  +
putchar c = ...
  +
</haskell>
  +
  +
would definitely be more appealing - for example, defining <code>readContents</code> as though it were a pure function:
   
 
<haskell>
 
<haskell>
Line 1,126: Line 1,197:
 
</haskell>
 
</haskell>
   
Defining readContents as a pure function will certainly simplify the code that uses it. But it will also create problems for the compiler:
+
will certainly simplify the code that uses it. However, those exact same problems are also lurking here:
   
  +
# Attempts to read the contents of files with the same name can be factored (''i.e.'' reduced to a single call) despite the fact that the file (or the current directory) can be changed between calls. Haskell considers all non-<code>IO</code> functions to be pure and feels free to merge multiple calls with the same parameters.
 
# This call is not inserted in a sequence of "world transformations", so the compiler doesn't know at what exact moment you want to execute this action. For example, if the file has one kind of contents at the beginning of the program and another at the end - which contents do you want to see? You have no idea when (or even if) this function is going to get invoked, because Haskell sees this function as pure and feels free to reorder the execution of any or all pure functions as needed.
 
# This call is not inserted in a sequence of "world transformations", so the compiler doesn't know at what exact moment you want to execute this action. For example, if the file has one kind of contents at the beginning of the program and another at the end - which contents do you want to see? You have no idea when (or even if) this function is going to get invoked, because Haskell sees this function as pure and feels free to reorder the execution of any or all pure functions as needed.
# Attempts to read the contents of files with the same name can be factored (''i.e.'' reduced to a single call) despite the fact that the file (or the current directory) can be changed between calls. Again, Haskell considers all non-IO functions to be pure and feels free to omit multiple calls with the same parameters.
 
   
So, implementing pure functions that interact with the Real World is
+
So, implementing supposedly-pure functions that interact with the '''Real World''' is
considered to be Bad Behavior. Good boys and girls never do it ;)
+
considered to be '''Bad Behavior'''. Nice programmers never do it ;-)
   
 
  +
Nevertheless, there are (semi-official) ways to use I/O actions inside
Nevertheless, there are (semi-official) ways to use IO actions inside
 
 
of pure functions. As you should remember this is prohibited by
 
of pure functions. As you should remember this is prohibited by
requiring the RealWorld "baton" in order to call an IO action. Pure functions don't have the baton, but there is a special "magic" procedure that produces this baton from nowhere, uses it to call an IO action and then throws the resulting "world" away! It's a little low-level magic :) This very special (and dangerous) procedure is:
+
requiring the <code>RealWorld</code> "baton" in order to call an I/O action. Pure functions don't have the baton, but there is a ''(ahem)'' "special" definition that produces this baton from nowhere, uses it to call an I/O action and then throws the resulting "world" away! It's a little low-level mirror-smoke. This particular (and dangerous) definition is:
   
 
<haskell>
 
<haskell>
Line 1,143: Line 1,213:
 
</haskell>
 
</haskell>
   
Let's look at its (possible) definition:
+
Let's look at how it ''could'' be defined:
   
 
<haskell>
 
<haskell>
Line 1,151: Line 1,221:
 
</haskell>
 
</haskell>
   
where 'createNewWorld' is an internal function producing a new value of
+
where <code>createNewWorld</code> is an private definition producing a new value of
the RealWorld type.
+
the <code>RealWorld</code> type.
   
Using unsafePerformIO, you can easily write pure functions that do
+
Using <code>unsafePerformIO</code>, you could easily write pure functions that do
 
I/O inside. But don't do this without a real need, and remember to
 
I/O inside. But don't do this without a real need, and remember to
follow this rule: the compiler doesn't know that you are cheating; it still
 
  +
follow this rule:
considers each non-IO function to be a pure one. Therefore, all the usual
 
optimization rules can (and will!) be applied to its execution. So
 
you must ensure that:
 
   
# The result of each call depends only on its arguments.
 
  +
* the compiler doesn't know that you are cheating; it still considers each non-<code>IO</code> function to be a pure one. Therefore, all the usual optimization rules can (and will!) be applied to its execution.
# You don't rely on side-effects of this function, which may be not executed if its results are not needed.
 
   
  +
So you must ensure that:
  +
  +
* The result of each call depends only on its arguments.
  +
* You don't rely on side-effects of this function, which may be not executed if its results are not needed.
   
 
Let's investigate this problem more deeply. Function evaluation in Haskell
 
Let's investigate this problem more deeply. Function evaluation in Haskell
is determined by a value's necessity - the language computes only the values that are really required to calculate the final result. But what does this mean with respect to the 'main' function? To "calculate the final world's" value, you need to perform all the intermediate IO actions that are included in the 'main' chain. By using 'unsafePerformIO' we call IO actions outside of this chain. What guarantee do we have that they will be run at all? None. The only time they will be run is if running them is required to compute the overall function result (which in turn should be required to perform some action in the
+
is determined by a value's necessity - the language computes only the values that are really required to calculate the final result. But what does this mean with respect to the <code>main</code> function? To "calculate the final world's" value, you need to perform all the intermediate I/O actions that are included in the <code>main</code> chain. By using <code>unsafePerformIO</code> we call I/O actions outside of this chain. What guarantee do we have that they will be run at all? None. The only time they will be run is if running them is required to compute the overall function result (which in turn should be required to perform some action in the <code>main</code> chain). This is an example of Haskell's evaluation-by-need strategy. Now you should clearly see the difference:
'main' chain). This is an example of Haskell's evaluation-by-need strategy. Now you should clearly see the difference:
 
   
- An IO action inside an IO procedure is guaranteed to execute as long as
 
  +
* An I/O action inside an I/O definition is guaranteed to execute as long as it is (directly or indirectly) inside the <code>main</code> chain - even when its result isn't used (because the implicit "world" value it returns ''will'' be used). You directly specify the order of the action's execution inside the I/O definition. Data dependencies are simulated via the implicit "world" values that are passed from each I/O action to the next.
it is (directly or indirectly) inside the 'main' chain - even when its result isn't used (because the implicit "world" value it returns ''will'' be used). You directly specify the order of the action's execution inside the IO procedure. Data dependencies are simulated via the implicit "world" values that are passed from each IO action to the next.
 
   
- An IO action inside 'unsafePerformIO' will be performed only if
 
  +
* An I/O action inside <code>unsafePerformIO</code> will be performed only if the result of this operation is really used. The evaluation order is not guaranteed and you should not rely on it (except when you're sure about whatever data dependencies may exist).
result of this operation is really used. The evaluation order is not
 
guaranteed and you should not rely on it (except when you're sure about
 
whatever data dependencies may exist).
 
   
 
  +
I should also say that inside the <code>unsafePerformIO</code> call you can organize
I should also say that inside 'unsafePerformIO' call you can organize
 
  +
a small internal chain of I/O actions with the help of the same binding
a small internal chain of IO actions with the help of the same binding
 
  +
operators and/or <code>do</code> syntactic sugar we've seen above. So here's how we'd rewrite our previous (pure!) definition of <code>one</code> using <code>unsafePerformIO</code>:
operators and/or 'do' syntactic sugar we've seen above. For example, here's a particularly convoluted way to compute the integer that comes after zero:
 
   
 
<haskell>
 
<haskell>
one :: Int
+
one :: Integer
 
one = unsafePerformIO $ do var <- newIORef 0
 
one = unsafePerformIO $ do var <- newIORef 0
 
modifyIORef var (+1)
 
modifyIORef var (+1)
Line 1,188: Line 1,253:
 
</haskell>
 
</haskell>
   
and in this case ALL the operations in this chain will be performed as
+
and in this case ''all'' the operations in this chain will be performed as
long as the result of the 'unsafePerformIO' call is needed. To ensure this,
+
long as the result of the <code>unsafePerformIO</code> call is needed. To ensure this,
the actual 'unsafePerformIO' implementation evaluates the "world" returned
+
the actual <code>unsafePerformIO</code> implementation evaluates the "world" returned
by the 'action':
+
by the <code>action</code>:
   
 
<haskell>
 
<haskell>
Line 1,198: Line 1,263:
 
</haskell>
 
</haskell>
   
(The 'seq' operation strictly evaluates its first argument before
+
(The <code>seq</code> operation strictly evaluates its first argument before
returning the value of the second one).
+
returning the value of the second one [[#readmore|[8]]]).
 
   
=== inlinePerformIO ===
+
=== '''inlinePerformIO''' ===
   
inlinePerformIO has the same definition as unsafePerformIO but with addition of INLINE pragma:
+
<code>inlinePerformIO</code> has the same definition as <code>unsafePerformIO</code> but with the addition of an <code>INLINE</code> pragma:
 
<haskell>
 
<haskell>
 
-- | Just like unsafePerformIO, but we inline it. Big performance gains as
 
-- | Just like unsafePerformIO, but we inline it. Big performance gains as
Line 1,213: Line 1,278:
 
</haskell>
 
</haskell>
   
Semantically inlinePerformIO = unsafePerformIO
+
Semantically <code>inlinePerformIO</code> = <code>unsafePerformIO</code>
 
in as much as either of those have any semantics at all.
 
in as much as either of those have any semantics at all.
   
The difference of course is that inlinePerformIO is even less safe than
+
The difference of course is that <code>inlinePerformIO</code> is even less safe than
unsafePerformIO. While ghc will try not to duplicate or common up
+
<code>unsafePerformIO</code>. While ghc will try not to duplicate or common up
different uses of unsafePerformIO, we aggressively inline
+
different uses of <code>unsafePerformIO</code>, we aggressively inline
inlinePerformIO. So you can really only use it where the IO content is
+
<code>inlinePerformIO</code>. So you can really only use it where the I/O content is
 
really properly pure, like reading from an immutable memory buffer (as
 
really properly pure, like reading from an immutable memory buffer (as
in the case of ByteStrings). However things like allocating new buffers
+
in the case of <code>ByteString</code>s). However things like allocating new buffers
should not be done inside inlinePerformIO since that can easily be
+
should not be done inside <code>inlinePerformIO</code> since that can easily be
 
floated out and performed just once for the whole program, so you end up
 
floated out and performed just once for the whole program, so you end up
 
with many things sharing the same buffer, which would be bad.
 
with many things sharing the same buffer, which would be bad.
   
So the rule of thumb is that IO things wrapped in unsafePerformIO have
+
So the rule of thumb is that I/O actions wrapped in <code>unsafePerformIO</code> have
to be externally pure while with inlinePerformIO it has to be really
+
to be externally pure while with <code>inlinePerformIO</code> it has to be really,
really pure or it'll all go horribly wrong.
+
''really'' pure or it'll all go horribly wrong.
   
 
That said, here's some really hairy code. This should frighten any pure
 
That said, here's some really hairy code. This should frighten any pure
Line 1,237: Line 1,302:
 
write !n body = Put $ \c buf@(Buffer fp o u l) ->
 
write !n body = Put $ \c buf@(Buffer fp o u l) ->
 
if n <= l
 
if n <= l
then write' c fp o u l
+
then write</code> c fp o u l
else write' (flushOld c n fp o u) (newBuffer c n) 0 0 0
+
else write</code> (flushOld c n fp o u) (newBuffer c n) 0 0 0
   
where {-# NOINLINE write' #-}
+
where {-# NOINLINE write</code> #-}
write' c !fp !o !u !l =
+
write</code> c !fp !o !u !l =
 
-- warning: this is a tad hardcore
 
-- warning: this is a tad hardcore
 
inlinePerformIO
 
inlinePerformIO
Line 1,256: Line 1,321:
 
This does not adhere to my rule of thumb above. Don't ask exactly why we
 
This does not adhere to my rule of thumb above. Don't ask exactly why we
 
claim it's safe :-) (and if anyone really wants to know, ask Ross
 
claim it's safe :-) (and if anyone really wants to know, ask Ross
Paterson who did it first in the Builder monoid)
+
Paterson who did it first in the <code>Builder</code> monoid)
   
=== unsafeInterleaveIO ===
+
=== '''unsafeInterleaveIO''' ===
   
But there is an even stranger operation called 'unsafeInterleaveIO' that
+
But there is an even stranger operation:
gets the "official baton", makes its own pirate copy, and then runs
 
an "illegal" relay-race in parallel with the main one! I can't talk further
 
about its behavior without causing grief and indignation, so it's no surprise
 
that this operation is widely used in countries that are hotbeds of software piracy such as Russia and China! ;) Don't even ask me - I won't say anything more about this dirty trick I use all the time ;)
 
   
One can use unsafePerformIO (not unsafeInterleaveIO) to perform I/O
 
  +
<haskell>
operations not in predefined order but by demand. For example, the
 
  +
unsafeInterleaveIO :: IO a -> IO a
following code:
 
  +
</haskell>
  +
  +
Don't let that type signature fool you - <code>unsafeInterleaveIO</code> also uses
  +
a dubiously-acquired baton which it uses to set up an underground
  +
relay-race for its unsuspecting parameter. If it happens, this seedy race
  +
then occurs alongside the offical <code>main</code> relay-race - if they collide,
  +
things will get ugly!
  +
  +
So how does <code>unsafeInterleaveIO</code> get that bootlegged baton? Typically by
  +
making a forgery of the offical one to keep for itself - it can do
  +
this because the I/O action <code>unsafeInterleaveIO</code> returns will be
  +
handed the offical baton in the <code>main</code> relay-race. But one
  +
miscreant realised there was a simpler way:
  +
  +
<haskell>
  +
unsafeInterleaveIO :: IO a -> IO a
  +
unsafeInterleaveIO a = return (unsafePerformIO a)
  +
</haskell>
  +
  +
Why bother with counterfeit copies of batons if you can just make them up?
  +
  +
At least you have some appreciation as to why <code>unsafeInterleaveIO</code> is, well
  +
'''unsafe!''' Just don't ask - to talk further is bound to cause grief and
  +
indignation. I won't say anything more about this ruffian I...use
  +
all the time (darn it!)
  +
  +
One can use <code>unsafePerformIO</code> (not <code>unsafeInterleaveIO</code>) to perform I/O
  +
operations not in some predefined order but by demand. For example, the following code:
   
 
<haskell>
 
<haskell>
Line 1,271: Line 1,363:
 
</haskell>
 
</haskell>
   
will perform getChar I/O call only when value of c is really required
+
will perform the <code>getChar</code> I/O call only when the value of <code>c</code> is really required
by code, i.e. it this call will be performed lazily as any usual
+
by the calling code, i.e. it this call will be performed lazily like any regular Haskell computation.
Haskell computation.
 
   
 
Now imagine the following code:
 
Now imagine the following code:
Line 1,281: Line 1,373:
 
</haskell>
 
</haskell>
   
Three chars inside this list will be computed on demand too, and this
+
The three characters inside this list will be computed on demand too, and this
 
means that their values will depend on the order they are consumed. It
 
means that their values will depend on the order they are consumed. It
is not that we usually need :)
+
is not what we usually want.
   
  +
<code>unsafeInterleaveIO</code> solves this problem - it performs I/O only on
  +
demand but allows you to define the exact ''internal'' execution order for parts
  +
of your data structure. It is why I wrote that <code>unsafeInterleaveIO</code> makes
  +
an illegal copy of the baton:
   
unsafeInterleaveIO solves this problem - it performs I/O only on
 
  +
* <code>unsafeInterleaveIO</code> accepts an I/O action as a parameter and returns another I/O action as the result:
demand but allows to define exact *internal* execution order for parts
 
of your datastructure. It is why I wrote that unsafeInterleaveIO makes
 
illegal copy of baton :)
 
   
First, unsafeInterleaveIO has (IO a) action as a parameter and returns
 
value of type 'a':
 
   
 
<haskell>
 
<haskell>
Line 1,298: Line 1,389:
 
</haskell>
 
</haskell>
   
Second, unsafeInterleaveIO don't perform any action immediately, it
+
* <code>unsafeInterleaveIO</code> doesn't perform any action immediately, it only creates a closure of type <code>a</code> which upon being needed will perform the action specified as the parameter.
only creates a box of type 'a' which on requesting this value will
 
perform action specified as a parameter.
 
   
Third, this action by itself may compute the whole value immediately
+
* this action by itself may compute the whole value immediately...or use <code>unsafeInterleaveIO</code> again to defer calculation of some sub-components:
or... use unsafeInterleaveIO again to defer calculation of some
 
sub-components:
 
   
 
<haskell>
 
<haskell>
Line 1,309: Line 1,400:
 
</haskell>
 
</haskell>
   
This code will be executed only at the moment when value of str is
+
This code will be executed only at the moment when the value of <code>str</code> is
really demanded. In this moment, getChar will be performed (with
+
really demanded. In this moment, <code>getChar</code> will be performed (with its
result assigned to c) and one more lazy IO box will be created - for s.
+
result assigned to <code>c</code>) and a new lazy-I/O closure will be created - for <code>s</code>.
This box again contains link to the myGetContents call
+
This new closure also contains a link to a <code>myGetContents</code> call.
   
Then, list cell returned that contains one char read and link to
+
Then the list cell is returned. It contains <code>Char</code> that was just read and a link to
myGetContents call as a way to compute rest of the list. Only at the
+
another <code>myGetContents</code> call as a way to compute rest of the list. Only at the
moment when next value in list required, this operation will be
+
moment when the next value in the list is required will this operation be performed again.
performed again
 
   
As a final result, we get inability to read second char in list before
+
As a final result, we can postpone the read of the second <code>Char</code> in the list before
first one, but lazy character of reading in whole. bingo!
+
the first one, but have lazy reading of characters as a whole - bingo!
   
   
PS: of course, actual code should include EOF checking. also note that
+
PS: of course, actual code should include EOF checking; also note that
you can read many chars/records at each call:
+
you can read multiple characters/records at each call:
   
 
<haskell>
 
<haskell>
Line 1,332: Line 1,423:
 
</haskell>
 
</haskell>
   
== A safer approach: the ST monad ==
 
  +
and we can rewrite <code>myGetContents</code> to avoid needing to
  +
use <code>unsafeInterleaveIO</code> where it's called:
   
We said earlier that we can use unsafePerformIO to perform computations that are totally pure but nevertheless interact with the Real World in some way. There is, however, a better way! One that remains totally pure and yet allows the use of references, arrays, and so on -- and it's done using, you guessed it, type magic. This is the ST monad.
 
 
The ST monad's version of unsafePerformIO is called runST, and it has a very unusual type.
 
 
<haskell>
 
<haskell>
runST :: (forall s . ST s a) -> a
 
  +
myGetContents = unsafeInterleaveIO $ do
</haskell>
 
  +
c <- replicateM 512 getChar
 
  +
s <- myGetContents
The s variable in the ST monad is the state type. Moreover, all the fun mutable stuff available in the ST monad is quantified over s:
 
  +
return (c++s)
<haskell>
 
newSTRef :: a -> ST s (STRef s a)
 
newArray_ :: Ix i => (i, i) -> ST s (STArray s i e)
 
</haskell>
 
 
So why does runST have such a funky type? Let's see what would happen if we wrote
 
<haskell>
 
makeSTRef :: a -> STRef s a
 
makeSTRef a = runST (newSTRef a)
 
</haskell>
 
This fails, because newSTRef a doesn't work for all state types s -- it only works for the s from the return type STRef s a.
 
 
This is all sort of wacky, but the result is that you can only run an ST computation where the output type is functionally pure, and makes no references to the internal mutable state of the computation. The ST monad doesn't have access to I/O operations like writing to the console, either -- only references, arrays, and suchlike that come in handy for pure computations.
 
 
Important note -- the state type doesn't actually mean anything. We never have a value of type s, for instance. It's just a way of getting the type system to do the work of ensuring purity for us, with smoke and mirrors.
 
 
It's really just type system magic: secretly, on the inside, runST runs a computation with the real world baton just like unsafePerformIO. Their internal implementations are almost identical: in fact, there's a function
 
<haskell>
 
stToIO :: ST RealWorld a -> IO a
 
</haskell>
 
 
The difference is that ST uses type system magic to forbid unsafe behavior like extracting mutable objects from their safe ST wrapping, but allowing purely functional outputs to be performed with all the handy access to mutable references and arrays.
 
 
So here's how we'd rewrite our function using unsafePerformIO from above:
 
 
<haskell>
 
oneST :: ST s Int -- note that this works correctly for any s
 
oneST = do var <- newSTRef 0
 
modifySTRef var (+1)
 
readSTRef var
 
 
one :: Int
 
one = runST oneST
 
 
</haskell>
 
</haskell>
   
Line 1,380: Line 1,436:
   
 
A little disclaimer: I should say that I'm not describing
 
A little disclaimer: I should say that I'm not describing
here exactly what a monad is (I don't even completely understand it myself) and my explanation shows only one _possible_ way to implement the IO monad in
+
here exactly what a monad is (I don't even completely understand it myself) and my explanation shows only one ''possible'' way to implement the I/O monad in
Haskell. For example, the hbc Haskell compiler implements IO monad via
+
Haskell. For example, the hbc compiler and the Hugs interpreter
continuations. I also haven't said anything about exception handling,
+
implements the I/O monad via continuations [[#readmore|[9]]]. I also haven't said anything about
which is a natural part of the "monad" concept. You can read the "All About
+
exception handling, which is a natural part of the "monad" concept. You can
Monads" guide to learn more about these topics.
+
read the [[All About Monads]] guide to learn more about these topics.
   
But there is some good news: first, the IO monad understanding you've just acquired will work with any implementation and with many other monads. You just can't work with RealWorld
 
  +
But there is some good news:
values directly.
 
   
Second, the IO monad implementation described here is really used in the GHC,
 
  +
* the I/O monad understanding you've just acquired will work with any implementation and with many other monads. You just can't work with <code>RealWorld</code> values directly.
yhc/nhc (Hugs/jhc, too?) compilers. Here is the actual IO definition
 
from the GHC sources:
 
   
  +
* the I/O monad implementation described here is similar to what GHC uses:
 
<haskell>
 
<haskell>
 
newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #))
 
newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #))
 
</haskell>
 
</haskell>
   
It uses the "State# RealWorld" type instead of our RealWorld, it uses the "(# #)" strict tuple for optimization, and it adds an IO data constructor
+
It uses the <code>State# RealWorld</code> type instead of our <code>RealWorld</code>, it uses the <code>(# ... #)</code> strict tuple for optimization, and it adds an <code>IO</code> data constructor
around the type. Nevertheless, there are no significant changes from the standpoint of our explanation. Knowing the principle of "chaining" IO actions via fake "state of the world" values, you can now easily understand and write low-level implementations of GHC I/O operations.
+
around the type. Nevertheless, there are no significant changes from the standpoint of our explanation. Knowing the principle of "chaining" I/O actions via fake "state of the world" values, you can now more easily understand and write low-level implementations of GHC I/O operations.
   
  +
Of course, other compilers e.g. yhc/nhc (jhc, too?) define <code>IO</code> in other ways.
   
 
=== The [[Yhc]]/nhc98 implementation ===
 
=== The [[Yhc]]/nhc98 implementation ===
Line 1,408: Line 1,463:
 
</haskell>
 
</haskell>
   
This implementation makes the "World" disappear somewhat, and returns Either a
+
This implementation makes the <code>World</code> disappear somewhat[[#readmore|[10]]], and returns <code>Either</code> a
result of type "a", or if an error occurs then "IOError". The lack of the World on the right-hand side of the function can only be done because the compiler knows special things about the IO type, and won't overoptimise it.
+
result of type <code>a</code>, or if an error occurs then <code>IOError</code>. The lack of the <code>World</code> on the right-hand side of the function can only be done because the compiler knows special things about the <code>IO</code> type, and won't overoptimise it.
   
   
== Further reading ==
 
   
[1] This tutorial is largely based on the Simon Peyton Jones' paper [http://research.microsoft.com/%7Esimonpj/Papers/marktoberdorf Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell]. I hope that my tutorial improves his original explanation of the Haskell I/O system and brings it closer to the point of view of beginning Haskell programmers. But if you need to learn about concurrency, exceptions and FFI in Haskell/GHC, the original paper is the best source of information.
 
  +
== <span id="readmore"></span>Further reading ==
  +
  +
[1] This tutorial is largely based on Simon Peyton Jones's paper [https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.9123&rep=rep1&type=pdf Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell]. I hope that my tutorial improves his original explanation of the Haskell I/O system and brings it closer to the point of view of new Haskell programmers. But if you need to learn about concurrency, exceptions and FFI in Haskell/GHC, the original paper is the best source of information.
   
 
[2] You can find more information about concurrency, FFI and STM at the [[GHC/Concurrency#Starting points]] page.
 
[2] You can find more information about concurrency, FFI and STM at the [[GHC/Concurrency#Starting points]] page.
Line 1,420: Line 1,476:
 
[3] The [[Arrays]] page contains exhaustive explanations about using mutable arrays.
 
[3] The [[Arrays]] page contains exhaustive explanations about using mutable arrays.
   
[4] Look also at the [[Tutorials#Using_monads|Using monads]] page, which contains tutorials and papers really describing these mysterious monads :)
+
[4] Look also at the [[Tutorials#Using_monads|Using monads]] page, which contains tutorials and papers really describing these mysterious monads.
   
 
[5] An explanation of the basic monad functions, with examples, can be found in the reference guide [http://members.chello.nl/hjgtuyl/tourdemonad.html A tour of the Haskell Monad functions], by Henk-Jan van Tuyl.
 
[5] An explanation of the basic monad functions, with examples, can be found in the reference guide [http://members.chello.nl/hjgtuyl/tourdemonad.html A tour of the Haskell Monad functions], by Henk-Jan van Tuyl.
   
[6] Official FFI specifiacations can be found on the page [http://www.cse.unsw.edu.au/~chak/haskell/ffi/ The Haskell 98 Foreign Function Interface 1.0: An Addendum to the Haskell 98 Report]
+
[6] Official FFI specifications can be found on the page [http://www.cse.unsw.edu.au/~chak/haskell/ffi/ The Haskell 98 Foreign Function Interface 1.0: An Addendum to the Haskell 98 Report]
   
 
[7] Using FFI in multithreaded programs described in paper [http://www.haskell.org/~simonmar/bib/concffi04_abstract.html Extending the Haskell Foreign Function Interface with Concurrency]
 
[7] Using FFI in multithreaded programs described in paper [http://www.haskell.org/~simonmar/bib/concffi04_abstract.html Extending the Haskell Foreign Function Interface with Concurrency]
  +
  +
[8] This particular behaviour is not a requirement of Haskell 2010, so the operation of <code>seq</code> may differ between various Haskell implementations - if you're not sure, staying within the I/O monad is the safest option.
  +
  +
[9] [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.3579&rep=rep1&type=pdf How to Declare an Imperative] by Phil Wadler provides an explanation of how this can be done.
  +
  +
[10] The <code>RealWorld</code> type can even be replaced e.g. <span style="color:darkred;">Functional I/O Using System Tokens</span> by Lennart Augustsson.
   
 
Do you have more questions? Ask in the [http://www.haskell.org/mailman/listinfo/haskell-cafe haskell-cafe mailing list].
 
Do you have more questions? Ask in the [http://www.haskell.org/mailman/listinfo/haskell-cafe haskell-cafe mailing list].
Line 1,435: Line 1,497:
   
 
Topics:
 
Topics:
* fixIO and 'mdo'
+
* <code>fixIO</code> and <code>mdo</code>
* Q monad
+
* <code>Q</code> monad
   
 
Questions:
 
Questions:
* split '>>='/'>>'/return section and 'do' section, more examples of using binding operators
+
* split <code>(>>=)</code>/<code>(>>)</code>/return section and <code>do</code> section, more examples of using binding operators
* IORef detailed explanation (==const*), usage examples, syntax sugar, unboxed refs
+
* <code>IORef</code> detailed explanation (==<code>const*</code>), usage examples, syntax sugar, unboxed refs
  +
* explanation of how the actual data "in" mutable references are inside <code>RealWorld</code>, rather than inside the references themselves (<code>IORef</code>, <code>IOArray</code> & co.)
 
* control structures developing - much more examples
 
* control structures developing - much more examples
* unsafePerformIO usage examples: global variable, ByteString, other examples
+
* <code>unsafePerformIO</code> usage examples: global variable, <code>ByteString</code>, other examples
* actual GHC implementation - how to write low-level routines on example of newIORef implementation
+
* how <code>unsafeInterLeaveIO</code> can be seen as a kind of concurrency, and therefore isn't so unsafe (unlike <code>unsafeInterleaveST</code> which really is unsafe)
  +
* discussion about different senses of <code>safe</code>/<code>unsafe</code> (like breaking equational reasoning vs. invoking undefined behaviour (so can corrupt the run-time system))
  +
* actual GHC implementation - how to write low-level routines based on example of <code>newIORef</code> implementation
   
This manual is collective work, so feel free to add more information to it yourself. The final goal is to collectively develop a comprehensive manual for using the IO monad.
+
This manual is collective work, so feel free to add more information to it yourself. The final goal is to collectively develop a comprehensive manual for using the I/O monad.
   
 
----
 
----

Latest revision as of 09:14, 12 January 2021

Haskell I/O can be a source of confusion and surprises for new Haskellers - if that's you, a good place to start is the Introduction to IO which can help you learn the basics (e.g. the syntax of I/O expressions) before continuing on.


While simple I/O code in Haskell looks very similar to its equivalents in imperative languages, attempts to write somewhat more complex code often result in a total mess. This is because Haskell I/O is really very different in how it actually works.

The following text is an attempt to explain the details of Haskell I/O implementations. This explanation should help you eventually learn all the smart I/O tips. Moreover, I've added a detailed explanation of various traps you might encounter along the way. After reading this text, you will be well on your way towards mastering I/O in Haskell.

Haskell is a pure language

Haskell is a pure language and even the I/O system can't break this purity. Being pure means that the result of any function call is fully determined by its arguments. Procedural entities like rand() or getchar() in C, which return different results on each call, are simply impossible to write in Haskell. Moreover, Haskell functions can't have side effects, which means that they can't effect any changes to the "real world", like changing files, writing to the screen, printing, sending data over the network, and so on. These two restrictions together mean that any function call can be replaced by the result of a previous call with the same parameters, and the language guarantees that all these rearrangements will not change the program result!

Let's compare this to C: optimizing C compilers try to guess which functions have no side effects and don't depend on mutable global variables. If this guess is wrong, an optimization can change the program's semantics! To avoid this kind of disaster, C optimizers are conservative in their guesses or require hints from the programmer about the purity of functions.

Compared to an optimizing C compiler, a Haskell compiler is a set of pure mathematical transformations. This results in much better high-level optimization facilities. Moreover, pure mathematical computations can be much more easily divided into several threads that may be executed in parallel, which is increasingly important in these days of multi-core CPUs. Finally, pure computations are less error-prone and easier to verify, which adds to Haskell's robustness and to the speed of program development using Haskell.

Haskell's purity allows the compiler to call only functions whose results are really required to calculate the final value of a top-level function (e.g. main) - this is called lazy evaluation. It's a great thing for pure mathematical computations, but how about I/O actions? A function like

putStrLn "Press any key to begin formatting"

can't return any meaningful result value, so how can we ensure that the compiler will not omit or reorder its execution? And in general: How we can work with stateful algorithms and side effects in an entirely lazy language? This question has had many different solutions proposed while Haskell was developed (see History of Haskell), with one solution eventually making its way into the current standard.

I/O in Haskell, simplified

Let's imagine that we want to implement the well-known getchar I/O operation in Haskell. What type should it have? Let's try:

getchar :: Char

get2chars = [getchar, getchar]

What will we get with getchar having just the Char type? You can see one problem in the definition of get2chars immediately:

  • because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "unnecessary" calls to getchar and use one returned value twice:


get2chars = let x = getchar in [x, x]  -- this should be a legitimate optimisation!

How can this problem be solved from the programmer's perspective? Let's introduce a fake parameter of getchar to make each call "different" from the compiler's point of view:

getchar :: Int -> Char

get2chars = [getchar 1, getchar 2]

Right away, this solves the first problem mentioned above - now the compiler will make two calls because it sees that the calls have different parameters. But there's another problem:

  • even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two characters in the order in which they were read, or in the opposite order? Nothing in the definition of get2chars answers this question.

We need to give the compiler some clue to determine which function it should call first. The Haskell language doesn't provide any way to specify the sequence needed to evaluate getchar 1 and getchar 2 - except for data dependencies! How about adding an artificial data dependency which prevents evaluation of the second getchar before the first one? In order to achieve this, we will return an additional fake result from getchar that will be used as a parameter for the next getchar call:

getchar :: Int -> (Char, Int)

get2chars _ = [a, b]  where (a, i) = getchar 1
                            (b, _) = getchar i

So far so good - now we can guarantee that a is read before b because reading b needs the value (i) that is returned by reading a!

We've added a fake parameter to get2chars but the problem is that the Haskell compiler is too smart! It can believe that the external getchar function is really dependent on its parameter but for get2chars it will see that we're just cheating because we throw it away! Therefore it won't feel obliged to execute the calls in the order we want.

How can we fix this? How about passing this fake parameter to the getchar function? In this case the compiler can't guess that it is really unused.

get2chars i0 = [a, b]  where (a, i1) = getchar i0
                             (b, i2) = getchar i1

Furthermore, get2chars has the same purity problems as the getchar function. If you need to call it two times, you need a way to describe the order of these calls. Consider this:

get4chars = [get2chars 1, get2chars 2]  -- order of calls to 'get2chars' isn't defined

We already know how to deal with this problem: get2chars should also return some fake value that can be used to order calls:

get2chars :: Int -> (String, Int)

get4chars i0 = (a++b)  where (a, i1) = get2chars i0
                             (b, i2) = get2chars i1

But what should the fake return value of get2chars be? If we use some integer constant, the excessively smart Haskell compiler will guess that we're cheating again. What about returning the value returned by getchar? See:

get2chars :: Int -> (String, Int)
get2chars i0 = ([a, b], i2)  where (a, i1) = getchar i0
                                   (b, i2) = getchar i1

While that does work, it's error-prone:

get2chars :: Int -> (String, Int)
get2chars i0 = ([a, b], i2)  where (a, i1) = getchar i2  -- this might take a while...
                                   (b, i2) = getchar i1

Using individual let-bindings is an improvement:

get2chars :: Int -> (String, Int)
get2chars i0 = let (a, i1) = getchar i2 in  -- error: i2 is undefined!
               let (b, i2) = getchar i1 in
               ([a, b], i2)

but only a minor one:

get2chars :: Int -> (String, Int)
get2chars i0 = let (a, i1) = getchar i0 in
               let (b, i2) = getchar i2 in  -- here we go again...
               ([a, b], i2)

So how in Haskell shall we prevent such mistakes from happening? With a monad!

What is a monad?

But what is a monad? For Haskell, it's a three-way partnership between:

  • a type: M a
  • an operator unit(M) :: a -> M a
  • an operator bind(M) :: M a -> (a -> M b) -> M b

where unit(M) and bind(M) satisify the monad laws.

This would translate literally into Haskell as:

class Monad m where
    unit :: a -> m a
    bind :: m a -> (a -> m b) -> m b

For now, we'll just define unit and bind directly - no type classes.

So how does something so vague abstract help us with I/O? Because this abstraction allows us to hide the manipulation of all those fake values - the ones we've been using to maintain the correct sequence of evaluation. We just need a suitable type:

type IO' a =  Int -> (a, Int)

and appropriate defintions for unit and bind:

unit       :: a -> IO' a
unit x     =  \i0 -> (x, i0)

bind       :: IO' a -> (a -> IO' b) -> IO' b
bind m k   =  \i0 -> let (x, i1) =  m i0 in
                     let (y, i2) =  k x i1 in
                     (y, i2)

Now for some extra changes to getchar and get2chars:

getchar      :: IO' Char    {-  = Int -> (Char, Int)  -}

get2chars    :: IO' String  {-  = Int -> (String, Int)  -}

get2chars =  \i0 -> let (a, i1) =  getchar i0 in
                    let (b, i2) =  getchar i1 in
                    let r       =  [a, b] in
                    (r, i2)

before we use unit and bind:

getchar   :: IO' Char

get2chars :: IO' String
get2chars =  getchar `bind` \a ->
             getchar `bind` \b ->
             unit [a, b]

We no longer have to mess up with those fake values directly! We just need to be sure that all the operations on I/O actions like unit and bind use them correctly. We can then make IO', unit, bind and (in this example) getchar into an abstract data type and just use those abstract I/O operations instead - only the Haskell implementation (e.g. compilers like ghc or jhc) needs to know how I/O actions actually work.

So there you have it - a miniature monadic I/O system in Haskell!

Running with the RealWorld

Warning: The following story about I/O is incorrect in that it cannot actually explain some important aspects of I/O (including interaction and concurrency). However, some people find it useful to begin developing an understanding.

The main Haskell function has the type:

main :: RealWorld -> ((), RealWorld)

where RealWorld is a fake type used instead of our Int. It's something like the baton passed in a relay race. When main calls some I/O action, it passes the RealWorld it received as a parameter. All I/O actions have similar types involving RealWorld as a parameter and result. To be exact, IO is a type synonym defined in the following way:

type IO a  =  RealWorld -> (a, RealWorld)

So, main just has type IO (), getChar has type IO Char and so on. You can think of the type IO Char as meaning "take the current RealWorld, do something to it, and return a Char and a (possibly changed) RealWorld". Let's look at main calling getChar two times:

getChar :: RealWorld -> (Char, RealWorld)

main :: RealWorld -> ((), RealWorld)
main world0 = let (a, world1) = getChar world0
                  (b, world2) = getChar world1
              in ((), world2)

Look at this closely: main passes the "world" it received to the first getChar. This getChar returns some new value of type RealWorld that gets used in the next call. Finally, main returns the "world" it got from the second getChar.

  • Is it possible here to omit any call of getChar if the Char it read is not used? No: we need to return the "world" that is the result of the second getChar and this in turn requires the "world" returned from the first getChar.
  • Is it possible to reorder the getChar calls? No: the second getChar can't be called before the first one because it uses the "world" returned from the first call.
  • Is it possible to duplicate calls? In Haskell semantics - yes, but real compilers never duplicate work in such simple cases (otherwise, the programs generated will not have any speed guarantees).

As we already said, RealWorld values are used like a baton which gets passed between all routines called by main in strict order. Inside each routine called, RealWorld values are used in the same way. Overall, in order to "compute" the world to be returned from main, we should perform each I/O action that is called from main, directly or indirectly. This means that each action inserted in the chain will be performed just at the moment (relative to the other I/O actions) when we intended it to be called. Let's consider the following program:

main = do a <- ask "What is your name?"
          b <- ask "How old are you?"
          return ()

ask s = do putStr s
           readLn

Now you have enough knowledge to rewrite it in a low-level way and check that each operation that should be performed will really be performed with the arguments it should have and in the order we expect.

But what about conditional execution? No problem. Let's define the well-known when operation:

when :: Bool -> IO () -> IO ()
when condition action world =
    if condition
      then action world
      else ((), world)

As you can see, we can easily include or exclude from the execution chain I/O actions depending on the data values. If condition will be False on the call of when, action will never be called because real Haskell compilers, again, never call functions whose results are not required to calculate the final result (i.e. here, the final "world" value of main).

Loops and more complex control structures can be implemented in the same way. Try it as an exercise!

Finally, you may want to know how much passing these RealWorld values around the program costs. It's free! These fake values exist solely for the compiler while it analyzes and optimizes the code, but when it gets to assembly code generation, it notices that this type is like (), so all these parameters and result values can be omitted from the final generated code - they're not needed any more!

(>>=) and do notation

All beginners (including me) start by thinking that do is some super-awesome statement that executes I/O actions. That's wrong - do is just syntactic sugar that simplifies the writing of definitions that use I/O (and also other monads, but that's beyond the scope of this tutorial). do notation eventually gets translated to a series of I/O actions passing "world" values around like we've manually written above. This simplifies the gluing of several I/O actions together.

You don't need to use do for just one action; for example,

  main = do putStr "Hello!"

is desugared to:

  main = putStr "Hello!"

Let's examine how to desugar a do-expression with multiple actions in the following example:

main = do putStr "What is your name?"
          putStr "How old are you?"
          putStr "Nice day!"

The do-expression here just joins several I/O actions that should be performed sequentially. It's translated to sequential applications of one of the so-called "binding operators", namely (>>):

main = (putStr "What is your name?")
       >> ( (putStr "How old are you?")
            >> (putStr "Nice day!")
          )

This binding operator just combines two I/O actions, executing them sequentially by passing the "world" between them:

(>>) :: IO a -> IO b -> IO b
(action1 >> action2) world0 =
   let (a, world1) = action1 world0
       (b, world2) = action2 world1
   in (b, world2)

If defining operators this way looks strange to you, read this definition as follows:

action1 >> action2 = action
  where
    action world0 = let (a, world1) = action1 world0
                        (b, world2) = action2 world1
                    in (b, world2)

Now you can substitute the definition of (>>) at the places of its usage and check that program constructed by the do desugaring is actually the same as we could write by manually manipulating "world" values.

A more complex example involves the binding of variables using <-:

main = do a <- readLn
          print a

This code is desugared into:

main = readLn
       >>= (\a -> print a)

where (>>=) corresponds to the bind operation in our miniature I/O system.

As you should remember, the (>>) binding operator silently ignores the value of its first action and returns as an overall result the result of its second action only. On the other hand, the (>>=) binding operator (note the extra = at the end) allows us to use the result of its first action - it gets passed as an additional parameter to the second one! Look at the definition:

(>>=) :: IO a -> (a -> IO b) -> IO b
(action >>= reaction) world0 =
   let (a, world1) = action world0
       (b, world2) = reaction a world1
   in (b, world2)
  • What does the type of reaction - namely a -> IO b - mean? By substituting the IO definition, we get a -> RealWorld -> (b, RealWorld). This means that reaction actually has two parameters - the type a actually used inside it, and the value of type RealWorld used for sequencing of I/O actions. That's always the case - any I/O definition has one more parameter compared to what you see in its type signature. This parameter is hidden inside the definition of the type synonym IO:
type IO a  =  RealWorld -> (a, RealWorld)
  • You can use these (>>) and (>>=) operations to simplify your program. For example, in the code above we don't need to introduce the variable, because the result of readLn can be send directly to print:
main = readLn >>= print

As you see, the notation:

 do x <- action1
    action2

where action1 has type IO a and action2 has type IO b, translates into:

 action1 >>= (\x -> action2)

where the second argument of (>>=) has the type a -> IO b. It's the way the <- binding is processed - the name on the left-hand side of <- just becomes a parameter of subsequent operations represented as one large I/O action. Note also that if action1 has type IO a then x will just have type a; you can think of the effect of <- as "unpacking" the I/O value of action1 into x. Note also that <- is not a true operator; it's pure syntax, just like do itself. Its meaning results only from the way it gets desugared.

Look at the next example:

main = do putStr "What is your name?"
          a <- readLn
          putStr "How old are you?"
          b <- readLn
          print (a,b)

This code is desugared into:

main = putStr "What is your name?"
       >> readLn
       >>= \a -> putStr "How old are you?"
       >> readLn
       >>= \b -> print (a,b)

I omitted the parentheses here; both the (>>) and the (>>=) operators are left-associative, but lambda-bindings always stretches as far to the right as possible, which means that the a and b bindings introduced here are valid for all remaining actions. As an exercise, add the parentheses yourself and translate this definition into the low-level code that explicitly passes "world" values. I think it should be enough to help you finally realize how the do translation and binding operators work.

Oh, no! I forgot the third monadic operator - return, which corresponds to unit in our miniature I/O system. It just combines its two parameters - the value passed and "world":

return :: a -> IO a
return a world0  =  (a, world0)

How about translating a simple example of return usage? Say,

main = do a <- readLn
          return (a*2)

Programmers with an imperative language background often think that return in Haskell, as in other languages, immediately returns from the I/O definition. As you can see in its definition (and even just from its type!), such an assumption is totally wrong. The only purpose of using return is to "lift" some value (of type a) into the result of a whole action (of type IO a) and therefore it should generally be used only as the last executed action of some I/O sequence. For example try to translate the following definition into the corresponding low-level code:

main = do a <- readLn
          when (a>=0) $ do
              return ()
          print "a is negative"

and you will realize that the print call is executed even for non-negative values of a. If you need to escape from the middle of an I/O definition, you can use an if expression:

main = do a <- readLn
          if (a>=0)
            then return ()
            else print "a is negative"

Moreover, Haskell layout rules allow us to use the following layout:

main = do a <- readLn
          if (a>=0) then return ()
            else do
          print "a is negative"
          ...

that may be useful for escaping from the middle of a longish do-expression.

Last exercise: implement a function liftM that lifts operations on plain values to the operations on monadic ones. Its type signature:

liftM :: (a -> b) -> (IO a -> IO b)

If that's too hard for you, start with the following high-level definition and rewrite it in low-level fashion:

liftM f action = do x <- action
                    return (f x)

Mutable data (references, arrays, hash tables...)

As you should know, every name in Haskell is bound to one fixed (immutable) value. This greatly simplifies understanding algorithms and code optimization, but it's inappropriate in some cases. As we all know, there are plenty of algorithms that are simpler to implement in terms of updatable variables, arrays and so on. This means that the value associated with a variable, for example, can be different at different execution points, so reading its value can't be considered as a pure function. Imagine, for example, the following code:

main = do let a0 = readVariable varA
              _  = writeVariable varA 1
              a1 = readVariable varA
          print (a0, a1)

Does this look strange?

  1. The two calls to readVariable look the same, so the compiler can just reuse the value returned by the first call.
  2. The result of the writeVariable call isn't used so the compiler can (and will!) omit this call completely.
  3. These three calls may be rearranged in any order because they appear to be independent of each other.

This is obviously not what was intended. What's the solution? You already know this - use I/O actions! Doing that guarantees:

  1. the result of the "same" action (such as readVariable varA) will not be reused
  2. each action will have to be executed
  3. the execution order will be retained as written

So, the code above really should be written as:

import Data.IORef
main = do varA <- newIORef 0  -- Create and initialize a new variable
          a0 <- readIORef varA
          writeIORef varA 1
          a1 <- readIORef varA
          print (a0, a1)

Here, varA has the type IORef Int which means "a variable (reference) in the I/O monad holding a value of type Int". newIORef creates a new variable (reference) and returns it, and then read/write actions use this reference. The value returned by the readIORef varA action depends not only on the variable involved but also on the moment this operation is performed so it can return different values on each call.

Arrays, hash tables and any other _mutable_ data structures are defined in the same way - for each of them, there's an operation that creates new "mutable values" and returns a reference to it. Then value-specific read and write operations in the I/O monad are used. The following code shows an example using mutable arrays:

import Data.Array.IO
main = do arr <- newArray (1,10) 37 :: IO (IOArray Int Int)
          a <- readArray arr 1
          writeArray arr 1 64
          b <- readArray arr 1
          print (a, b)

Here, an array of 10 elements with 37 as the initial value at each location is created. After reading the value of the first element (index 1) into a this element's value is changed to 64 and then read again into b. As you can see by executing this code, a will be set to 37 and b to 64.

Other state-dependent operations are also often implemented with I/O actions. For example, a random number generator should return a different value on each call. It looks natural to give it a type involving IO:

rand :: IO Int

Moreover, when you import C routines you should be careful - if this routine is impure, i.e. its result depends on something in the "real world" (file system, memory contents...), internal state and so on, you should give it an IO type. Otherwise, the compiler can "optimize" repetitive calls to the definition with the same parameters!

For example, we can write a non-IO type for:

foreign import ccall
   sin :: Double -> Double

because the result of sin depends only on its argument, but

foreign import ccall
   tell :: Int -> IO Int

If you will declare tell as a pure function (without IO) then you may get the same position on each call!

Encapsulated mutable data: ST

If you're going to be doing things like sending text to a screen or reading data from a scanner, IO is the type to start with - you can then customise or add existing or new I/O operations as you see fit. But what if that shiny-new (or classic) algorithm you're working on really only needs mutable state - then having to drag that IO type from main all the way down to wherever you're implementing the algorithm can get quite annoying.

Fortunately there is a better way! One that remains totally pure and yet allows the use of references, arrays, and so on - and it's done using, you guessed it, Haskell's versatile type system (and one extension). It is the ST type, and it too is monadic!

So what's the big difference between the ST and IO types? In one word - runST:

runST :: (forall s . ST s a) -> a

Yes - it has a very unusual type. But that type allows you to run your stateful computation as if it was a pure definition!

The s type variable in ST is the type of the local state. Moreover, all the fun mutable stuff available for ST is quantified over s:

newSTRef :: a -> ST s (STRef s a)
newArray_ :: Ix i => (i, i) -> ST s (STArray s i e)

So why does runST have such a funky type? Let's see what would happen if we wrote

makeSTRef :: a -> STRef s a
makeSTRef a = runST (newSTRef a)

This fails, because newSTRef a doesn't work for all state types s - it only works for the s from the return type STRef s a.

This is all sort of wacky, but the result is that you can only run an ST computation where the output type is functionally pure, and makes no references to the internal mutable state of the computation. In exchange for that, there's no access to I/O operations like writing to the console - only references, arrays, and such that come in handy for pure computations.

Important note - the state type doesn't actually mean anything. We never have a value of type s, for instance. It's just a way of getting the type system to do the work of ensuring purity is preserved.

On the inside, runST runs a computation with a baton similar to RealWorld for the IO type. Once the computation has completed runST separates the resulting value from the final baton. This value is then returned by runST. The internal implementations are so similar there's there's a function:

stToIO :: ST RealWorld a -> IO a

The difference is that ST uses the type system to forbid unsafe behavior like extracting mutable objects from their safe ST wrapping, but allowing purely functional outputs to be performed with all the handy access to mutable references and arrays.

For example, here's a particularly convoluted way to compute the integer that comes after zero:

oneST :: ST s Integer -- note that this works correctly for any s
oneST = do var <- newSTRef 0
           modifySTRef var (+1)
           readSTRef var

one :: Int
one = runST oneST

I/O actions as values

By this point you should understand why it's impossible to use I/O actions inside non-I/O (pure) functions. Such functions just don't get a "baton"; they don't know any "world" value to pass to an I/O action. The RealWorld type is an abstract datatype, so pure functions also can't construct RealWorld values by themselves, and it's a strict type, so undefined also can't be used. So, the prohibition of using I/O actions inside pure functions is maintained by the type system (as it usually is in Haskell).

But while pure code can't execute I/O actions, it can work with them as with any other functional values - they can be stored in data structures, passed as parameters, returned as results, collected in lists, and partially applied. But an I/O action will remain a functional value because we can't apply it to the last argument - of type RealWorld.

In order to execute the I/O action we need to apply it to some RealWorld value. That can be done only inside other I/O actions, in their "actions chains". And real execution of this action will take place only when this action is called as part of the process of "calculating the final value of world" for main. Look at this example:

main world0 = let get2chars = getChar >> getChar
                  ((), world1) = putStr "Press two keys" world0
                  (answer, world2) = get2chars world1
              in ((), world2)

Here we first bind a value to get2chars and then write a binding involving putStr. But what's the execution order? It's not defined by the order of the let bindings, it's defined by the order of processing "world" values! You can arbitrarily reorder those local bindings - the execution order will be defined by the data dependency with respect to the "world" values that get passed around. Let's see what this main looks like in the do notation:

main = do let get2chars = getChar >> getChar
          putStr "Press two keys"
          get2chars
          return ()

As you can see, we've eliminated two of the let bindings and left only the one defining get2chars. The non-let actions are executed in the exact order in which they're written, because they pass the "world" value from action to action as we described above. Thus, this version of the function is much easier to understand because we don't have to mentally figure out the data dependency of the "world" value.

Moreover, I/O actions like get2chars can't be executed directly because they are functions with a RealWorld parameter. To execute them, we need to supply the RealWorld parameter, i.e. insert them in the main chain, placing them in some do sequence executed from main (either directly in the main function, or indirectly in an I/O function called from main). Until that's done, they will remain like any function, in partially evaluated form. And we can work with I/O actions as with any other functions - bind them to names (as we did above), save them in data structures, pass them as function parameters and return them as results - and they won't be performed until you give them that inaugural RealWorld parameter!

Example: a list of I/O actions

Let's try defining a list of I/O actions:

ioActions :: [IO ()]
ioActions = [(print "Hello!"),
             (putStr "just kidding"),
             (getChar >> return ())
            ]

I used additional parentheses around each action, although they aren't really required. If you still can't believe that these actions won't be executed immediately, just recall the real type of this list:

ioActions :: [RealWorld -> ((), RealWorld)]

Well, now we want to execute some of these actions. No problem, just insert them into the main chain:

main = do head ioActions
          ioActions !! 1
          last ioActions

Looks strange, right? Really, any I/O action that you write in a do-expression (or use as a parameter for the (>>)/(>>=) operators) is an expression returning a result of type IO a for some type a. Typically, you use some function that has the type x -> y -> ... -> IO a and provide all the x, y, etc. parameters. But you're not limited to this standard scenario - don't forget that Haskell is a functional language and you're free to compute the functional value required (recall that IO a is really a function type) in any possible way. Here we just extracted several functions from the list - no problem. This functional value can also be constructed on-the-fly, as we've done in the previous example - that's also OK. Want to see this functional value passed as a parameter? Just look at the definition of when. Hey, we can buy, sell, and rent these I/O actions just like we can with any other functional values! For example, let's define a function that executes all the I/O actions in the list:

sequence_ :: [IO a] -> IO ()
sequence_ [] = return ()
sequence_ (x:xs) = do x
                      sequence_ xs

No mirrors or smoke - we just extract I/O actions from the list and insert them into a chain of I/O operations that should be performed one after another (in the same order that they occurred in the list) to "compute the final world value" of the entire sequence_ call.

With the help of sequence_, we can rewrite our last main function as:

main = sequence_ ioActions

Haskell's ability to work with I/O actions as with any other (functional and non-functional) values allows us to define control structures of arbitrary complexity. Try, for example, to define a control structure that repeats an action until it returns the False result:

while :: IO Bool -> IO ()
while action = ???

Most programming languages don't allow you to define control structures at all, and those that do often require you to use a macro-expansion system. In Haskell, control structures are just trivial functions anyone can write.

Example: returning an I/O action as a result

How about returning an I/O action as the result of a function? Well, we've done this for each I/O definition - they all return I/O actions that need a RealWorld value to be performed. While we usually just execute them as part of a higher-level I/O definition, it's also possible to just collect them without actual execution:

main = do let a = sequence ioActions
              b = when True getChar
              c = getChar >> getChar
          putStr "These let-bindings are not executed!"

These assigned I/O actions can be used as parameters to other definitions, or written to global variables, or processed in some other way, or just executed later, as we did in the example with get2chars.

But how about returning a parameterized I/O action from an I/O definition? Here's a definition that returns the i'th byte from a file represented as a Handle:

readi h i = do hSeek h AbsoluteSeek i
               hGetChar h

So far so good. But how about a definition that returns the i'th byte of a file with a given name without reopening it each time?

readfilei :: String -> IO (Integer -> IO Char)
readfilei name = do h <- openFile name ReadMode
                    return (readi h)

As you can see, it's an I/O definition that opens a file and returns...an I/O action that will read the specified byte. But we can go further and include the readi body in readfilei:

readfilei name = do h <- openFile name ReadMode
                    let readi h i = do hSeek h AbsoluteSeek i
                                       hGetChar h
                    return (readi h)

That's a little better. But why do we add h as a parameter to readi if it can be obtained from the environment where readi is now defined? An even shorter version is this:

readfilei name = do h <- openFile name ReadMode
                    let readi i = do hSeek h AbsoluteSeek i
                                     hGetChar h
                    return readi

What have we done here? We've build a parameterized I/O action involving local names inside readfilei and returned it as the result. Now it can be used in the following way:

main = do myfile <- readfilei "test"
          a <- myfile 0
          b <- myfile 1
          print (a,b)

This way of using I/O actions is very typical for Haskell programs - you just construct one or more I/O actions that you need, with or without parameters, possibly involving the parameters that your "constructor" received, and return them to the caller. Then these I/O actions can be used in the rest of the program without any knowledge about your internal implementation strategy. One thing this can be used for is to partially emulate the OOP (or more precisely, the ADT) programming paradigm.

Example: a memory allocator generator

As an example, one of my programs has a module which is a memory suballocator. It receives the address and size of a large memory block and returns two specialised I/O operations - one to allocate a subblock of a given size and the other to free the allocated subblock:

memoryAllocator :: Ptr a -> Int -> IO (Int -> IO (Ptr b),
                                       Ptr c -> IO ())

memoryAllocator buf size = do ......
                              let alloc size = do ...
                                                  ...
                                  free ptr = do ...
                                                ...
                              return (alloc, free)

How this is implemented? alloc and free work with references created inside the memoryAllocator definition. Because the creation of these references is a part of the memoryAllocator I/O-action chain, a new independent set of references will be created for each memory block for which memoryAllocator is called:

memoryAllocator buf size = do start <- newIORef buf
                              end <- newIORef (buf `plusPtr` size)
                              ...

These two references are read and written in the alloc and free definitions (we'll implement a very simple memory allocator for this example):

      ...
      let alloc size = do addr <- readIORef start
                          writeIORef start (addr `plusPtr` size)
                          return addr
                          
      let free ptr = do writeIORef start ptr

What we've defined here is just a pair of closures that use state available at the moment of their definition. As you can see, it's as easy as in any other functional language, despite Haskell's lack of direct support for impure functions.

The following example uses the operations returned by memoryAllocator, to simultaneously allocate/free blocks in two independent memory buffers:

main = do buf1 <- mallocBytes (2^16)
          buf2 <- mallocBytes (2^20)
          (alloc1, free1) <- memoryAllocator buf1 (2^16)
          (alloc2, free2) <- memoryAllocator buf2 (2^20)
          ptr11 <- alloc1 100
          ptr21 <- alloc2 1000
          free1 ptr11
          free2 ptr21
          ptr12 <- alloc1 100
          ptr22 <- alloc2 1000

Example: emulating OOP with record types

Let's implement the classical OOP example: drawing figures. There are figures of different types: circles, rectangles and so on. The task is to create a heterogeneous list of figures. All figures in this list should support the same set of operations: draw, move and so on. We will define these operations using I/O actions. Instead of a "class" let's define a structure containing implementations of all the operations required:

data Figure = Figure { draw :: IO (),
                       move :: Displacement -> IO ()
                     }

type Displacement = (Int, Int)  -- horizontal and vertical displacement in points

The constructor of each figure's type should just return a Figure record:

circle    :: Point -> Radius -> IO Figure
rectangle :: Point -> Point -> IO Figure

type Point = (Int, Int)  -- point coordinates
type Radius = Int        -- circle radius in points

We will "draw" figures by just printing their current parameters. Let's start with a simplified implementation of the circle and rectangle constructors, without actual move support:

circle center radius = do
    let description = "  Circle at "++show center++" with radius "++show radius
    return $ Figure { draw = putStrLn description }

rectangle from to = do
    let description = "  Rectangle "++show from++"-"++show to)
    return $ Figure { draw = putStrLn description }

As you see, each constructor just returns a fixed draw operation that prints parameters with which the concrete figure was created. Let's test it:

drawAll :: [Figure] -> IO ()
drawAll figures = do putStrLn "Drawing figures:"
                     mapM_ draw figures

main = do figures <- sequence [circle (10,10) 5,
                               circle (20,20) 3,
                               rectangle (10,10) (20,20),
                               rectangle (15,15) (40,40)]
          drawAll figures


Now let's define "full-featured" figures that can actually be moved around. In order to achieve this, we should provide each figure with a mutable variable that holds each figure's current screen location. The type of this variable will be IORef Point. This variable should be created in the figure constructor and manipulated in I/O operations (closures) enclosed in the Figure record:

circle center radius = do
    centerVar <- newIORef center
    
    let drawF = do center <- readIORef centerVar
                   putStrLn ("  Circle at "++show center
                             ++" with radius "++show radius)
                   
    let moveF (addX,addY) = do (x,y) <- readIORef centerVar
                               writeIORef centerVar (x+addX, y+addY)
                               
    return $ Figure { draw=drawF, move=moveF }
   
rectangle from to = do
    fromVar <- newIORef from
    toVar   <- newIORef to

    let drawF = do from <- readIORef fromVar
                   to   <- readIORef toVar
                   putStrLn ("  Rectangle "++show from++"-"++show to)
                   
    let moveF (addX,addY) = do (fromX,fromY) <- readIORef fromVar
                               (toX,toY)     <- readIORef toVar
                               writeIORef fromVar (fromX+addX, fromY+addY)
                               writeIORef toVar   (toX+addX, toY+addY)

    return $ Figure { draw=drawF, move=moveF }

Now we can test the code which moves figures around:

main = do figures <- sequence [circle (10,10) 5,
                               rectangle (10,10) (20,20)]
          drawAll figures
          mapM_ (\fig -> move fig (10,10)) figures
          drawAll figures

It's important to realize that we are not limited to including only I/O actions in a record that's intended to simulate a C++/Java-style interface. The record can also include values, IORefs, pure functions - in short, any type of data. For example, we can easily add to the Figure interface fields for area and origin:

data Figure = Figure { draw :: IO (),
                       move :: Displacement -> IO (),
                       area :: Double,
                       origin :: IORef Point
                     }

Exception handling (under development)

Although Haskell provides a set of exception raising/handling features comparable to those in popular OOP languages (C++, Java, C#), this part of the language receives much less attention. This is for two reasons:

  • you just don't need to worry as much about them - most of the time it just works "behind the scenes".
  • Haskell, lacking OOP-style inheritance, doesn't allow the programmer to easily subclass exception types, therefore limiting the flexibility of exception handling.

The Haskell RTS raises more exceptions than traditional languages - pattern match failures, calls with invalid arguments (such as head []) and computations whose results depend on special values undefined and error "...." all raise their own exceptions:

  • example 1:
main = print (f 2)

f 0 = "zero"
f 1 = "one"
  • example 2:
main = print (head [])
  • example 3:
main = print (1 + (error "Value that wasn't initialized or cannot be computed"))

This allows the writing of programs in a much more error-prone way.

Interfacing with C/C++ and foreign libraries (under development)

While Haskell is great at algorithm development, speed isn't its best side. We can combine the best of both worlds, though, by writing speed-critical parts of program in C and the rest in Haskell. We just need a way to call C functions from Haskell and vice versa, and to marshal data between both worlds.

We also need to interact with the C world for using Windows/Linux APIs, linking to various libraries and DLLs. Even interfacing with other languages often requires going through C world as a "common denominator". Chapter 8 of the Haskell 2010 report provides a complete description of interfacing with C.

We will learn FFI via a series of examples. These examples include C/C++ code, so they need C/C++ compilers to be installed, the same will be true if you need to include code written in C/C++ in your program (C/C++ compilers are not required when you just need to link with existing libraries providing APIs with C calling convention). On Unix (and Mac OS?) systems, the system-wide default C/C++ compiler is typically used by GHC installation. On Windows, no default compilers exist, so GHC is typically shipped with a C compiler, and you may find on the download page a GHC distribution bundled with C and C++ compilers. Alternatively, you may find and install a GCC/MinGW version compatible with your GHC installation.

If you need to make your C/C++ code as fast as possible, you may compile your code by Intel compilers instead of GCC. However, these compilers are not free, moreover on Windows, code compiled by Intel compilers may not interact correctly with GHC-compiled code, unless one of them is put into DLLs (due to object file incompatibility).

More links:

C->Haskell
A lightweight tool for implementing access to C libraries from Haskell.
HSFFIG
Haskell FFI Binding Modules Generator (HSFFIG) is a tool that takes a C library header (".h") and generates Haskell Foreign Functions Interface import declarations for items (functions, structures, etc.) the header defines.
MissingPy
MissingPy is really two libraries in one. At its lowest level, MissingPy is a library designed to make it easy to call into Python from Haskell. It provides full support for interpreting arbitrary Python code, interfacing with a good part of the Python/C API, and handling Python objects. It also provides tools for converting between Python objects and their Haskell equivalents. Memory management is handled for you, and Python exceptions get mapped to Haskell Dynamic exceptions. At a higher level, MissingPy contains Haskell interfaces to some Python modules.
HsLua
A Haskell interface to the Lua scripting language

Calling functions

We begin by learning how to call C functions from Haskell and Haskell functions from C. The first example consists of three files:

main.hs:

{-# LANGUAGE ForeignFunctionInterface #-}

main = do print "Hello from main"
          c_function

haskell_function = print "Hello from haskell_function"

foreign import ccall safe "prototypes.h"
    c_function :: IO ()

foreign export ccall
    haskell_function :: IO ()

vile.c:

#include <stdio.h>
#include "prototypes.h"

void c_function (void)
{
  printf("Hello from c_function\n");
  haskell_function();
}

prototypes.h:

extern void c_function (void);
extern void haskell_function (void);

It may be compiled and linked in one step by ghc:

 ghc --make main.hs vile.c

Or, you may compile C module(s) separately and link in ".o" files (this may be preferable if you use make and don't want to recompile unchanged sources; ghc's --make option provides smart recompilation only for ".hs" files):

 ghc -c vile.c
 ghc --make main.hs vile.o

You may use gcc/g++ directly to compile your C/C++ files but I recommend to do linking via ghc because it adds a lot of libraries required for execution of Haskell code. For the same reason, even if your main routine is written in C/C++, I recommend calling it from the Haskell function main - otherwise you'll have to explicitly init/shutdown the GHC RTS (run-time system).

We use the foreign import declaration to import foreign routines into our Haskell world, and foreign export to export Haskell routines into the external world. Note that import creates a new Haskell symbol (from the external one), while export uses a Haskell symbol previously defined. Technically speaking, both types of declarations create a wrapper that converts the names and calling conventions from C to Haskell or vice versa.

All about the foreign declaration

The ccall specifier in foreign declarations means the use of the C (not C++ !) calling convention. This means that if you want to write the external function in C++ (instead of C) you should add export "C" specification to its declaration - otherwise you'll get linking errors. Let's rewrite our first example to use C++ instead of C:

prototypes.h:

#ifdef __cplusplus
extern "C" {
#endif

extern void c_function (void);
extern void haskell_function (void);

#ifdef __cplusplus
}
#endif

Compile it via:

 ghc --make main.hs vile.cpp

where "vile.cpp" is just a renamed copy of "vile.c" from the first example. Note that the new "prototypes.h" is written to allow compiling it both as C and C++ code. When it's included from "vile.cpp", it's compiled as C++ code. When GHC compiles "main.hs" via the C compiler (enabled by the -fvia-C option), it also includes "prototypes.h" but compiles it in C mode. It's why you need to specify ".h" files in foreign declarations - depending on which Haskell compiler you use, these files may be included to check consistency of C and Haskell declarations.

The quoted part of the foreign declaration may also be used to import or export a function under another name - for example,

foreign import ccall safe "prototypes.h CFunction"
    c_function :: IO ()

foreign export ccall "HaskellFunction"
    haskell_function :: IO ()

specifies that the C function called CFunction will become known as the Haskell function c_function, while the Haskell function haskell_function will be known in the C world as HaskellFunction. It's required when the C name doesn't conform to Haskell naming requirements.

Although the Haskell FFI standard tells about many other calling conventions in addition to ccall (e.g. cplusplus, jvm, net) current Haskell implementations support only ccall and stdcall. The latter, also called the "Pascal" calling convention, is used to interface with WinAPI:

foreign import stdcall unsafe "windows.h SetFileApisToOEM"
  setFileApisToOEM :: IO ()

And finally, about the safe/unsafe specifier: a C function imported with the unsafe keyword is called directly and the Haskell runtime is stopped while the C function is executed (when there are several OS threads executing the Haskell program, only the current OS thread is delayed). This call doesn't allow recursively entering into the Haskell world by calling any Haskell function - the Haskell RTS is just not prepared for such an event. However, unsafe calls are as quick as calls in the C world. It's ideal for "momentary" calls that quickly return back to the caller.

When safe is specified, the C function is called in a safe environment - the Haskell execution context is saved, so it's possible to call back to Haskell and, if the C call takes a long time, another OS thread may be started to execute Haskell code (of course, in threads other than the one that called the C code). This has its own price, though - around 1000 CPU ticks per call.

You can read more about interaction between FFI calls and Haskell concurrency in [7].

Marshalling simple types

Calling by itself is relatively easy; the real problem of interfacing languages with different data models is passing data between them. In this case, there is no guarantee that Haskell's Int is represented in memory the same way as C's int, nor Haskell's Double the same as C's double and so on. While on some platforms they are the same and you can write throw-away programs relying on these, the goal of portability requires you to declare imported and exported functions using special types described in the FFI standard, which are guaranteed to correspond to C types. These are:

import Foreign.C.Types (               -- equivalent to the following C type:
         CChar, CUChar,                --  char/unsigned char
         CShort, CUShort,              --  short/unsigned short
         CInt, CUInt, CLong, CULong,   --  int/unsigned/long/unsigned long
         CFloat, CDouble...)           --  float/double

Now we can import and export typeful C/Haskell functions:

foreign import ccall unsafe "math.h"
    c_sin :: CDouble -> CDouble

Note that pure C functions (those whose results depend only on their arguments) are imported without IO in their return type. The const specifier in C is not reflected in Haskell types, so appropriate compiler checks are not performed.

All these numeric types are instances of the same classes as their Haskell cousins (Ord, Num, Show and so on), so you may perform calculations on these data directly. Alternatively, you may convert them to native Haskell types. It's very typical to write simple wrappers around imported and exported functions just to provide interfaces having native Haskell types:

-- |Type-conversion wrapper around c_sin
sin :: Double -> Double
sin = fromRational . c_sin . toRational

Memory management

Marshalling strings

import Foreign.C.String (   -- representation of strings in C
         CString,           -- = Ptr CChar
         CStringLen)        -- = (Ptr CChar, Int)
foreign import ccall unsafe "string.h"
    c_strlen :: CString -> IO CSize     -- CSize defined in Foreign.C.Types and is equal to size_t
-- |Type-conversion wrapper around c_strlen 
strlen :: String -> Int
strlen = ....

Marshalling composite types

A C array may be manipulated in Haskell as StorableArray.

There is no built-in support for marshalling C structures and using C constants in Haskell. These are implemented in the c2hs preprocessor, though.

Binary marshalling (serializing) of data structures of any complexity is implemented in the library module "Binary".

Dynamic calls

DLLs

because i don't have experience of using DLLs, can someone write into this section? Ultimately, we need to consider the following tasks:

  • using DLLs of 3rd-party libraries (such as ziplib)
  • putting your own C code into a DLL to use in Haskell
  • putting Haskell code into a DLL which may be called from C code

The dark side of the I/O monad

Unless you are a systems developer, postgraduate CS student, or have alternate (and eminent!) verificable qualifications you should have no need whatsoever for this section - here is just one tiny example of what can go wrong if you don't know what you are doing. Look for other solutions!

unsafePerformIO

Do you remember that initial attempt to define getchar?

getchar :: Char

get2chars = [getchar, getchar]

Let's also recall the problems arising from this faux-definition:

  1. Because the Haskell compiler treats all functions as pure (not having side effects), it can avoid "unnecessary" calls to getchar and use one returned value twice;
  2. Even if it does make two calls, there is no way to determine which call should be performed first. Do you want to return the two characters in the order in which they were read, or in the opposite order? Nothing in the definition of get2chars answers this question.

Despite these problems, programmers coming from an imperative language background often look for a way to do this - disguise one or more I/O actions as a pure definition. Having seen procedural entities similar in appearance to:

void putchar(char c);

the thought of just writing:

putchar :: Char -> ()
putchar c = ...

would definitely be more appealing - for example, defining readContents as though it were a pure function:

readContents :: Filename -> String

will certainly simplify the code that uses it. However, those exact same problems are also lurking here:

  1. Attempts to read the contents of files with the same name can be factored (i.e. reduced to a single call) despite the fact that the file (or the current directory) can be changed between calls. Haskell considers all non-IO functions to be pure and feels free to merge multiple calls with the same parameters.
  2. This call is not inserted in a sequence of "world transformations", so the compiler doesn't know at what exact moment you want to execute this action. For example, if the file has one kind of contents at the beginning of the program and another at the end - which contents do you want to see? You have no idea when (or even if) this function is going to get invoked, because Haskell sees this function as pure and feels free to reorder the execution of any or all pure functions as needed.

So, implementing supposedly-pure functions that interact with the Real World is considered to be Bad Behavior. Nice programmers never do it ;-)

Nevertheless, there are (semi-official) ways to use I/O actions inside of pure functions. As you should remember this is prohibited by requiring the RealWorld "baton" in order to call an I/O action. Pure functions don't have the baton, but there is a (ahem) "special" definition that produces this baton from nowhere, uses it to call an I/O action and then throws the resulting "world" away! It's a little low-level mirror-smoke. This particular (and dangerous) definition is:

unsafePerformIO :: IO a -> a

Let's look at how it could be defined:

unsafePerformIO :: (RealWorld -> (a, RealWorld)) -> a
unsafePerformIO action = let (a, world1) = action createNewWorld
                         in a

where createNewWorld is an private definition producing a new value of the RealWorld type.

Using unsafePerformIO, you could easily write pure functions that do I/O inside. But don't do this without a real need, and remember to follow this rule:

  • the compiler doesn't know that you are cheating; it still considers each non-IO function to be a pure one. Therefore, all the usual optimization rules can (and will!) be applied to its execution.

So you must ensure that:

  • The result of each call depends only on its arguments.
  • You don't rely on side-effects of this function, which may be not executed if its results are not needed.

Let's investigate this problem more deeply. Function evaluation in Haskell is determined by a value's necessity - the language computes only the values that are really required to calculate the final result. But what does this mean with respect to the main function? To "calculate the final world's" value, you need to perform all the intermediate I/O actions that are included in the main chain. By using unsafePerformIO we call I/O actions outside of this chain. What guarantee do we have that they will be run at all? None. The only time they will be run is if running them is required to compute the overall function result (which in turn should be required to perform some action in the main chain). This is an example of Haskell's evaluation-by-need strategy. Now you should clearly see the difference:

  • An I/O action inside an I/O definition is guaranteed to execute as long as it is (directly or indirectly) inside the main chain - even when its result isn't used (because the implicit "world" value it returns will be used). You directly specify the order of the action's execution inside the I/O definition. Data dependencies are simulated via the implicit "world" values that are passed from each I/O action to the next.
  • An I/O action inside unsafePerformIO will be performed only if the result of this operation is really used. The evaluation order is not guaranteed and you should not rely on it (except when you're sure about whatever data dependencies may exist).

I should also say that inside the unsafePerformIO call you can organize a small internal chain of I/O actions with the help of the same binding operators and/or do syntactic sugar we've seen above. So here's how we'd rewrite our previous (pure!) definition of one using unsafePerformIO:

one :: Integer
one = unsafePerformIO $ do var <- newIORef 0
                           modifyIORef var (+1)
                           readIORef var

and in this case all the operations in this chain will be performed as long as the result of the unsafePerformIO call is needed. To ensure this, the actual unsafePerformIO implementation evaluates the "world" returned by the action:

unsafePerformIO action = let (a,world1) = action createNewWorld
                         in (world1 `seq` a)

(The seq operation strictly evaluates its first argument before returning the value of the second one [8]).

inlinePerformIO

inlinePerformIO has the same definition as unsafePerformIO but with the addition of an INLINE pragma:

-- | Just like unsafePerformIO, but we inline it. Big performance gains as
-- it exposes lots of things to further inlining
{-# INLINE inlinePerformIO #-}
inlinePerformIO action = let (a, world1) = action createNewWorld
                         in (world1 `seq` a)
#endif

Semantically inlinePerformIO = unsafePerformIO in as much as either of those have any semantics at all.

The difference of course is that inlinePerformIO is even less safe than unsafePerformIO. While ghc will try not to duplicate or common up different uses of unsafePerformIO, we aggressively inline inlinePerformIO. So you can really only use it where the I/O content is really properly pure, like reading from an immutable memory buffer (as in the case of ByteStrings). However things like allocating new buffers should not be done inside inlinePerformIO since that can easily be floated out and performed just once for the whole program, so you end up with many things sharing the same buffer, which would be bad.

So the rule of thumb is that I/O actions wrapped in unsafePerformIO have to be externally pure while with inlinePerformIO it has to be really, really pure or it'll all go horribly wrong.

That said, here's some really hairy code. This should frighten any pure functional programmer...

write :: Int -> (Ptr Word8 -> IO ()) -> Put ()
write !n body = Put $ \c buf@(Buffer fp o u l) ->
  if n <= l
    then write</code> c fp o u l
    else write</code> (flushOld c n fp o u) (newBuffer c n) 0 0 0

  where {-# NOINLINE write</code> #-}
        write</code> c !fp !o !u !l =
          -- warning: this is a tad hardcore
          inlinePerformIO
            (withForeignPtr fp
              (\p -> body $! (p `plusPtr` (o+u))))
          `seq` c () (Buffer fp o (u+n) (l-n))

it's used like:

word8 w = write 1 (\p -> poke p w)

This does not adhere to my rule of thumb above. Don't ask exactly why we claim it's safe :-) (and if anyone really wants to know, ask Ross Paterson who did it first in the Builder monoid)

unsafeInterleaveIO

But there is an even stranger operation:

unsafeInterleaveIO :: IO a -> IO a

Don't let that type signature fool you - unsafeInterleaveIO also uses a dubiously-acquired baton which it uses to set up an underground relay-race for its unsuspecting parameter. If it happens, this seedy race then occurs alongside the offical main relay-race - if they collide, things will get ugly!

So how does unsafeInterleaveIO get that bootlegged baton? Typically by making a forgery of the offical one to keep for itself - it can do this because the I/O action unsafeInterleaveIO returns will be handed the offical baton in the main relay-race. But one miscreant realised there was a simpler way:

unsafeInterleaveIO   :: IO a -> IO a
unsafeInterleaveIO a =  return (unsafePerformIO a)

Why bother with counterfeit copies of batons if you can just make them up?

At least you have some appreciation as to why unsafeInterleaveIO is, well unsafe! Just don't ask - to talk further is bound to cause grief and indignation. I won't say anything more about this ruffian I...use all the time (darn it!)

One can use unsafePerformIO (not unsafeInterleaveIO) to perform I/O operations not in some predefined order but by demand. For example, the following code:

do let c = unsafePerformIO getChar
   do_proc c

will perform the getChar I/O call only when the value of c is really required by the calling code, i.e. it this call will be performed lazily like any regular Haskell computation.

Now imagine the following code:

do let s = [unsafePerformIO getChar, unsafePerformIO getChar, unsafePerformIO getChar]
   do_proc s

The three characters inside this list will be computed on demand too, and this means that their values will depend on the order they are consumed. It is not what we usually want.

unsafeInterleaveIO solves this problem - it performs I/O only on demand but allows you to define the exact internal execution order for parts of your data structure. It is why I wrote that unsafeInterleaveIO makes an illegal copy of the baton:

  • unsafeInterleaveIO accepts an I/O action as a parameter and returns another I/O action as the result:


do str <- unsafeInterleaveIO myGetContents
  • unsafeInterleaveIO doesn't perform any action immediately, it only creates a closure of type a which upon being needed will perform the action specified as the parameter.
  • this action by itself may compute the whole value immediately...or use unsafeInterleaveIO again to defer calculation of some sub-components:
myGetContents = do
   c <- getChar
   s <- unsafeInterleaveIO myGetContents
   return (c:s)

This code will be executed only at the moment when the value of str is really demanded. In this moment, getChar will be performed (with its result assigned to c) and a new lazy-I/O closure will be created - for s. This new closure also contains a link to a myGetContents call.

Then the list cell is returned. It contains Char that was just read and a link to another myGetContents call as a way to compute rest of the list. Only at the moment when the next value in the list is required will this operation be performed again.

As a final result, we can postpone the read of the second Char in the list before the first one, but have lazy reading of characters as a whole - bingo!


PS: of course, actual code should include EOF checking; also note that you can read multiple characters/records at each call:

myGetContents = do
   c <- replicateM 512 getChar
   s <- unsafeInterleaveIO myGetContents
   return (c++s)

and we can rewrite myGetContents to avoid needing to use unsafeInterleaveIO where it's called:

myGetContents = unsafeInterleaveIO $ do
   c <- replicateM 512 getChar
   s <- myGetContents
   return (c++s)

Welcome to the machine: the actual GHC implementation

A little disclaimer: I should say that I'm not describing here exactly what a monad is (I don't even completely understand it myself) and my explanation shows only one possible way to implement the I/O monad in Haskell. For example, the hbc compiler and the Hugs interpreter implements the I/O monad via continuations [9]. I also haven't said anything about exception handling, which is a natural part of the "monad" concept. You can read the All About Monads guide to learn more about these topics.

But there is some good news:

  • the I/O monad understanding you've just acquired will work with any implementation and with many other monads. You just can't work with RealWorld values directly.
  • the I/O monad implementation described here is similar to what GHC uses:
newtype IO a = IO (State# RealWorld -> (# State# RealWorld, a #))

It uses the State# RealWorld type instead of our RealWorld, it uses the (# ... #) strict tuple for optimization, and it adds an IO data constructor around the type. Nevertheless, there are no significant changes from the standpoint of our explanation. Knowing the principle of "chaining" I/O actions via fake "state of the world" values, you can now more easily understand and write low-level implementations of GHC I/O operations.

Of course, other compilers e.g. yhc/nhc (jhc, too?) define IO in other ways.

The Yhc/nhc98 implementation

data World = World
newtype IO a = IO (World -> Either IOError a)

This implementation makes the World disappear somewhat[10], and returns Either a result of type a, or if an error occurs then IOError. The lack of the World on the right-hand side of the function can only be done because the compiler knows special things about the IO type, and won't overoptimise it.


Further reading

[1] This tutorial is largely based on Simon Peyton Jones's paper Tackling the awkward squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell. I hope that my tutorial improves his original explanation of the Haskell I/O system and brings it closer to the point of view of new Haskell programmers. But if you need to learn about concurrency, exceptions and FFI in Haskell/GHC, the original paper is the best source of information.

[2] You can find more information about concurrency, FFI and STM at the GHC/Concurrency#Starting points page.

[3] The Arrays page contains exhaustive explanations about using mutable arrays.

[4] Look also at the Using monads page, which contains tutorials and papers really describing these mysterious monads.

[5] An explanation of the basic monad functions, with examples, can be found in the reference guide A tour of the Haskell Monad functions, by Henk-Jan van Tuyl.

[6] Official FFI specifications can be found on the page The Haskell 98 Foreign Function Interface 1.0: An Addendum to the Haskell 98 Report

[7] Using FFI in multithreaded programs described in paper Extending the Haskell Foreign Function Interface with Concurrency

[8] This particular behaviour is not a requirement of Haskell 2010, so the operation of seq may differ between various Haskell implementations - if you're not sure, staying within the I/O monad is the safest option.

[9] How to Declare an Imperative by Phil Wadler provides an explanation of how this can be done.

[10] The RealWorld type can even be replaced e.g. Functional I/O Using System Tokens by Lennart Augustsson.

Do you have more questions? Ask in the haskell-cafe mailing list.

To-do list

If you are interested in adding more information to this manual, please add your questions/topics here.

Topics:

  • fixIO and mdo
  • Q monad

Questions:

  • split (>>=)/(>>)/return section and do section, more examples of using binding operators
  • IORef detailed explanation (==const*), usage examples, syntax sugar, unboxed refs
  • explanation of how the actual data "in" mutable references are inside RealWorld, rather than inside the references themselves (IORef, IOArray & co.)
  • control structures developing - much more examples
  • unsafePerformIO usage examples: global variable, ByteString, other examples
  • how unsafeInterLeaveIO can be seen as a kind of concurrency, and therefore isn't so unsafe (unlike unsafeInterleaveST which really is unsafe)
  • discussion about different senses of safe/unsafe (like breaking equational reasoning vs. invoking undefined behaviour (so can corrupt the run-time system))
  • actual GHC implementation - how to write low-level routines based on example of newIORef implementation

This manual is collective work, so feel free to add more information to it yourself. The final goal is to collectively develop a comprehensive manual for using the I/O monad.