Iteratee I/O

Iteratee I/O is a way to avoid the problems that can occur with lazy I/O. They work by making the I/O operations explicit, making their behavior easier to reason about.

The problem with lazy I/O

As a beginner, you probably used Haskell's lazy I/O through the System.IO module. However, while it is good enough for simple programs, its unpredictability makes it unsuitable for practical use.

For example, a common beginner mistake is to close a file before one has finished reading it:

wrong = do
    fileData <- withFile "test.txt" ReadMode hGetContents
    putStr fileData

The problem is withFile closes the handle before fileData is forced. The correct way is to pass all the code to withFile:

right = withFile "test.txt" ReadMode $ \handle -> do
    fileData <- hGetContents handle
    putStr fileData

Here, the data is consumed before withFile finishes.

Although this is easily fixed, the type system does not enforce the correct solution. Even worse, if you use the former code, it won't even raise an error – it will just fail silently and return an empty string. Many years passed before a satisfactory solution to the streaming data problem was found.

How iteratees work

When you "step" an iteratee, it reads a chunk of data, updates the internal state and returns a new iteratee along with the data it read. Because an iteratee is simply a function with state, many iteratees can be composed together to form a pipeline.

Some implementations also provide a resource management layer that releases resources automatically when they are no longer needed. This is very useful in a server, where sockets and file handles are scarce.

Implementations

iteratee

The original iteratee library, by Oleg Kiselyov.

iterIO

Yet another implementation.

enumerator

Used in Snap. It does not use any extensions, so it will work with most Haskell compilers.

pipes

Elegant streaming library with many unique features

conduit

Popular implementation designed with practical use in mind, created by the author of Yesod. Recently heavily influenced by pipes.

liboleg

An evolving collection of Oleg Kiselyov's Haskell modules (depends on the package unix and will therefore not compile on Windows systems).

io-streams

Focuses on streaming IO and having a simpler type framework then the Conduit and Pipes packages.

Essays by Oleg

• Oleg's writings: Incremental multi-level input processing with left-fold enumerator: predictable, high-performance, safe, and elegant
• An implementation by Oleg, iteratees on Chars and Strings
• A general library by Oleg

Other discussions

• Understandings of Iteratees
• The Monad.Reader Issue 16; see the section "Iteratee: Teaching an Old Fold New Tricks" by John W. Lato
• Yesod Book: Conduits
• Lazy IO considered harmful; way to go, Left-fold enumerator!
• A Darcs repository of an alternative implementation
• Stanford CS240h lecture on iteratee

Users of Iteratee I/O

• Snap: The Snap web framework
• yaml: Low-level binding to the libyaml C library
• usb 0.4: Communicate with USB devices
• sstable: SSTables in Haskell
• WAI: a Web Application Interface for haskell web frameworks (used by Yesod).

See also

• Enumerator and iteratee
• Iteratee