Difference between revisions of "Pipes"

From HaskellWiki
Jump to: navigation, search
(pipes-parse 1.0.0)
(Announcements: pipes-attoparsec-
Line 136: Line 136:
* [http://www.haskellforall.com/2013/06/pipes-parse-100-pushback-delimited.html pipes-parse-1.0.0]: Pushback, delimited parsers, resumable parsing, and lenses
* [http://www.haskellforall.com/2013/06/pipes-parse-100-pushback-delimited.html pipes-parse-1.0.0]: Pushback, delimited parsers, resumable parsing, and lenses
* [https://groups.google.com/d/msg/haskell-pipes/3A-RbzCUovw/9GEIDi5WRbwJ pipes-attoparsec-]: New API, interleaved parsing support, built on top of pipes-parse.
== Upcoming libraries ==
== Upcoming libraries ==

Revision as of 07:14, 4 June 2013

Use pipes to create elegant streaming programs within Haskell. Build streaming components and connect them them together to process both events and data.


Pipes is the official Hackage category for the pipes ecosystem.


The pipes package is the core library which emphasizes:

  • Elegance and Simplicity
  • Speed
  • Lightweight dependencies

The pipes library offers many unique features not available in other streaming libraries:

  • Bidirectional streams
  • Extension framework
  • Elegant ListT interconversions
  • Correctness proofs for the entire library

Read the official pipes tutorial to learn more.

Source code hosted on GitHub


The pipes-safe package adds resource management and exception safety to the pipes ecosystem. pipes-safe offers many high-level features such as:

  • Complete exception safety, including asynchronous exceptions
  • Termination safety
  • Resources are freed in reverse order of acquisition

Additionally, pipes-safe is the only streaming library that provides truly native exception handling, allowing you to seamlessly handle exceptions and resume uninterrupted streaming.

Read the official pipes-safe tutorial to learn more.

Source code hosted on GitHub


The pipes-concurrency package adds powerful and light-weight concurrency primitives to the pipes ecosystem. Use these primitives to:

  • Build reactive event-driven programs
  • Merge and split streams
  • Communicate between multiple concurrent pipelines
  • Manage multiple resources simultaneously

The pipes-concurrency library offers several features unavailable in other streaming libraries such as:

  • Deadlock safety
  • Cyclic communication graphs
  • Dynamically changing graph topologies

Read the official pipes-concurrency tutorial to learn more.

Source code hosted on GitHub


The pipes-parse package defines the generic machinery necessary for common parsing tasks using pipes:

  • Detect and handle end of input
  • Save unused input for later steps
  • Mix proxies with different leftover buffers using lenses
  • Nesting and delimiting parsers to a subset of the input
  • Interruptible and resumable parsing

Read the official pipes-parse tutorial to learn more.

Source code hosted on GitHub

Community-contributed libraries


The pipes-attoparsec library converts attoparsec parsers to pipes for high-performance incremental parsing.

Source code hosted on GitHub


The pipes-network library converts server and client sockets to pipes to seamlessly stream data over any network.

Source code hosted on GitHub


The pipes-network-tls allows streaming through TLS-secured network connections, exposing a similar API to the one exposed by pipes-network.

Source code hosted on GitHub


The pipes-zlib enables compression and decompression of strict ByteString streams using the zlib codec.

Source code hosted on GitHub


Besides the usual Haskell community channels such as the official mailing lists or the Haskell subreddit, you can ask for help, suggest improvements, or discuss about the Pipes ecosystem at the “haskell-pipes” mailing list at Google Groups.


In chronological order.

  • pipes-1.0.0: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single Category.
  • pipes-2.0.0: The introduction of Frames, later deprecated in favor of pipes-safe.
  • pipes-2.1.0: Transition of Frames to indexed monads, later deprecated in favor of pipes-safe.
  • pipes-2.3.0: The introduction of bidirectional Proxies, which evolved into modern pipes.
  • pipes-2.4.0: The release of the proxy transformer extension system.
  • pipes-3.0.0: Major API simplification and consolidation.
  • pipes-3.2.0: ListT integration, Codensity proxy transformer, and ArrowChoice primitives.

Upcoming libraries

  • pipes-string: Bytestring and Text support
  • pipes-free: Pipe suspension and single-stepping pipes
  • pipes-attoparsec: Improve the current pipes-attoparsec library by relying on the upcoming pipes-parse for interleaved parsing support.