Difference between revisions of "Pipes"
(→Announcements: add pipes-network-0.1.0 announce) |
m |
||
(18 intermediate revisions by 3 users not shown) | |||
Line 3: | Line 3: | ||
[[Category:Packages]] |
[[Category:Packages]] |
||
+ | The pipes library is a clean and powerful stream processing library that lets you build and connect reusable streaming components. |
||
− | Use pipes to create elegant streaming programs within Haskell. Build streaming components and connect them them together to process both events and data. |
||
== Libraries == |
== Libraries == |
||
Line 11: | Line 11: | ||
=== pipes === |
=== pipes === |
||
− | The [http://hackage.haskell.org/package/pipes pipes] package is the core library which |
+ | The [http://hackage.haskell.org/package/pipes pipes] package is the core library which provides reusable primitives for stream programming. Use these as building blocks for more sophisticated streaming abstractions. |
⚫ | |||
− | * Elegance and Simplicity |
||
− | * Speed |
||
− | * Lightweight dependencies |
||
− | |||
− | The pipes library offers many unique features not available in other streaming libraries: |
||
− | |||
− | * Bidirectional streams |
||
− | * Extension framework |
||
− | * Elegant ListT interconversions |
||
− | * Correctness proofs for the entire library |
||
− | |||
⚫ | |||
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Library GitHub] |
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Library GitHub] |
||
Line 30: | Line 19: | ||
=== pipes-safe === |
=== pipes-safe === |
||
− | The [http://hackage.haskell.org/package/pipes-safe pipes-safe] package adds resource management and exception safety to the pipes ecosystem. pipes-safe |
+ | The [http://hackage.haskell.org/package/pipes-safe pipes-safe] package adds resource management and exception safety to the pipes ecosystem. Use pipes-safe to safely acquire and release resources deterministically within a pipeline. |
− | |||
− | * Complete exception safety, including asynchronous exceptions |
||
− | * Termination safety |
||
− | * Resources are freed in reverse order of acquisition |
||
− | |||
− | Additionally, pipes-safe is the only streaming library that provides truly native exception handling, allowing you to seamlessly handle exceptions and resume uninterrupted streaming. |
||
− | |||
⚫ | |||
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Safe-Library GitHub] |
Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Safe-Library GitHub] |
||
Line 44: | Line 25: | ||
=== pipes-concurrency === |
=== pipes-concurrency === |
||
− | The [http://hackage.haskell.org/package/pipes-concurrency pipes-concurrency] package adds |
+ | The [http://hackage.haskell.org/package/pipes-concurrency pipes-concurrency] package adds concurrency primitives to the pipes ecosystem with built-in deadlock safety. Use these primitives to: |
* Build reactive event-driven programs |
* Build reactive event-driven programs |
||
− | * Merge and |
+ | * Merge and broadcast streams |
* Communicate between multiple concurrent pipelines |
* Communicate between multiple concurrent pipelines |
||
− | * Manage multiple resources simultaneously |
||
⚫ | |||
− | The pipes-concurrency library offers several features unavailable in other streaming libraries such as: |
||
⚫ | |||
− | * Deadlock safety |
||
− | * Cyclic communication graphs |
||
− | * Dynamically changing graph topologies |
||
+ | === pipes-parse === |
||
− | Read the [http://hackage.haskell.org/packages/archive/pipes-concurrency/1.0.0/doc/html/Control-Proxy-Concurrent-Tutorial.html official pipes-concurrency tutorial] to learn more. |
||
+ | The [http://hackage.haskell.org/package/pipes-parse pipes-parse] package defines the generic machinery necessary for common parsing tasks. Use pipes-parse to: |
||
⚫ | |||
+ | |||
+ | * handle leftovers and end of input, |
||
+ | * interrupt and resume streaming, and |
||
+ | * sub-divide streams without collecting elements in memory. |
||
+ | |||
+ | Source code hosted on [https://github.com/Gabriel439/Haskell-Pipes-Parse-Library GitHub] |
||
== Community-contributed libraries == |
== Community-contributed libraries == |
||
+ | |||
+ | Listed in alphabetical order. |
||
+ | |||
+ | === pipes-aeson === |
||
+ | |||
+ | The [http://hackage.haskell.org/package/pipes-aeson pipes-aeson] library allows you to encode and decode JSON values flowing through streams, possibly |
||
+ | interleaving other stream effects while doing it. |
||
+ | |||
+ | Source code is hosted on [https://github.com/k0001/pipes-aeson GitHub] |
||
=== pipes-attoparsec === |
=== pipes-attoparsec === |
||
− | The [http://hackage.haskell.org/package/pipes-attoparsec pipes-attoparsec] library converts [http://hackage.haskell.org/package/attoparsec attoparsec] parsers to pipes for high-performance |
+ | The [http://hackage.haskell.org/package/pipes-attoparsec pipes-attoparsec] library converts [http://hackage.haskell.org/package/attoparsec attoparsec] parsers to pipes for high-performance incremental parsing. |
Source code hosted on [https://github.com/k0001/pipes-attoparsec GitHub] |
Source code hosted on [https://github.com/k0001/pipes-attoparsec GitHub] |
||
+ | |||
+ | === pipes-binary === |
||
+ | |||
+ | The [http://hackage.haskell.org/package/pipes-binary pipes-binary] library allows streams of binary data to be encoded and decoded using the <code>Binary</code> instances from the [http://hackage.haskell.org/package/binary binary] package. |
||
+ | |||
+ | Source code is hosted on [https://github.com/k0001/pipes-binary GitHub] |
||
=== pipes-network === |
=== pipes-network === |
||
Line 74: | Line 73: | ||
Source code hosted on [https://github.com/k0001/pipes-network GitHub] |
Source code hosted on [https://github.com/k0001/pipes-network GitHub] |
||
+ | |||
+ | === pipes-network-tls === |
||
+ | |||
+ | The [http://hackage.haskell.org/package/pipes-network-tls pipes-network-tls] allows streaming through TLS-secured network connections, exposing a similar API to the one exposed by [http://hackage.haskell.org/package/pipes-network pipes-network]. |
||
+ | |||
+ | Source code hosted on [https://github.com/k0001/pipes-network-tls GitHub] |
||
=== pipes-zlib === |
=== pipes-zlib === |
||
Line 80: | Line 85: | ||
Source code hosted on [https://github.com/k0001/pipes-zlib GitHub] |
Source code hosted on [https://github.com/k0001/pipes-zlib GitHub] |
||
+ | |||
+ | == Community == |
||
+ | |||
+ | Besides the usual Haskell community channels such as the [[Mailing_lists|official mailing lists]] or the [http://reddit.com/r/haskell Haskell subreddit], you can ask for help, suggest improvements, or discuss about the Pipes ecosystem at the [https://groups.google.com/group/haskell-pipes “haskell-pipes” mailing list at Google Groups]. |
||
== Announcements == |
== Announcements == |
||
− | In chronological order |
+ | In chronological order: |
* [http://www.reddit.com/r/haskell/comments/ohjg7/a_new_approach_to_iteratees/ pipes-1.0.0]: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single [http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Category.html Category]. |
* [http://www.reddit.com/r/haskell/comments/ohjg7/a_new_approach_to_iteratees/ pipes-1.0.0]: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single [http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Category.html Category]. |
||
Line 108: | Line 117: | ||
* [https://groups.google.com/forum/?fromgroups=#!topic/haskell-pipes/O4_5dZ7WwXE pipes-zlib-0.2.0.0]: Zlib compression/decompression support for pipes |
* [https://groups.google.com/forum/?fromgroups=#!topic/haskell-pipes/O4_5dZ7WwXE pipes-zlib-0.2.0.0]: Zlib compression/decompression support for pipes |
||
+ | |||
+ | * [https://groups.google.com/d/msg/haskell-pipes/XGb-8x5YWY0/41BUbJagnuEJ pipes-network-tls-0.1.0.0]: Stream through TLS-secured network connections. |
||
+ | |||
+ | * [http://www.haskellforall.com/2013/06/pipes-parse-100-pushback-delimited.html pipes-parse-1.0.0]: Pushback, delimited parsers, resumable parsing, and lenses |
||
+ | |||
+ | * [https://groups.google.com/d/msg/haskell-pipes/3A-RbzCUovw/9GEIDi5WRbwJ pipes-attoparsec-0.2.0.0]: New API, interleaved parsing support, built on top of pipes-parse. |
||
+ | |||
+ | * [https://groups.google.com/d/msg/haskell-pipes/2CQ9eiPUxjA/kBqcQboPdjoJ pipes-aeson-0.1.0.0]: Encode and decode JSON streams. |
||
+ | |||
+ | * [https://groups.google.com/d/msg/haskell-pipes/7I8KcMW3zHU/5Mi8PKbuv2sJ pipes-binary-0.1.0.0]: Encode and decode binary streams using the <code>binary</code> package. |
||
+ | |||
+ | * [http://www.haskellforall.com/2013/09/pipes-40-simpler-types-and-api.html pipes-4.0.0]: Simpler types and API |
||
== Upcoming libraries == |
== Upcoming libraries == |
||
+ | * pipes-bytestring: ByteString support. Source code is hosted at [https://github.com/Gabriel439/Haskell-Pipes-ByteString-Library GitHub] |
||
− | * pipes-parse: Parsing support for the pipes ecosystem |
||
+ | |||
⚫ | |||
+ | |||
+ | == Videos == |
||
+ | * [http://www.youtube.com/watch?v=2jdJGdA7AYs Pipes], by Oliver Charles. London Haskell user group, 18th September 2013. |
||
⚫ | |||
+ | == See also == |
||
− | * pipes-free: Pipe suspension and single-stepping pipes |
||
+ | * [http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit The core flaw of pipes and conduit] (blog article) |
||
− | * pipes-network-tls: TLS-secured alternative to pipes-network |
Revision as of 22:22, 23 October 2013
The pipes library is a clean and powerful stream processing library that lets you build and connect reusable streaming components.
Libraries
Pipes is the official Hackage category for the pipes ecosystem.
pipes
The pipes package is the core library which provides reusable primitives for stream programming. Use these as building blocks for more sophisticated streaming abstractions.
Read the official pipes tutorial to learn more.
Source code hosted on GitHub
pipes-safe
The pipes-safe package adds resource management and exception safety to the pipes ecosystem. Use pipes-safe to safely acquire and release resources deterministically within a pipeline.
Source code hosted on GitHub
pipes-concurrency
The pipes-concurrency package adds concurrency primitives to the pipes ecosystem with built-in deadlock safety. Use these primitives to:
- Build reactive event-driven programs
- Merge and broadcast streams
- Communicate between multiple concurrent pipelines
Read the official pipes-concurrency tutorial to learn more.
Source code hosted on GitHub
pipes-parse
The pipes-parse package defines the generic machinery necessary for common parsing tasks. Use pipes-parse to:
- handle leftovers and end of input,
- interrupt and resume streaming, and
- sub-divide streams without collecting elements in memory.
Source code hosted on GitHub
Community-contributed libraries
Listed in alphabetical order.
pipes-aeson
The pipes-aeson library allows you to encode and decode JSON values flowing through streams, possibly interleaving other stream effects while doing it.
Source code is hosted on GitHub
pipes-attoparsec
The pipes-attoparsec library converts attoparsec parsers to pipes for high-performance incremental parsing.
Source code hosted on GitHub
pipes-binary
The pipes-binary library allows streams of binary data to be encoded and decoded using the Binary
instances from the binary package.
Source code is hosted on GitHub
pipes-network
The pipes-network library converts server and client sockets to pipes to seamlessly stream data over any network.
Source code hosted on GitHub
pipes-network-tls
The pipes-network-tls allows streaming through TLS-secured network connections, exposing a similar API to the one exposed by pipes-network.
Source code hosted on GitHub
pipes-zlib
The pipes-zlib enables compression and decompression of strict ByteString streams using the zlib codec.
Source code hosted on GitHub
Community
Besides the usual Haskell community channels such as the official mailing lists or the Haskell subreddit, you can ask for help, suggest improvements, or discuss about the Pipes ecosystem at the “haskell-pipes” mailing list at Google Groups.
Announcements
In chronological order:
- pipes-1.0.0: The original announcement, which first introduced the concept of unifying sources and sinks and transducers into a single Category.
- pipes-2.0.0: The introduction of Frames, later deprecated in favor of pipes-safe.
- pipes-2.1.0: Transition of Frames to indexed monads, later deprecated in favor of pipes-safe.
- pipes-2.3.0: The introduction of bidirectional Proxies, which evolved into modern pipes.
- pipes-2.4.0: The release of the proxy transformer extension system.
- pipes-2.5.0: Major performance improvements.
- pipes-3.0.0: Major API simplification and consolidation.
- pipes-safe-1.0.0: Resource safety and exception management
- pipes-network-0.1.0: Use TCP network sockets as pipes streams.
- pipes-3.2.0: ListT integration, Codensity proxy transformer, and ArrowChoice primitives.
- pipes-concurrency-1.0.0: Concurrency support for pipes
- pipes-zlib-0.2.0.0: Zlib compression/decompression support for pipes
- pipes-network-tls-0.1.0.0: Stream through TLS-secured network connections.
- pipes-parse-1.0.0: Pushback, delimited parsers, resumable parsing, and lenses
- pipes-attoparsec-0.2.0.0: New API, interleaved parsing support, built on top of pipes-parse.
- pipes-aeson-0.1.0.0: Encode and decode JSON streams.
- pipes-binary-0.1.0.0: Encode and decode binary streams using the
binary
package.
- pipes-4.0.0: Simpler types and API
Upcoming libraries
- pipes-bytestring: ByteString support. Source code is hosted at GitHub
- pipes-text: Text support
Videos
- Pipes, by Oliver Charles. London Haskell user group, 18th September 2013.
See also
- The core flaw of pipes and conduit (blog article)