Ecosyste.ms: Repos
An open API service providing repository metadata for many open source software ecosystems.
Package Usage: hackage: streaming
This package contains two modules, Streaming
and Streaming.Prelude.
The principal module, Streaming.Prelude, exports an elementary streaming prelude focused on
a simple "source" or "producer" type, namely Stream (Of a) m r.
This is a sort of effectful version of
([a],r) in which successive elements of type a arise from some sort of monadic
action before the succession ends with a value of type r.
Everything in the library is organized to make
programming with this type as simple as possible,
by the simple expedient of making it as close to Prelude
and Data.List as possible. Thus for example
the trivial program
sums the first three valid integers from user input. Similarly,
upper-cases the first two lines from stdin as they arise,
and sends them to stdout. And so on,
with filtering, mapping, breaking, chunking, zipping, unzipping, replicating
and so forth:
we program with streams of Ints or Strings directly as
if they constituted something like a list. That's because streams really do constitute something
like a list, and the associated operations can mostly have the same names.
(A few, like reverse, don't stream and thus disappear;
others like unzip are here given properly streaming formulation for the first time.)
And we everywhere
oppose "extracting a pure list from IO",
which is the origin of typical Haskell memory catastrophes.
Basically any case where you are
tempted to use mapM, replicateM, traverse or sequence
with Haskell lists, you would do better to use something like
Stream (Of a) m r. The type signatures are a little fancier, but
the programs themselves are mostly the same. In fact, they are mostly simpler. Thus,
consider the trivial demo program mentioned in
this SO question
The new user notices that this exhausts memory, and worries about the efficiency of Haskell IORefs.
But of course it exhausts memory! Look what it says!
The problem is immediately cured by writing
which really does what the other program was meant to do,
uses no more memory than hello-world, and is simpler anyway, since it
doesn't involve the detour of "extracting a list from IO". Almost
every use of list mapM, replicateM, traverse and sequence produces
this problem on a smaller scale. People get used to it, as if it were
characteristic of Haskell programs to use a lot of memory. But in truth
"extracting a list or sequence from IO" is mostly just bad practice pure and simple.
Of course, mapM, replicateM, traverse and sequence make sense for lists,
under certain conditions! But unsafePerformIO also makes sense under
certain conditions.
The Streaming module exports the general type,
Stream f m r, which can be used to stream successive distinct
steps characterized by any
functor f, though we are mostly interested in organizing computations
of the form Stream (Of a) m r. The streaming-IO libraries have
various devices for dealing
with effectful variants of [a] or ([a],r) in which the emergence of
successive elements somehow depends on IO. But it is only with
the general type Stream f m r, or some equivalent,
that one can envisage (for example) the connected streaming of their
sorts of stream - as one makes lists of lists in the Haskell
Prelude and Data.List. One needs some such type if we are
to express properly streaming equivalents of e.g.
to mention a few obviously desirable operations.
(This is explained more elaborately in the readme below.)
One could of course throw something
like the present Stream type on top of a prior stream concept: this is how pipes and
pipes-group (which are very much our model here) use FreeT.
But once one grasps the iterable stream concept needed to express
those functions then one will also see that,
with it, one is already in possession of a complete
elementary streaming library - since one possesses Stream ((,) a) m r
or equivalently Stream (Of a) m r. This
is the type of a 'generator' or 'producer' or 'source' or whatever
you call an effectful stream of items.
The present Streaming.Prelude is thus the simplest streaming library that can replicate anything like the API of the Prelude and Data.List.
The emphasis of the library is on interoperation; for
the rest its advantages are: extreme simplicity, re-use of
intuitions the user has gathered from mastery of Prelude and
Data.List, and a total and systematic rejection of type synonyms.
The two conceptual pre-requisites are some
comprehension of monad transformers and some familiarity
with 'rank 2 types'. It is hoped that experimentation with this
simple material, starting with the ghci examples in Streaming.Prelude,
will give people who are new to these concepts some
intuition about their importance. The most fundamental purpose of the
library is to express elementary streaming ideas without reliance on
a complex framework, but in a way that integrates transparently with
the rest of Haskell, using ideas - e.g. rank 2 types, which are here
implicit or explicit in most mapping - that the user can carry elsewhere,
rather than chaining her understanding to the curiosities of
a so-called streaming IO framework (as necessary as that is for certain purposes.)
See the
readme
below for further explanation, including the examples linked there.
Elementary usage can be divined from the ghci examples in
Streaming.Prelude and perhaps from this rough beginning of a
tutorial.
Note also the
streaming bytestring
and
streaming utils
packages. Questions about usage can be put
raised on StackOverflow with the tag [haskell-streaming],
or as an issue on Github, or on the
pipes list
(the package understands itself as part of the pipes 'ecosystem'.)
The simplest form of interoperation with
pipes
is accomplished with this isomorphism:
Interoperation with
io-streams
is thus:
With
conduit
one might use, e.g.:
These conversions should never be more expensive than a single >-> or =$=.
The simplest interoperation with regular Haskell lists is provided by, say
The latter of course accumulates the whole list in memory, and is mostly what we are trying
to avoid. Every use of Prelude.mapM f should be reconceived as using the
composition Streaming.toList_ . Streaming.mapM f . Streaming.each with a view to
considering whether the accumulation required by Streaming.toList_ is really necessary.
Here are the results of some
microbenchmarks
based on the
benchmarks
included in the machines package:
41 versions
Latest release: 11 months ago
17 dependent packages
45,137 downloads total
View more package details: https://packages.ecosyste.ms/registries/hackage.haskell.org/packages/streaming
View more repository details: https://repos.ecosyste.ms/hosts/GitHub/repositories/haskell-streaming%2Fstreaming
Dependent Repos 129
global-access-public/streaming-sync
computes synchronization operations out of an old and a fresh stream- * streaming-sync.cabal
Last synced: 4 months ago
etiennejf/plutus-apps Fork of IntersectMBO/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 113 MB - Last synced: 4 months ago - Pushed: 4 months ago
ustenzel/biohazard-tools
Tools involving mostly BAM files.- >=0.1.4.2 && <0.3 biohazard-tools.cabal
- >=0.1.4.2 && <0.3 biohazard-tools.cabal
Size: 2.74 MB - Last synced: over 1 year ago
ustenzel/biohazard-random
tools that may or may not be a good idea- >=0.1.4.2 && <0.3 biohazard-random.cabal
Size: 590 KB - Last synced: over 1 year ago
google/codeworld
Educational computer programming environment using Haskell- * codeworld-available-pkgs/codeworld-available-pkgs.cabal
Size: 7.18 MB - Last synced: 12 days ago - Pushed: over 1 year ago
SamProtas/composable-associations
Library providing polymorphic composition of serialized haskell data to reduce boilerplate.- 0.1.4.5 composable-associations/cabal.config
- 0.1.4.5 composable-associations-aeson/cabal.config
Size: 47.9 KB - Last synced: about 1 month ago - Pushed: about 3 years ago
ustenzel/jivebunny
a probabilistic demultiplexer for Illumina sequencing data- >=0.1.4.2 && <0.3 jivebunny.cabal
Size: 652 KB - Last synced: over 1 year ago
nftdot.io/cardano-ledger
The ledger implementation and specifications of the Cardano blockchain.- * eras/byron/ledger/impl/cardano-ledger-byron.cabal
- * eras/byron/ledger/impl/cardano-ledger-byron.cabal
- * eras/byron/ledger/impl/test/cardano-ledger-byron-test.cabal
Last synced: over 1 year ago
global-access-public/one-password
One password API access- * one-password.cabal
- * one-password.cabal
Last synced: 4 months ago
ustenzel/genomfart
genotype caller for ancient DNA- >=0.1.4.2 && <0.3 genomfart.cabal
Size: 921 KB - Last synced: over 1 year ago
dnaumenko/learning_notes
Collection of exercises from various books/sites- 0.1.4.5 learning_haskell/cabal.config
Size: 4.29 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
bjpop/language-python-test
Code for testing the language-python library- 0.1.4.3 cabal.config
Size: 1.53 MB - Last synced: 22 days ago - Pushed: over 5 years ago
global-access-public/influxdb
haskell metrics with influxDB 2.0- * influx.cabal
Last synced: 4 months ago
change-metrics/lentille 📦
Issues tracker crawler for Monocle- <0.3 lentille/lentille.cabal
- <0.3 lentille-bugzilla/lentille-bugzilla.cabal
- <0.3 lentille-github/lentille-github.cabal
Size: 399 KB - Last synced: about 1 year ago - Pushed: almost 3 years ago
rmourey26/ouroboros-network
An implementation of the Ouroboros family of consensus algorithms, with its networking support- * ouroboros-consensus/ouroboros-consensus.cabal
- * ouroboros-consensus-byron/ouroboros-consensus-byron.cabal
Last synced: about 1 year ago
rmourey26/plutus
The Plutus language implementation and tools- * playground-common/playground-common.cabal
Last synced: about 1 year ago
zliu41/cardano-ledger Fork of input-output-hk/cardano-ledger
The ledger implementation and specifications of the Cardano blockchain.- * eras/byron/ledger/impl/cardano-ledger-byron.cabal
- * eras/byron/ledger/impl/cardano-ledger-byron.cabal
Size: 68.5 MB - Last synced: 6 months ago - Pushed: 6 months ago
sumitraja/upnp-client
UPNP Client primarily targeted at getting content off a ContentDirectory- 0.1.4.0 cabal.config
Size: 4.57 MB - Last synced: over 1 year ago
LightAndLight/sage
Efficient parser combinators for Haskell- * sage.cabal
- * sage.cabal
Size: 94.7 KB - Last synced: about 1 month ago - Pushed: about 1 month ago
paolino/accessing-salesforce
- * wreq-experiment.cabal
- * wreq-experiment.cabal
Size: 13.7 KB - Last synced: about 1 year ago - Pushed: almost 2 years ago
input-output-hk/cardano-ledger-1-tech-writing-tweaks Fork of input-output-hk/cardano-ledger
The ledger implementation and specifications of the Cardano blockchain.- * eras/byron/ledger/impl/test/cardano-ledger-byron-test.cabal
Size: 37.3 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
jacobstern/hamilton
- 0.1.4.5 cabal.config
- 0.1.4.5 hamilton/cabal.config
Size: 856 KB - Last synced: over 1 year ago
elaforge/karya
music sequencer and generalized notation- 0.2.3.0 cabal.config
Size: 179 MB - Last synced: about 2 months ago - Pushed: 2 months ago
karelfv/alpinelinux_aports Fork of alpinelinux/aports
[MIRROR] Alpine packages build scripts- 0.2.3.1 testing/darcs/cabal.config
- 0.2.3.1 testing/stack/cabal.config
Size: 434 MB - Last synced: about 1 year ago - Pushed: about 1 year ago
rmourey26/cardano-ledger-specs
A formal specification and executable model of the ledger rules introduced by the Shelley release- * byron/ledger/impl/cardano-ledger-byron.cabal
- * byron/ledger/impl/cardano-ledger-byron.cabal
Last synced: about 1 year ago
dcSpark/fracada-il-primo
Advanced Plutus fractionalized NFT protocol which supports "NFT Bag" functionality.- * fracada-il-primo.cabal
- * fracada-il-primo.cabal
Size: 20 MB - Last synced: about 1 month ago - Pushed: 5 months ago
victorlcasilva/aports
- 0.2.3.1 testing/darcs/cabal.config
- 0.2.3.1 testing/stack/cabal.config
Size: 252 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
tom.stejskal/apinette-gui
Simple tool to query REST API servers.- * apinette.cabal
Last synced: about 1 year ago
choener/ViennaRNA-extras
ViennaRNA v2 extensions in Haskell- >=0.1 ViennaRNA-extras.cabal
Size: 34.2 KB - Last synced: 4 months ago - Pushed: almost 2 years ago
tonmoyislam250/mineaports
- 0.2.3.1 testing/darcs/cabal.config
- 0.2.3.1 testing/stack/cabal.config
Size: 181 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
datahaskell/dh-core
Functional data science- * datasets/datasets.cabal
- * datasets/datasets.cabal
Last synced: over 1 year ago
steshaw/intellij-haskell
IntelliJ plugin for Haskell- 0.1.4.3 src/main/resources/cabal/cabal.config
Last synced: over 1 year ago
marklnichols/Strat
Haskell library for strategy game AI- 0.1.4.5 cabal.config
Last synced: over 1 year ago
YoEight/eventstore-hs
Haskell GetEventStore TCP Client- * eventstore.cabal
- * eventstore.cabal
Last synced: over 1 year ago
mattias.br/sillyWikipediaSpider
My first Haskell project (yay!): crawls through wikipedia, builds a graph with articles as nodes and links as edges. Does graph analysis. Not completed yet because I'm still horrible at fp...- * wiki.cabal
Last synced: over 1 year ago
lae/haskell-dns-message
Mirrored from https://github.com/laech/haskell-dns-message.git- 0.1.4.5 cabal.config
Last synced: over 1 year ago
jb55/streaming-http-example
Streaming http/wai examples- * streaming-http-example.cabal
Size: 1.95 KB - Last synced: about 1 year ago - Pushed: over 8 years ago
alpinelinux/aports
[MIRROR] Alpine packages build scripts- 0.2.3.1 testing/darcs/cabal.config
- 0.2.3.1 testing/stack/cabal.config
Size: 413 MB - Last synced: 5 days ago - Pushed: 5 days ago
polysemy-research/polysemy-zoo
:monkey::panda_face: Experimental, user-contributed effects and interpreters for polysemy- ==0.2.* polysemy-zoo.cabal
- ==0.2.* polysemy-zoo.cabal
Size: 205 KB - Last synced: 23 days ago - Pushed: 10 months ago
mathiasverraes/playingwithprojections-haskell
Playing with Projections - Haskell- * playingwithprojections.cabal
Size: 228 KB - Last synced: 24 days ago - Pushed: almost 4 years ago
EventStore/EventStoreDB-Client-Haskell Fork of YoEight/eventstore
EventStore Haskell TCP Client- * eventstore.cabal
- * eventstore.cabal
Size: 1.21 MB - Last synced: 29 days ago - Pushed: 10 months ago
input-output-hk/cardano-node-native-tokens-folder-TW Fork of input-output-hk/cardano-node
The core component that is used to participate in a Cardano decentralised blockchain.- * bench/tx-generator/tx-generator.cabal
Size: 23.3 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
catch-21/plutus-apps Fork of IntersectMBO/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 230 MB - Last synced: about 1 month ago - Pushed: about 1 year ago
mlabs-haskell/plutus-apps Fork of IntersectMBO/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 111 MB - Last synced: 25 days ago - Pushed: over 1 year ago
ilap/cardano-ledger-specs Fork of input-output-hk/cardano-ledger
A formal specification and executable model of the ledger rules introduced by the Shelley release- * byron/ledger/impl/cardano-ledger-byron.cabal
- * byron/ledger/impl/cardano-ledger-byron.cabal
- * byron/ledger/impl/test/cardano-ledger-byron-test.cabal
Size: 35.1 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
WingRiders/plutus-apps
- * playground-common/playground-common.cabal
Size: 108 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
bgamari/build-env
- 0.2.3.1 tests/SEEDS/cabal.config
Size: 271 KB - Last synced: 26 days ago - Pushed: about 1 year ago
joinplank/plutus-apps Fork of input-output-hk/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 115 MB - Last synced: 9 months ago - Pushed: about 1 year ago
TristanCacqueray/monocle Fork of change-metrics/monocle
Detect anomalies in your GitHub/Gerrit projects- >=0.2 haskell/monocle.cabal
Size: 6.83 MB - Last synced: 4 months ago - Pushed: 4 months ago
Shiba-Chain/plutus-apps Fork of input-output-hk/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 108 MB - Last synced: 8 months ago - Pushed: over 1 year ago
vdukhovni/streaming-bytestring Fork of haskell-streaming/streaming-bytestring
effectful sequences of bytes; an alternative no-lazy-io implementation of Data.ByteString.Lazy- >=0.1.4.0 && <0.3 streaming-bytestring.cabal
- * streaming-bytestring.cabal
Size: 569 KB - Last synced: about 1 year ago - Pushed: over 1 year ago
Ali-Hill/plutus-apps Fork of input-output-hk/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 114 MB - Last synced: about 1 year ago - Pushed: about 1 year ago
etiennejf/plutus-libs Fork of tweag/cooked-validators
- * cooked-validators/cooked-validators.cabal
- * cooked-validators/cooked-validators.cabal
- * examples/examples.cabal
- * examples/examples.cabal
Size: 1.43 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
Tharun-tharun/plutus Fork of ahaxu/plutus
The Plutus language implementation and tools- * freer-extras/freer-extras.cabal
- * marlowe/marlowe.cabal
Size: 68.6 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
effectfully/plutus-prototype Fork of IntersectMBO/plutus
- * marlowe/marlowe.cabal
- * playground-common/playground-common.cabal
Size: 118 MB - Last synced: 23 days ago - Pushed: 23 days ago
rmourey26/cardano-ledger-specs Fork of IntersectMBO/cardano-ledger
A formal specification and executable model of the ledger rules introduced by the Shelley release- * byron/ledger/impl/cardano-ledger-byron.cabal
- * byron/ledger/impl/cardano-ledger-byron.cabal
- * byron/ledger/impl/test/cardano-ledger-byron-test.cabal
Size: 74.1 MB - Last synced: 4 months ago - Pushed: 5 months ago
j-mueller/plutus-libs Fork of tweag/cooked-validators
- * cooked-validators/cooked-validators.cabal
- * cooked-validators/cooked-validators.cabal
- * examples/examples.cabal
- * examples/examples.cabal
Size: 1.08 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
j-mueller/plutus-apps Fork of etiennejf/plutus-apps
The Plutus application platform- * playground-common/playground-common.cabal
Size: 114 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
steshaw/plutus-libs Fork of tweag/cooked-validators
- * cooked-validators/cooked-validators.cabal
- * cooked-validators/cooked-validators.cabal
Size: 4.33 MB - Last synced: about 1 year ago - Pushed: over 1 year ago
bitpaint/embassy-os Fork of Start9Labs/embassy-os
A graphical operating system for running self-hosted software.- * agent/ambassador-agent.cabal
Size: 10.4 MB - Last synced: about 1 year ago - Pushed: over 1 year ago