Re-export

Generally speaking, yielding values from inside a Conduit requires some allocation for constructors. This can introduce an overhead, similar to the overhead needed to represent a list of values instead of a vector. This overhead is even more severe when talking about unboxed values.

This combinator allows you to overcome this overhead, and efficiently fill up vectors. It takes two parameters. The first is the size of each mutable vector to be allocated. The second is a function. The function takes an argument which will yield the next value into a mutable vector.

Under the surface, this function uses a number of tricks to get high performance. For more information on both usage and implementation, please see: https://www.fpcomplete.com/user/snoyberg/library-documentation/vectorbuilder

Since: conduit-1.3.0

Same as linesUnbounded, but for ASCII/binary data.

Since: conduit-1.3.0

Convert a stream of arbitrarily-chunked textual data into a stream of data where each chunk represents a single line. Note that, if you have unknownuntrusted input, this function is unsafe/, since it would allow an attacker to form lines of massive length and exhaust memory.

Since: conduit-1.3.0

Same as unlines, but operates on ASCII/binary data.

Since: conduit-1.3.0

Insert a newline character after each incoming chunk of data.

Since: conduit-1.3.0

Same as line, but operates on ASCII/binary data.

Since: conduit-1.3.0

Stream in the entirety of a single line.

Like takeExactly, this will consume the entirety of the line regardless of the behavior of the inner Conduit.

Since: conduit-1.3.0

Decode a stream of binary data as UTF8, replacing any invalid bytes with the Unicode replacement character.

Since: conduit-1.3.0

Decode a stream of binary data as UTF8.

Since: conduit-1.3.0

Encode a stream of text as UTF8.

Since: conduit-1.3.0

concatMapM with an accumulator.

Since: conduit-1.3.0

Monadic mapAccumWhileC.

Analog of scanl for lists, monadic.

Since: conduit-1.3.0

Apply a monadic action on all values in a stream.

This Conduit can be used to perform a monadic side-effect for every value, whilst passing the value through the Conduit as-is.

iterM f = mapM (\a -> f a >>= \() -> return a)

Since: conduit-1.3.0

Keep only elements in the chunked stream passing a given monadic predicate.

Since: conduit-1.3.0

Keep only values in the stream passing a given monadic predicate.

Since: conduit-1.3.0

Apply the monadic function to each value in the stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMapM, mapMaybeM, and mapFoldableM.

Since: conduit-1.3.0

Apply a monadic monomorphic transformation to all elements in a chunked stream.

Unlike mapME, this will work on types like ByteString and Text which are MonoFunctor but not Functor.

Since: conduit-1.3.0

Apply a monadic transformation to all elements in a chunked stream.

Since: conduit-1.3.0

Apply a monadic transformation to all values in a stream.

If you do not need the transformed values, and instead just want the monadic side-effects of running the action, see mapM_.

Since: conduit-1.3.0

Split input into chunk of size chunkSize

If the input does not split into chunks exactly, the remainder will be leftover (see also chunksOfE)

Since: conduit-1.3.0

Split input into chunk of size chunkSize

The last element may be smaller than the chunkSize (see also chunksOfExactlyE which will not yield this last element)

Since: conduit-1.3.0

Sliding window of values 1,2,3,4,5 with window size 2 gives [1,2],[2,3],[3,4],[4,5]

Best used with structures that support O(1) snoc.

Since: conduit-1.3.0

Insert the given value between each two values in the stream.

Since: conduit-1.3.0

concatMap with an accumulator.

Since: conduit-1.3.0

mapWhileC with a break condition dependent on a strict accumulator. Equivalently, mapAccum as long as the result is Right. Instead of producing a leftover, the breaking input determines the resulting accumulator via Left.

Analog of scanl for lists.

Since: conduit-1.3.0

Break up a stream of values into vectors of size n. The final vector may be smaller than n if the total number of values is not a strict multiple of n. No empty vectors will be yielded.

Since: conduit-1.3.0

Map values as long as the result is Just.

Since: conduit-1.3.0

Keep only elements in the chunked stream passing a given predicate.

Since: conduit-1.3.0

Keep only values in the stream passing a given predicate.

Since: conduit-1.3.0

Flatten out a stream by yielding the values contained in an incoming MonoFoldable as individually yielded values.

Since: conduit-1.3.0

Same as takeExactly, but for chunked streams.

Since: conduit-1.3.0

Consume precisely the given number of values and feed them downstream.

This function is in contrast to take, which will only consume up to the given number of values, and will terminate early if downstream terminates early. This function will discard any additional values in the stream if they are unconsumed.

Note that this function takes a downstream ConduitT as a parameter, as opposed to working with normal fusion. For more information, see http://www.yesodweb.com/blog/2013/10/core-flaw-pipes-conduit, the section titled "pipes and conduit: isolate".

Since: conduit-1.3.0

Stream all elements downstream that match the given predicate in a chunked stream.

Same caveats regarding downstream termination apply as with takeE.

Since: conduit-1.3.0

Stream all values downstream that match the given predicate.

Same caveats regarding downstream termination apply as with take.

Since: conduit-1.3.0

Stream up to n number of elements downstream in a chunked stream.

Note that, if downstream terminates early, not all values will be consumed. If you want to force exactly the given number of values to be consumed, see takeExactlyE.

Since: conduit-1.3.0

Stream up to n number of values downstream.

Note that, if downstream terminates early, not all values will be consumed. If you want to force exactly the given number of values to be consumed, see takeExactly.

Since: conduit-1.3.0

Apply the function to each element in the chunked stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMap, mapMaybe, and mapFoldable.

Since: conduit-1.3.0

Apply the function to each value in the stream, resulting in a foldable value (e.g., a list). Then yield each of the individual values in that foldable value separately.

Generalizes concatMap, mapMaybe, and mapFoldable.

Since: conduit-1.3.0

Apply a monomorphic transformation to all elements in a chunked stream.

Unlike mapE, this will work on types like ByteString and Text which are MonoFunctor but not Functor.

Since: conduit-1.3.0

Apply a transformation to all elements in a chunked stream.

Since: conduit-1.3.0

Apply a transformation to all values in a stream.

Since: conduit-1.3.0

sinkHandle applied to stderr.

Since: conduit-1.3.0

sinkHandle applied to stdout.

Since: conduit-1.3.0

Print all incoming values to stdout.

Since: conduit-1.3.0

Apply the provided monadic mapping function and monoidal combine all elements in the chunked stream.

Since: conduit-1.3.0

Apply the provided monadic mapping function and monoidal combine all values.

Since: conduit-1.3.0

A monadic strict left fold on a chunked stream.

Since: conduit-1.3.0

A monadic strict left fold.

Since: conduit-1.3.0

Apply the action to all elements in the chunked stream.

Note: the same caveat as with mapM_C applies. If you don't want to consume the values, you can use iterM:

iterM (omapM_ f)

Since: conduit-1.3.0

Apply the action to all values in the stream.

Note: if you want to pass the values instead of consuming them, use iterM instead.

Since: conduit-1.3.0

Find the first matching value.

Since: conduit-1.3.0

Get the product of all elements in the chunked stream.

Since: conduit-1.3.0

Get the product of all values in the stream.

Since: conduit-1.3.0

Get the sum of all elements in the chunked stream.

Since: conduit-1.3.0

Get the sum of all values in the stream.

Since: conduit-1.3.0

True if there are no elements in the chunked stream.

This function may remove empty leading chunks from the stream, but otherwise will not modify it.

Since: conduit-1.3.0

True if there are no values in the stream.

This function does not modify the stream.

Since: conduit-1.3.0

Get the smallest element in the chunked stream, if present.

Since: conduit-1.3.0

Get the smallest value in the stream, if present.

Since: conduit-1.3.0

Get the largest element in the chunked stream, if present.

Since: conduit-1.3.0

Get the largest value in the stream, if present.

Since: conduit-1.3.0

Count how many elements in the chunked stream pass the given predicate.

Since: conduit-1.3.0

Count how many values in the stream pass the given predicate.

Since: conduit-1.3.0

Count how many elements are in the chunked stream.

Since: conduit-1.3.0

Count how many values are in the stream.

Since: conduit-1.3.0

Retrieve the last element in the chunked stream, if present.

Since: conduit-1.3.0

Same as lastC, but returns a default value if none are available from the stream.

Since: conduit-1.3.0

Retrieve the last value in the stream, if present.

Since: conduit-1.3.0

View the next element in the chunked stream without consuming it.

Since: conduit-1.3.0

View the next value in the stream without consuming it.

Since: conduit-1.3.0

Get the next element in the chunked stream.

Since: conduit-1.3.0

Same as headC, but returns a default value if none are available from the stream.

Since: conduit-1.3.0

Take a single value from the stream, if available.

Since: conduit-1.3.0

Are no elements in the chunked stream equal to the given element?

Stops consuming as soon as a match is found.

Since: conduit-1.3.0

Are no values in the stream equal to the given value?

Stops consuming as soon as a match is found.

Since: conduit-1.3.0

Are any elements in the chunked stream equal to the given element?

Stops consuming as soon as a match is found.

Since: conduit-1.3.0

Are any values in the stream equal to the given value?

Stops consuming as soon as a match is found.

Since: conduit-1.3.0

Alternatively combine all values in the stream.

Since: conduit-1.3.0

Are any elements in the chunked stream True?

Consumption stops once the first True is encountered.

Since: conduit-1.3.0

Are any values in the stream True?

Consumption stops once the first True is encountered.

Since: conduit-1.3.0

Are all elements in the chunked stream True?

Consumption stops once the first False is encountered.

Since: conduit-1.3.0

Are all values in the stream True?

Consumption stops once the first False is encountered.

Since: conduit-1.3.0

Check that at least one element in the chunked stream returns True.

Subject to shortcut logic: at the first True, consumption of the stream will stop.

Since: conduit-1.3.0

Check that at least one value in the stream returns True.

Subject to shortcut logic: at the first True, consumption of the stream will stop.

Since: conduit-1.3.0

Check that all elements in the chunked stream return True.

Subject to shortcut logic: at the first False, consumption of the stream will stop.

Since: conduit-1.3.0

Check that all values in the stream return True.

Subject to shortcut logic: at the first False, consumption of the stream will stop.

Since: conduit-1.3.0

Apply the provided mapping function and monoidal combine all elements of the chunked stream.

Since: conduit-1.3.0

Apply the provided mapping function and monoidal combine all values.

Since: conduit-1.3.0

A strict left fold on a chunked stream.

Since: conduit-1.3.0

A strict left fold.

Since: conduit-1.3.0

Monoidally combine all elements in the chunked stream.

Since: conduit-1.3.0

Monoidally combine all values in the stream.

Since: conduit-1.3.0

Drop all elements in the chunked stream which match the given predicate.

Note: you likely want to use it with monadic composition. See the docs for dropC.

Since: conduit-1.3.0

Drop all values which match the given predicate.

Note: you likely want to use it with monadic composition. See the docs for dropC.

Since: conduit-1.3.0

Drop a certain number of elements from a chunked stream.

Note: you likely want to use it with monadic composition. See the docs for dropC.

Since: conduit-1.3.0

Ignore a certain number of values in the stream.

Note: since this function doesn't produce anything, you probably want to use it with (>>) instead of directly plugging it into a pipeline:

>>> runConduit $ yieldMany [1..5] .| dropC 2 .| sinkList
[]
>>> runConduit $ yieldMany [1..5] .| (dropC 2 >> sinkList)
[3,4,5]

Since: conduit-1.3.0

sourceHandle applied to stdin.

Since: conduit-1.3.0

Perform the given action n times, yielding each result.

Since: conduit-1.3.0

Repeatedly run the given action and yield all values it produces, until the provided predicate returns False.

Since: conduit-1.3.0

Repeatedly run the given action and yield all values it produces.

Since: conduit-1.3.0

Produce a finite stream consisting of n copies of the given value.

Since: conduit-1.3.0

Produce an infinite stream consisting entirely of the given value.

Since: conduit-1.3.0

Produces an infinite stream of repeated applications of f to x.

Since: conduit-1.3.0

Enumerate from a value to a final value, inclusive, via succ.

This is generally more efficient than using Prelude's enumFromTo and combining with sourceList since this avoids any intermediate data structures.

Since: conduit-1.3.0

Generate a producer from a seed value.

Since: conduit-1.3.0

Run a consuming conduit repeatedly, only stopping when there is no more data available from upstream.

In contrast to peekForever, this function will ignore empty chunks of data. So for example, if a stream of data contains an empty ByteString, it is still treated as empty, and the consuming function is not called.

Since: conduit-1.3.0

Run a consuming conduit repeatedly, only stopping when there is no more data available from upstream.

Since: conduit-1.3.0

Consume a source with a strict accumulator, in a way piecewise defined by a controlling stream. The latter will be evaluated until it terminates.

>>> let f a s = liftM (:s) $ mapC (*a) =$ CL.take a
>>> reverse $ runIdentity $ yieldMany [0..3] $$ mapAccumS f [] (yieldMany [1..])
[[],[1],[4,6],[12,15,18]] :: [[Int]]

An unsafe, but possibly more efficient buffer allocation strategy: reuse the buffer, if it is big enough for the next build step to execute.

Since: conduit-1.3.0

The simplest buffer allocation strategy: whenever a buffer is requested, allocate a new one that is big enough for the next build step to execute.

NOTE that this allocation strategy may spill quite some memory upon direct insertion of a bytestring by the builder. Thats no problem for garbage collection, but it may lead to unreasonably high memory consumption in special circumstances.

Since: conduit-1.3.0

Since: conduit-1.3.0

A conduit that incrementally executes builders and passes on the filled chunks as bytestrings to an inner sink.

INV: All bytestrings passed to the inner sink are non-empty.

Since: conduit-1.3.0

Incrementally execute builders on the given buffer and pass on the filled chunks as bytestrings. Note that, if the given buffer is too small for the execution of a build step, a larger one will be allocated.

WARNING: This conduit yields bytestrings that are NOT referentially transparent. Their content will be overwritten as soon as control is returned from the inner sink!

Since: conduit-1.3.0

Same as builderToByteString, but input and output are wrapped in Flush.

Since: conduit-1.3.0

Incrementally execute builders and pass on the filled chunks as bytestrings.

Since: conduit-1.3.0

sinkFile specialized to ByteString to help with type inference.

Since: conduit-1.3.0

Same as await, but discards any leading onull values.

Since: conduit-1.3.0

Consume and discard all remaining values in the stream.

Subject to fusion

Since: conduit-1.3.0

Same as sinkBuilder, but afterwards convert the builder to its lazy representation.

Alternatively, this could be considered an alternative to sinkLazy, with the following differences:

• This function will allow multiple input types, not just the strict version of the lazy structure.
• Some buffer copying may occur in this version.

Subject to fusion

Since: conduit-1.3.0

Sink incoming values into a vector, up until size maxSize. Subsequent values will be left in the stream. If there are less than maxSize values present, returns a Vector of smaller size.

Note that using this function is more memory efficient than sinkList and then converting to a Vector, as it avoids intermediate list constructors.

Subject to fusion

Since: conduit-1.3.0

Sink incoming values into a vector, growing the vector as necessary to fit more elements.

Note that using this function is more memory efficient than sinkList and then converting to a Vector, as it avoids intermediate list constructors.

Subject to fusion

Since: conduit-1.3.0

Consume all values from the stream and return as a list. Note that this will pull all values into memory.

Subject to fusion

Since: conduit-1.3.0

Consume all incoming strict chunks into a lazy sequence. Note that the entirety of the sequence will be resident at memory.

This can be used to consume a stream of strict ByteStrings into a lazy ByteString, for example.

Subject to fusion

Since: conduit-1.3.0

Deeply stream the contents of the given directory.

This works the same as sourceDirectory, but will not return directories at all. This function also takes an extra parameter to indicate whether symlinks will be followed.

Since: conduit-1.3.0

Stream the contents of the given directory, without traversing deeply.

This function will return all of the contents of the directory, whether they be files, directories, etc.

Note that the generated filepaths will be the complete path, not just the filename. In other words, if you have a directory foo containing files bar and baz, and you use sourceDirectory on foo, the results will be foo/bar and foo/baz.

Since: conduit-1.3.0

Same as withSinkFile, but lets you use a Builder.

Since: conduit-1.3.0

Like withBinaryFile, but provides a sink to write bytes to.

Since: conduit-1.3.0

Like withBinaryFile, but provides a source to read bytes from.

Since: conduit-1.3.0

An alternative to sinkHandle. Instead of taking a pre-opened Handle, it takes an action that opens a Handle (in write mode), so that it can open it only when needed and close it as soon as possible.

Since: conduit-1.3.0

Stream incoming Flushes, executing them on IO.Handle Note that this function does not automatically close the Handle when processing completes

Since: conduit-1.3.0

Stream incoming builders, executing them directly on the buffer of the given Handle. Note that this function does not automatically close the Handle when processing completes. Pass flush to flush the buffer.

Since: conduit-1.3.0

Stream all incoming data to the given Handle. Note that this function does not flush and will not close the Handle when processing completes.

Since: conduit-1.3.0

Same as sinkTempFile, but will use the default temp file directory for the system as the first argument.

Since: conduit-1.3.0

Stream data into a temporary file in the given directory with the given filename pattern, and return the temporary filename. The temporary file will be automatically deleted when exiting the active ResourceT block, if it still exists.

Since: conduit-1.3.0

Like sinkFileCautious, but uses the with pattern instead of MonadResource.

Since: conduit-1.3.0

Cautious version of sinkFile. The idea here is to stream the values to a temporary file in the same directory of the destination file, and only on successfully writing the entire file, moves it atomically to the destination path.

In the event of an exception occurring, the temporary file will be deleted and no move will be made. If the application shuts down without running exception handling (such as machine failure or a SIGKILL), the temporary file will remain and the destination file will be untouched.

Since: conduit-1.3.0

Stream all incoming data to the given file.

Since: conduit-1.3.0

Same as sourceFile. The alternate name is a holdover from an older version, when sourceFile was more polymorphic than it is today.

Since: conduit-1.3.0

An alternative to sourceHandle. Instead of taking a pre-opened Handle, it takes an action that opens a Handle (in read mode), so that it can open it only when needed and close it as soon as possible.

Since: conduit-1.3.0

Same as sourceHandle, but instead of allocating a new buffer for each incoming chunk of data, reuses the same buffer. Therefore, the ByteStrings yielded by this function are not referentially transparent between two different yields.

This function will be slightly more efficient than sourceHandle by avoiding allocations and reducing garbage collections, but should only be used if you can guarantee that you do not reuse a ByteString (or any slice thereof) between two calls to await.

Since: conduit-1.3.0

Stream the contents of a Handle as binary data. Note that this function will not automatically close the Handle when processing completes, since it did not acquire the Handle in the first place.

Since: conduit-1.3.0

Stream the contents of a file as binary data.

Since: conduit-1.3.0

Generate a producer by yielding each of the strict chunks in a LazySequence.

For more information, see toChunks.

Subject to fusion

Since: conduit-1.3.0

Yield each of the values contained by the given MonoFoldable.

This will work on many data structures, including lists, ByteStrings, and Vectors.

Subject to fusion

Since: conduit-1.3.0

A buffer allocation strategy (buf0, nextBuf) specifies the initial buffer to use and how to compute a new buffer nextBuf minSize buf with at least size minSize from a filled buffer buf. The double nesting of the IO monad helps to ensure that the reference to the filled buffer buf is lost as soon as possible, but the new buffer doesn't have to be allocated too early.

Since: conduit-1.3.0

Execute RWST in the base monad

Since 1.0.11

Evaluate RWST in the base monad

Since 1.0.11

Run RWST in the base monad

Since 1.0.11

Wrap the base monad in RWST

Since 1.0.11

Execute RWST in the base monad

Since 1.0.11

Evaluate RWST in the base monad

Since 1.0.11

Run RWST in the base monad

Since 1.0.11

Wrap the base monad in RWST

Since 1.0.11

Execute WriterT in the base monad

Since 1.0.11

Run WriterT in the base monad

Since 1.0.11

Wrap the base monad in WriterT

Since 1.0.11

Execute WriterT in the base monad

Since 1.0.11

Run WriterT in the base monad

Since 1.0.11

Wrap the base monad in WriterT

Since 1.0.11

Execute StateT in the base monad

Since 1.0.11

Evaluate StateT in the base monad

Since 1.0.11

Run StateT in the base monad

Since 1.0.11

Wrap the base monad in StateT

Since 1.0.11

Execute StateT in the base monad

Since 1.0.11

Evaluate StateT in the base monad

Since 1.0.11

Run StateT in the base monad

Since 1.0.11

Wrap the base monad in StateT

Since 1.0.11

Run ReaderT in the base monad

Since 1.0.11

Wrap the base monad in ReaderT

Since 1.0.11

Run MaybeT in the base monad

Since 1.0.11

Wrap the base monad in MaybeT

Since 1.0.11

Catch an exception in the base monad

Since 1.1.0

Run CatchT in the base monad

Since 1.1.0

Catch an error in the base monad

Since 1.2.12

Run ExceptT in the base monad

Since 1.2.12

Wrap the base monad in ExceptT

Since 1.2.12

Run a pipeline which acquires resources with ResourceT, and then run the ResourceT transformer. This is equivalent to runResourceT . runConduit.

Since: conduit-1.2.8