Module org.elasticsearch.server
Package org.elasticsearch.common.io.stream

Class ReleasableBytesStreamOutput

java.lang.Object
java.io.OutputStream
org.elasticsearch.common.io.stream.StreamOutput
org.elasticsearch.common.io.stream.BytesStream
org.elasticsearch.common.io.stream.BytesStreamOutput
org.elasticsearch.common.io.stream.ReleasableBytesStreamOutput
All Implemented Interfaces:
Closeable, Flushable, AutoCloseable, Releasable
public class ReleasableBytesStreamOutput extends BytesStreamOutput implements Releasable
A @link StreamOutput that accumulates the resulting data in memory, using BigArrays to avoids frequent reallocation & copying of the internal data once the resulting data grows large enough whilst avoiding excessive overhead in the final result for small objects.

A ReleasableBytesStreamOutput behaves similarly to a BytesStreamOutput except that it accumulates data using the provided BigArrays, which typically should be a recycling instance and thus the resulting bytes must be explicitly released when no longer needed. As with the BytesStreamOutput it uses a thread-locally-cached buffer for some of its writes and pushes data to the underlying array in small chunks, causing frequent calls to BigArrays.resize(org.elasticsearch.common.util.ByteArray, long). If the array is large enough (≥8kiB) then the resize operations happen in-place, obtaining a recycled 16kiB page and appending it to the array, but for smaller arrays these resize operations allocate a completely fresh byte[] into which they copy the entire contents of the old one.

As above, smaller arrays grow slowly into freshly-allocated byte[] arrays with a target of 12.5% overhead. It may be worth adding a BufferedStreamOutput wrapper to reduce the frequency of the resize operations, especially if a suitable buffer is already allocated and available.

This is different from a RecyclerBytesStreamOutput which only uses recycled 16kiB pages and never itself allocates a raw byte[]. However, note that by default a ReleasableBytesStreamOutput uses PageCacheRecycler.PAGE_SIZE_IN_BYTES for its expectedSize so that it also always starts by using a recycled page rather than a slow-growing fresh byte[].

The resulting ReleasableBytesReference is a view over the underlying byte[] pages and involves no significant extra allocation to obtain. It is oversized: The worst case for overhead is when the data is one byte more than a 16kiB page and therefore the result must retain two pages even though all but one byte of the second page is unused. The recycling BigArrays also switches to using recycled pages at half a page (8kiB) which also carries around 50% overhead. For smaller objects the overhead will be 12.5%.

Any memory allocated in this way is tracked by the org.elasticsearch.common.breaker subsystem if using a suitably-configured BigArrays.

Please note, closing this stream will release the bytes that are in use by any ReleasableBytesReference returned from BytesStreamOutput.bytes(), so this stream should only be closed after the bytes have been output or copied elsewhere.

  • Constructor Details

    • ReleasableBytesStreamOutput

      public ReleasableBytesStreamOutput(BigArrays bigarrays)
      Create a ReleasableBytesStreamOutput, acquiring from the given BigArrays a single recycled page for the initial buffer, and growing the buffer as needed.

    • ReleasableBytesStreamOutput

      public ReleasableBytesStreamOutput(int expectedSize, BigArrays bigArrays)
      Create a ReleasableBytesStreamOutput, allocating an initial buffer of size expectedSize from the given BigArrays, and growing the buffer as needed.

      Note that if expectedSize < PageCacheRecycler.PAGE_SIZE_IN_BYTES / 2 then this will allocate a new byte[] rather than using a recycled page, and will keep on allocating new byte[] instances, copying the contents, until the contents reach PageCacheRecycler.PAGE_SIZE_IN_BYTES / 2. In the worst case this can be over 40 allocations before it gets big enough to start using recycled pages. This is probably not what you want.

  • Method Details