Module org.elasticsearch.server

Package org.elasticsearch.common.io.stream

Class BytesStreamOutput

java.lang.Object

java.io.OutputStream

org.elasticsearch.common.io.stream.StreamOutput

org.elasticsearch.common.io.stream.BytesStream

org.elasticsearch.common.io.stream.BytesStreamOutput

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

Direct Known Subclasses:

ReleasableBytesStreamOutput

public class BytesStreamOutput
extends BytesStream

A @link StreamOutput that accumulates the resulting data in memory, using BigArrays to avoids frequent reallocation & copying of the internal data once the resulting data grows large enough whilst avoiding excessive overhead in the final result for small objects.

A BytesStreamOutput accumulates data using a non-recycling BigArrays and, as with an OutputStreamStreamOutput, it uses a thread-locally-cached buffer for some of its writes and pushes data to the underlying array in small chunks, causing frequent calls to BigArrays.resize(org.elasticsearch.common.util.ByteArray, long). If the array is large enough (≥16kiB) then the resize operations happen in-place, allocating a new 16kiB byte[] and appending it to the array, but for smaller arrays these resize operations allocate a completely fresh byte[] into which they copy the entire contents of the old one.

BigArrays.resize(org.elasticsearch.common.util.ByteArray, long) grows smaller arrays more slowly than a ByteArrayOutputStream, with a target of 12.5% overhead rather than 100%, which means that a sequence of smaller writes causes more allocations and copying overall. It may be worth adding a BufferedStreamOutput wrapper to reduce the frequency of the resize operations, especially if a suitable buffer is already allocated and available.

The resulting BytesReference is a view over the underlying byte[] pages and involves no significant extra allocation to obtain. It is oversized: The worst case for overhead is when the data is one byte more than a 16kiB page and therefore the result must retain two pages even though all but one byte of the second page is unused. For smaller objects the overhead will be 12.5%.

Any memory allocated in this way is untracked by the org.elasticsearch.common.breaker subsystem unless the caller takes steps to add this tracking themselves.

Field Summary

Fields

Modifier and Type	Field	Description
protected final BigArrays	bigArrays
protected ByteArray	bytes
protected int	count

Fields inherited from class org.elasticsearch.common.io.stream.StreamOutput

GENERIC_LIST_HEADER

Constructor Summary

Constructors

Modifier	Constructor	Description
	BytesStreamOutput()	Create a non recycling BytesStreamOutput with an initial capacity of 0.
	BytesStreamOutput(int expectedSize)	Create a non recycling BytesStreamOutput with enough initial pages acquired to satisfy the capacity given by expected size.
protected	BytesStreamOutput(int expectedSize, BigArrays bigArrays)

Method Summary

Modifier and Type	Method	Description
BytesReference	bytes()
void	close()	Closes this stream to further operations.
BytesReference	copyBytes()	Like bytes() but copies the bytes to a freshly allocated buffer.
protected void	ensureCapacity(long offset)
void	flush()	Forces any buffered output to be written.
long	position()
void	reset()
void	seek(long position)
int	size()	Returns the current size of the buffer.
void	skip(int length)
void	writeByte(byte b)	Writes a single byte.
void	writeBytes(byte[] b, int offset, int length)	Writes an array of bytes.
void	writeGenericString(String value)	Write a String within StreamOutput.writeGenericValue(java.lang.Object), represented as the type code byte 0 followed by the string itself written as if with StreamOutput.writeString(java.lang.String).
void	writeOptionalString(String str)	Write a possibly-null String, represented as a invalid reference boolean which is false if the string is null, or else true if it is not null, and in this latter case it is followed by the string itself written as if with StreamOutput.writeString(java.lang.String).
void	writeString(String str)	Write a String: its length in Unicode code units (chars) (NB not bytes) written using StreamOutput.writeVInt(int) followed by the sequence of characters themselves encoded as UTF-8.

Methods inherited from class org.elasticsearch.common.io.stream.BytesStream

writeWithSizePrefix

Methods inherited from class org.elasticsearch.common.io.stream.StreamOutput

checkWriteable, getTransportVersion, legacyWriteWithSizePrefix, setTransportVersion, write, write, writeArray, writeArray, writeBigInteger, writeBoolean, writeByteArray, writeBytes, writeBytes, writeBytesRef, writeBytesReference, writeCollection, writeCollection, writeDouble, writeDoubleArray, writeDoubleLE, writeEnum, writeEnumSet, writeException, writeFloat, writeFloatArray, writeGenericList, writeGenericMap, writeGenericNull, writeGenericValue, writeGeoPoint, writeInstant, writeInt, writeIntArray, writeIntLE, writeLong, writeLongArray, writeLongLE, writeMap, writeMap, writeMap, writeMapValues, writeMapValues, writeMapWithConsistentOrder, writeMissingString, writeMissingWriteable, writeNamedWriteable, writeNamedWriteableCollection, writeOptional, writeOptionalArray, writeOptionalArray, writeOptionalBoolean, writeOptionalByteArray, writeOptionalBytesReference, writeOptionalCollection, writeOptionalCollection, writeOptionalDouble, writeOptionalEnum, writeOptionalException, writeOptionalFloat, writeOptionalFloatArray, writeOptionalInstant, writeOptionalInt, writeOptionalLong, writeOptionalMap, writeOptionalNamedWriteable, writeOptionalNamedWriteableCollection, writeOptionalSecureString, writeOptionalStringArray, writeOptionalStringCollection, writeOptionalText, writeOptionalTimeValue, writeOptionalVInt, writeOptionalVLong, writeOptionalWriteable, writeOptionalZoneId, writeSecureString, writeShort, writeStringArray, writeStringArrayNullable, writeStringCollection, writeText, writeTimeValue, writeVInt, writeVIntArray, writeVLong, writeVLongArray, writeWriteable, writeZLong, writeZoneId

Methods inherited from class java.io.OutputStream

nullOutputStream, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

bigArrays

protected final BigArrays bigArrays

bytes

@Nullable
protected ByteArray bytes

count

protected int count

Constructor Details

BytesStreamOutput

public BytesStreamOutput()

Create a non recycling BytesStreamOutput with an initial capacity of 0.

BytesStreamOutput

public BytesStreamOutput(int expectedSize)

Create a non recycling BytesStreamOutput with enough initial pages acquired to satisfy the capacity given by expected size.

Parameters:

expectedSize - the expected maximum size of the stream in bytes.

BytesStreamOutput

protected BytesStreamOutput(int expectedSize,
 BigArrays bigArrays)

Method Details

position

public long position()

Specified by:

position in class StreamOutput

Returns:

the current position of the stream, in bytes. Increases as data is written to the stream. If the stream is seekable then the seek operation updates the current position().

writeByte

public void writeByte(byte b)

Description copied from class: StreamOutput

Writes a single byte.

Specified by:

writeByte in class StreamOutput

writeBytes

public void writeBytes(byte[] b,
 int offset,
 int length)

Description copied from class: StreamOutput

Writes an array of bytes.

Specified by:

writeBytes in class StreamOutput

Parameters:

b - the bytes to write

offset - the offset in the byte array

length - the number of bytes to write

writeString

public void writeString(String str)
                 throws IOException

Description copied from class: StreamOutput

Write a String: its length in Unicode code units (chars) (NB not bytes) written using StreamOutput.writeVInt(int) followed by the sequence of characters themselves encoded as UTF-8.

May be performance-critical, so subclasses must specify an explicit implementation. If performance is unimportant, consider using StreamOutputHelper.writeString(java.lang.String, java.io.OutputStream).

Specified by:

writeString in class StreamOutput

Throws:

IOException

writeOptionalString

public void writeOptionalString(@Nullable
 String str)
                         throws IOException

Description copied from class: StreamOutput

Write a possibly-null String, represented as a

invalid reference

boolean

which is false if the string is null, or else true if it is not null, and in this latter case it is followed by the string itself written as if with StreamOutput.writeString(java.lang.String).

May be performance-critical, so subclasses must specify an explicit implementation. If performance is unimportant, consider using StreamOutputHelper.writeOptionalString(java.lang.String, java.io.OutputStream).

Specified by:

writeOptionalString in class StreamOutput

Throws:

IOException

writeGenericString

public void writeGenericString(String value)
                        throws IOException

Description copied from class: StreamOutput

Write a String within StreamOutput.writeGenericValue(java.lang.Object), represented as the type code byte 0 followed by the string itself written as if with StreamOutput.writeString(java.lang.String).

May be performance-critical, so subclasses must specify an explicit implementation. If performance is unimportant, consider using StreamOutputHelper.writeGenericString(java.lang.String, java.io.OutputStream).

Specified by:

writeGenericString in class StreamOutput

Throws:

IOException

reset

public void reset()

flush

public void flush()

Description copied from class: StreamOutput

Forces any buffered output to be written.

Specified by:

flush in interface Flushable

Specified by:

flush in class StreamOutput

seek

public void seek(long position)

Specified by:

seek in class BytesStream

skip

public void skip(int length)

close

public void close()

Description copied from class: StreamOutput

Closes this stream to further operations.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in class StreamOutput

size

public int size()

Returns the current size of the buffer.

Returns:

the value of the count field, which is the number of valid bytes in this output stream.

See Also:

• ByteArrayOutputStream.size()

bytes

public BytesReference bytes()

Specified by:

bytes in class BytesStream

copyBytes

public BytesReference copyBytes()

Like bytes() but copies the bytes to a freshly allocated buffer.

Returns:

copy of the bytes in this instances

ensureCapacity

protected void ensureCapacity(long offset)