Module org.elasticsearch.compute
Package org.elasticsearch.compute.data

Class OrdinalBytesRefBlock

java.lang.Object
org.elasticsearch.compute.data.AbstractNonThreadSafeRefCounted
org.elasticsearch.compute.data.OrdinalBytesRefBlock
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.lucene.util.Accountable, Writeable, Block, BytesRefBlock, org.elasticsearch.core.RefCounted, org.elasticsearch.core.Releasable, BlockLoader.Block
public final class OrdinalBytesRefBlock extends AbstractNonThreadSafeRefCounted implements BytesRefBlock
A BytesRefBlock consists of a pair: an IntBlock for ordinals and a BytesRefVector for the dictionary. Compared to the regular BytesRefBlock, this block is slower due to indirect access and consume more memory because of the additional ordinals block. However, they offer significant speed improvements and reduced memory usage when byte values are frequently repeated

  • Constructor Details

  • Method Details

    • isDense

      public boolean isDense()
      Returns true if this ordinal block is dense enough to enable optimizations using its ordinals

    • isDense

      public static boolean isDense(long totalPositions, long dictionarySize)

    • getOrdinalsBlock

      public IntBlock getOrdinalsBlock()

    • getDictionaryVector

      public BytesRefVector getDictionaryVector()

    • getBytesRef

      public org.apache.lucene.util.BytesRef getBytesRef(int valueIndex, org.apache.lucene.util.BytesRef dest)
      Description copied from interface: BytesRefBlock
      Retrieves the BytesRef value stored at the given value index.

      Values for a given position are between getFirstValueIndex(position) (inclusive) and getFirstValueIndex(position) + getValueCount(position) (exclusive).

      Specified by:
      getBytesRef in interface BytesRefBlock
      Parameters:
      valueIndex - the value index
      dest - the destination
      Returns:
      the data value (as a BytesRef)

    • asVector

      public OrdinalBytesRefVector asVector()
      Description copied from interface: Block
      Returns an efficient dense single-value view of this block. Null, if the block is not dense single-valued. That is, if mayHaveNulls returns true, or getTotalValueCount is not equal to getPositionCount.
      Specified by:
      asVector in interface Block
      Specified by:
      asVector in interface BytesRefBlock
      Returns:
      an efficient dense single-value view of this block

    • asOrdinals

      public OrdinalBytesRefBlock asOrdinals()
      Description copied from interface: BytesRefBlock
      Returns an ordinal bytesref block if this block is backed by a dictionary and ordinals; otherwise, returns null. Callers must not release the returned block as no extra reference is retained by this method.
      Specified by:
      asOrdinals in interface BytesRefBlock

    • slice

      public BytesRefBlock slice(int beginInclusive, int endExclusive)
      Description copied from interface: Block
      Return a subset of this Block from position beginInclusive to position endExclusive. This may return the same instance if the range covers all positions, but if it does it will RefCounted.incRef() it.

      NOTE: Implementations will not try to optimize zero length slices as we expect them to be rare.

      Specified by:
      slice in interface Block
      Specified by:
      slice in interface BytesRefBlock

    • filter

      public BytesRefBlock filter(boolean mayContainDuplicates, int... positions)
      Description copied from interface: Block
      Creates a new block that only exposes the positions provided.
      Specified by:
      filter in interface Block
      Specified by:
      filter in interface BytesRefBlock
      Parameters:
      mayContainDuplicates - may the positions array contain duplicate positions?
      positions - the positions to retain
      Returns:
      a filtered block

    • keepMask

      public BytesRefBlock keepMask(BooleanVector mask)
      Description copied from interface: Block
      Build a Block with the same values as this Block, but replacing all values for which mask.getBooleanValue(position) returns false with null. The mask vector must be at least as long as this Block.
      Specified by:
      keepMask in interface Block
      Specified by:
      keepMask in interface BytesRefBlock

    • lookup

      public org.elasticsearch.core.ReleasableIterator<BytesRefBlock> lookup(IntBlock positions, ByteSizeValue targetBlockSize)
      Description copied from interface: Block
      Builds an Iterator of new Blocks with the same Block.elementType() as this Block whose values are copied from positions in this Block. It has the same number of positions as the positions parameter.

      For example, if this block contained [a, b, [b, c]] and were called with the block [0, 1, 1, [1, 2]] then the result would be [a, b, b, [b, b, c]].

      This process produces count(this) * count(positions) values per positions which could be quite large. Instead of returning a single Block, this returns an Iterator of Blocks containing all of the promised values.

      The returned ReleasableIterator may retain a reference to the positions parameter. Close it to release those references.

      This block is built using the same BlockFactory as was used to build the positions parameter.

      Specified by:
      lookup in interface Block
      Specified by:
      lookup in interface BytesRefBlock

    • deepCopy

      public OrdinalBytesRefBlock deepCopy(BlockFactory blockFactory)
      Description copied from interface: BytesRefBlock
      Make a deep copy of this Block using the provided BlockFactory, likely copying all data.
      Specified by:
      deepCopy in interface Block
      Specified by:
      deepCopy in interface BytesRefBlock

    • closeInternal

      protected void closeInternal()
      Description copied from class: AbstractNonThreadSafeRefCounted
      This is called when the number of references reaches zero. This is where resources should be released (adjusting circuit breakers if needed).
      Specified by:
      closeInternal in class AbstractNonThreadSafeRefCounted

    • getTotalValueCount

      public int getTotalValueCount()
      Description copied from interface: Block
      Returns the total number of values in this block not counting nulls. This powers the COUNT aggregation and is used to report the number of fields loaded by ESQL.
      Specified by:
      getTotalValueCount in interface Block
      Returns:
      the total number of values in this block not counting nulls

    • getPositionCount

      public int getPositionCount()
      Description copied from interface: Block
      Returns the number of positions (rows) in this block. See class javadoc for the usual way to iterate these positions.
      Specified by:
      getPositionCount in interface Block
      Returns:
      the number of positions (rows) in this block

    • getFirstValueIndex

      public int getFirstValueIndex(int position)
      Description copied from interface: Block
      Returns the index of the first value for the given position. See class javadoc for the usual way to iterate these positions.

      For densely packed data this will return its parameter unchanged. For fields with null values or multivalued fields, this will shift. Here's an example: 

      
     0   <---+
     1       | Values at first position
     2       |
     3   <---+
     5   <---- Value at second position
     6   <---+ Values at third position
     7   <---+

      This represents three rows. The first has the value [0, 1, 2, 3]. The second has the value 5. The third has the value [6, 7]. This method will return 0 for the first position, 4 for the second, and 5 for the third.

      Specified by:
      getFirstValueIndex in interface Block
      Returns:
      the index of the first value for the given position

    • getValueCount

      public int getValueCount(int position)
      Description copied from interface: Block
      Returns the number of values for the given position. See class javadoc for the usual way to iterate these positions.

      For densely packed data this will return 1. For nulls this will return 0. For multivalued fields, this will return the number of values. Here's an example: 

      
     0   <---+
     1       | Values at first position
     2       |
     3   <---+
     5   <---- Value at second position
     6   <---+ Values at third position
     7   <---+

      This represents three rows. The first has the value [0, 1, 2, 3]. The second has the value 5. The third has the value [6, 7]. This method will return 4 for the first position, 1 for the second, and 2 for the third.

      Specified by:
      getValueCount in interface Block
      Returns:
      the number of values for the given position

    • elementType

      public ElementType elementType()
      Description copied from interface: Block
      Returns the element type of this block.
      Specified by:
      elementType in interface Block
      Returns:
      the element type of this block

    • blockFactory

      public BlockFactory blockFactory()
      Description copied from interface: Block
      The block factory associated with this block.
      Specified by:
      blockFactory in interface Block

    • allowPassingToDifferentDriver

      public void allowPassingToDifferentDriver()
      Description copied from interface: Block
      Before passing a Block to another Driver, it is necessary to switch the owning block factory to its parent, which is associated with the global circuit breaker. This ensures that when the new driver releases this Block, it returns memory directly to the parent block factory instead of the local block factory of this Block. This is important because the local block factory is not thread safe and doesn't support simultaneous access by more than one thread.
      Specified by:
      allowPassingToDifferentDriver in interface Block

    • isNull

      public boolean isNull(int position)
      Specified by:
      isNull in interface Block
      Parameters:
      position - the position
      Returns:
      true if the value stored at the given position is null, false otherwise

    • mayHaveNulls

      public boolean mayHaveNulls()
      Specified by:
      mayHaveNulls in interface Block
      Returns:
      true if some values might be null. False, if all values are guaranteed to be not null.

    • areAllValuesNull

      public boolean areAllValuesNull()
      Specified by:
      areAllValuesNull in interface Block
      Returns:
      true if all values in this block are guaranteed to be null.

    • mayHaveMultivaluedFields

      public boolean mayHaveMultivaluedFields()
      Description copied from interface: Block
      Can this block have multivalued fields? Blocks that return false will never return more than one from Block.getValueCount(int). This may return true for Blocks that do not have multivalued fields, but it will always answer quickly.
      Specified by:
      mayHaveMultivaluedFields in interface Block

    • doesHaveMultivaluedFields

      public boolean doesHaveMultivaluedFields()
      Description copied from interface: Block
      Does this block have multivalued fields? Unlike Block.mayHaveMultivaluedFields() this will never return a false positive. In other words, if this returns true then there are positions for which Block.getValueCount(int) will return more than 1. This will answer quickly if it can but may have to check all positions.
      Specified by:
      doesHaveMultivaluedFields in interface Block

    • mvOrdering

      public Block.MvOrdering mvOrdering()
      Description copied from interface: Block
      How are multivalued fields ordered?
      Specified by:
      mvOrdering in interface Block

    • expand

      public OrdinalBytesRefBlock expand()
      Description copied from interface: Block
      Expand multivalued fields into one row per value. Returns the same block if there aren't any multivalued fields to expand. The returned block needs to be closed by the caller to release the block's resources.
      Specified by:
      expand in interface Block
      Specified by:
      expand in interface BytesRefBlock

    • ramBytesUsed

      public long ramBytesUsed()
      Specified by:
      ramBytesUsed in interface org.apache.lucene.util.Accountable

    • equals

      public boolean equals(Object o)
      Description copied from interface: BytesRefBlock
      Compares the given object with this block for equality. Returns true if and only if the given object is a BytesRefBlock, and both blocks are equal.
      Specified by:
      equals in interface BytesRefBlock
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Description copied from interface: BytesRefBlock
      Returns the hash code of this block, as defined by BytesRefBlock.hash(BytesRefBlock).
      Specified by:
      hashCode in interface BytesRefBlock
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object