Module org.elasticsearch.compute
Package org.elasticsearch.compute.data

Class DocVector

java.lang.Object
org.elasticsearch.compute.data.AbstractNonThreadSafeRefCounted
org.elasticsearch.compute.data.DocVector
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.lucene.util.Accountable, Vector, org.elasticsearch.core.RefCounted, org.elasticsearch.core.Releasable
public final class DocVector extends AbstractNonThreadSafeRefCounted implements Vector
Vector where each entry references a lucene document.

  • Field Details

    • SHARD_SEGMENT_DOC_MAP_PER_ROW_OVERHEAD

      public static final int SHARD_SEGMENT_DOC_MAP_PER_ROW_OVERHEAD
      Per position memory cost to build the shard segment doc map required to load fields out of order.
      See Also:

  • Constructor Details

  • Method Details

    • config

      public static DocVector.Config config()
      Fancy wrapper around the ctor for DocVector.

    • shards

      public IntVector shards()

    • segments

      public IntVector segments()

    • docs

      public IntVector docs()

    • singleSegment

      public boolean singleSegment()
      Does this vector contain only documents from a single segment.

    • mayContainDuplicates

      public boolean mayContainDuplicates()
      Can this vector reference duplicate documents? Some BlockLoaders will run more slowly if this is true. These BlockLoaders will return incorrect results if there are duplicates and this is false. This exists because of a hierarchy of speeds:
      • We can better optimize some BlockLoaders when they receive DocVectors that don't contain duplicates.
      • It's rare that we want to load from duplicate doc ids. We don't need to spend that much time optimizing it.
      • We sometimes really want to load from duplicate doc ids to minimize total amount of loading we have to do in fairly specific cases like resolving dimension values after time series aggregations.

    • singleSegmentNonDecreasing

      public boolean singleSegmentNonDecreasing()
      Does this vector contain only documents from a single segment and are the documents always non-decreasing? Non-decreasing here means almost monotonically increasing.

    • shardSegmentDocMapForwards

      public int[] shardSegmentDocMapForwards()
      Map from the positions in this page to the positions in lucene's native order for loading doc values.

    • shardSegmentDocMapBackwards

      public int[] shardSegmentDocMapBackwards()
      Reverse of shardSegmentDocMapForwards. If you load doc values in the "forward" order then you can call Block.filter(boolean, int...) on the loaded values with this array to put them in the same order as this Page.

    • sizeOfSegmentDocMap

      public static long sizeOfSegmentDocMap(int positionCount)

    • asBlock

      public DocBlock asBlock()
      Description copied from interface: Vector
      Returns a new Block containing this vector.
      Specified by:
      asBlock in interface Vector
      Returns:
      a new Block containing this vector

    • slice

      public DocVector slice(int beginInclusive, int endExclusive)
      Description copied from interface: Vector
      Return a subset of this Vector from position beginInclusive to position endExclusive. This may return the same instance if the range covers all positions, but if it does it will RefCounted.incRef() it.

      NOTE: Implementations will not try to optimize zero length slices as we expect them to be rare.

      Specified by:
      slice in interface Vector

    • filter

      public DocVector filter(boolean mayContainDuplicates, int... positions)
      Description copied from interface: Vector
      Creates a new vector that only exposes the positions provided. Materialization of the selected positions is avoided.
      Specified by:
      filter in interface Vector
      Parameters:
      mayContainDuplicates - may the positions array contain duplicate positions?
      positions - the positions to retain
      Returns:
      a filtered vector

    • deepCopy

      public DocVector deepCopy(BlockFactory blockFactory)
      Description copied from interface: Vector
      Make a deep copy of this Block using the provided BlockFactory, likely copying all data.
      Specified by:
      deepCopy in interface Vector

    • keepMask

      public DocBlock keepMask(BooleanVector mask)
      Description copied from interface: Vector
      Build a Block the same values as this Vector, but replacing all values for which mask.getBooleanValue(position) returns false with null. The mask vector must be at least as long as this Vector.
      Specified by:
      keepMask in interface Vector

    • lookup

      public org.elasticsearch.core.ReleasableIterator<? extends Block> lookup(IntBlock positions, ByteSizeValue targetBlockSize)
      Description copied from interface: Vector
      Builds an Iterator of new Blocks with the same Vector.elementType() as this Vector whose values are copied from positions in this Vector. It has the same number of positions as the positions parameter.

      For example, if this vector contained [a, b, c] and were called with the block [0, 1, 1, [1, 2]] then the result would be [a, b, b, [b, c]].

      This process produces count(positions) values per positions which could be quite large. Instead of returning a single Block, this returns an Iterator of Blocks containing all of the promised values.

      The returned ReleasableIterator may retain a reference to the positions parameter. Close it to release those references.

      This block is built using the same BlockFactory as was used to build the positions parameter.

      Specified by:
      lookup in interface Vector

    • elementType

      public ElementType elementType()
      Description copied from interface: Vector
      Returns the element type of this vector.
      Specified by:
      elementType in interface Vector
      Returns:
      the element type of this vector

    • isConstant

      public boolean isConstant()
      Description copied from interface: Vector
      Returns true iff this vector is a constant vector - returns the same constant value for every position.
      Specified by:
      isConstant in interface Vector
      Returns:
      true iff this vector is a constant vector - returns the same constant value for every position

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • ramBytesEstimated

      public static long ramBytesEstimated(IntVector shards, IntVector segments, IntVector docs, int[] shardSegmentDocMapForwards, int[] shardSegmentDocMapBackwards)

    • ramBytesUsed

      public long ramBytesUsed()
      Specified by:
      ramBytesUsed in interface org.apache.lucene.util.Accountable

    • allowPassingToDifferentDriver

      public void allowPassingToDifferentDriver()
      Description copied from interface: Vector
      Before passing a Vector to another Driver, it is necessary to switch the owning block factory to its parent, which is associated with the global circuit breaker. This ensures that when the new driver releases this Vector, it returns memory directly to the parent block factory instead of the local block factory of this Block. This is important because the local block factory is not thread safe and doesn't support simultaneous access by more than one thread.
      Specified by:
      allowPassingToDifferentDriver in interface Vector

    • closeInternal

      public void closeInternal()
      Description copied from class: AbstractNonThreadSafeRefCounted
      This is called when the number of references reaches zero. This is where resources should be released (adjusting circuit breakers if needed).

    • shardRefCounted

      public org.elasticsearch.core.RefCounted shardRefCounted(int position)

    • newFixedBuilder

      public static DocVector.FixedBuilder newFixedBuilder(BlockFactory blockFactory, int estimatedSize)
      Builds a doc vector with a fixed length DocBlock.

    • getPositionCount

      public final int getPositionCount()
      Description copied from interface: Vector
      Returns the number of positions (rows) in this vector. See class javadoc for the usual way to iterate these positions.
      Specified by:
      getPositionCount in interface Vector
      Returns:
      the number of positions (rows) in this vector

    • blockFactory

      public BlockFactory blockFactory()
      Description copied from interface: Vector
      The block factory associated with this vector.
      Specified by:
      blockFactory in interface Vector