Module org.elasticsearch.server
Package org.elasticsearch.index.codec.tsdb.pipeline

Class BlockFormat

java.lang.Object
org.elasticsearch.index.codec.tsdb.pipeline.BlockFormat
public final class BlockFormat extends Object
Defines how each block of encoded values is written to the data file.

Data file layout: 

   +------------------+----------------------------------------+
   | Block 0          | [bitmap][payload][stage metadata]      |
   | Block 1          | [bitmap][payload][stage metadata]      |
   | ...              | ...                                    |
   | Block N-1        | [bitmap][payload][stage metadata]      |
   +------------------+----------------------------------------+
   | Block Offsets     | DirectMonotonicWriter encoded offsets |
   +------------------+----------------------------------------+

Each block contains:

  • bitmap: 1 byte (<= 8 stages) or 2 bytes (> 8 stages) indicating which stages were applied
  • payload: the encoded values written by the terminal payload stage
  • stage metadata: per-stage metadata written by transformation stages (e.g., GCD divisor)

The layout is designed for sequential decoding: the bitmap comes first so the decoder immediately knows which stages to reverse, followed by the payload and then stage metadata in reverse stage order (see EncodingContext.writeStageMetadata(org.apache.lucene.store.DataOutput)). This means the decoder can read every section in a single forward pass with no seeking or buffering. See FieldDescriptor for the metadata file format that describes pipeline configuration.

  • Method Details

    • writeBlock

      public static void writeBlock(org.apache.lucene.store.DataOutput out, long[] values, PayloadEncoder payloadStage, EncodingContext context) throws IOException
      Writes a block of encoded values to the data output.
      Parameters:
      out - the data output stream
      values - the values to encode
      payloadStage - the terminal payload encoder
      context - the encoding context with block metadata
      Throws:
      IOException - if an I/O error occurs

    • readBlock

      public static int readBlock(org.apache.lucene.store.DataInput in, long[] values, PayloadDecoder payloadStage, DecodingContext context, int payloadPosition) throws IOException
      Reads a block of encoded values from the data input.
      Parameters:
      in - the data input stream
      values - the output array to populate
      payloadStage - the terminal payload decoder
      context - the decoding context with block metadata
      payloadPosition - the pipeline position of the payload stage
      Returns:
      the number of values decoded
      Throws:
      IOException - if an I/O error occurs