Package org.elasticsearch.xpack.esql.datasource.parquet

Class ParquetFormatReader

java.lang.Object

org.elasticsearch.xpack.esql.datasource.parquet.ParquetFormatReader

All Implemented Interfaces:

Closeable, AutoCloseable, FormatReader, RangeAwareFormatReader

public class ParquetFormatReader
extends Object
implements RangeAwareFormatReader

FormatReader implementation for Parquet files.

Uses Parquet's native ParquetFileReader with our StorageObject abstraction. Produces ESQL Page batches directly without requiring Arrow as an intermediate format.

Key features:

• Works with any StorageProvider (HTTP, S3, local)
• Efficient columnar reading with column projection
• No Hadoop dependencies in the core path
• Direct conversion from Parquet to ESQL blocks

Nested Class Summary

Nested classes/interfaces inherited from interface org.elasticsearch.xpack.esql.datasources.spi.FormatReader

FormatReader.SchemaResolution

Nested classes/interfaces inherited from interface org.elasticsearch.xpack.esql.datasources.spi.RangeAwareFormatReader

RangeAwareFormatReader.SplitRange

Field Summary

Fields inherited from interface org.elasticsearch.xpack.esql.datasources.spi.FormatReader

NO_LIMIT

Constructor Summary

Constructors

Constructor	Description
ParquetFormatReader(BlockFactory blockFactory)

Method Summary

Modifier and Type	Method	Description
AggregatePushdownSupport	aggregatePushdownSupport()
void	close()
List<RangeAwareFormatReader.SplitRange>	discoverSplitRanges(StorageObject object)
List<String>	fileExtensions()
FilterPushdownSupport	filterPushdownSupport()
String	formatName()
SourceMetadata	metadata(StorageObject object)
CloseableIterator<Page>	read(StorageObject object, FormatReadContext context)
CloseableIterator<Page>	readRange(StorageObject object, List<String> projectedColumns, int batchSize, long rangeStart, long rangeEnd, List<Attribute> resolvedAttributes, ErrorPolicy errorPolicy)	Reads only row groups whose starting position falls within [rangeStart, rangeEnd).
ParquetFormatReader	withPushedFilter(Object pushedFilter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.elasticsearch.xpack.esql.datasources.spi.FormatReader

defaultErrorPolicy, defaultSchemaResolution, read, readAsync, schema, supportsNativeAsync, withConfig, withSchema

Constructor Details

ParquetFormatReader

public ParquetFormatReader(BlockFactory blockFactory)

Method Details

withPushedFilter

public ParquetFormatReader withPushedFilter(Object pushedFilter)

Specified by:

withPushedFilter in interface FormatReader

filterPushdownSupport

public FilterPushdownSupport filterPushdownSupport()

Specified by:

filterPushdownSupport in interface FormatReader

metadata

public SourceMetadata metadata(StorageObject object)
                        throws IOException

Specified by:

metadata in interface FormatReader

Throws:

IOException

read

public CloseableIterator<Page> read(StorageObject object,
 FormatReadContext context)
                             throws IOException

Specified by:

read in interface FormatReader

Throws:

IOException

aggregatePushdownSupport

public AggregatePushdownSupport aggregatePushdownSupport()

Specified by:

aggregatePushdownSupport in interface FormatReader

formatName

public String formatName()

Specified by:

formatName in interface FormatReader

fileExtensions

public List<String> fileExtensions()

Specified by:

fileExtensions in interface FormatReader

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

discoverSplitRanges

public List<RangeAwareFormatReader.SplitRange> discoverSplitRanges(StorageObject object)
                                                            throws IOException

Specified by:

discoverSplitRanges in interface RangeAwareFormatReader

Throws:

IOException

readRange

public CloseableIterator<Page> readRange(StorageObject object,
 List<String> projectedColumns,
 int batchSize,
 long rangeStart,
 long rangeEnd,
 List<Attribute> resolvedAttributes,
 ErrorPolicy errorPolicy)
                                  throws IOException

Reads only row groups whose starting position falls within [rangeStart, rangeEnd). errorPolicy is accepted for interface compliance but not applied — Parquet errors are structural (corrupt page, schema mismatch) rather than row-level.

Specified by:

readRange in interface RangeAwareFormatReader

Throws:

IOException