Package org.elasticsearch.xpack.esql.inference

Class InferenceOperator

java.lang.Object

org.elasticsearch.compute.operator.AsyncOperator<InferenceOperator.OngoingInferenceResult>

org.elasticsearch.xpack.esql.inference.InferenceOperator

All Implemented Interfaces:

Closeable, AutoCloseable, Operator, org.elasticsearch.core.Releasable

Direct Known Subclasses:

CompletionOperator, RerankOperator, TextEmbeddingOperator

public abstract class InferenceOperator
extends AsyncOperator<InferenceOperator.OngoingInferenceResult>

An abstract asynchronous operator that performs throttled bulk inference execution.

The InferenceOperator integrates with the compute framework and supports throttled bulk execution of inference requests. It transforms input Page into inference requests, asynchronously executes them, and uses the responses to build a new Page.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	InferenceOperator.BulkInferenceOperation	Manages the execution of inference requests for a single input page.
static final record	InferenceOperator.BulkInferenceRequestItem	Represents a single inference request with metadata for result building.
static interface	InferenceOperator.BulkInferenceRequestItemIterator
static final record	InferenceOperator.BulkInferenceResponseItem	Represents a completed inference response with metadata for result building.
static final record	InferenceOperator.OngoingInferenceResult	Represents the result of a bulk inference operation for a single input page.
static interface	InferenceOperator.OutputBuilder

Nested classes/interfaces inherited from class org.elasticsearch.compute.operator.AsyncOperator

AsyncOperator.Status

Nested classes/interfaces inherited from interface org.elasticsearch.compute.operator.Operator

Operator.OperatorFactory

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_MAX_OUTSTANDING_PAGES
static final int	DEFAULT_MAX_OUTSTANDING_REQUESTS

Fields inherited from interface org.elasticsearch.compute.operator.Operator

MIN_TARGET_PAGE_SIZE, NOT_BLOCKED, TARGET_PAGE_SIZE

Constructor Summary

Constructors

Constructor	Description
InferenceOperator(DriverContext driverContext, InferenceService inferenceService, InferenceOperator.BulkInferenceRequestItemIterator.Factory inferenceRequestsFactory, InferenceOperator.OutputBuilder outputBuilder)	Constructs a new InferenceOperator with default throttling parameters.
InferenceOperator(DriverContext driverContext, InferenceService inferenceService, InferenceOperator.BulkInferenceRequestItemIterator.Factory inferenceRequestsFactory, InferenceOperator.OutputBuilder outputBuilder, int maxOutstandingPages, int maxOutstandingInferenceRequests)	Constructs a new InferenceOperator.

Method Summary

Modifier and Type	Method	Description
protected void	doClose()
Page	getOutput()
protected String	inferenceId()
protected void	performAsync(Page input, ActionListener<InferenceOperator.OngoingInferenceResult> listener)
protected void	releaseFetchedOnAnyThread(InferenceOperator.OngoingInferenceResult ongoingInferenceResult)

Methods inherited from class org.elasticsearch.compute.operator.AsyncOperator

addInput, canProduceMoreDataWithoutExtraInput, close, fetchFromBuffer, finish, isBlocked, isFinished, needsInput, releasePageOnAnyThread, status, status

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_MAX_OUTSTANDING_PAGES

public static final int DEFAULT_MAX_OUTSTANDING_PAGES

See Also:

• Constant Field Values

DEFAULT_MAX_OUTSTANDING_REQUESTS

public static final int DEFAULT_MAX_OUTSTANDING_REQUESTS

See Also:

• Constant Field Values

Constructor Details

InferenceOperator

public InferenceOperator(DriverContext driverContext,
 InferenceService inferenceService,
 InferenceOperator.BulkInferenceRequestItemIterator.Factory inferenceRequestsFactory,
 InferenceOperator.OutputBuilder outputBuilder,
 int maxOutstandingPages,
 int maxOutstandingInferenceRequests)

Constructs a new InferenceOperator.

Parameters:

driverContext - The driver context.

inferenceService - The inference service to use for executing inference requests.

maxOutstandingPages - The maximum number of pages processed in parallel.

maxOutstandingInferenceRequests - The maximum number of inference requests to be run in parallel.

InferenceOperator

public InferenceOperator(DriverContext driverContext,
 InferenceService inferenceService,
 InferenceOperator.BulkInferenceRequestItemIterator.Factory inferenceRequestsFactory,
 InferenceOperator.OutputBuilder outputBuilder)

Constructs a new InferenceOperator with default throttling parameters. Use default max outstanding pages and requests settings.

Parameters:

driverContext - The driver context.

inferenceService - The inference service to use for executing inference requests.

inferenceRequestsFactory - Factory for creating inference request iterators from input pages.

outputBuilder - Builder for converting inference responses into output pages.

Method Details

performAsync

protected void performAsync(Page input,
 ActionListener<InferenceOperator.OngoingInferenceResult> listener)

Specified by:

performAsync in class AsyncOperator<InferenceOperator.OngoingInferenceResult>

getOutput

public Page getOutput()

releaseFetchedOnAnyThread

protected void releaseFetchedOnAnyThread(InferenceOperator.OngoingInferenceResult ongoingInferenceResult)

Specified by:

releaseFetchedOnAnyThread in class AsyncOperator<InferenceOperator.OngoingInferenceResult>

doClose

protected void doClose()

Specified by:

doClose in class AsyncOperator<InferenceOperator.OngoingInferenceResult>

inferenceId

protected String inferenceId()