roboto.experimental.topics.decode.scan_task

Module Contents

roboto.experimental.topics.decode.scan_task.ScanTaskDecoder

Decodes one scan task into a stream of native-order RecordBatches.

Called with the scan task, its partition (for the timestamp designation), the window translated into the partition’s stored-time domain, the projection restricted to the scan task’s subtree. Produced timestamps are in the stored-time domain; the executor applies the partition’s time_offset_ns.

roboto.experimental.topics.decode.scan_task.make_scan_task_decoder(params)

Bind execution inputs into a ScanTaskDecoder.

The returned decoder decodes one scan task (one file, one format), filtered and projected, dispatching on the scan task’s server-declared format.

Timestamps are keyed off the partition’s timestamp designation and produced in the stored-time domain; the caller applies the partition’s time_offset_ns. Rows are filtered to the decoder’s window (inclusive on both ends) and projected to its projection_paths.

MCAP scan tasks decode through the Rust mcap_codec batch decoder, which reads each column’s Arrow type from the file’s own embedded schema – no ingestion-declared schema tree is needed. Parquet decodes column-wise from its own file footer.

Parameters:

params (roboto.experimental.topics.decode.common.ScanTaskDecodeParams) – Execution inputs (URL resolution and cache policy).

Returns:

A decoder that, given a scan task, its plan partition, an inclusive time window, and the projected field paths the scan task covers, returns the decoded scan task; consume its batches once.

Raises:

• The returned decoder raises –

• NotImplementedError – The scan task’s storage format has no decoder, or a Parquet scan task is designated an envelope-derived timestamp.

Return type:

ScanTaskDecoder