roboto.experimental.topics#

Topics APIs in active refinement; see roboto.experimental for the stability contract.

Submodules#

Package Contents#

class roboto.experimental.topics.FieldAddress(/, **data)#

Bases: pydantic.BaseModel

Addresses a schema field, and the subtree nested under it, by exactly one of two forms.

A path names the field by its path_in_schema components directly (no string delimiter, so a component may itself contain a .); a field_id names it opaquely and resolves server-side to the same path. Either form designates the field and every field nested under it.

Parameters:: data (Any)

field_id: str | None = None#: The field’s opaque id (sf_*).

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: tuple[str, Ellipsis] | None = None#: The field’s path_in_schema components; () addresses the schema root.

roboto.experimental.topics.PLAN_VERSION: int = 1#

Contract version stamped on every plan.

ReadPlan validation refuses a plan whose version it does not recognize, so a consumer on an older contract fails at parse time instead of misreading a newer plan.

class roboto.experimental.topics.ReadPlan(/, **data)#

Bases: pydantic.BaseModel

Resolves a read of one topic over a time window into the files to fetch and how to interpret them.

Parameters:: data (Any)

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

next_page_token: str | None = None#

always None today, meaning the plan is complete inline.

The field exists so a later contract revision can page large partition sets without a breaking envelope change.

Type:: Reserved pagination seam

partitions: tuple[ReadPlanPartition, Ellipsis] = ()#: One entry per partition in the window, each its own fetch-and-interpret plan.

plan_version: int = 1#: Contract version of this plan. Validation refuses a version this model does not recognize.

projection: ReadPlanProjection#: The output fields a consumer projects decoded rows to.

schema_: ReadPlanSchemaRef | None = None#

The resolved schema on a non-empty plan. Serializes as schema.

None exactly when the plan is empty: the window contains no partitions, or a schema_id/schema_checksum matches no in-window partition (data may exist in the window under a different schema).

topic_id: str#: The topic this plan reads.

window: ReadPlanWindow#: The time window the plan resolves over.

class roboto.experimental.topics.ReadPlanExtent(/, **data)#

Bases: pydantic.BaseModel

A partition’s time bounds, clipped to the plan window.

Parameters:: data (Any)

max: int#: Inclusive upper bound, in absolute Unix-epoch nanoseconds.

min: int#: Inclusive lower bound, in absolute Unix-epoch nanoseconds.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class roboto.experimental.topics.ReadPlanFieldRef(/, **data)#

Bases: pydantic.BaseModel

A schema field named by both its id and its path components in the schema.

Parameters:: data (Any)

field_id: str#: Identifier of the schema field.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: tuple[str, Ellipsis]#: The field’s path components within the schema, from the root to the field.

class roboto.experimental.topics.ReadPlanObjectRef(/, **data)#

Bases: pydantic.BaseModel

Points to the file backing a scan task. A consumer fetches the file’s bytes from it.

Parameters:: data (Any)

fs_node_id: str#: Identifier of the backing file. This id is stable, so a consumer can cache on it.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class roboto.experimental.topics.ReadPlanPartition(/, **data)#

Bases: pydantic.BaseModel

Everything needed to fetch and interpret one in-window partition’s bytes.

Parameters:: data (Any)

extent: ReadPlanExtent#: The partition’s time bounds, clipped to the window.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scan_tasks: tuple[ReadPlanScanTask, Ellipsis] = ()#: The files to read for this partition; empty when the partition has no readable data.

time_offset_ns: int#: Offset a consumer adds to each decoded row timestamp; the same for every row in the partition.

timestamp: ReadPlanTimestamp#: Where this partition’s row timestamps come from.

topic_part_id: str#: Identifier of the partition.

class roboto.experimental.topics.ReadPlanProjection(/, **data)#

Bases: pydantic.BaseModel

The output fields the plan resolves rows to.

The projection takes exactly one of two forms: either every field in the schema (all is true, and the field list is left implicit so the plan need not enumerate a large schema) or an explicit fields list.

Parameters:: data (Any)

all: bool = False#: True when the projection covers every field in the schema.

classmethod all_fields()#

Return a projection covering every field in the schema.

Return type:: ReadPlanProjection

fields: tuple[ReadPlanFieldRef, Ellipsis] | None = None#: The resolved field set when the read is narrowed; None when all is true.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod narrowed(fields)#

Return a projection narrowed to an explicit field set.

Parameters:: fields (Iterable[ReadPlanFieldRef])
Return type:: ReadPlanProjection

class roboto.experimental.topics.ReadPlanRequest(/, **data)#

Bases: pydantic.BaseModel

The body of a read-plan request: the logical read question to resolve into a physical plan.

Parameters:: data (Any)

end_time: int#: Inclusive window upper bound, absolute Unix-epoch nanoseconds.

fields_exclude: tuple[FieldAddress, Ellipsis] | None = None#: Field subtrees to drop from the projection; None drops none.

fields_include: tuple[FieldAddress, Ellipsis] | None = None#: Field subtrees to project; None projects every field.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prefer: RepresentationPreference | None = None#: Per-subtree representation preference; None applies default selection everywhere.

schema_checksum: str | None = None#: Schema to use, by checksum, or None.

schema_id: str | None = None#: Schema to use, by id, or None to default to the sole in-window schema.

start_time: int#: Inclusive window lower bound, absolute Unix-epoch nanoseconds.

timeline_source_id: str | None = None#: Timeline source to resolve partition extents with, by id, or None.

timeline_source_name: str | None = None#: Timeline source to resolve partition extents with, by name, or None.

class roboto.experimental.topics.ReadPlanScanTask(/, **data)#

Bases: pydantic.BaseModel

One file to open, with the format and transformations needed to interpret it.

Which of the representations satisfying the governing selector backs a scan task is service policy and may change between releases; only the selector’s hard-filter matching rule is contract.

Parameters:: data (Any)

format: roboto.domain.topics.RepresentationStorageFormat#: The format the bytes are stored in; selects the decoder a consumer applies.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: ReadPlanObjectRef#: The single file this scan task resolves to.

precedence: int#: Where two scan tasks’ scopes overlap, the one with the higher precedence wins.

scope: ReadPlanFieldRef | None = None#: The field subtree this scan task covers; None covers the whole schema.

transformations: tuple[str, Ellipsis] = ()#: Transformations applied to produce this variant, in order; empty on the original.

class roboto.experimental.topics.ReadPlanSchemaRef(/, **data)#

Bases: pydantic.BaseModel

Identifies the single topic schema the plan uses.

Parameters:: data (Any)

checksum: str#: Checksum of the schema’s content. A consumer can cache the schema by this value.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

schema_id: str#: Identifier of the resolved topic schema.

class roboto.experimental.topics.ReadPlanTimestamp(/, **data)#

Bases: pydantic.BaseModel

Where a partition’s row timestamps come from.

Timestamps are either read out of a schema field (kind is "schema_field", and field names which one) or taken from the storage envelope (message log or publish time), in which case no schema field is involved and field is None.

Parameters:: data (Any)

field: ReadPlanFieldRef | None = None#: The schema field timestamps are read from; set exactly when kind is "schema_field".

kind: roboto.domain.topics.TimelineSourceKind#

from a schema field, or from the storage envelope.

Type:: How timestamps are sourced

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class roboto.experimental.topics.ReadPlanWindow(/, **data)#

Bases: pydantic.BaseModel

The absolute time window, in Unix-epoch nanoseconds, the plan resolves over.

Parameters:: data (Any)

end: int#: Inclusive upper bound, in nanoseconds since the Unix epoch.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

start: int#: Inclusive lower bound, in nanoseconds since the Unix epoch.

class roboto.experimental.topics.RepresentationOverride(/, **data)#

Bases: pydantic.BaseModel

Applies a representation selector to one field subtree, overriding the request default.

Parameters:: data (Any)

field: FieldAddress#: The subtree this override scopes to.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

selector: roboto.experimental.topics.record.RepresentationSelector#: The selector to apply within that subtree.

class roboto.experimental.topics.RepresentationPreference(/, **data)#

Bases: pydantic.BaseModel

Selects which stored variant of each field to read, per subtree.

A default selector applies to every field unless a more specific override covers it. Where several overrides cover a field, the one whose addressed subtree is the longest prefix of the field’s path wins; this rule is selector_for().

The governing selector and its matching rule are contract: a selector never substitutes a non-matching variant, and a read fails when a selector that sets any criterion is satisfied by no stored representation for a requested field — the plan never silently omits a field an explicit requirement covers. Which of the representations that satisfy the selector the service ultimately schedules is service policy and may change between releases.

Parameters:: data (Any)

default: roboto.experimental.topics.record.RepresentationSelector#: The selector applied to any field no override covers; matches anything when unset.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

overrides: tuple[RepresentationOverride, Ellipsis] = ()#: Per-subtree selector overrides, resolved longest-matching-prefix wins.

selector_for(field_path)#

Resolve the selector that governs the field at field_path, longest-matching-prefix wins.

An override applies when its addressed subtree path is a prefix of field_path; among applicable overrides the deepest subtree wins, and a field no override covers gets default.

Parameters:: field_path (tuple[str, Ellipsis]) – The path_in_schema components of the field whose selector is being resolved.
Returns:: The governing selector.
Raises:: ValueError – An override addresses its subtree by field_id. Resolving a field_id to a path takes the schema, which this value object does not hold; resolve every override address to its path form first.
Return type:: roboto.experimental.topics.record.RepresentationSelector

class roboto.experimental.topics.RepresentationRecord(/, **data)#

Bases: pydantic.BaseModel

One stored variant of a topic partition’s data, optionally narrowed to a subset of its fields.

A representation pairs a stored file with the data of a single topic partition. field_id narrows it to one field and the fields nested under it; None covers every field in the partition.

The same partition can have several representations that differ in storage_format, content_format, and transformations. A consumer picks the one whose attributes suit it: a viewer of image data, for example, may prefer a JPEG- or PNG-encoded variant over the untransformed original.

Parameters:: data (Any)

content_format: str | None = None#: The format of the data inside the stored file. For image data, this may be the image encoding (e.g. "jpeg", "png") on a transformed variant. None when unspecified.

created: datetime.datetime | None = None#

created_by: str#

field_id: str | None = None#: The field this representation is narrowed to, covering that field and the fields nested under it. None when the representation covers every field in the partition.

fs_node_id: str#: Identifier of the file backing this representation.

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

modified: datetime.datetime | None = None#

modified_by: str#

org_id: str#

representation_id: str#

storage_format: roboto.domain.topics.RepresentationStorageFormat#: Container the representation data is stored in (e.g. MCAP, Parquet).

topic_part_id: str#: Identifier of the topic partition this representation belongs to.

transformations: list[str] = None#

The transformations applied to the source data to produce this variant, in the order applied. Empty on the untransformed original.

Each entry is a "<kind>:<param>" string whose <kind> is a TransformationKind member, e.g. ["downsample:0.5", "encode:jpeg"].

class roboto.experimental.topics.RepresentationSelector(/, **data)#

Bases: pydantic.BaseModel

Selects which stored variant of a field to read when several are available.

A selector has three optional criteria — storage_format, content_format, and transformations — one for each attribute on which stored variants of the same field can differ (see RepresentationRecord). A criterion that is set is a requirement a variant must meet to be selected; a criterion left None places no requirement, and any value is acceptable.

A selector never falls back to a variant other than the one it describes. If any criterion is set and no stored variant of a requested field meets every requirement — whether the variants that exist all fall short, or the field has no stored variant at all — the read fails with an error rather than quietly leave out the field. Only under a selector with no criteria set is a field with no stored variant simply absent from the result; such a selector requires nothing, so nothing requested is missing.

Successor to roboto.domain.topics.RepresentationSelector, which get_data() uses. This class keeps the same no-fallback matching, and adds an explicit storage_format criterion and a more expressive way to specify required transformations.

Parameters:: data (Any)

content_format: str | None = None#

Required content encoding (e.g. "jpeg") by scalar equality; None does not constrain it.

There is no legacy carve-out: a representation whose content_format is None does not satisfy an explicit request.

matches(representation)#

Return whether representation satisfies every set axis of this selector.

Parameters:: representation (RepresentationRecord)
Return type:: bool

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod raw()#

Select the untransformed original (a representation with no transformations).

Return type:: RepresentationSelector

storage_format: roboto.domain.topics.RepresentationStorageFormat | None = None#: Required container (e.g. MCAP, Parquet) by scalar equality; None does not constrain it.

transformations: tuple[str, Ellipsis] | None = None#

Required transformations; None does not constrain, () requires the untransformed original.

A non-empty tuple is all-of: every token must be satisfied by some descriptor on the representation, which may carry additional transformations. A token is either a bare kind (e.g. "downsample"), satisfied by any descriptor whose kind prefix equals it, or a full "<kind>:<param>" descriptor (e.g. "encode:jpeg"), satisfied only by an exact match. The grammar is open string matching; the recognized vocabulary (TransformationKind) is enforced by the service at the request boundary, which rejects a token naming an unrecognized kind.