roboto.experimental.ingestion_rules.record

Module Contents

class roboto.experimental.ingestion_rules.record.DefaultIngestionRuleRecord(/, **data)

Bases: pydantic.BaseModel

Wire format of a rule Roboto offers every org as a starting point for a new rule, and whether the caller’s org already has it.

POST /v1/ingestion-rules/defaults creates those with for_new_orgs set that are not already_present.

Parameters:

data (Any)

already_present: bool

Whether the org has a rule with this name, or a rule with one of these include patterns.

category: str = 'Other'

The group the rule is listed under, such as ROS or Tabular.

exclude_patterns: list[str]

for_new_orgs: bool = True

Whether sign-up offers this rule and POST /v1/ingestion-rules/defaults creates it. When false, the rule is only offered as a starting point for a new rule.

include_patterns: list[str]

invocation: roboto.domain.triggers.ActionInvocationSpec

name: str

needs_settings: bool = False

Whether the rule’s action likely needs settings for the org’s files, such as which column holds the timestamps, before it ingests them.

class roboto.experimental.ingestion_rules.record.ExistingFilesPreview(/, **data)

Bases: pydantic.BaseModel

Files already uploaded that a rule matches, that are not ingested, and whose current upload no rule’s trigger has run on: ones uploaded, or uploaded again, before the rule existed or while its auto-ingest was off. Editing a file’s metadata or tags does not make it one of these.

Response of GET /v1/ingestion-rules/<rule_id>/existing-files.

Parameters:

data (Any)

complete_session_count: int = 0

Complete sessions holding any of these files. After a session’s files are ingested, it is announced as fully ingested again, and the triggers on session.ingested start for it.

count: int

count_is_lower_bound: bool = False

Whether there may be more than count; counting stops after a fixed number of files.

sample_paths: list[str]

Up to 10 of the files’ paths. Empty for a caller who is not an org admin.

class roboto.experimental.ingestion_rules.record.IngestFileOutcome

Bases: roboto.compat.StrEnum

What POST /v1/ingestion-rules/ingest did with one file.

AlreadyRunning = 'already_running'

An ingestion run on the file is still queued or running, so no new one was started.

AutoIngestOff = 'auto_ingest_off'

Rules match the file, but none of them auto-ingests.

NoMatchingRule = 'no_matching_rule'

No rule in the file’s org matches its path.

NotFound = 'not_found'

No such file in the caller’s org, the caller may not edit it, or it is not in a dataset (rules apply only to dataset files).

NotStarted = 'not_started'

A matching rule’s trigger declined the file (it is disabled, say).

Started = 'started'

A matching rule’s trigger started ingesting the file.

class roboto.experimental.ingestion_rules.record.IngestFileResult(/, **data)

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

Parameters:

data (Any)

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

file_id: str

invocation_id: str | None = None

The invocation started, when outcome is started.

outcome: IngestFileOutcome

rule_id: str | None = None

The rule whose trigger ran, or would have.

class roboto.experimental.ingestion_rules.record.IngestionRuleRecord(/, **data)

Bases: pydantic.BaseModel

Wire format of an ingestion rule: which file paths in an org are ingestable, and how they are ingested.

A file whose path matches any of include_patterns and none of exclude_patterns is marked ingestable when it is created. The rule owns one trigger (trigger_id) that runs invocation on each matching upload while auto_ingest is on.

Parameters:

data (Any)

auto_ingest: bool

whether the rule’s trigger is enabled. Turning this off disables the trigger and leaves matching files ingestable.

Type:

Whether matching uploads are ingested automatically

created: datetime.datetime

created_by: str

description: str | None = None

exclude_patterns: list[str]

Git wildmatch patterns. A file matching any of them is not ingestable by this rule.

include_patterns: list[str]

Git wildmatch patterns, e.g. **/*.mcap. A file matching any of them is ingestable.

invocation: roboto.domain.triggers.ActionInvocationSpec | None = None

How each matching upload is ingested. The trigger passes the uploaded file as the first input, followed by whatever invocation.invocation_input selects. None only on a rule that does not auto-ingest.

modified: datetime.datetime

modified_by: str

name: str

org_id: str

rule_id: str

trigger_id: str | None = None

The trigger that runs invocation on matching uploads, enabled while auto_ingest is on. Only the rule can enable, disable, edit or delete it. None while the rule has no invocation.