Concepts#
Roboto makes it easy to manage, process and analyze your data.
For example, you can:
Upload log files into datasets and share them with your team
Add tags and metadata to datasets e.g.
crashorsw_version=3.4.2Visualize log data from multiple files in interactive dashboards
Generate artifacts by post-processing log files with custom actions
Create events on slices of data to mark important moments
Search data to find relevant logs and edge cases
Learn the basics of Roboto, including core concepts and essential terminology below.
Overview#
In Roboto, you create datasets and upload (or import) log files to them.
Log files are ingested using specialized actions. Roboto includes built-in ingestion actions for common robotics formats like ros, px4 and parquet. It’s also possible to create ingestion actions for custom log formats too.
During ingestion, log files are processed to index and optimize their constituent topics. Once ingested, topic data can be queried and visualized.
Additionally, you can create and invoke custom actions to post-process your data. For example, you can automatically generate events or detailed reports.
Key Terms#
Action#
An action is a reusable function to process, transform or analyze data in Roboto. Actions can be invoked manually, or automatically with triggers.
The underlying code for an action needs to be in a Docker image and pushed to a registry. Actions range from simple data transformations to sophisticated algorithms and can also programmatically add tags, metadata or events to items in Roboto.
During creation, actions must be assigned a unique name by which they can be referenced. Additionally, actions can be defined with any number of optional or required parameters.
See the Process Data with Actions user guide for additional information.
Collection#
A collection is a higher-level container for grouping datasets together. Collections can also be used to group files from several distinct datasets together.
For example, a collection may be created with image files from several different source datasets.
Upon creation, collections are assigned a unique identifier beginning with the cl_ prefix.
See the roboto collections CLI for additional information.
Dataset#
A dataset contains files in a directory structure similar to Dropbox or Google Drive.
Typically, a dataset stores the files from a single robot activity, such as a drone flight or an autonomous vehicle mission, however, it’s versatile enough to be a general-purpose assembly of files.
Every file in a dataset, whether it’s raw or generated, resides under a unique key prefix in a Roboto-managed or user-provided storage bucket. Upon creation, datasets are assigned a unique identifier beginning with the ds_ prefix.
Datasets can have associated tags and metadata to improve search and capture important context. For example, you could tag a dataset with crash or add metadata such as sw_version=3.4.2.
See the roboto datasets CLI for additional information.
Device#
A device represents a robot, upload station, or other non-human entity which can programmatically interact with Roboto.
Devices are referenced by a user-provided device_id. This ID can be any string, but needs to be unique within an org. If available, existing robot serial numbers or other unique identifiers are an excellent choice for device_id.
Users can provision scoped access tokens for devices, which enables them to programmatically create datasets, upload files, and perform other operations.
When a device creates a dataset or uploads a file using its access token, that device’s device_id is automatically added to the resource as first-class metadata.
See the Devices page for additional information.
Event#
An event serves as a “time anchor,” enabling you to link key entities—such as datasets, files, topics, and message paths—to a specific timespan.
Events are a powerful way to mark important moments of interest (e.g., Discharged or Takeoff) and correlate them with the underlying data. You can assign tags and metadata to events, making them easier to discover and organize.
Events can be created manually in the visualizer, programmatically via the SDK, or automatically by actions. Once created, they are visualized on relevant plots and the timeline, helping you to quickly locate and analyze these key moments during review.
Additionally, by creating events, you can extract the associated data through the SDK, enabling more targeted analysis and streamlined workflows.
See the Create Events on Data user guide for additional information.
File#
A file is an individual data item that has been uploaded or imported to a dataset in Roboto.
An example file might be report.pdf or recording.bag, but any file type is theoretically supported.
Files that contain structured time-series data can be ingested by corresponding actions. This indexes their constituent topic data for visualization and search.
Files can also have associated tags and metadata to improve search and capture important context.
Image#
A Docker image is a package that includes all of the source code and dependencies necessary to run an action.
After definition, a Docker image can be associated with an action. Images must be pushed to Roboto’s registry, or hosted on a public registry such as Docker Hub. We plan to support images hosted in private registries too.
See the roboto images CLI for additional information.
Ingestion#
In Roboto, ingestion refers to the process of indexing and summarizing log files. A log file must be ingested before the data it contains can be visualized or queried within Roboto.
Ingestion happens through specialized actions, which process raw log files into an optimized format when necessary. During ingestion, log files are analyzed to identify their constituent topics, which represent structured time-series data.
A successfully ingested log file has a blue chevron next to it in the dataset file browser. Clicking it reveals the file’s topics.
Roboto provides built-in ingestion actions for common robotics formats such as ros and px4. You can find these in the Action Hub. It’s also possible to create ingestion actions for custom log formats. We plan to provide examples of these in the near future. Contact us to learn more.
See the IngestionStatus class in the SDK for additional information.
Invocation#
An invocation is a specific execution of an action, initiated either manually or automatically by a trigger.
For an action to be invoked, the required data to be processed must be specified, and values for any action parameters need to be provided.
During the execution of an invocation, live logs can be viewed in the Roboto platform or CLI.
Upon creation, invocations are assigned a unique identifier beginning with the iv_ prefix.
See the roboto invocations CLI for additional information.
Topic#
A topic is a sequence of structured time-series data linked to a source file. Each topic follows a defined schema, where the message paths represent the individual fields or signals within that schema.
The data items within a topic are called messages, and each message is structured according to the topic’s schema. Conceptually, topics and messages resemble ROS topics and ROS messages.
For example, a topic named battery_status might contain message paths like voltage_v and current_a, each representing individual time-series signals.
In Roboto, topic data can be visualized in plots, maps etc., and statistical summaries and metrics can be queried.
Trigger#
A trigger is a rule that automatically invokes an action when specific events or conditions occur.
For example, a trigger could be defined to invoke an action when any new dataset is uploaded and tagged with the word failure.
During creation, triggers must be given a unique name by which they can be referenced, as well as corresponding parameters.
See the roboto triggers CLI for additional information.