roboto.domain.topics.http_range_reader#

Seekable HTTP byte-range reader for efficient partial access to remote files.

This module provides a seekable file-like object backed by HTTP Range requests, enabling efficient random access to remote files (e.g., MCAP files on S3) without downloading the entire file.

Module Contents#

class roboto.domain.topics.http_range_reader.HttpRangeReader(url, read_ahead_size=_READ_AHEAD_SIZE)#

A seekable, buffered byte-range reader backed by an HTTP URL.

Uses HTTP range requests so only the requested byte ranges are fetched, allowing efficient partial access to remote files (e.g., reading just the MCAP summary/index section at the end of a file without downloading the full data payload).

Reads are satisfied from an in-memory sparse cache. HTTP requests are only issued on a cache miss, fetching _READ_AHEAD_SIZE bytes at a time. Unlike a simple single-buffer approach, this cache retains all fetched regions, so seeking back to previously-read data doesn’t trigger re-fetches.

This class implements the IO[bytes] protocol methods needed by mcap.reader.

Uses urllib3 connection pooling to reuse HTTP connections across requests, reducing TCP handshake and TLS negotiation overhead.

Parameters:

  • url (str)

  • read_ahead_size (int)

close()#

Close the reader and release resources.

Return type:

None

prefetch_range(start, end)#

Prefetch a byte range using parallel HTTP requests.

Parameters:

  • start (int) – Start byte offset (inclusive)

  • end (int) – End byte offset (inclusive)

Return type:

None

read(size=-1)#
Parameters:

size (int)

Return type:

bytes

readable()#
Return type:

bool

seek(offset, whence=0)#
Parameters:

  • offset (int)

  • whence (int)

Return type:

int

seekable()#
Return type:

bool

property size: int#

Get the total size of the remote file in bytes.

Return type:

int

tell()#
Return type:

int

writable()#
Return type:

bool

roboto.domain.topics.http_range_reader.as_io_bytes(reader)#

Cast an HttpRangeReader to typing.IO[bytes] for type-checking purposes.

Parameters:

reader (HttpRangeReader)

Return type:

IO[bytes]

roboto.domain.topics.http_range_reader.logger#
On this page