roboto.domain.topics.sparse_buffer#

Seekable read-only buffer backed by sparse in-memory byte regions.

Provides a file-like interface (IO[bytes]) over non-contiguous byte data, allowing efficient representation of partially-fetched remote files. Only the regions that have been explicitly loaded are stored in memory.

Module Contents#

class roboto.domain.topics.sparse_buffer.SparseBuffer(file_size)#

A seekable, read-only file-like object backed by sparse in-memory byte regions.

Stores fetched byte regions and provides a standard IO[bytes] interface for reading from them. Regions are automatically merged when they overlap or are adjacent, keeping the internal representation compact.

This is intended to be used as: 1. The cache backend for HttpRangeReader (sparse storage with smart fetching) 2. The stream for mcap.reader.SeekingReader after bulk-fetching byte ranges

Example

>>> buf = SparseBuffer(file_size=1000)
>>> buf.add_region(0, b"MCAP_MAGIC")  # header
>>> buf.add_region(900, b"footer_data")  # footer
>>> buf.seek(0)
0
>>> buf.read(10)
b'MCAP_MAGIC'
Parameters:

file_size (int)

add_region(offset, data)#

Store a byte region at the given file offset.

Merges with any overlapping or adjacent existing regions.

Parameters:

  • offset (int) – Byte offset within the virtual file.

  • data (bytes) – Raw bytes to store at that offset.

Return type:

None

clear()#

Remove all cached regions.

Return type:

None

find_region(start, size)#

Check if [start, start+size) is fully contained in a cached region.

Parameters:

  • start (int) – Start byte offset.

  • size (int) – Number of bytes.

Returns:

The requested bytes if fully cached, None otherwise.

Return type:

bytes | None

read(size=-1)#

Read up to size bytes from the current position.

If the current position is within a cached region, returns available bytes (may be fewer than requested if the region ends before size bytes). If the current position is not in any cached region (a gap), returns b””.

This allows callers to detect partial hits and fetch missing data: - len(result) == size: fully satisfied - 0 < len(result) < size: partial hit, more data may be needed - len(result) == 0: gap at current position, caller should fetch

Parameters:

size (int) – Maximum number of bytes to read. -1 means read to end of file.

Returns:

Bytes read from cached regions, or b”” if at a gap or past EOF.

Return type:

bytes

readable()#

Return True - this buffer supports reading.

Return type:

bool

property regions: list[tuple[int, int]]#

List of (start, end) byte ranges currently cached.

End is exclusive. Useful for fetch planning and debugging.

Return type:

list[tuple[int, int]]

seek(offset, whence=0)#

Move the read position.

Parameters:

  • offset (int) – Byte offset relative to the position indicated by whence.

  • whence (int) – 0=SEEK_SET (start), 1=SEEK_CUR (current), 2=SEEK_END (end).

Returns:

The new absolute position.

Return type:

int

seekable()#

Return True - this buffer supports seeking.

Return type:

bool

property size: int#

Total size of the virtual file.

Return type:

int

tell()#

Return the current read position.

Return type:

int

writable()#

Return False - this buffer is read-only.

Return type:

bool

On this page