roboto.fs.object_store#

Submodules#

Package Contents#

type roboto.fs.object_store.CredentialProvider = Callable[[], Credentials]#
class roboto.fs.object_store.Credentials#

Bases: TypedDict

This interface is driven by botocore.credentials.RefreshableCredentials

access_key: str#
expiry_time: str | None#
region: str#
secret_key: str#
token: str#
class roboto.fs.object_store.FutureLike#

Bases: Protocol[T_co]

Protocol for future-like objects.

done()#

Return True if the future is done.

Return type:

bool

result()#

Wait for and return the result.

Return type:

T_co

class roboto.fs.object_store.ObjectStore#

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...
classmethod create(credential_provider, **kwargs)#
Parameters:

credential_provider (CredentialProvider)

Return type:

ObjectStore

get(source_uri, destination, on_progress=None)#

Downloads a file from a cloud URI to a local path.

Parent directories of the destination path are created automatically if they don’t exist.

Parameters:

  • source_uri (str) – Full URI (e.g., ‘s3://my-bucket/folder/data.csv’)

  • destination (pathlib.Path) – Local path where the file should be saved.

  • on_progress (Optional[OnProgress]) – Optional callback to be periodically called with the number of bytes downloaded.

Return type:

FutureLike[None]

put(source, destination_uri, on_progress=None)#

Uploads a local file to a specific cloud URI.

Parameters:

  • source (pathlib.Path) – Local path to the file.

  • destination_uri (str) – Full URI (e.g., ‘s3://my-bucket/folder/data.csv’)

  • on_progress (Optional[OnProgress]) – Optional callback to be periodically called with the number of bytes uploaded.

Return type:

FutureLike[None]

type roboto.fs.object_store.OnProgress = Callable[[int], None]#
class roboto.fs.object_store.S3Store(s3_client, transfer_config=None)#

Bases: roboto.fs.object_store.object_store.ObjectStore

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...
Parameters:

  • s3_client (botocore.client.BaseClient)

  • transfer_config (Optional[boto3.s3.transfer.TransferConfig])

classmethod create(credential_provider, **kwargs)#

Factory function to assemble an S3Store with refreshable credentials.

Parameters:

credential_provider (roboto.fs.object_store.object_store.CredentialProvider)

Return type:

S3Store

get(source_uri, destination, on_progress=None)#

Downloads a file from a cloud URI to a local path.

Parent directories of the destination path are created automatically if they don’t exist.

Parameters:

  • source_uri (str) – Full URI (e.g., ‘s3://my-bucket/folder/data.csv’)

  • destination (pathlib.Path) – Local path where the file should be saved.

  • on_progress (Optional[roboto.fs.object_store.object_store.OnProgress]) – Optional callback to be periodically called with the number of bytes downloaded.

Return type:

roboto.fs.object_store.object_store.FutureLike[None]

put(source, destination_uri, on_progress=None)#

Uploads a local file to a specific cloud URI.

Parameters:

  • source (pathlib.Path) – Local path to the file.

  • destination_uri (str) – Full URI (e.g., ‘s3://my-bucket/folder/data.csv’)

  • on_progress (Optional[roboto.fs.object_store.object_store.OnProgress]) – Optional callback to be periodically called with the number of bytes uploaded.

Return type:

roboto.fs.object_store.object_store.FutureLike[None]

class roboto.fs.object_store.StoreRegistry#
classmethod get_store_for_uri(uri, credential_provider, **kwargs)#
Parameters:

  • uri (str)

  • credential_provider (roboto.fs.object_store.object_store.CredentialProvider)

Return type:

roboto.fs.object_store.object_store.ObjectStore

classmethod register(scheme)#

Class Decorator. Registers the decorated class to handle the specific URI scheme.

Parameters:

scheme (str)

On this page