common.storages.fsspecs.google_drive

GoogleDriveFileSystem Objects

class GoogleDriveFileSystem(AbstractFileSystem)

[view_source]

__init__

def __init__(credentials: GcpCredentials = None,
             trash_delete: bool = True,
             access: Optional[Literal["full_control",
                                      "read_only"]] = "full_control",
             spaces: Optional[Literal["drive", "appDataFolder",
                                      "photos"]] = "drive",
             **kwargs: Any)

[view_source]

Google Drive as a file-system.

The gdrive url has following format: gdrive://<root_file_id/<file_path> Where <root_file_id> is a file id of the folder where the <file_path> is present.

Google Drive provides consistency when file ids are used. Changes are reflected immediately. In case of listings (ls) the consistency is eventual. Changes are reflected with a delay. As ls is used to retrieve file id from file name, we will be unable to build consistent filesystem with google drive API. Use this with care.

Based on original fsspec Google Drive implementation: https://github.com/fsspec/gdrivefs

Arguments:

• credentials GcpCredentials - Google Service credentials. If not provided, anonymous credentials are used
• trash_delete bool - If True sends files to trash on rm. If False, deletes permanently. Note that permanent delete is not available for shared drives access (Optional[Literal["full_control", "read_only"]]): One of "full_control", "read_only". spaces (Optional[Literal["drive", "appDataFolder", "photos"]]): Category of files to search, can be 'drive', 'appDataFolder' and 'photos'. Of these, only the first is general. **kwargs: Passed to the parent.

connect

def connect() -> None

[view_source]

Connect to Google Drive.

mkdir

def mkdir(path: str, create_parents: Optional[bool] = True) -> None

[view_source]

Create a directory.

Arguments:

• path str - The directory to create. create_parents (Optional[bool]): Whether to create parent directories if they don't exist. Defaults to True.

makedirs

def makedirs(path: str, exist_ok: Optional[bool] = True) -> None

[view_source]

Create a directory and all its parent components.

Arguments:

• path str - The directory to create.
• exist_ok Optional[bool] - Whether to raise an error if the directory already exists. Defaults to True.

rm

def rm(path: str,
       recursive: Optional[bool] = True,
       maxdepth: Optional[int] = None) -> None

[view_source]

Remove files or directories.

Arguments:

• path str - The file or directory to remove.
• recursive Optional[bool] - Whether to remove directories recursively. Defaults to True.
• maxdepth Optional[int] - The maximum depth to remove directories.

rmdir

def rmdir(path: str) -> None

[view_source]

Remove a directory.

Arguments:

• path str - The directory to remove.

export

def export(path: str, mime_type: str) -> Any

[view_source]

Convert a Google-native file to other format and download

mime_type is something like "text/plain"

ls

def ls(path: str, detail: Optional[bool] = False) -> Any

[view_source]

List files in a directory.

Arguments:

• path str - The directory to list.
• detail Optional[bool] - Whether to return detailed file information. Defaults to False.

Returns:

• Any - Files in the directory data.

path_to_file_id

def path_to_file_id(path: str,
                    parent_id: Optional[str] = None,
                    parent_path: str = "") -> str

[view_source]

Get the file ID from a path.

Arguments:

• path str - The path to get the file ID from.
• parent_id Optional[str] - The parent directory id to search.
• parent_path Optional[str] - Path corresponding to parent id

Returns:

• str - The file ID.

GoogleDriveFile Objects

class GoogleDriveFile(AbstractBufferedFile)

[view_source]

__init__

def __init__(fs: GoogleDriveFileSystem,
             path: str,
             mode: Optional[str] = "rb",
             block_size: Optional[int] = DEFAULT_BLOCK_SIZE,
             autocommit: Optional[bool] = True,
             **kwargs: Any)

[view_source]

A Google Drive file.

Arguments:

• fs AbstractFileSystem - The file system to open the file from.
• path str - The file to open.
• mode Optional[str] - The mode to open the file in. Defaults to "rb".
• block_size Optional[str] - The block size to use. Defaults to DEFAULT_BLOCK_SIZE.
• autocommit Optional[bool] - Whether to automatically commit the file. Defaults to True.
• **kwargs - Passed to the parent.

commit

def commit() -> None

[view_source]

If not auto-committing, finalize the file.

discard

def discard() -> None

[view_source]

Cancel in-progress multi-upload.