Skip to main content
Version: 0.5.4

common.storages.load_storage

LoadStorage Objects​

class LoadStorage(DataItemStorage, VersionedStorage)

[view_source]

NORMALIZED_FOLDER​

folder within the volume where load packages are stored

LOADED_FOLDER​

folder to keep the loads that were completely processed

NEW_PACKAGES_FOLDER​

folder where new packages are created

list_new_jobs​

def list_new_jobs(load_id: str) -> Sequence[str]

[view_source]

Lists all jobs in new jobs folder of normalized package storage and checks if file formats are supported

list_normalized_packages​

def list_normalized_packages() -> Sequence[str]

[view_source]

Lists all packages that are normalized and will be loaded or are currently loaded

list_loaded_packages​

def list_loaded_packages() -> Sequence[str]

[view_source]

List packages that are completely loaded

list_failed_jobs_in_loaded_package​

def list_failed_jobs_in_loaded_package(load_id: str) -> Sequence[LoadJobInfo]

[view_source]

List all failed jobs and associated error messages for a completed load package with load_id

commit_schema_update​

def commit_schema_update(load_id: str, applied_update: TSchemaTables) -> None

[view_source]

Marks schema update as processed and stores the update that was applied at the destination

import_new_job​

def import_new_job(load_id: str,
                   job_file_path: str,
                   job_state: TJobState = "new_jobs") -> None

[view_source]

Adds new job by moving the job_file_path into new_jobs of package load_id

maybe_remove_completed_jobs​

def maybe_remove_completed_jobs(load_id: str) -> None

[view_source]

Deletes completed jobs if delete_completed_jobs config flag is set. If package has failed jobs, nothing gets deleted.

get_load_package_info​

def get_load_package_info(load_id: str) -> LoadPackageInfo

[view_source]

Gets information on normalized OR loaded package with given load_id, all jobs and their statuses.

get_load_package_state​

def get_load_package_state(load_id: str) -> TLoadPackageState

[view_source]

Gets state of normlized or loaded package with given load_id, all jobs and their statuses.

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.