lazyscribe.prefect package¶

Submodules¶

Prefect experiment tasks.

class lazyscribe.prefect.experiment.LazyExperiment(project: Path | None = None, author: str | None = 'runner', **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe experiments.

This task is only meant for creating new experiments.

Parameters¶

projectPath: The path to the project JSON associated with the project.
authorstr, optional (default getpass.getuser()): The author of the experiment.
**kwargs: Keyword arguments for prefect.Task.

load_artifact(name: str, validate: bool = True, **kwargs)[source]¶

Add a load_artifact task.

Parameters¶

namestr: The name of the artifact to load.
validatebool, optional (default True): Whether or not to validate the runtime environment against the artifact metadata.
**kwargsdict: Keyword arguments for the handler read function.

Returns¶

object: The artifact.

log_artifact(name: str, value: Any, handler: str, fname: str | None = None, overwrite: bool = False, **kwargs)[source]¶

Add a log_artifact task.

Parameters¶

namestr: The name of the artifact.
valueAny: The object to persist to the filesystem.
handlerstr: The name of the handler to use for the object.
fnamestr, optional (default None): The filename for the artifact. If not provided, it will be derived from the name of the artifact and the builtin suffix for each handler.
overwritebool, optional (default False): Whether or not to overwrite an existing artifact with the same name. If set to True, the previous artifact will be removed and overwritten with the current artifact.
**kwargsdict: Keyword arguments for the write function of the handler.

log_metric(name: str, value: float | int)[source]¶

Add a log_metric task.

Parameters¶

namestr: Name of the metric.
valueint or float: Value of the metric.

log_parameter(name: str, value: Any)[source]¶

Add a log_parameter task.

Parameters¶

namestr: Name of the parameter.
valueAny: The parameter itself.

log_test(name: str, description: str | None = None, flow: Flow | None = None) → Iterator[Task][source]¶

Add a lazyscribe.prefect.LazyTest task to the flow.

On exit from the context handler, an additional task will be added to append the test to the experiment itself.

Parameters¶

namestr: Name of the test.
descriptionstr, optional (default None): A description of the test.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

LazyTest: An instantiated lazyscribe.prefect.LazyTest object. This task has already been added to the flow.

run(name: str | None = None, project: Path | None = None, author: str | None = None) → Experiment[source]¶

Instantiate a new experiment.

Parameters¶

namestr, optional (default None): The name of the experiment. Defaults to the task name.
projectPath, optional (default None): The project JSON path. Defaults to the class attribute.
authorstr, optional (default None): The author. Defaults to the class attribute.

Returns¶

Experiment: The lazyscribe.Experiment class.

tag(*args, overwrite: bool = False)[source]¶

Add a add_tag task.

Parameters¶

*args: The tags.
overwritebool, optional (default False): Whether to add or overwrite the new tags.

Prefect project tasks.

class lazyscribe.prefect.project.LazyProject(fpath: str = 'project.json', mode: Literal['r', 'a', 'w', 'w+'] = 'w', author: str | None = None, storage_options: dict | None = None, **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe projects.

Parameters¶

fpathstr, optional (default “project.json”): The location of the project file. If not file exists, this will be the location of the output JSON file.
mode{“r”, “a”, “w”, “w+”}, optional (default “w”): The mode for opening the project. See lazyscribe.Project for reference.
authorstr, optional (default None): The project author.
storage_optionsdict, optional (default None): Storage options to pass to the filesystem initialization.
**kwargs: Keyword arguments for prefect.Task.

append(other: Task, **kwargs)[source]¶

Add an append_experiment task to the flow.

Parameters¶

otherTask: The new experiment to add.
**kwargs: Keyword arguments for prefect.task().

Add a lazyscribe.prefect.LazyExperiment to the flow.

On exit from the context handler, an additional task will be added to append the experiment to the project itself.

Parameters¶

namestr: The name of the experiment.
projectstr, optional (default None): The location of the project JSON. If you have parameterized the project location, re-supply it here.
authorstr, optional (default None): The author of the project. If you have parameterized the author, please re-supply it here.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

LazyExperiment: An instantiated lazyscribe.prefect.LazyExperiment object. This task has already been added to the flow.

merge(other: Project, flow: Flow | None = None)[source]¶

Add a merge_projects task to the flow.

Parameters¶

otherProject: The other project.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

Task: The bound lazyscribe.prefect.merge_projects() task.

run(fpath: str | None = None, mode: Literal['r', 'a', 'w', 'w+'] | None = None, author: str | None = None, storage_options: dict | None = None) → Project[source]¶

Instantiate a lazyscribe.Project.

Parameters¶

fpathstr, optional (default None): The location of the project JSON file.
modestr, optional (default None): The mode for opening the project.
authorstr, optional (default None): The author for the project.
storage_optionsdict, optional (default None): Storage options to pass to the filesystem initialization.

Returns¶

Project: The instantiated project.

save(flow: Flow | None = None)[source]¶

Add a save_project task to the flow.

Parameters¶

flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

to_tabular(flow: Flow | None = None) → Task[source]¶

Add a project_to_tabular task to the flow.

Parameters¶

flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

Task: The project-level list.
Task: The test-level list.

Prefect test tasks.

class lazyscribe.prefect.test.LazyTest(description: str | None = None, **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe tests.

This task should only be used to instantiate new tests.

Parameters¶

descriptionstr, optional (default None): Description of the test.
**kwargs: Keyword arguments for prefect.Task.

log_metric(name: str, value: float | int)[source]¶

Add a log_metric task.

Parameters¶

namestr: The name of the metric.
valuefloat or int: The metric value.

log_parameter(name: str, value: Any)[source]¶

Add a log_parameter task.

Parameters¶

namestr: The name of the parameter.
valueany: The parameter value.

run(name: str | None = None, description: str | None = None) → Test[source]¶

Instantiate a new lazyscribe.test.Test object.

Parameters¶

namestr, optional (default None): The name of the test. Defaults to the task name.
descriptionstr, optional (default None): Description of the test.

Returns¶

Test: Instantiated lazyscribe.test.Test object.

Module contents¶

Import the tasks.

class lazyscribe.prefect.LazyExperiment(project: Path | None = None, author: str | None = 'runner', **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe experiments.

This task is only meant for creating new experiments.

Parameters¶

projectPath: The path to the project JSON associated with the project.
authorstr, optional (default getpass.getuser()): The author of the experiment.
**kwargs: Keyword arguments for prefect.Task.

load_artifact(name: str, validate: bool = True, **kwargs)[source]¶

Add a load_artifact task.

Parameters¶

namestr: The name of the artifact to load.
validatebool, optional (default True): Whether or not to validate the runtime environment against the artifact metadata.
**kwargsdict: Keyword arguments for the handler read function.

Returns¶

object: The artifact.

log_artifact(name: str, value: Any, handler: str, fname: str | None = None, overwrite: bool = False, **kwargs)[source]¶

Add a log_artifact task.

Parameters¶

namestr: The name of the artifact.
valueAny: The object to persist to the filesystem.
handlerstr: The name of the handler to use for the object.
fnamestr, optional (default None): The filename for the artifact. If not provided, it will be derived from the name of the artifact and the builtin suffix for each handler.
overwritebool, optional (default False): Whether or not to overwrite an existing artifact with the same name. If set to True, the previous artifact will be removed and overwritten with the current artifact.
**kwargsdict: Keyword arguments for the write function of the handler.

log_metric(name: str, value: float | int)[source]¶

Add a log_metric task.

Parameters¶

namestr: Name of the metric.
valueint or float: Value of the metric.

log_parameter(name: str, value: Any)[source]¶

Add a log_parameter task.

Parameters¶

namestr: Name of the parameter.
valueAny: The parameter itself.

log_test(name: str, description: str | None = None, flow: Flow | None = None) → Iterator[Task][source]¶

Add a lazyscribe.prefect.LazyTest task to the flow.

On exit from the context handler, an additional task will be added to append the test to the experiment itself.

Parameters¶

namestr: Name of the test.
descriptionstr, optional (default None): A description of the test.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

LazyTest: An instantiated lazyscribe.prefect.LazyTest object. This task has already been added to the flow.

run(name: str | None = None, project: Path | None = None, author: str | None = None) → Experiment[source]¶

Instantiate a new experiment.

Parameters¶

namestr, optional (default None): The name of the experiment. Defaults to the task name.
projectPath, optional (default None): The project JSON path. Defaults to the class attribute.
authorstr, optional (default None): The author. Defaults to the class attribute.

Returns¶

Experiment: The lazyscribe.Experiment class.

tag(*args, overwrite: bool = False)[source]¶

Add a add_tag task.

Parameters¶

*args: The tags.
overwritebool, optional (default False): Whether to add or overwrite the new tags.

class lazyscribe.prefect.LazyProject(fpath: str = 'project.json', mode: Literal['r', 'a', 'w', 'w+'] = 'w', author: str | None = None, storage_options: dict | None = None, **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe projects.

Parameters¶

fpathstr, optional (default “project.json”): The location of the project file. If not file exists, this will be the location of the output JSON file.
mode{“r”, “a”, “w”, “w+”}, optional (default “w”): The mode for opening the project. See lazyscribe.Project for reference.
authorstr, optional (default None): The project author.
storage_optionsdict, optional (default None): Storage options to pass to the filesystem initialization.
**kwargs: Keyword arguments for prefect.Task.

append(other: Task, **kwargs)[source]¶

Add an append_experiment task to the flow.

Parameters¶

otherTask: The new experiment to add.
**kwargs: Keyword arguments for prefect.task().

Add a lazyscribe.prefect.LazyExperiment to the flow.

On exit from the context handler, an additional task will be added to append the experiment to the project itself.

Parameters¶

namestr: The name of the experiment.
projectstr, optional (default None): The location of the project JSON. If you have parameterized the project location, re-supply it here.
authorstr, optional (default None): The author of the project. If you have parameterized the author, please re-supply it here.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

LazyExperiment: An instantiated lazyscribe.prefect.LazyExperiment object. This task has already been added to the flow.

merge(other: Project, flow: Flow | None = None)[source]¶

Add a merge_projects task to the flow.

Parameters¶

otherProject: The other project.
flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

Task: The bound lazyscribe.prefect.merge_projects() task.

run(fpath: str | None = None, mode: Literal['r', 'a', 'w', 'w+'] | None = None, author: str | None = None, storage_options: dict | None = None) → Project[source]¶

Instantiate a lazyscribe.Project.

Parameters¶

fpathstr, optional (default None): The location of the project JSON file.
modestr, optional (default None): The mode for opening the project.
authorstr, optional (default None): The author for the project.
storage_optionsdict, optional (default None): Storage options to pass to the filesystem initialization.

Returns¶

Project: The instantiated project.

save(flow: Flow | None = None)[source]¶

Add a save_project task to the flow.

Parameters¶

flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

to_tabular(flow: Flow | None = None) → Task[source]¶

Add a project_to_tabular task to the flow.

Parameters¶

flowFlow, optional (default None): A prefect.Flow object. If not supplied, this function will retrieve a flow from prefect.context.

Returns¶

Task: The project-level list.
Task: The test-level list.

class lazyscribe.prefect.LazyTest(description: str | None = None, **kwargs)[source]¶

Bases: Task

Prefect integration for logging lazyscribe tests.

This task should only be used to instantiate new tests.

Parameters¶

descriptionstr, optional (default None): Description of the test.
**kwargs: Keyword arguments for prefect.Task.

log_metric(name: str, value: float | int)[source]¶

Add a log_metric task.

Parameters¶

namestr: The name of the metric.
valuefloat or int: The metric value.

log_parameter(name: str, value: Any)[source]¶

Add a log_parameter task.

Parameters¶

namestr: The name of the parameter.
valueany: The parameter value.

run(name: str | None = None, description: str | None = None) → Test[source]¶

Instantiate a new lazyscribe.test.Test object.

Parameters¶

namestr, optional (default None): The name of the test. Defaults to the task name.
descriptionstr, optional (default None): Description of the test.

Returns¶

Test: Instantiated lazyscribe.test.Test object.