Save and examine experiment artifacts¶

In this guide, we will walk through how you can save artifacts associated with your experiment.

Persistence¶

Every experiment has a path attribute, with the pathlib.Path to a unique folder based on experiment slug:

from lazyscribe import Project

project = Project(fpath="project.json", mode="w")
with project.log("My experiment") as exp:
    exp.path

This will return pathlib.Path("./my-experiment-YYYYMMDDHHMMSS"), where the datetime corresponds to the created_at attribute.

Important

Unless you log an artifact, this directory will not be created automatically.

To associate an artifact with your experiment, use lazyscribe.experiment.Experiment.log_artifact(). Serialization is delegated to a subclass of lazyscribe.artifacts.base.Artifact.

Persisting a scikit-learn estimator with pickle.¶

from lazyscribe import Project
from sklearn.svm import SVC

project = Project(fpath="project.json", mode="w")
with project.log("My experiment") as exp:
    X, y = ...
    model = SVC()
    model.fit(X, y)
    exp.log_artifact(name="estimator", value=model, handler="pickle")

In the case of code failures, we want to minimize the chance that you need to clean up orphaned experiment data. For this reason, artifacts are not persisted to the filesystem when you call lazyscribe.experiment.Experiment.log_artifact(). Artifacts are only saved when you call lazyscribe.project.Project.save().

We have a selection of builtin artifact handlers, specified below:

Builtin artifact handlers¶
Class	Alias	Description	Additional requirements
`lazyscribe.artifacts.json.JSONArtifact`	json	Artifacts written using `json.dump()` and read using `json.load()`	N/A
`lazyscribe.artifacts.pickle:PickleArtifact`	pickle	Artifacts written using `pickle.dump()` and read using `pickle.load()`	N/A

We also provide first-party supported artifact handlers (install via pip):

First-party supported artifact handlers¶
Alias	Description	Package Installation
joblib	Artifacts written using `joblib.dump()` and read using `joblib.load()`	lazyscribe-joblib
csv	Artifacts written to CSV files using PyArrow	lazyscribe-arrow
parquet	Artifacts written to parquet files using PyArrow	lazyscribe-arrow
onnx	Artifacts written to ONNX model objects	lazyscribe-onnx
yaml	Artifacts written using `yaml.dump()` and read using `yaml.load()`. You can specify the dumper using the `Dumper` keyword argument and the loader using the `Loader` keyword argument. Defaults to `yaml.FullDumper` and `yaml.SafeLoader` respectively if not specified.	lazyscribe-yaml

Loading and validation¶

To load an artifact, use lazyscribe.experiment.Experiment.load_artifact().

from lazyscribe import Project

project = Project("project.json", mode="r")
exp = project["my-experiment"]
model = exp.load_artifact(name="estimator")

When an artifact is persisted to the filesystem, the handler may save environment parameters to use for validation when attempting to load the artifact into python. For example, when persisting a scikit-learn model object with the "joblib" handler from lazyscribe-joblib, it will include the scikit-learn and joblib versions in the artifact metadata. If the metadata doesn’t match with a handler constructed in the current runtime environment, lazyscribe will raise an error. You can disable validation using validate=False:

model = exp.load_artifact(name="estimator", validate=False)