Understand experiments · Lesson 07
Run the fraud notebook and read retained history
The video executes the reviewed fraud notebook and asks which data and settings produced its saved model. It then replaces the current CSV and deliberately damages a disposable copy of the evidence store.
By the end of this lesson
- Separate the intended Plan from observed Run evidence.
- Read a retained Run by its exact identity.
- Explain artifact retention and the recorded integrity refusal.
Watch the lesson
English · 6:15 · Recorded with Apizr 0.4.6 development candidate.
Chapters in this recording
| Time | Chapter |
|---|---|
| 00:00 | What produced this model file? |
| 00:27 | Plan, execution, Run |
| 00:58 | Execute the reviewed notebook |
| 02:03 | Intended values and observed values |
| 02:36 | Read the retained record by identity |
| 03:26 | Does history silently follow the current CSV? |
| 04:17 | Variant: altered retained evidence is refused |
| 05:12 | Your turn: what should you preserve? |
| 05:35 | A Run is a reviewable execution record |
| 06:04 | Continue with the written guide |
Read the recording transcript and commands
This is the retained English narration, in the original chapter order. Commands keep the recorded version and prerequisites; setup additions in the lesson are labelled separately.
00:00 — What produced this model file?
You have reviewed the fraud notebook. Now your colleague asks which data and settings produced the saved model on your disk. A model filename alone cannot answer that. In this lesson we execute the trusted notebook and read a retained experiment record. We will connect the intended setup, the observed metrics and the selected model identity, then check what happens when the current CSV is replaced.
00:27 — Plan, execution, Run
Think of the lifecycle as three separate steps. A Plan identifies the intended source, inputs and controls. A fresh local Python process executes the workload in the selected environment. The Run records observed facts, including status, selected metrics and output fingerprints. The process has host access, so this is the point at which trusting the notebook matters. Apizr does not install pandas or scikit-learn for you.
00:58 — Execute the reviewed notebook
Read the command before running it. Experiment run selects the saved notebook. The output option gives the model a stable logical name and points to its actual file. JSON goes into run a dot JSON so that later commands can use the exact Run identity. During execution the forest is trained and its held-out metrics are calculated. A successful exit says that this execution completed; model quality still needs scientific interpretation.
apizr experiment run fraud_detection.ipynb --output model=artifacts/model.joblib --format json > run-a.json
python lab.py run run-a.json
02:03 — Intended values and observed values
Do not merge the Plan and Run into one undifferentiated configuration. A parameter candidate in source and a runtime parameter observation answer different questions. Likewise, a seed in the Plan is not automatically an observed seed in the Run. Look for origin and missing evidence rather than filling the gap yourself. The worker Python and package versions describe the execution environment; requirements text alone describes a specification.
02:36 — Read the retained record by identity
List helps discover the retained Run, while show selects one exact identity. We save the complete record and display a short projection so that each line remains readable. Find the three observed metric values, the input digest and the selected model digest. These values come from this actual execution. Precision and recall describe different errors; neither can be replaced by the label successful. Keep the complete JSON when a reviewer needs more than this projection.
apizr experiment list
RUN_A=$(python -c 'import json; print(json.load(open("run-a.json"))["run_digest"])')
apizr experiment show "$RUN_A" --format json > retained-a.json
python lab.py run retained-a.json
03:26 — Does history silently follow the current CSV?
Now we deliberately replace the current CSV with cohort B without rerunning the notebook. Predict the effect on Run A. The retained input identity stays tied to A, while the file at the same path now has another digest. This is why a content identity matters. The local history records evidence; it is not a dataset archive or a model registry. If the underlying bytes are overwritten, the recorded digest does not recreate them.
RUN_A=$(python -c 'import json; print(json.load(open("run-a.json"))["run_digest"])')
cp data/transactions-b.csv data/transactions.csv
python lab.py retain
apizr experiment show "$RUN_A" --format json > retained-a-after.json
04:17 — Variant: altered retained evidence is refused
Now test the integrity boundary instead of trusting a happy-path report. We copy the local store and append whitespace to one retained Run in that disposable copy. The JSON may still be syntactically readable, but its bytes no longer match the canonical record and filename identity. Show must refuse it. The original history remains available. This demonstrates byte integrity of retained local evidence, not a signed attestation or authorization for another user to trust the experiment.
cp -R .apizr/experiments/v1 damaged-store
RUN_A=$(python -c 'import json; print(json.load(open("run-a.json"))["run_digest"])'); printf " " >> "damaged-store/runs/$RUN_A.json"
Expected refusal: exit 2.
apizr experiment show "$RUN_A" --store damaged-store --format json > refused.json 2> refused.log
python lab.py refusal
05:12 — Your turn: what should you preserve?
Pause and answer. You have a Run digest and a selected model fingerprint. Tomorrow you overwrite the model file. Can show restore the previous model? What should you retain separately if another engineer must serve that exact model later? Explain the difference between identifying an artifact and storing the artifact bytes.
05:35 — A Run is a reviewable execution record
Show cannot recreate overwritten model bytes. Keep the selected data and model through your own artifact retention process when they must be reused. The Run lets you verify which identities and observations were recorded, and it refuses corrupt retained evidence. You can now explain what produced this observed result and what must be preserved alongside it. Next we record cohort B and compare the two histories.
06:04 — Continue with the written guide
The current written guide remains authoritative. Continue there for the detailed steps. Thank you for following Apizr Academy.
1. Execute the reviewed workload
Use a fresh fraud-detection/ copy from the recorded source materials, with cohort A active. Install the notebook's scientific requirements in the selected development environment. The process executes trusted code with host access; Apizr does not install pandas or scikit-learn for it.
apizr experiment run fraud_detection.ipynb --output model=artifacts/model.joblib --format json > run-a.json
python lab.py run run-a.json
apizr experiment list
RUN_A=$(python -c 'import json; print(json.load(open("run-a.json"))["run_digest"])')
apizr experiment show "$RUN_A" --format json > retained-a.json
python lab.py run retained-a.json
The Plan identifies intended source, inputs and controls. The Run contains the observed outcome, metrics and selected output identity. Read parameter origins rather than assuming every source candidate or declared seed became a runtime observation. Success means this execution completed; precision, recall and AUC still require interpretation.
2. Replace the CSV without rewriting Run A
cp data/transactions-b.csv data/transactions.csv
python lab.py retain
apizr experiment show "$RUN_A" --format json > retained-a-after.json
The file at the same path now has different bytes. The retained record remains bound to the input of A. Its fingerprints identify data and the selected model; they do not archive either file's contents.
3. Test the recorded integrity boundary
Work on a copied store, keeping the original history untouched:
cp -R .apizr/experiments/v1 damaged-store
printf ' ' >> "damaged-store/runs/$RUN_A.json"
apizr experiment show "$RUN_A" --store damaged-store --format json > refused.json 2> refused.log
The final command is expected to be refused with exit 2: syntactically readable JSON can still violate canonical bytes and the stored identity. This demonstrates local evidence integrity, not a signature or independent authorization.
Practice
Answer the video's artifact-retention question. If tomorrow's training overwrites model.joblib, can show restore A's model? What must another engineer preserve to serve exactly that model later?
Check your reasoning
The record cannot recreate overwritten model bytes. Retain the actual selected model and needed input bytes through your artifact-retention process, alongside the Run. Their hashes let you check identity; the copied-store refusal shows why altering a retained record is not a valid way to change its evidence.