What you will do
You will understand the reference inputs, protocol choices, fitting/scoring boundary, and conditions for reusing fitted parameters.
Before you start
- A completed, retained household source run with verified configuration, population, world, and model references.
- Reviewed historical FX and observed-price mappings with comparable geography, frequency, currencies, and quantity units.
- A supported Linux runtime with Landlock/seccomp isolation and the installed calibration CLI. Unsupported isolation blocks fitting.
Step / 01
Start from the exact household source
Open Calibration and evaluation from your project, or choose Evaluate this configuration against historical prices on a completed household run. The workbench loads and verifies its exact retained input closure. Expired or erased sources cannot be reconstructed from metadata.
Select the historical FX series and price targets, matching each configured good to comparable observations. Fixed base prices, the FX anchor, and import shares remain part of the reference context. The fit does not jointly estimate import share and pass-through because this equation identifies their product.
Confirm quantity comparability explicitly. Missing periods, currency differences, package sizes, and unsupported coverage are not repaired through implicit conversion or synthetic historical values.
Step / 02
Lock the protocol before fitting
Declare strictly ordered, non-overlapping training, validation, and final holdout windows. Choose nonzero coefficient bounds inside [0, 1], integer lag candidates, minimum row counts, coverage, design energy, and lag-tie tolerance. Select a constant-anchor or explicitly declared coefficient/lag comparator.
Training estimates the bounded coefficient. Validation compares eligible lags on common usable dates. Missing historical context, unidentifiable exposure, insufficient support, or ambiguous lag selection remains unavailable rather than yielding an arbitrary coefficient.
Declare target-specific acceptance criteria before looking at scores: relative error tolerances, comparator improvement, required backtest pass fraction, and an in-range policy. Unselected criteria do not become a passing assessment. Specify the intended FX range and horizon; execution outside observed support remains extrapolation.
Step / 03
Keep fitting and scoring separate
Review the protocol, native-isolation capability, coverage, and exposure diagnostics, then create the study from its reviewed hash. The backend locks source and observation-partition identities before queueing.
Each fitting child receives only training/validation observations and necessary historical context. Linux filesystem and network restrictions enforce that capability. All fits and backtests finish and their outputs are sealed before the control plane supplies final holdout to scoring. The scorer verifies the fit identity and only scores; it does not refit on holdout.
Owner-level exposure history survives copied data, new projects, and report deletion. Reused holdout is exploratory and cannot establish fresh untouched evaluation. Pipeline isolation does not prove that a person never previously looked at the uploaded observations.
Step / 04
Read error, support, and uncertainty together
Compare fitted, original-config, and declared-comparator errors on identical usable holdout dates. Relative RMSE/MAE divide price errors by the fixed base price. Missing dates reduce coverage. Boundary estimates, unavailable targets, failed criteria, and out-of-range FX remain visible in the report.
Optional moving-block residual bootstrap retains joint coefficient vectors under explicitly declared assumptions, block size, draws, seed, and interval level. It is separate from paired stochastic-replica variation and explicit structural/parameter sensitivity. Increasing replicas cannot repair model misspecification or population uncertainty.
The retained WFP/World Bank example improves holdout error over its constant-anchor comparator for the available targets, but fails their declared tolerance and uses holdout FX outside training support. It demonstrates a replayable comparison and visible rejection, not a validated forecast.
Step / 05
Reuse fitted points through a new reviewed binding
Inspect the verified report, paged residuals, draws, scope diagnostics, and exact replay. Creating a fixed-estimate binding is an explicit owner action. Prepare a new scenario with its verified fitted points and lag references while preserving the source units and model context.
The proposal appends new reviewed versions; it does not overwrite the source run or frozen world. Fitted intervals are never silently converted to independent uniform parameters. Changed FX levels or horizons receive the assessment's extrapolation diagnostics.
Calibration evidence depends on retained sources. Erasure invalidates dependent bindings and derivative evidence, while exposure metadata remains so deleting a study cannot reset holdout history.