Get model performance
GET /v1/models/{id}/performance
apiKeyX-Spavik-Api-Key<token>Spavik integration API key (sk-spv-api-...), sent in X-Spavik-Api-Key.
id*stringModel identifier or slug
days?integervalue <= 36530User-Agent?stringInformation about the agent originating the request, when known. Browsers supply their own value. Optional diagnostic metadata, declared by the caller and never used for authorization. Omit unknown values. Values are trimmed and truncated to the documented length.
length <= 1024X-Spavik-Client-Name?stringHTTP caller: SDK, MCP server or direct application (for example spavik-sdk-python, spavik-sdk-javascript or spavik-mcp). Optional diagnostic metadata, declared by the caller and never used for authorization. Omit unknown values. Values are trimmed and truncated to the documented length.
length <= 128X-Spavik-Client-Version?stringVersion of that SDK, MCP server or direct application. Optional diagnostic metadata, declared by the caller and never used for authorization. Omit unknown values. Values are trimmed and truncated to the documented length.
length <= 64Success
application/json- response
source?"client_reported_predictions""client_reported_predictions"quality_reports?array<>model_id?stringversion?integertask?string"classification""regression"pending_outcomes?integerrefresh_suggested?booleantraining_engine?stringtraining_metrics?Scores on evaluated observations. Classification: AUC, Accuracy, AveragePrecision (binary only), LogLoss and MacroF1. Regression: MAE, RMSE, R². Forecasting adds MASE, Bias, BiasRatio and Pinball. Undefined scores are null; see metric_context.undefined_reasons. Secondary scores do not change the primary assessment.
training_evaluation?Engine evaluation metadata, forwarded without recalculating the split. Null when not supplied by the engine. Row positions are zero-based in the received data, excluding its header and before dropping missing targets.
evaluation?training_classification_report?|Per-class scores on actual predicted labels. Null for regression, older engines or unavailable evaluation. At training these describe the holdout; an augmented version retains its initial evaluation. In performance these describe only predictions matched to received outcomes, not the entire prediction population. All classes known to the evaluated versions and all observed/predicted labels are included. The AUC assessment is unchanged.
training_calibration?|Descriptive calibration on held-out test rows, before the final persistence fit. Does not recalibrate probabilities. Null for regression or missing historical diagnostics. Retained as the initial evaluation on same-engine refreshes; cleared when the engine changes.
observed?Measure suited to the task. For classification, metric is accuracy and value the share of exact answers (higher is better). For regression, metric is mean_absolute_error and value the average gap between predicted and actual (lower is better). baseline is what the training reference scores on the same outcomes, using each prediction version. Missing training parameters give a null baseline. Read metrics, baselines, metric_context and assessment for the complete task-specific evaluation.
import requestsurl = "https://api.spavik.co/v1/models/string/performance"response = requests.request("GET", url)print(response.text){ "source": "client_reported_predictions", "quality_reports": [ { "version": 0, "engine": "string", "created_at": "2019-08-24T14:15:22Z", "report": {} } ], "model_id": "string", "version": 0, "task": "classification", "pending_outcomes": 0, "refresh_suggested": true, "training_engine": "string", "training_metrics": { "AUC": 0, "Accuracy": 0, "AveragePrecision": 0, "LogLoss": 0, "MacroF1": 0, "MAE": 0, "RMSE": 0, "R²": 0, "MASE": 0, "Bias": 0, "BiasRatio": 0, "Pinball": 0, "property1": 0, "property2": 0 }, "training_evaluation": { "metric_context": { "undefined_reasons": { "property1": "string", "property2": "string" }, "class_order": [ "string" ], "positive_class": "string", "auc_average": "binary", "average_precision_method": "non_interpolated", "macro_f1_average": "unweighted_defined_classes", "log_loss_epsilon": 0 }, "protocol_version": 0, "interpretation": "descriptive_only", "kind": "random_holdout", "split": { "seed": 0, "shuffle": true, "test_fraction_requested": 0, "stratified": true, "stratification_reason": "string", "train_rows": [ 0 ], "test_rows": [ 0 ] }, "sampling": { "method": "none", "seed": 0, "n_input": 0, "n_missing_target": 0, "n_eligible": 0, "n_selected": 0, "max_samples_requested": 0, "effective_cap": 0 }, "class_counts": { "eligible": [ { "class": "string", "count": 0 } ], "sampled": [ { "class": "string", "count": 0 } ], "train": [ { "class": "string", "count": 0 } ], "test": [ { "class": "string", "count": 0 } ] }, "auc": { "variant": "string", "class_order": [ "string" ], "positive_class": "string", "available": true, "unavailable_reason": "string" } }, "evaluation": { "kind": "matched_outcomes", "since": "2019-08-24T14:15:22Z", "limit": 0, "fetched_count": 0, "readable_count": 0, "evaluated_count": 0, "limit_reached": true }, "training_classification_report": { "evaluated_count": 0, "decision_rule": { "kind": "argmax", "description": "string", "class_order": [ "string" ], "threshold": 0, "tie_break": "first_in_class_order" }, "per_class": [ { "class": "string", "support": 0, "predicted_count": 0, "true_positive": 0, "precision": 0, "recall": 0, "f1": 0, "undefined_reasons": { "precision": "no_predicted_samples", "recall": "no_actual_samples", "f1": "no_actual_or_predicted_samples" } } ] }, "training_calibration": { "available": true, "unavailable_reason": "no_evaluated_samples", "evaluated_count": 0, "evaluated_on": "test", "kind": "one_vs_rest", "interpretation": "descriptive_only", "binning": "uniform", "n_bins": 10, "bin_boundaries": "left_closed_right_open_last_closed", "per_class": [ { "class": "string", "bins": [ { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 }, { "lower": 0, "upper": 0, "count": 0, "positive_count": 0, "mean_predicted_probability": 0, "observed_frequency": 0 } ] } ] }, "observed": { "metric": "accuracy", "compared": 0, "value": 0, "baseline": 0, "better_than_baseline": true, "classification_report": { "evaluated_count": 0, "decision_rule": { "kind": "argmax", "description": "string", "class_order": [ "string" ], "threshold": 0, "tie_break": "first_in_class_order" }, "per_class": [ { "class": "string", "support": 0, "predicted_count": 0, "true_positive": 0, "precision": 0, "recall": 0, "f1": 0, "undefined_reasons": { "precision": "no_predicted_samples", "recall": "no_actual_samples", "f1": "no_actual_or_predicted_samples" } } ] }, "metrics": { "AUC": 0, "Accuracy": 0, "AveragePrecision": 0, "LogLoss": 0, "MacroF1": 0, "MAE": 0, "RMSE": 0, "R²": 0, "MASE": 0, "Bias": 0, "BiasRatio": 0, "Pinball": 0, "property1": 0, "property2": 0 }, "metric_context": { "undefined_reasons": { "property1": "string", "property2": "string" }, "class_order": [ "string" ], "positive_class": "string", "auc_average": "binary", "average_precision_method": "non_interpolated", "macro_f1_average": "unweighted_defined_classes", "log_loss_epsilon": 0 }, "baselines": { "property1": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" }, "property2": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" } }, "assessment": { "status": "better_than_baseline", "metric": "AUC", "message": "string" }, "by_engine": [ { "version": 0, "engine": "string", "compared": 0, "metrics": { "AUC": 0, "Accuracy": 0, "AveragePrecision": 0, "LogLoss": 0, "MacroF1": 0, "MAE": 0, "RMSE": 0, "R²": 0, "MASE": 0, "Bias": 0, "BiasRatio": 0, "Pinball": 0, "property1": 0, "property2": 0 }, "metric_context": { "undefined_reasons": { "property1": "string", "property2": "string" }, "class_order": [ "string" ], "positive_class": "string", "auc_average": "binary", "average_precision_method": "non_interpolated", "macro_f1_average": "unweighted_defined_classes", "log_loss_epsilon": 0 }, "baselines": { "property1": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" }, "property2": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" } }, "assessment": { "status": "better_than_baseline", "metric": "AUC", "message": "string" }, "classification_report": { "evaluated_count": 0, "decision_rule": { "kind": "argmax", "description": "string", "class_order": [ "string" ], "threshold": 0, "tie_break": "first_in_class_order" }, "per_class": [ { "class": "string", "support": 0, "predicted_count": 0, "true_positive": 0, "precision": 0, "recall": 0, "f1": 0, "undefined_reasons": { "precision": "no_predicted_samples", "recall": "no_actual_samples", "f1": "no_actual_or_predicted_samples" } } ] } } ], "by_version": [ { "version": 0, "compared": 0, "value": 0, "classification_report": { "evaluated_count": 0, "decision_rule": { "kind": "argmax", "description": "string", "class_order": [ "string" ], "threshold": 0, "tie_break": "first_in_class_order" }, "per_class": [ { "class": "string", "support": 0, "predicted_count": 0, "true_positive": 0, "precision": 0, "recall": 0, "f1": 0, "undefined_reasons": { "precision": "no_predicted_samples", "recall": "no_actual_samples", "f1": "no_actual_or_predicted_samples" } } ] }, "metrics": { "AUC": 0, "Accuracy": 0, "AveragePrecision": 0, "LogLoss": 0, "MacroF1": 0, "MAE": 0, "RMSE": 0, "R²": 0, "MASE": 0, "Bias": 0, "BiasRatio": 0, "Pinball": 0, "property1": 0, "property2": 0 }, "metric_context": { "undefined_reasons": { "property1": "string", "property2": "string" }, "class_order": [ "string" ], "positive_class": "string", "auc_average": "binary", "average_precision_method": "non_interpolated", "macro_f1_average": "unweighted_defined_classes", "log_loss_epsilon": 0 }, "baselines": { "property1": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" }, "property2": { "floor": 0, "class_order": [ "string" ], "probabilities": [ 0 ], "undefined_reason": "string", "relative_error_reduction": 0, "reduction_undefined_reason": "string", "label": "string", "higher_better": true, "fit_on": "train", "prediction": 0, "constant_floor": 0, "constant_label": "string" } }, "assessment": { "status": "better_than_baseline", "metric": "AUC", "message": "string" } } ] }}How is this guide?
Last updated on