Anomaly detection

Score how unusual new data is with an autoencoder: reconstruction error and a learned threshold.

2 nodes. Right-click any node in the editor to read this documentation in the app.

Anomaly Detection

🚨 Novelty Threshold

id anomaly_novelty_threshold · Anomaly Detection · Python export: yes

Learn what 'normal' error looks like, then flag what exceeds it. The threshold comes from the errors on healthy data: the maximum, a percentile (default 99), or mean + k (sample) standard deviations. Rows of the new errors above it are flagged is_anomaly. With no new errors wired in, the healthy errors themselves are scored. Colour a Scatter Plot by is_anomaly to see the flagged rows.

Inputs

Port	Type	Description
train_errors	DataFrame | ndarray	Reconstruction errors on healthy data; sets the threshold.
new_errors	DataFrame | ndarray	Errors to score against the threshold (optional: blank scores the healthy errors).

Outputs

Port	Type	Description
flags	DataFrame	Columns: window, error, threshold, is_anomaly.

Fields

Field	Type	Default	Choices
Threshold method	select	percentile	percentile, max, mean_k_std
Percentile (percentile method)	float	99.0
k (mean_k_std method)	float	3.0

🔍 Reconstruction Error

id anomaly_reconstruction_error · Anomaly Detection · Python export: yes

How badly an autoencoder rebuilt each row: the mean squared (or absolute) difference between a row and its reconstruction. Rows the model saw during training reconstruct well; unfamiliar rows reconstruct badly, which makes the error a novelty score. Wire in the original data and Run Model's output.

Inputs

Port	Type	Description
original	ndarray | DataFrame | Tensor	The data that went into the model.
reconstruction	ndarray | DataFrame | Tensor	The model's output for it (same shape as original).

Outputs

Port	Type	Description
errors	DataFrame	One column, 'error': the reconstruction error of each row.

Fields

Field	Type	Default	Choices
Error measure	select	mse	mse, mae