Alex Tech Topic — Issue #4

Demystifying Anomaly Detection | Understand the Norm and Detect Abnormalities in Your Metrics

Alexandre Couëdelo
3 min readNov 28, 2022

In the last newsletter, I briefly covered the Alert threshold topic based on what I learnt from writing Static vs Dynamic thresholds.

I realise that the crux of the issue with using dynamic thresholds was the lack of expertise in anomaly detection. The Anomaly detection is not limited to Alerts but asks the question:

Given a set of data, how do I know if something is abnormal?

AI art generated with the subject as input

Anomaly Detection

Anomaly detection is the process of identifying metrics, events or observations that do not conform to an expected pattern or are unexpected in some way. In other words, it is the identification of irregular behaviour in a dataset.

Simply put, anomaly selection is a classic data science problem. My point is that Infrastructure engineers and SRE alike should be more aware of those data science problem-solving techniques.

There are two popular ways to detect anomalies:

  • Statistical Methods: Use historical data to identify normal patterns of behaviour.
  • Machine Learning Methods: Use algorithms to learn what constitutes normal behaviour for a…

--

--

Alexandre Couëdelo

Software Supply Chain and Automation Specialist (aka. DevOps).