An Improved Methodology for Outlier Detection in Dynamic Data Sets

A time series Kalman filter (TSKF) is proposed that successfully handles outlier detection in dynamic systems, where normal process changes often mask the existence of outliers. The TSKF method combines a time series model fitting procedure with a modified Kalman filter to deal with additive outlier (AO) and innovational outlier (IO) detection problems in dynamic process data set. Compared with current outlier detection methods, the new method enjoys the following advantages: (a) no prior knowledge of the process model is needed; (b) it has a high break-down point; (c) it is easy to tune; (d) it can be applied to both univariate and multivariate outlier detection; (e) it is applicable to both on-line and off-line operation; (f) it cleans outliers while maintains the integrity of the original data set.