Adaptive forecaster for despiking of time-series data. More...

#include <AdaptiveDespiker.h>

Public Member Functions
	AdaptiveDespiker ()
	Constructor. More...

void	setOutlierProbability (float val)

float	getOutlierProbability () const

void	setDiscLevelMultiplier (float val)

float	getDiscLevelMultiplier () const

float	getDiscLevel () const
	If a value is more than discrLevel * sigma away from the mean, then it is considered a spike, i.e. More...

float	despike (float u, bool *spike)
	Pass a value u, and return a forecasted value, along with a boolean indicating whether AdaptiveDespiker considers it a spike. More...

float	forecast () const
	Forecast a value based on the previous point and correlation and mean. More...

void	reset ()
	Reset the statistics. More...

Static Public Member Functions
static float	discrLevel (float prob)
	Compute a discrimination level from a Gaussan probability. More...

static float	adjustLevel (float corr)
	Adjust the discrimination level based on the current correlation. More...

Private Member Functions
void	initStatistics (float u)

void	incrementStatistics (float u)

void	updateStatistics (float u)

Static Private Member Functions
static bool	staticInit ()

static void	spline (float x, float y, int n, double yp1, double ypn, float *y2)
	Copied from "Numerical Recipies in C", the only changes being arrays are indexed from 0, due to personal taste, and doubles are passed instead of floats. More...

static double	splint (float xa, float ya, float *y2a, int n, double x) throw (std::range_error)

Private Attributes
float	_prob
	Prob for detection of outlier in good Gaussian data. More...

float	_levelMultiplier
	Increase detection level with this factor. More...

float	_maxMissingFreq
	If more than _maxMissingFreq of the recent 10 data points are missing data points, don't substitute forecasted data or update forecast statistics. More...

float	_u1
	Last point. More...

double	_mean1
	Previous mean. More...

double	_mean2
	Current mean. More...

double	_var1
	Previous variance. More...

double	_var2
	Current variance. More...

double	_corr
	Correlation. More...

double	_initLevel
	Discrimination level. More...

double	_level

float	_missfreq
	Running ave of freq of missing data. More...

int	_msize
	Memory size. More...

size_t	_npts
	Number of points processed. More...

Static Private Attributes
static const size_t	STATISTICS_SIZE = 100

static const int	ADJUST_TABLE_SIZE = 100

static const int	LEN_ERFC_ARRAY = 140

static float	_adj [ADJUST_TABLE_SIZE][2]

static bool	staticInitDone = AdaptiveDespiker::staticInit()

Detailed Description

Adaptive forecaster for despiking of time-series data.

Algorithim taken from "A Statistical Data Screening Procedure" by Jorgen Hojstrup, Dept of Meteorology and Wind Energy, RISO National Laboratory, Denmark. [Published as: Meas. Sci. Technol (UK). v4 (1993) p. 153-157]

Constructor & Destructor Documentation

AdaptiveDespiker::AdaptiveDespiker ( )

Constructor.

References setDiscLevelMultiplier(), and setOutlierProbability().

Member Function Documentation

float AdaptiveDespiker::adjustLevel ( float corr )

static

Adjust the discrimination level based on the current correlation.

References _adj, and ADJUST_TABLE_SIZE.

Referenced by incrementStatistics(), and updateStatistics().

float AdaptiveDespiker::despike	(	float	u,
		bool *	spike
	)

Pass a value u, and return a forecasted value, along with a boolean indicating whether AdaptiveDespiker considers it a spike.

The input value is added to the AdaptiveDespiker statistics for forecasting.

References _level, _maxMissingFreq, _missfreq, _npts, _var2, forecast(), incrementStatistics(), initStatistics(), STATISTICS_SIZE, and updateStatistics().

float AdaptiveDespiker::discrLevel ( float prob )

static

Compute a discrimination level from a Gaussan probability.

If a value is more than discrLevel * sigma away from the mean, then it is considered a spike, i.e. it's probability exceeded the given value.

References LEN_ERFC_ARRAY.

Referenced by setDiscLevelMultiplier(), and setOutlierProbability().

float nidas::core::AdaptiveDespiker::forecast ( ) const

inline

Forecast a value based on the previous point and correlation and mean.

References _corr, _mean2, and _u1.

Referenced by despike().

float nidas::core::AdaptiveDespiker::getDiscLevel ( ) const

inline

If a value is more than discrLevel * sigma away from the mean, then it is considered a spike, i.e.

it's probability exceeded getOutlierProbability().

References _level.

Referenced by nidas::dynld::isff::Wind3D::getDiscLevel().

float nidas::core::AdaptiveDespiker::getDiscLevelMultiplier ( ) const

inline

References _levelMultiplier.

Referenced by nidas::dynld::isff::Wind3D::getDiscLevelMultiplier().

float nidas::core::AdaptiveDespiker::getOutlierProbability ( ) const

inline

References _prob.

Referenced by nidas::dynld::isff::Wind3D::getOutlierProbability().

void AdaptiveDespiker::incrementStatistics ( float u )

private

References _corr, _initLevel, _level, _mean1, _mean2, _missfreq, _msize, _npts, _u1, _var1, _var2, adjustLevel(), and STATISTICS_SIZE.

Referenced by despike().

void AdaptiveDespiker::initStatistics ( float u )

private

References _corr, _mean1, _mean2, _missfreq, _npts, _u1, _var1, and _var2.

Referenced by despike().

void AdaptiveDespiker::reset ( )

Reset the statistics.

References _initLevel, _level, _missfreq, and _npts.

void AdaptiveDespiker::setDiscLevelMultiplier ( float val )

Parameters

val	Discrimination level fudge factor. The computation of discrimination level from the Gaussian probability is typically too small and one can multiply the resultant level by this factor. We've typically used 2.5 in practice.

References _initLevel, _level, _levelMultiplier, _prob, and discrLevel().

Referenced by AdaptiveDespiker().

void AdaptiveDespiker::setOutlierProbability ( float val )

Parameters

val	Probability of an outlier. This is converted to a discrimination level based on a Gaussian distribution.

See Also

References _initLevel, _level, _levelMultiplier, _prob, and discrLevel().

Referenced by AdaptiveDespiker().

void AdaptiveDespiker::spline	(	float *	x,
		float *	y,
		int	n,
		double	yp1,
		double	ypn,
		float *	y2
	)

staticprivate

Copied from "Numerical Recipies in C", the only changes being arrays are indexed from 0, due to personal taste, and doubles are passed instead of floats.

Given arrays a[0..n-1] and y[0..n-1] containing a tabulated function, i.e. yi = f(xi), with x1 < x2 < ... < xn-1 and given values yp1 and ypn for the first derivative of the interpolating function at points 1 and n-1 respectively, this routine returns an array y2[0..n-1] that contains the second derivatives of the interpolatimng function at the tabulated points xi. If yp1 and/or ypn are equal to 1.e30 or larger, the routine is signalled to set the corresponding boundary condition for a natural spline, with zero second derivative at that boundary.