A simple, nearest-point resampler, for generating merged samples from variables from one or more sample sources. More...

#include <NearestResamplerAtRate.h>

Inheritance diagram for nidas::core::NearestResamplerAtRate:

[legend]

Public Member Functions
	NearestResamplerAtRate (const std::vector< const Variable * > &vars, bool nansVariable=true)
	Constructor. More...

	NearestResamplerAtRate (const std::vector< Variable * > &vars, bool nansVariable=true)

	~NearestResamplerAtRate ()

void	setRate (double val)
	Set the requested output rate, in Hz. More...

double	getRate () const

void	setMiddleTimeTags (bool val)
	If true, generate output timetags that are the middle of the requested output periods. More...

bool	getMiddleTimeTags () const

void	setFillGaps (bool val)
	Should output records of all missing data (nans), be generated, or just discarded. More...

bool	getFillGaps () const

SampleSource *	getRawSampleSource ()
	Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples. More...

SampleSource *	getProcessedSampleSource ()
	Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples. More...

std::list< const SampleTag * >	getSampleTags () const
	Get the SampleTag of my merged output sample. More...

SampleTagIterator	getSampleTagIterator () const
	Implementation of SampleSource::getSampleTagIterator(). More...

void	addSampleClient (SampleClient *client) throw ()
	Implementation of SampleSource::addSampleClient(). More...

void	removeSampleClient (SampleClient *client) throw ()
	Remove a SampleClient from this SampleSource. More...

void	addSampleClientForTag (SampleClient client, const SampleTag ) throw ()
	Add a Client for a given SampleTag. More...

void	removeSampleClientForTag (SampleClient client, const SampleTag ) throw ()
	Remove a SampleClient for a given SampleTag from this SampleSource. More...

int	getClientCount () const throw ()
	How many SampleClients are currently in my list. More...

void	flush () throw ()
	Implementation of Resampler::flush(). More...

const SampleStats &	getSampleStats () const

void	connect (SampleSource *input) throw (nidas::util::InvalidParameterException)
	Connect the resampler to a SampleSource. More...

void	disconnect (SampleSource *input) throw ()

bool	receive (const Sample *s) throw ()
	Method called to pass a sample to this client. More...

Private Member Functions
void	ctorCommon (const std::vector< const Variable * > &vars, bool nansVariable)
	Common tasks of constructors. More...

void	addSampleTag (const SampleTag *tag) throw ()
	Add a SampleTag to this SampleSource. More...

void	removeSampleTag (const SampleTag *tag) throw ()

void	sendSample (dsm_time_t) throw ()

NearestResamplerAtRate &	operator= (const NearestResamplerAtRate &)
	No assignment. More...

	NearestResamplerAtRate (const NearestResamplerAtRate &x)
	No copy. More...

Private Attributes
SampleSourceSupport	_source

SampleTag	_outSample

std::vector< Variable * >	_reqVars
	Requested variables. More...

std::map< Variable *, unsigned int >	_outVarIndices
	Index of each requested output variable in the output sample. More...

std::map< dsm_sample_id_t, std::vector< unsigned int > >	_inmap
	For each input sample, first index of variable data values to be read. More...

std::map< dsm_sample_id_t, std::vector< unsigned int > >	_lenmap
	For each input sample, length of variables to read. More...

std::map< dsm_sample_id_t, std::vector< unsigned int > >	_outmap
	For each input sample, index into output sample of each variable. More...

int	_ndataValues

int	_outlen

double	_rate

int	_deltatUsec
	The output deltaT, 1/rate in microseconds. More...

int	_deltatUsecD10
	DeltaT over 10. More...

int	_deltatUsecD2
	DeltaT over 2. More...

bool	_exactDeltatUsec
	True if simple integer math is used to increment output sample time tags. More...

bool	_middleTimeTags
	If true, generate output timetags that are the middle of the requested output periods. More...

dsm_time_t	_outputTT

dsm_time_t	_nextOutputTT

dsm_time_t *	_prevTT

dsm_time_t *	_nearTT

float *	_prevData

float *	_nearData

int *	_samplesSinceOutput

SampleT< float > *	_osamp

bool	_fillGaps

std::map< dsm_sample_id_t, unsigned int >	_ttOutOfOrder

Detailed Description

A simple, nearest-point resampler, for generating merged samples from variables from one or more sample sources.

The output sample time tags will be evenly spaced at the given rate, and values of variables are merged into the output sample by associating those values with the nearest time tag to the output time tags.

The only requirement is that the samples which are fed to the receive() method should be sorted in time. It they aren't sorted some data will be lost. NearestResamplerAtRate does not need to know sampling rates, and the sampling rates of the input variables, including the master variable, may vary.

This resampler operates in two modes, based on the value of the setMiddleTimeTags() attribute.

If getMiddleTimeTags() is true, generate output timetags that are at the middle of the requested output periods. For example, for a rate=20, deltaT=0.05 sec, the output timetags will be 00:00:00.025, 00:00:00.075, etc. The output sample at 00:00:00.025 contains the nearest input values from the time period centered at 00:00:00.025.

If getMiddleTimeTags() is false, the output time tags for the above example would be 00:00:00.00, 00:00:00.05, etc. The output sample at 00:00:00.00 contains the nearest input values from the time period centered at 00:00:00.00.

Because of assumed input timetag jitter (inaccuracy), the nearest point matching algorithm is a bit forgiving. When matching for the nearest points to time t, input samples will be matched whose time tags are between t - 0.9*deltatT <= inputTimeTag <= t + 0.9*deltaT. If more than one input sample lies in the window, then the nearest one is used. This input window is of size 1.8 * deltaT, rather than 1.0 * deltaT, which is what one might expect. Therefore an input point could be matched with two output points. In this example, it is possible that an output sample at 00:00:00.025 could contain an input value from the previous day.

Constructor & Destructor Documentation

NearestResamplerAtRate::NearestResamplerAtRate	(	const std::vector< const Variable * > &	vars,
		bool	nansVariable = `true`
	)

Constructor.

References ctorCommon().

NearestResamplerAtRate::NearestResamplerAtRate	(	const std::vector< Variable * > &	vars,
		bool	nansVariable = `true`
	)

References ctorCommon().

NearestResamplerAtRate::~NearestResamplerAtRate ( )

References _nearData, _nearTT, _osamp, _prevData, _prevTT, _reqVars, _samplesSinceOutput, and nidas::core::SampleT< DataT >::freeReference().

nidas::core::NearestResamplerAtRate::NearestResamplerAtRate ( const NearestResamplerAtRate & x )

private

No copy.

Member Function Documentation

void nidas::core::NearestResamplerAtRate::addSampleClient	(	SampleClient *	client	)
throw	(
	)

inlinevirtual

Implementation of SampleSource::addSampleClient().

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::addSampleClient().

void nidas::core::NearestResamplerAtRate::addSampleClientForTag	(	SampleClient *	client,
		const SampleTag *
	)
throw	(
	)

inlinevirtual

Add a Client for a given SampleTag.

Implementation of SampleSource::addSampleClient().

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::addSampleClient().

void nidas::core::NearestResamplerAtRate::addSampleTag	(	const SampleTag *	tag	)
throw	(
	)

inlineprivatevirtual

Add a SampleTag to this SampleSource.

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::addSampleTag().

Referenced by ctorCommon().

void NearestResamplerAtRate::connect	(	SampleSource *	input	)
throw	(	nidas::util::InvalidParameterException
	)

virtual

Connect the resampler to a SampleSource.

Implements nidas::core::Resampler.

References GET_DSM_ID, GET_SPS_ID, nidas::core::SampleTag::getDataIndex(), nidas::core::SampleTag::getDSMId(), nidas::core::SampleTag::getId(), nidas::core::Variable::getLength(), nidas::core::Variable::getName(), nidas::core::Variable::getSampleTag(), nidas::core::SampleTag::getSpSId(), nidas::core::SampleTag::getVariableIterator(), nidas::core::VariableIterator::hasNext(), nidas::core::VariableIterator::next(), and WLOG.

void NearestResamplerAtRate::ctorCommon	(	const std::vector< const Variable * > &	vars,
		bool	nansVariable
	)

private

Common tasks of constructors.

References _ndataValues, _nearData, _nearTT, _nextOutputTT, _outlen, _outputTT, _outSample, _outVarIndices, _prevData, _prevTT, _reqVars, _samplesSinceOutput, addSampleTag(), nidas::core::SampleTag::addVariable(), nidas::core::floatNAN, GET_DSM_ID, GET_SPS_ID, nidas::core::SampleTag::getDSMId(), nidas::core::SampleTag::getId(), nidas::core::Project::getInstance(), nidas::core::Variable::getLength(), nidas::core::Variable::getName(), nidas::core::Variable::getSampleTag(), nidas::core::SampleTag::getSpSId(), nidas::core::Project::getUniqueSampleId(), nidas::core::SampleTag::setDSMId(), nidas::core::Variable::setName(), setRate(), nidas::core::SampleTag::setSampleId(), nidas::core::Variable::setType(), nidas::core::Variable::setUnits(), and nidas::core::Variable::WEIGHT.

Referenced by NearestResamplerAtRate().

void NearestResamplerAtRate::disconnect	(	SampleSource *	input	)
throw	(
	)

virtual

Implements nidas::core::Resampler.

void NearestResamplerAtRate::flush	(	)
throw	(
	)

virtual

Implementation of Resampler::flush().

Implements nidas::core::Resampler.

References _nextOutputTT, and sendSample().

int nidas::core::NearestResamplerAtRate::getClientCount	(	)	const
throw	(
	)

inlinevirtual

How many SampleClients are currently in my list.

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::getClientCount().

bool nidas::core::NearestResamplerAtRate::getFillGaps ( ) const

inline

References _fillGaps.

bool nidas::core::NearestResamplerAtRate::getMiddleTimeTags ( ) const

inline

References _middleTimeTags.

SampleSource* nidas::core::NearestResamplerAtRate::getProcessedSampleSource ( )

inlinevirtual

Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples.

SampleClients use this method to get a pointer to whatever sample source they are interested in. Derived classes can return NULL if they are not a SampleSource of processed samples.

Implements nidas::core::SampleSource.

References _source.

double nidas::core::NearestResamplerAtRate::getRate ( ) const

inline

References _rate.

SampleSource* nidas::core::NearestResamplerAtRate::getRawSampleSource ( )

inlinevirtual

Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples.

SampleClients use this method to get a pointer to whatever sample source they are interested in. Derived classes can return NULL if they are not a SampleSource of raw samples.

Implements nidas::core::SampleSource.

const SampleStats& nidas::core::NearestResamplerAtRate::getSampleStats ( ) const

inlinevirtual

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::getSampleStats().

SampleTagIterator nidas::core::NearestResamplerAtRate::getSampleTagIterator ( ) const

inlinevirtual

Implementation of SampleSource::getSampleTagIterator().

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::getSampleTagIterator().

std::list<const SampleTag*> nidas::core::NearestResamplerAtRate::getSampleTags ( ) const

inlinevirtual

Get the SampleTag of my merged output sample.

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::getSampleTags().

NearestResamplerAtRate& nidas::core::NearestResamplerAtRate::operator= ( const NearestResamplerAtRate & )

private

No assignment.

bool NearestResamplerAtRate::receive	(	const Sample *	s	)
throw	(
	)

virtual

Method called to pass a sample to this client.

This method is typically called by a SampleSource for each of its SampleClients when it has a sample ready. Returns true: success false: sample rejected. This is meant to signal a warning-type situation - like a socket not being available temporarily. True errors will be thrown as an IOException.

Implements nidas::core::SampleClient.

References nidas::core::DOUBLE_ST, nidas::core::FLOAT_ST, nidas::util::UTime::format(), GET_DSM_ID, GET_SHORT_ID, GET_SPS_ID, USECS_PER_SEC, and WLOG.

void nidas::core::NearestResamplerAtRate::removeSampleClient	(	SampleClient *	c	)
throw	(
	)

inlinevirtual

Remove a SampleClient from this SampleSource.

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::removeSampleClient().

void nidas::core::NearestResamplerAtRate::removeSampleClientForTag	(	SampleClient *	c,
		const SampleTag *
	)
throw	(
	)

inlinevirtual

Remove a SampleClient for a given SampleTag from this SampleSource.

The pointer to the SampleClient must remain valid, until after it is removed.

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::removeSampleClient().

void nidas::core::NearestResamplerAtRate::removeSampleTag	(	const SampleTag *	tag	)
throw	(
	)

inlineprivatevirtual

Implements nidas::core::SampleSource.

References _source, and nidas::core::SampleSourceSupport::removeSampleTag().

void NearestResamplerAtRate::sendSample	(	dsm_time_t	tt	)
throw	(
	)

private

References nidas::core::floatNAN, and USECS_PER_SEC.

Referenced by flush().

void nidas::core::NearestResamplerAtRate::setFillGaps ( bool val )

inline

Should output records of all missing data (nans), be generated, or just discarded.

References _fillGaps.

Referenced by DataPrep::run().

void nidas::core::NearestResamplerAtRate::setMiddleTimeTags ( bool val )

inline

If true, generate output timetags that are the middle of the requested output periods.

For example, for a rate=20, deltaT=0.05 sec, the output timetags will be 00:00:00.025, 00:00:00.075, etc. The sample at 00:00:00.025 contains the nearest input values from the time period centered at 00:00:00.025 If false, the output time tags for the above example would be 00:00:00.00, 00:00:00.05, etc. The sample at 00:00:00.00 contains the nearest input values from the time period centerd at 00:00:00.00, i.e. points from the previous day could be used.

References _middleTimeTags.

Referenced by DataPrep::run().

void NearestResamplerAtRate::setRate ( double val )

Set the requested output rate, in Hz.

For rates < 1 it is best to choose a value such that 10^6/rate is an integer. If you really want rate=1/3 Hz, specify rate to 7 significant figures, 0.3333333, and you will avoid round off errors in the time tag. Output rates > 1 should be integers, or of a value with enough significant figures such that 10^6/rate is an integer. Support for other rates could be added if (really) necessary.

References _deltatUsec, _deltatUsecD10, _deltatUsecD2, _exactDeltatUsec, _rate, and USECS_PER_SEC.

Referenced by ctorCommon(), and DataPrep::run().

Member Data Documentation

int nidas::core::NearestResamplerAtRate::_deltatUsec

private

The output deltaT, 1/rate in microseconds.

Referenced by setRate().

int nidas::core::NearestResamplerAtRate::_deltatUsecD10

private

DeltaT over 10.

A fudge factur used for doing nearest point alignments.

Referenced by setRate().

int nidas::core::NearestResamplerAtRate::_deltatUsecD2

private

DeltaT over 2.

Referenced by setRate().

bool nidas::core::NearestResamplerAtRate::_exactDeltatUsec

private

True if simple integer math is used to increment output sample time tags.

This will be the case if rate <= 1.0 or 1/rate is within 0.02 microseconds if an integer.

Referenced by setRate().

bool nidas::core::NearestResamplerAtRate::_fillGaps

private

Referenced by getFillGaps(), and setFillGaps().

std::map<dsm_sample_id_t,std::vector<unsigned int> > nidas::core::NearestResamplerAtRate::_inmap

private

For each input sample, first index of variable data values to be read.

std::map<dsm_sample_id_t,std::vector<unsigned int> > nidas::core::NearestResamplerAtRate::_lenmap

private

For each input sample, length of variables to read.

bool nidas::core::NearestResamplerAtRate::_middleTimeTags

private

If true, generate output timetags that are the middle of the requested output periods.

For example, for a rate=20, deltaT=0.05 sec, the output timetags will be 00:00:00.025, 00:00:00.075, etc. The sample at 00:00:00.025 contains the nearest input values from the period 00:00:00.0 to 00:00:00.05 If false, the output time tags for the above example would be 00:00:00.00, 00:00:00.05, etc. The sample at 00:00:00.00 contains the nearest input values from 23:59:59.975 to 00:00:00.025.

Referenced by getMiddleTimeTags(), and setMiddleTimeTags().

int nidas::core::NearestResamplerAtRate::_ndataValues

private

Referenced by ctorCommon().

float* nidas::core::NearestResamplerAtRate::_nearData

private

Referenced by ctorCommon(), and ~NearestResamplerAtRate().

dsm_time_t* nidas::core::NearestResamplerAtRate::_nearTT

private

Referenced by ctorCommon(), and ~NearestResamplerAtRate().

dsm_time_t nidas::core::NearestResamplerAtRate::_nextOutputTT

private