nidas v1.2.3
|
An implementation of a SampleInput. More...
#include <SampleInputStream.h>
Classes | |
struct | ReadResult |
Tuple for all the possible results of iostream reads. More... | |
Public Types | |
typedef nidas::core::dsm_time_t | dsm_time_t |
Static Public Member Functions | |
static const XMLCh * | getNamespaceURI () |
Protected Member Functions | |
SampleInputStream (SampleInputStream &x, nidas::core::IOChannel *iochannel) | |
Copy constructor, with a new, connected IOChannel. | |
Protected Attributes | |
nidas::core::IOChannel * | _iochan |
nidas::core::SampleSourceSupport | _source |
Private Member Functions | |
nidas::core::Sample * | nextSample () |
Unpack the next sample from the InputStream. | |
nidas::core::Sample * | nextSample (bool keepreading, bool searching=false, dsm_time_t search_time=LONG_LONG_MIN) |
Unpack the next sample from the InputStream buffer or by reading more data if keepreading is true. | |
void | checkUnexpectedEOF () |
ReadResult | readBlock (bool keepreading, char *&ptr, size_t &lentoread) |
Read a block into memory, updating the given block pointer and length counter accordingly. | |
ReadResult | read (bool keepreading, char *ptr, size_t lentoread) |
All reads of the iostream go through here. | |
void | handleNewInput () |
Whenever we are at the start of a new input, we need to start over with parsing the input header. | |
nidas::core::Sample * | handleEOF (bool keepreading) |
When EOF has been caught, then either we need to return the last pending sample, or else we need to throw the exception. | |
void | closeBlocks () |
nidas::core::Sample * | sampleFromHeader () throw () |
Check the current header for validity and generate a sample for it. | |
SampleInputStream (const SampleInputStream &x) | |
No regular copy. | |
SampleInputStream & | operator= (const SampleInputStream &x) |
No assignment. | |
Private Attributes | |
nidas::core::DSMService * | _service |
Service that has requested my input. | |
nidas::core::IOStream * | _iostream |
const nidas::core::DSMConfig * | _dsm |
bool | _expectHeader |
bool | _inputHeaderParsed |
nidas::core::SampleHeader | _sheader |
size_t | _headerToRead |
char * | _hptr |
nidas::core::Sample * | _samp |
Will be non-null if we have previously read part of a sample from the stream. | |
nidas::core::Sample * | _sampPending |
The currently pending sample. | |
size_t | _dataToRead |
How many bytes left to read from the stream into the data portion of samp. | |
char * | _dptr |
Pointer into the data portion of samp where we will read next. | |
bool | _skipSample |
This is set if the data for the current sample header should be read but skipped, because the sample is the first after a block of bad samples and thus the time at the front of the sample might still be corrupt. | |
BlockStats | _block |
Information about the current block of samples, good or bad. | |
size_t | _badSamples |
Number of bad samples in the stream so far, which is to say number of bytes checked which did not contain a reasonable sample header. | |
size_t | _goodSamples |
Number of good samples in the stream so far. | |
nidas::core::SampleInputHeader | _inputHeader |
nidas::core::BadSampleFilter | _bsf |
SampleInputStream * | _original |
bool | _raw |
std::string | _last_name |
Keep the current name of the input stream so it can be referenced even after the input stream has advanced to a new input name. | |
nidas::util::EOFException | _eofx |
bool | _ateof |
Static Private Attributes | |
static XMLCh * | namespaceURI = 0 |
An implementation of a SampleInput.
The readSamples method converts raw bytes from the iochannel into Samples.
If a SampleClient has requested processed Samples via addProcessedSampleClient, then SampleInputStream will pass Samples to the respective DSMSensor for processing, and then the DSMSensor passes the processed Samples to the SampleClient:
iochannel -> readSamples method -> DSMSensor -> SampleClient
If a SampleClient has requested non-processed Samples, via the simple addSampleClient method, then SampleInput stream passes the Samples straight to the SampleClient:
iochannel -> readSamples method -> SampleClient
SampleInputStream::SampleInputStream | ( | nidas::core::IOChannel * | iochannel, |
bool | raw = false ) |
Constructor.
iochannel | The IOChannel that we use for data input. SampleInputStream will own the pointer to the IOChannel, and will delete it in ~SampleInputStream(). If it is a null pointer, then it must be set within the fromDOMElement method. |
References _iochan, _iostream, nidas::core::IOChannel::getBufferSize(), and setIOChannel().
|
virtual |
References _iochan, _samp, _sampPending, close(), and nidas::core::Sample::freeReference().
|
protected |
Copy constructor, with a new, connected IOChannel.
References _iochan, _iostream, nidas::core::IOChannel::getBufferSize(), and setIOChannel().
|
private |
No regular copy.
|
inlinevirtual |
Implementation of SampleSource::addSampleClient().
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::addSampleClient().
|
inlinevirtual |
Add a Client for a given SampleTag.
Implementation of SampleSource::addSampleClient().
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::addSampleClientForTag().
|
inlinevirtual |
Implementation of SampleInput::addSampleTag().
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::addSampleTag().
|
private |
References _dataToRead, _headerToRead, _last_name, _samp, _sheader, CNAME, nidas::core::Sample::getDataByteLength(), nidas::core::SampleHeader::getSizeOf(), and WLOG.
Referenced by read().
|
virtual |
Create a clone, with a new, connected IOChannel.
Reimplemented in nidas::dynld::RawSampleInputStream.
References SampleInputStream().
|
virtual |
nidas::util::IOException |
Implements nidas::core::SampleInput.
References _iochan, _iostream, nidas::core::IOChannel::close(), and closeBlocks().
Referenced by NidsMerge::run(), nidas::dynld::raf::SyncServer::stop(), and ~SampleInputStream().
|
private |
References _badSamples, _block, _bsf, _goodSamples, _last_name, nidas::dynld::BlockStats::block_start, CNAME, DLOG, nidas::core::BadSampleFilter::filterBadSamples(), nidas::core::getSample(), nidas::util::LogContextState::log(), LOG_DEBUG, LOG_WARNING, and nidas::dynld::BlockStats::nbytes.
Referenced by close(), and handleNewInput().
|
virtual |
Implementation of IOChannelRequester::connected.
One can use this method to notify SampleInputStream that the IOChannel is connected, which will cause SampleInputStream to open the IOStream.
Implements nidas::core::IOChannelRequester.
References nidas::core::getSample().
|
inline |
Distribute a sample to my clients.
One could use this to insert a sample into the stream.
References _source, and nidas::core::SampleSourceSupport::distribute().
Referenced by nidas::dynld::raf::SyncServer::handleSample().
|
virtual |
Implementation of SampleSource::flush(), unpacks and distributes any samples currently in the read buffer.
Implements nidas::core::SampleSource.
References _source, CNAME, nidas::core::SampleSourceSupport::distribute(), nidas::core::SampleSourceSupport::getClientCount(), nextSample(), and VLOG.
nidas::util::InvalidParameterException |
Implements nidas::core::DOMable.
References _iochan, nidas::core::IOChannel::createIOChannel(), nidas::core::getSample(), setIOChannel(), and WLOG.
Referenced by nidas::dynld::RawSampleInputStream::fromDOMElement().
|
inline |
References _badSamples.
|
inlinevirtual |
How many SampleClients are currently in my list.
)
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getClientCount().
What DSM am I connnected to? May be NULL if it cannot be determined.
Implements nidas::core::SampleInput.
References _dsm.
|
inline |
References _expectHeader.
|
virtual |
Implements nidas::core::SampleInput.
References _iochan, and nidas::core::IOChannel::getFd().
|
inline |
References _inputHeader.
Referenced by nidas::dynld::raf::SyncRecordReader::init().
|
virtual |
Implements nidas::core::SampleInput.
References _iochan, and nidas::core::IOChannel::getName().
Referenced by parseInputHeader(), readInputHeader(), and sampleFromHeader().
References nidas::core::DOMable::namespaceURI.
Referenced by nidas::dynld::UDPSampleOutput::getProjectDOM(), nidas::core::Dataset::toDOMElement(), nidas::core::ProjectConfig::toDOMElement(), nidas::core::Dataset::toDOMParent(), nidas::core::Datasets::toDOMParent(), nidas::core::ProjectConfig::toDOMParent(), nidas::core::ProjectConfigs::toDOMParent(), nidas::core::DSMConfig::toDOMParent(), nidas::core::DSMSensor::toDOMParent(), nidas::core::Project::toDOMParent(), nidas::core::SampleTag::toDOMParent(), nidas::core::Site::toDOMParent(), nidas::core::ServerSocket::toDOMParent(), nidas::core::Variable::toDOMParent(), nidas::core::Datasets::writeXML(), and nidas::core::ProjectConfigs::writeXML().
|
inlinevirtual |
Implements nidas::core::SampleInput.
References _original.
|
inlinevirtual |
Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples.
SampleClients use this method to get a pointer to whatever sample source they are interested in. Derived classes can return NULL if they are not a SampleSource of processed samples.
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getProcessedSampleSource().
|
inlinevirtual |
Several objects in NIDAS can be both a SampleSource of raw Samples and processed Samples.
SampleClients use this method to get a pointer to whatever sample source they are interested in. Derived classes can return NULL if they are not a SampleSource of raw samples.
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getRawSampleSource().
|
inlinevirtual |
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getSampleStats().
|
inlinevirtual |
Implementation of SampleSource::getSampleTagIterator().
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getSampleTagIterator().
|
inlinevirtual |
Get the output SampleTags.
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::getSampleTags().
When EOF has been caught, then either we need to return the last pending sample, or else we need to throw the exception.
HOWEVER, because nextSample() is not allowed to throw an IOException when keepreading is false, since in general it does not make sense to trigger EOF when nothing is being read, this handler only throws the exception when keepreading is true.
References _eofx, _sampPending, CNAME, DLOG, and nidas::core::getSample().
Referenced by nextSample(), and readSamples().
|
private |
Whenever we are at the start of a new input, we need to start over with parsing the input header.
This method sets up the right state to start over on a new file.
References _inputHeaderParsed, _iostream, nidas::core::IOStream::backup(), closeBlocks(), CNAME, DLOG, nidas::core::IOStream::getNumInputBytes(), and VLOG.
Referenced by read().
|
virtual |
nidas::util::IOException |
Implements nidas::core::SampleInput.
References _iochan, and nidas::core::IOChannel::isNonBlocking().
|
private |
Unpack the next sample from the InputStream.
This method does not perform any physical reads, so it should not throw EOFException or IOException.
References nextSample().
Referenced by flush(), nextSample(), readSample(), readSamples(), and search().
|
private |
Unpack the next sample from the InputStream buffer or by reading more data if keepreading
is true.
If searching
is set, then reading stops after the first sample header found whose time tag is after search_time
, as described in the search() method.
nidas::util::IOException |
References _ateof, _bsf, _dataToRead, _dptr, _headerToRead, _hptr, _inputHeaderParsed, _samp, _sampPending, _sheader, _skipSample, CNAME, DLOG, nidas::core::BadSampleFilter::filterBadSamples(), nidas::util::UTime::format(), nidas::core::Sample::freeReference(), nidas::core::SampleHeader::getDataByteLength(), nidas::core::Sample::getDataByteLength(), nidas::core::SampleHeader::getRawId(), nidas::core::getSample(), nidas::core::SampleHeader::getSizeOf(), nidas::core::SampleHeader::getTimeTag(), nidas::core::Sample::getTimeTag(), nidas::core::Sample::getVoidDataPtr(), handleEOF(), readBlock(), readInputHeader(), sampleFromHeader(), nidas::core::SampleHeader::setDataByteLength(), nidas::core::SampleHeader::setRawId(), nidas::core::SampleHeader::setTimeTag(), VLOG, and WLOG.
|
private |
No assignment.
bool SampleInputStream::parseInputHeader | ( | ) |
nidas::util::IOException |
References _bsf, _dataToRead, _expectHeader, _headerToRead, _hptr, _inputHeader, _inputHeaderParsed, _iostream, _samp, _sheader, nidas::core::IOStream::backup(), CNAME, DLOG, nidas::core::Sample::freeReference(), getName(), nidas::core::IOStream::getNumInputBytes(), nidas::core::SampleHeader::getSizeOf(), nidas::core::SampleInputHeader::parse(), PLOG, and nidas::core::BadSampleFilter::skipNidasHeader().
Referenced by readInputHeader().
|
private |
All reads of the iostream go through here.
There are three kinds of reads needed:
In all these cases, we need to catch if we hit the end of a file to know if the file ended unexpectedly. Also, if we hit the end of a file and a sample is still pending (because filtering is enabled and a sample is not good until it passes the filter and is succeeded by eof or another good sample), then we also need to return that sample. This method records the eof exception but does not pass it on, waiting instead for it to be thrown after any pending sample has been returned by nextSample(). Callers can test ReadResult members for which cases have occurred.
References _ateof, _eofx, _iostream, nidas::core::IOStream::available(), checkUnexpectedEOF(), CNAME, DLOG, nidas::core::getSample(), handleNewInput(), nidas::core::IOStream::isNewInput(), nidas::core::IOStream::read(), nidas::core::IOStream::readBuf(), and VLOG.
Referenced by readBlock(), and readSamples().
|
private |
Read a block into memory, updating the given block pointer and length counter accordingly.
If keepreading
is false, only read into the block what is available from the iostream buffer. Return the ReadResult of the last read. Always return if the current input ends or eof is reached.
References nidas::core::getSample(), and read().
Referenced by nextSample().
void SampleInputStream::readInputHeader | ( | ) |
Read archive information at beginning of input stream or file.
nidas::util::IOException |
References _inputHeaderParsed, _iostream, _last_name, CNAME, DLOG, getName(), nidas::core::IOStream::getNumInputBytes(), parseInputHeader(), and nidas::core::IOStream::read().
Referenced by nidas::dynld::raf::SyncRecordReader::init(), and nextSample().
|
virtual |
Read the next sample from the InputStream.
The caller must call freeReference on the sample when they're done with it. This method may perform zero or more reads of the IOChannel.
nidas::util::IOException |
Implements nidas::core::SampleInput.
References nextSample().
Referenced by nidas::dynld::raf::SyncRecordReader::init(), and nidas::dynld::raf::SyncRecordReader::read().
|
virtual |
Read a buffer of data, serialize the data into samples, and distribute() samples to the receive() method of my SampleClients and DSMSensors.
This will perform only one physical read of the underlying IOChannel and so is appropriate to use when a select() or poll() has determined that there is data available on the file descriptor, or when the physical device is configured for non-blocking reads.
nidas::util::IOException |
Implements nidas::core::SampleInput.
References _ateof, _iostream, _source, nidas::core::IOStream::available(), CNAME, nidas::core::SampleSourceSupport::distribute(), handleEOF(), nextSample(), read(), and VLOG.
|
inlinevirtual |
Remove a SampleClient from this SampleSource.
)
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::removeSampleClient().
|
inlinevirtual |
Remove a SampleClient for a given SampleTag from this SampleSource.
The pointer to the SampleClient must remain valid, until after it is removed.
)
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::removeSampleClientForTag().
|
inlinevirtual |
)
Implements nidas::core::SampleSource.
References _source, and nidas::core::SampleSourceSupport::removeSampleTag().
|
virtual |
nidas::util::IOException |
Implements nidas::core::SampleInput.
References _iochan, _service, nidas::core::getSample(), and nidas::core::IOChannel::requestConnection().
|
private |
Check the current header for validity and generate a sample for it.
References _badSamples, _block, _bsf, _goodSamples, _headerToRead, _hptr, _iostream, _sheader, _skipSample, nidas::util::LogContextState::active(), nidas::dynld::BlockStats::addGoodSample(), CNAME, nidas::dynld::BlockStats::endBadBlock(), nidas::core::BadSampleFilter::filterBadSamples(), ftime(), nidas::core::SampleHeader::getDataByteLength(), nidas::core::SampleHeader::getId(), getName(), nidas::core::IOStream::getNumInputBytes(), nidas::core::getSample(), nidas::core::SampleHeader::getSizeOf(), nidas::core::SampleHeader::getTimeTag(), nidas::core::SampleHeader::getType(), nidas::dynld::BlockStats::good, nidas::core::BadSampleFilter::invalidSampleHeader(), nidas::util::LogContextState::log(), LOG_DEBUG, LOG_WARNING, nidas::dynld::BlockStats::nbytes, nidas::dynld::BlockStats::start_time, nidas::dynld::BlockStats::startBadBlock(), and WLOG.
Referenced by nextSample().
void SampleInputStream::search | ( | const nidas::util::UTime & | tt | ) |
Search forward until a sample header is read whose time is greater than or equal to tt.
Leaves the InputStream positioned so that the next call to readSample() or readSamples() will read the rest of the sample.
nidas::util::IOException |
References CNAME, DLOG, and nextSample().
|
inline |
Replace the bad sample filter rules for this stream with bsf
.
References _bsf, and nidas::core::getSample().
References _expectHeader, and nidas::core::getSample().
Referenced by setIOChannel().
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setFilterBadSamples().
|
virtual |
Set the IOChannel for this SampleInputStream.h.
References _dsm, _iochan, nidas::core::IOChannel::close(), nidas::core::IOChannel::getConnectionInfo(), nidas::core::Project::getInstance(), nidas::core::getSample(), setExpectHeader(), and nidas::core::IOChannel::writeNidasHeader().
Referenced by fromDOMElement(), SampleInputStream(), and SampleInputStream().
Implements nidas::core::SampleInput.
References _source, and nidas::core::SampleSourceSupport::setKeepStats().
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMaxDsmId().
Referenced by nidas::dynld::raf::SyncServer::openStream().
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMaxSampleLength().
|
inline |
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMaxSampleTime().
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMinDsmId().
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMinSampleLength().
|
inline |
See BadSampleFilter.
References _bsf, and nidas::core::BadSampleFilter::setMinSampleTime().
nidas::util::IOException |
Implements nidas::core::SampleInput.
References _iochan, and nidas::core::IOChannel::setNonBlocking().
|
virtualinherited |
Add my content into a DOMElement.
xercesc::DOMException |
Reimplemented in nidas::core::DSMConfig, nidas::core::DSMSensor, nidas::core::Project, nidas::core::SampleTag, nidas::core::Site, nidas::core::ServerSocket, and nidas::core::Variable.
Referenced by nidas::core::ServerSocket::toDOMParent().
|
virtualinherited |
Create a DOMElement and append it to the parent.
xercesc::DOMException |
Reimplemented in nidas::core::DSMConfig, nidas::core::DSMSensor, nidas::core::Project, nidas::core::SampleTag, nidas::core::Site, nidas::core::ServerSocket, and nidas::core::Variable.
|
private |
Referenced by nextSample(), read(), and readSamples().
|
private |
Number of bad samples in the stream so far, which is to say number of bytes checked which did not contain a reasonable sample header.
Referenced by closeBlocks(), getBadSamples(), and sampleFromHeader().
|
private |
Information about the current block of samples, good or bad.
Referenced by closeBlocks(), and sampleFromHeader().
|
private |
|
private |
How many bytes left to read from the stream into the data portion of samp.
Referenced by checkUnexpectedEOF(), nextSample(), and parseInputHeader().
|
private |
Pointer into the data portion of samp where we will read next.
Referenced by nextSample().
|
private |
Referenced by getDSMConfig(), and setIOChannel().
|
private |
Referenced by handleEOF(), and read().
|
private |
Referenced by getExpectHeader(), parseInputHeader(), and setExpectHeader().
|
private |
Number of good samples in the stream so far.
Referenced by closeBlocks(), and sampleFromHeader().
|
private |
Referenced by checkUnexpectedEOF(), nextSample(), parseInputHeader(), and sampleFromHeader().
|
private |
Referenced by nextSample(), parseInputHeader(), and sampleFromHeader().
|
private |
Referenced by getInputHeader(), and parseInputHeader().
|
private |
Referenced by handleNewInput(), nextSample(), parseInputHeader(), and readInputHeader().
|
protected |
|
private |
Referenced by close(), handleNewInput(), parseInputHeader(), read(), readInputHeader(), readSamples(), sampleFromHeader(), SampleInputStream(), and SampleInputStream().
|
private |
Keep the current name of the input stream so it can be
referenced even after the input stream has advanced to a new input name.
Referenced by checkUnexpectedEOF(), closeBlocks(), and readInputHeader().
|
private |
Referenced by getOriginal().
|
private |
|
private |
Will be non-null if we have previously read part of a sample from the stream.
Referenced by checkUnexpectedEOF(), nextSample(), parseInputHeader(), and ~SampleInputStream().
|
private |
The currently pending sample.
When filtering is active, the pending sample is held until the succeeding sample header is confirmed to be good.
Referenced by handleEOF(), nextSample(), and ~SampleInputStream().
|
private |
Service that has requested my input.
Referenced by requestConnection().
|
private |
Referenced by checkUnexpectedEOF(), nextSample(), parseInputHeader(), and sampleFromHeader().
|
private |
This is set if the data for the current sample header should be read but skipped, because the sample is the first after a block of bad samples and thus the time at the front of the sample might still be corrupt.
Referenced by nextSample(), and sampleFromHeader().
|
protected |
Referenced by addSampleClient(), addSampleClientForTag(), addSampleTag(), distribute(), flush(), getClientCount(), getProcessedSampleSource(), getRawSampleSource(), getSampleStats(), getSampleTagIterator(), getSampleTags(), readSamples(), removeSampleClient(), removeSampleClientForTag(), removeSampleTag(), and setKeepStats().
|
staticprivateinherited |
Referenced by nidas::core::DOMable::getNamespaceURI().