Data-Driven Models

This is a proposal to move the HIAPER display to be completely data-driven. The display side should completely describe its model (the data dependencies) to the datastore library, and then it waits and responds to notifications from the datastore upon changes to the model. We are very close to this now. The goal is for the datastore to be like one big dataset engine. Views (such as Plots) assemble models for the data they need, including derivations, then they let the datastore signal them when the view needs to be updated.

DataSet Models

Right now the update notification must pass back into the PlotPage level, so that all of the datasets can be consolidated into a list and passed back into the datastore to be updated at one time. In fact, the DataDomain for the PlotPage and the list of DataSets together represent the model for which the PlotPage and its Plots are the view. By creating an abstraction which encapsulates this dataset model and registering the model with the datastore, the datastore will have all the information it needs to signal when the model has changed. The PlotPage and the plots need only connect to signals from the model which tell them when the view needs to be updated because the data in the model have changed.

When the datastore receives a realtime notification of a new record of data from the database, then it searches all of the models to find the ones which need to be updated because their domain overlaps the received data. For example, if the model domain is open-ended and new data have been received, then that model needs to be updated. The datastore can update the datasets in all of those models all at once, in a single consolidated call, then trigger a notification on those models. There's no need to pass any steps off to the view just to see if the datasets or domain has changed. If they had, then the model would have changed and the datastore would no longer see a dependency. In this scheme, individual plots can listen for changes on their particular model, since they can redraw themselves independently of their container, plot page, plot window, whatever. If in a PlotWindow of several plots only one plot shows realtime data, then only that one plot should be redrawn.

As views (plots) add and remove datasets, they change their model, and that could trigger an implicit replot. A model could be controlled from multiple places, and on any change to a plot's model the plot just responds by replotting. This gets us closer to generic controllers, such as a single GUI for changing any model. The model being modified could be for a Plot or for the input to a derivation or for the input to a netcdf exporter, but the basic code for setup and control of the model will be the same in any case.

This also helps with the issue of multiple PlotPage instances causing multiple calls to updateDataSets rather than one consolidated call as was originally intended. When the datastore side receives an update of new data, via either a broadcast or a database notify, it can immediately update all of the dataset models whose domains include the new data. In other words, for the realtime case, all models with open-ended domains will have the new data filled into their datasets, and their update signals will fire. It does not matter how the models are partitioned among PlotPage instances.

The notifications from the datastore side still need to include details about the nature of the update, such as exactly which datasets in the model changed and over what domain. Then the views can make intelligent choices about whether a redraw is really necessary. For example, don't redraw a plot when the changed domain is outside the zoomed region. Or if data were only added, only plot the new points. Derivations may be able to choose to only compute the new data and not the existing data.

Of course, model changes are not driven only from below the datastore. The user can also change the time domain. When multiple plots are sharing the same time domain, such as when on a PlotPage, a change of the domain should trigger a single replot in all of the models which depend upon that domain. This is like a hierarchy of dataset models which essentially parallels the plot hierarchy. The top model contains only the domain, and it contains other models which inherit a dependency on the root, domain model. For a PlotPage, the root model contains only the data domain, and when that domain changes, all of the models for the plots on that page need to be updated. The datastore can still traverse the model hierarchy updating all of the datasets according to their domain, where the domain in most cases is inherited from the root of the model. So the challenge here is defining a dataset model which can be linked to other models to describe the dependencies completely, so the datastore can determine which models' views need to be updated.

Incidentally, PlotPage has a signal userChangedPlotDomain() that is not used anywhere. Is it needed?

Data Sources as Models for Asynchronous Opens

The set of data sources in the datastore can also be considered a model to which the rest of the application can listen for updates. The MainWindow menus which need to be updated when a datasource is added can listen for such an event from the datastore (with an indication of whether it opened or failed and why). Then we're closer to opening a datasource asynchronously. As much of the opening part as possible can happen in a separate thread in re-entrant code, then when finished pass the thread can pass the result back to the datastore through the event queue. The result is either a failure message or the pointer to the new datasource. In the main application thread, the event handler actually hooks the new datasource into the datasource tree and fires the signals.

Task List

Table 2.1. Tasks for Data-Driven Models

DaysTaskStatus
2Extend the DataDomain class to include the notions of open-ended, inclusive boundaries. Essentially this means moving those same notions from what the PlotTimeDialog already describes into the DataDomain class.Not done.
5Define the DataSetModel class. The DataSetModel encapsulates a set of DataSets, a DataDomain, and an optional parent DataSetModel on which it depends. It also provides an interface to listen for changes to the model, by which clients can receive notices on changes and information about what changed. It may suffice to just extend the DataSetSelection class, or else just define a DataSetModel as a collection of DataSetSelection instances.Not done.
4Change the Plot class to create a DataSetModel instance comprising its DataSet instances. Register its model with the datastore, then connect to the model's update notices.Not done.