High Rate ISFF data for HVAMS03

Introduction

The full HVAMS03 dataset is archived in binary files, which contain the raw, uncalibrated sensor output. The HVAMS03 high-rate archive files are available by FTP (soon), on DVD, or from the NCAR MassStore.

A Java application, which calibrates this sensor output into ASCII or NetCDF form, is available for download. Using this application you can write variables for time periods of interest into text or NetCDF files for further processing using any analysis software you choose.

Therefore, in order to generate high rate ISFF data, you must do the following. Further information on these steps is contained in the table of contents, to the left.

  1. Install Java on your system. It must be at least version 1.3. You can install either the JRE (Java Runtime Environment) or the Java Developers Kit, which is variously known as the JDK, the SDK or the J2SE.
  2. Download and install the ISFF java application.
  3. Get your hands on the appropriate binary archive files, by FTP, DVD, or direct transfer from the NCAR mass store.
  4. Run the ISFF java application.

Archive DataSet Structure and Size

The binary data files were archived in a directory called "all". The java application expects to find the files in a directory with that name, so when copying or downloading the files to your system, put them in a directory called all.

The file names have the following format: sN_yymmdd_HHMMSS.dat where N is the station number (1-9). yymmdd_HHMMSS are the the two digit year, month, day, hour, minute and second fields, representing the start time of the data in the file, in GMT.

Each file contains up to 24 hours of data, usually starting at 00:00:00 GMT of the day. One day's data from a station is about 20 Mbyte. The size of the entire dataset for the project is 9.3 Gbyte.

DVD Disks

The high rate data can be copied to DVD+R disks, which can be shipped to investigators on request.

Direct Mass Store Access

If you have direct access to the NCAR MassStore, the binary files are archived under /ATD/DATA/2003/HVAMS03/isff/raw_data/all. Use the msrcp command from a UNIX system to copy files from the mass store.

For example, to copy all files for October 26th to your hard disk:

cd /some_data_directory
mkdir all
cd all
msrcp 'mss:/ATD/DATA/2003/HVAMS03/isff/raw_data/all/*031026*' .
You must use quotes around the name of the mass store file mask, and note the trailing period (after a space). Also note the files were placed in an all directory. In this case you would set RAWDATADIR to /some_data_directory (see section on Running Java App).
Last modified: Thursday, 30-Apr-2009
© NCAR/Earth Observing Laboratory
This page was prepared by Gordon Maclean, NCAR Research Technology Facility