NIWOT02: Logbook Entries

NIWOT02: Site all Messages, 8 Entries..

Return to Logbook Contents Page
Entry Date Title Site Author #Graphics
32 Wed 23-Oct-2002merging SDR dataallmaclean
31 Wed 23-Oct-2002SDR dataallmaclean
30 Tue 22-Oct-2002chronologyallmaclean
27 Mon 07-Oct-2002Time delays for the Hydra intakesalldelany
16 Thu 05-Sep-2002pam stations upallmilitzer
11 Tue 03-Sep-2002pam stations outallmaclean
6 Fri 16-Aug-2002EVE bug causing serial errors and lost dataallmaclean
5 Fri 16-Aug-2002Freewave configallmilitzer

32: LOG, Site all, Wed 23-Oct-2002 09:48:40 MDT, merging SDR data

Here are the scripts used in preparing and merging the SDR data with the RF
data.  They are on $PAM/scripts unless noted otherwise.  Passing the -4
option fixes the channel 4 length problem.

set_project NIWOT02

cd to_some_scratch_space

mkdir -p stn3/disk1

cd stn3/disk1
# read flash drive
cp /mnt/pccard/niwot* .

# this script creates directory new_names, containing symbolic
# links back to the actual SDR files.  The symbolic links have names
# like 020815.00000 indicating the actual start date of the file
assign_date_sdr -4

cd new_names

# strip ETX from SDR files, pass data into ingestor, which does
# necessary sscanf's on ASCII channels based on channel_config, and
# generates samples that ASTER data system can handle.  Script expects
# to read files called YYMMDD.HHMMSS so that it can determine the date.
# script runs archive, which connects to the ingestor and archives
# the files on $RAWDATADIR/all
convert_sdr -4 stn3 0208*

# Merge the sdr and rf data sets. This script assumes the sdr data is on 
# /scr/isff/aster/projects/NIWOT02/raw_data/sdr
# the rf data is on
# /scr/isff/aster/projects/NIWOT02/raw_data
# It creates new merged files on
# /scr/isff/aster/projects/NIWOT02/raw_data/new


31: LOG, Site all, Wed 23-Oct-2002 09:27:37 MDT, SDR data
The SDR disks were not monitored closely during NIWOT02, since we had
RF communications.

However the SDR data will fill some holes:
1. Aug 9-14 at station 3 when the SDR was operating but
   the RF was not.
2. At the beginning of the project EVE, was writing channel 4 data
   (containing days since Jan 1, 1970 and station #) with a wrong
   length value of 2, when it should have been 3.  This caused
   the RF ingest program to loose data

John gave me two SDR (IBM microdrives) disks from each station.
Disk2 from station 1 only contains July lab data. Disk5 from stn1
has data starting Aug 14.

The two disks from station 3 have useful August data.  It appears that
the only disk swap was at stn3, on Aug 14.

Since the SDRs contain the early data, they must have quit writing
once they filled up, and (fortunately) did not delete old files.

Here are ls -l of each disk, followed by the time values provided by
$PAM/scripts/sdr_show_date -4

Stn1, disk #2
  ls -l:
  -rwxrwxr-x    1 maclean  sssf     19370467 Jan  1  2018 niwot101.dat
  -rwxrwxr-x    1 maclean  sssf     19611939 Jan  2  2018 niwot102.dat
  -rwxrwxr-x    1 maclean  sssf      6307840 Jan  2  2018 niwot103.dat

  file          date
  niwot101.dat  020725.221635   had chan 4 length problem (2 should be 3)
  niwot102.dat  020726.061629   had chan 4 length problem (2 should be 3)
  niwot102.dat  020726.141637   had chan 4 length problem (2 should be 3)

  Since the dates are July, this disk is of lab data.

Stn1 disk #5

  total 348072
  -rwxrwxr-x    1 maclean  sssf     55336649 Aug  5  1980 niwot102.dat
  -rwxrwxr-x    1 maclean  sssf     55223871 Aug  6  1980 niwot103.dat
  -rwxrwxr-x    1 maclean  sssf     55311905 Aug  7  1980 niwot104.dat
  -rwxrwxr-x    1 maclean  sssf     55262628 Aug  8  1980 niwot105.dat
  -rwxrwxr-x    1 maclean  sssf     55541780 Aug  9  1980 niwot106.dat
  -rwxrwxr-x    1 maclean  sssf     53141504 Aug  9  1980 niwot107.dat
  -rwxrwxr-x    1 maclean  sssf     26583040 Aug 11  1980 niwot108.dat
  drwxrwxr-x    2 maclean  sssf         8192 Aug 14 12:50 recycled

  file          date
  niwot102.dat 020814.203341    had chan 4 length problem (2 should be 3)
  niwot103.dat 020815.204005    had chan 4 length problem (2 should be 3)
  niwot104.dat 020816.204015    had chan 4 length problem (2 should be 3)
  niwot105.dat 020817.204011    had chan 4 length problem (2 should be 3)
  niwot106.dat 020818.204022    had chan 4 length problem (2 should be 3)
  niwot107.dat 020819.204023    had chan 4 length problem (2 should be 3)
  niwot108.dat 020820.194505    didn't need fix
  recycled/dd12.01z  020622.080124  IHOP data?
  recycled/dd13.32z  020622.003231  IHOP data?

  The RF archive contains data from 020814.212108.  However it suffers
  from the channel 4 problem so converting these files will gain some

Stn3 disk #1

  total 309760
  -rwxrwxr-x    1 maclean  sssf     41435136 Jan  1  2021 niwot10g.dat
  -rwxrwxr-x    1 maclean  sssf        57344 Dec 21  1914 niwot10h.dat
  -rwxrwxr-x    1 maclean  sssf     55762860 Jan  2  2010 niwot301.dat
  -rwxrwxr-x    1 maclean  sssf     55782410 Jan  3  2010 niwot302.dat
  -rwxrwxr-x    1 maclean  sssf     55777648 Jan  4  2010 niwot303.dat
  -rwxrwxr-x    1 maclean  sssf     55774014 Jan  5  2010 niwot304.dat
  -rwxrwxr-x    1 maclean  sssf     52591448 Jan  6  2010 niwot305.dat

  file          date
  niwot10g.dat  020730.232109   had chan 4 length problem (2 should be 3)
  niwot10h.dat  020731.173038   had chan 4 length problem (2 should be 3)
  niwot301.dat  020809.205547   had chan 4 length problem (2 should be 3)
  niwot302.dat  020810.205454   had chan 4 length problem (2 should be 3)
  niwot303.dat  020811.205456   had chan 4 length problem (2 should be 3)
  niwot304.dat  020812.205501   had chan 4 length problem (2 should be 3)
  niwot305.dat  020813.205507   had chan 4 length problem (2 should be 3)

  RF archive contains data from 020814.212236, so these files will help.

Stn3 disk #6
  -rwxrwxr-x    1 maclean  sssf     55546305 Jan  7  2010 niwot306.dat
  -rwxrwxr-x    1 maclean  sssf     55795404 Jan  8  2010 niwot307.dat
  -rwxrwxr-x    1 maclean  sssf     36233216 Jan  8  2010 niwot308.dat
  -rwxrwxr-x    1 maclean  sssf        98304 Jan  8  2010 niwot309.dat
  -rwxrwxr-x    1 maclean  sssf     55787828 Jan  9  2010 niwot30a.dat
  -rwxrwxr-x    1 maclean  sssf     55787385 Jan 10  2010 niwot30b.dat
  -rwxrwxr-x    1 maclean  sssf     55791484 Jan 11  2010 niwot30c.dat
  -rwxrwxr-x    1 maclean  sssf     15966208 Jan 12  2010 niwot30d.dat

  file          date
  niwot306.dat  020814.193618   had chan 4 length problem (2 should be 3)
  niwot307.dat  020815.193600   had chan 4 length problem (2 should be 3)
  niwot308.dat  020816.193606   had chan 4 length problem (2 should be 3)
  niwot309.dat  020817.111148   had chan 4 length problem (2 should be 3)
  niwot30a.dat  020817.111449   had chan 4 length problem (2 should be 3)
  niwot30b.dat  020818.111501   had chan 4 length problem (2 should be 3)
  niwot30c.dat  020819.111505   had chan 4 length problem (2 should be 3)
  niwot30d.dat  020820.111511   had chan 4 length problem (2 should be 3)
        IO error reading niwot30d from microdrive

  recycled/dd12.01z  020622.080124  IHOP data (we have these files on CD)
  recycled/dd13.32z  020622.003231  IHOP data

30: LOG, Site all, Tue 22-Oct-2002 15:20:26 MDT, chronology

Here is a chronology based upon the data archive, system logs
(and anybody's notes or recollection?):

Aug  7 17:41 MDT  Reboot of aster. No more reboots are seen in the system
                  log until Aug 24  stn1,3 ingest timeouts until Aug 14 15:18
                  Therefore aster must have been installed in the CUFF trailer
                  on Aug 7.

Aug  9 14:55 MDT  Date of earliest station 3 (east) SDR data

Aug 14 14:33 MDT  Date of earliest station 1 (west) SDR data

Aug 14 11:54 MDT  First cosmos message in system log: first archive file
Aug 14 15:18 MDT  last select timeouts for stn1 and 3, receiving RF data


27: CO2, Site all, Mon 07-Oct-2002 10:07:33 MDT, Time delays for the Hydra intakes
24 Sep''02. Tony D and Don L visited Niwot02 site and undertook to measure the
time delays for the Hydra.

The AC power for the relays, and hence solenoids, was switched off. 
The AC power for the Opto22 and the 12 VDC for the sample valve and the 
sample pump were maintained.
The AC daisy-chain and the corresponding connections from the solenoids 
were disconnected. Zip cords were connected in their stead. Thus on the flip 
of a switch on the power block a single solenoid could be activated.

The flow through the buffer volumes was continuous and the choice of 
solenoid merely changed the choice of sampled volume. 
The solenoids were activated at the instant of an Opto22 transition and the 
instruction to breathe into the intake was also given, by radio at the 
instant of an Opto22 transition. Generally the buffer volume was sampled 
for 80 x 2 seconds befor the breath instruction was given. 
then 30 seconds of breathe was sampled. A 10 - 15 minute period was allocated 
to allow the pulse of CO2 to travel the length of the Hydra intake line.
 A table of delay times is gien in 
16: EVEs, Site all, Thu 05-Sep-2002 08:47:03 MDT, pam stations up
9/4/02 afternoon ~14-15:30 MDT.
The PAM stations were down because both A/C power supplies
died; probably from lightning.
The dead supplies were replaced with identical units at
both Stn1/3.   The stations started up normally and RF to
ASTER was ok.
I added a ground to help protect the replacement supplies.
11: EVEs, Site all, Tue 03-Sep-2002 15:14:30 MDT, pam stations out
stn3 died on Aug 31, 18:14 MDT, and stn1 died Sep 1, 10:13 MDT.

John and Tony are going up there today (Sep 3) to figure out
what is wrong.
6: EVEs, Site all, Fri 16-Aug-2002 18:46:18 MDT, EVE bug causing serial errors and lost data
There is an EVE bug related to the new contents of
fast-out channel 4. EVE is still reporting a length of 2, and it
should be 3, because it also contains a 1 byte station number.

I verified this, both by looking at the fast out data, and looking
at the code in fastOutput.c

This is why we see large numbers of serial errors in the
check_aster output. 

It also causes more problems for station 3 than station 1.

After reading the timetag, channel number=4 and length=2 (incorrect)
it then reads the next 2 bytes as data and expects the next byte to
be 0x03  (ETX).  In the case of station 1, the next byte is 0x1,
and so the ingestor records a serial error, then looks forward
for the ETX, which is the next byte. So for station 1 no data is lost, 
but we see serial errors.

For station 3, the ingestor gets an ETX when it expects one, but
it is actually the station number because both are 0x3.  
The next byte is actally the ETX, but it treats that as part of 
the next sample's timetag.  This next sample will then be garbage,
which is why we're losing sonic data, and seeing CSAT3 diag values 
of 16 (indicating a missing sample).

5: Communications, Site all, Fri 16-Aug-2002 18:45:12 MDT, Freewave config
Copied from an email from John:
 The RF seems less than optimal but the cockpits
didn't appear to be losing anything.  Time will tell.
The connects look like this:
Stn1 West:
  FW2105 (slave@pam) - ~9m antenna 64-degreesTrue - FW2101 (master@base) - usb0
  uses lower antenna (short/thin cable) at base
Stn3 East:
  FW2088 (slave@pam) - ~9m antenna 47-degreesTrue - FW2095 (master@base) - usb1
  uses upper antenna (long/thick cable) at base
The upper base antenna could be raised quite a bit, but the lower not.
Otherwise we could go with the 2.4 FWs which could work better in the cluttered
treeish conditions, but they'd require some programming and other antennas.

NOTE: We need another Freewave to USB de9/m-f PC cable to
replace the one I borrowed from Andy.