t-rex-iss-mapr: SOFTWARE Messages: 10 Entries..

The MAPR and ISS-MAPR systems were backed up to DVD with mondo.  Copies of the
DVDs are in the CD case in the middle drawers.

backed up data at MAPR at 22 UTC march 29th

These are the changes made on mapr to install a watchdog mechanism.

The flower icon which starts maprdisplay now runs a script called 'md' instead.  md starts maprdisplay in the background and logs the output to a dated directory under /home/iss/md-logs.  Try 'md log' to see the output from the currently running mapr session.  The flower icon automatically starts maprdisplay with the wind200_rass100 schedule.  So if you want to run with a different schedule, you must use the 'Restart' menu button in the upper left.

The md script also registers a cron task which checks every 15 minutes that mapr is running, unless mapr was last shut down normally.  If maprdisplay must be restarted, md restarts it with the schedule it was last running, then sends an email to me.

The md script cannot tell if maprdisplay is hung, only if it is still running or not.  I've changed maprdisplay to use an alarm() call if it fails to finish processing a time series within 60 seconds.  In other words, if maprdisplay hangs, the system sends it a signal after 60 seconds, and the signal causes the program to abort.  At the next 15-minute watchdog check, md should detect the missing maprdisplay and restart it.  The alarm() technique has not been tested yet.  It does not appear to cause any harm, but it will be hard to demonstrate that it works for the particular kinds of hangs that have been happening.  For the moment, it seemed the safest, most expedient, and most likely technique to succeed in recovering from hangs.

As reported by James, satellite internet was slow, still working but very slow, causing data transfers to fail.  I noticed the motosat D2 was not answering at http://192.168.1.250 like it should, and the Ready light on the D2 was off.  The DW6000 status page (http://192.168.0.1) showed both RX and TX operational, but it also showed a large count of failed transmissions.

Here's what I did:

I stowed the dish from the D2 Stow button.

I turned off the linksys router, D2, and DW6000.

I turned them back on in this order: DW6000, D2, linksys.

The D2 still did not answer at 192.168.1.250, so I cycled the power on it a second time.  Then I finally could browse to its web page.

Started a search with the D2 Search button.

The search finished in a few minutes, the Ready light came back on, and Internet traffic seemed normal again.  The signal strength on the D2 looks low (as in, the blue bar is short), but all other status messages on the D2 and DW6000 indicate things are operational.

I ran two manual transfers, including one of the last 10 hours of data.  For ncfca, I only transferred the last two hours.

I have been configuring the GTS TEMP message transmission.

1. Added a second call to AspenQC in init.d/sounding, which
generates the TEMP message and emails it to the NWS gateway.
The message also gets CC'd to me, but maybe we'll change this
to a more suitable authority.

2. Modified aspen.xml , to set correct TEMP properties (enable NNNN,
set the ICAO identifier, set the call sign, select message type = 2
for TEMP MOBILE, set abreviated hader to NNXX00).

3. Created a bunch of bogus soundings on GAUS, simply by copying
and renaming one of the soundings in training/, to debug the
scheme.

4. That abreviated header is for testing purposes. When we verify
that the messages are getting accepted at NWS, the abreviated header
will be changed to a legitimate one.

A Maxtor 300GB disk is now available om MAPR as /media/usbdisk.

The motosat system keeps dropping and reacquiring the 
signal. The transmit signal goes out on the dw6000, and
then the ready light on the motosat goes off. Looking
on the motosat status page, you see that the transmitter
has been turned off because the receive signal is no good.

Watching it for about half an hour, I see that the receive
signal comes and goes. When it drops out, it takes awhile 
to lock onto receive again. As soon as it does, the
transmitter will be enabled and the ready light will
appear on the motosat.

The dish is looking through heavy clouds. It is spitting rain
occasionally. Maybe that is the problem?

I did a search, and motosat was happy with what it found.
When reception is good, the signal quality is very high.

/iss/ds was rsync'ed to the raid drive.

cd /iss/ds
rsync -av -e ssh . mapr:/raid/data/data_source

For some reason Juerg was looking through the data and suddenly the workstation rebooted. He was only making foreground background changes. Unfortunately this meant we lost the sounding log started by Gary and Serena.