IHOP02: EVE Messages: 12 Entries..

06/19/02 09:30 - 10:10 Local

Reason:	Double check station was ok, swap sdr

SDR Microdrive #6 was removed, #4 installed.
backup_sdr was run at the base later.

Checked gas cylinders and decided to swap in new N2 based
upon usage over the last couple of weeks.
bottle	psi	hg"
1/4	1950	5"
1/3	1950	5.8"
1/2	1840	5.6"
1/1	1930	6"
N2	2700	6.5"   (old had been 500 /6")

Site 3
06/19/02 07:35 - 08:15 Local, visit


The File System crashed last night at about 0Z.
This may have occured/begun when the ramdisk copy metppf
file to /card was attempted based upon examination of the
syslog file.  Same old symptom: file sizes = 0 and inability
to access them.  I didn't notice if the /card led light was
lit indicating that board had locked up causing this...

Otherwise the system kept running including GOES and continued to
write to metppf for some reason....strange.

Note:	'Reboot' did not recover this problem
	Apparently the take-down of the running processes locked
	up (fileout) and meanwhile the watchdog pulse task was
	still going preventing the w.d.board from doing the reset.
	'Reset' button on CPU did solve this problem.
	It was not necessary to power cycle the system.
	Change take-down task-delete the watchdog up-front.

From syslog:
06/19-00:00:00 0x14d218 (METPPF-Fil): ERROR:
	FileToCard: Can't create "/CARD/METPPF.169"
	       0x14d218 (METPPF-Fil): ERROR:
	File Stat Call - /RAMDISK/METPPF.165


POTENTIAL METPPF PROBLEM:
I didn't notice this when booting up after creating the
sdr-hr config file, but the additional spaced used for that may
have prevented eve from properly calculating and starting the main
metppf save process.
From syslog....
06/19-12:58:07 0x177670 (eveMain): *** EVE Booting PROM May 14 2002 *** 
06/19-12:58:12 0x177670 (eveMain): Purging '/RAMDISK/METPPF.164-9'
06/19-12:58:20 0x177670 (eveMain): ERROR:   FILE 'METPPF' Ignoring Request!!!
             Space available/uncommitted 57590 < fileSize needed 66816 bytes
06/19-12:58:21 0x177670 (eveMain): *** EVE config 125 running ***

This problem may have also happened at 2, and at 6 when starting up
the SDR local storage.  I did put some safeguards in to force it to
write what it can, but maybe not in this case and I should be able
to determine that when visiting site 6 today.


See Also: Entry #265 for sdr-hr summary 

Site 3 SDR_OEM High Rate Data Acquired: Shield On

Begin:	06/19/02 07:35 - 08:15 Local, visit
End:	06/19/02 12:00 - 12:10 Local, visit

Copied	config.dat to config.sav
Created config.hr3, Used this during run.
Returned to original config.dat at end.
There should be no visible effects in slow data.

Data Collected: MicroDrive#3
File:	st3_061a.dat, ~6mB, ~4hours

See Also: Entry #265 for sdr-hr summary 

SDR_OEM High Rate Data Acquired: Shield On

Begin:	06/18/02 17:30 - 18:20 Local, visit
End:	06/19/02 10:55 - 11:10 Local, visit

Copied	config.dat to config.sav
Created config.hr2, Used this during run.
Returned to original config.dat at end.
There should be no visible effects in slow data.

Data Collected: MicroDrive#5
File:	st2_0619.dat, ~25.5mB, ~17hours

See Also: Entry #265 for sdr-hr summary 
	  Entry #264 for rnet coef change

Summary of SDR_OEM High Rate Data Acquired
Purpose: to look for noise imposed on the data by GOES RF noise



Site  Shield	~Begin(Z)    ~End(Z)     ~Len.	  File(s)	~mB	Drive#

1	Primary 'High-Rate' Data Station had continuous recording. See 
	other logs.

2	On	6/18,23:00   6/19,16:00  17:00	st2_0619.dat	25.5	5
	Off	6/27,20:25   6/28,04:25  08:00	st2z0629.off	12.0
	Off	6/28,04:25   6/28,12:25  08:00	st2z062a.off	12.0
	Off	6/28,12:25   6/28,15:44  03:19	st2z062b.off	 5.0

3	On	6/19,13:00   6/19,17:05   4:05  st3_0619.dat	 6.0	3
		6/21,22:35   6/22,18:55  20:25  st3_0622.dat	30.6    5
	Off	File not recorded properly in field...ran later in Bldr...
	Off	7/02,23:00   7/03,14:45  15:45  st3_0702.dat    22.5    3

4	On	6/20,21:50   6/21,19:00  21:10	st4_0621.dat	32	5
	Off	6/27,00:37   6/27,08:37  08:00 	st4z0627.off    11.9
	Off	6/27,08:37   6/27,16:10  07:33	st4z0628.off	11.4


5	On	6/21,17:56   6/21,18:01	 00:05	st5_0622.dat	 0.13	3
		     18:10   6/22,18:10  24:00	st5_0624.dat	39.3
		6/22,18:10        19:25  01:15	st5_0625.dat	 2.1
	Off	File not recorded properly in field...ran later in Bldr...
	Off	7/08,18:15   7/09,14:55  20:40  st5_0708.off	31.4	3

6       On	6/19,22:10   6/20,19:50  21:40  st6_061c.dat	35.5	3
	Off	6/26				st6z0627.off	  .0
	Off	6/26,23:20   6/27,07:20  08:00	st6z0628.off	12.1
	Off	6/27,07:20   6/27,13:42  06:20	st6z0629.off	10.5


7	On	6/1x,15:13   6/1x,15:31  00:18  04191216.00z	 0.4
		6/1x,15:35   6/1x+,21:15 29:40  04191217.00z	44.8
		6/1x,21:44   6/1x,21:46  00:02  06180001.00z	 0.06
	On	6/20,21:50   6/21,05:50	 08:00	06202151.00z	12.0	2
		6/21,05:50   6/21,13:50  08:00	06202152.00z	12.0	2
		6/21,13:50   6/21,21:10  07:20	06202153.00z	11.1	2
		6/21,21:15   6/21,21:28  00:13	06202154.00z	 0.3	2
		6/21,21:30   6/22,04:00  06:30	06202155.00z	 9.7	2
	Off	6/25,22:41   6/26,06:41  08:00	st7z0626.off	12.0
	Off	6/26,06:41   6/26,14:41  08:00	st7z0627.off	12.0
	Off	6/26,14:41   6/26,20:28  05:50	st7z0627.off	 8.1

8	Primary 'High-Rate' Data Station had continuous recording. See 
	other logs.

9	On	6/23,21:05   6/24,21:04  24:00	st9_0623.dat	23.2	3
	On	6/24,21:05   6/25,20:30  23:30	st9_0623.dat	22.5	3
	Off	File not recorded properly in field...ran later in Bldr...
		10hz sonic as it was during teardown...
	Off	7/03,17:25   ....        24:00  st9_0703.off	21	3
	Off	7/04,17:25   ....        24:00  st9_0704.off	21	3
	Off	7/05,17:25   ....        24:00  st9_0705.off	21	3
	Off	7/06,17:25   ....        24:00  st9_0706.off	21	3
	Off	7/07,17:25   ....        24:00  st9_0707.off	21.	3
	Off	7/08,17:25   7/08,17:40  24:00  st9_0708.off	 0.2	3

6/17/02 !6:20-50 Local

During Site visit, upon arrival, we noticed that the EVE file system
had crashed.  Nevertheless data collection / goes was running OK.

The evidence of this is file-size =0 for all files when doing a 'dir'
command for either the ramdisk or /card.
Upon checking the syslog file I saw many entries indicating 'metppf
file access fail' for the 5-minute writes after the crash.
Unfortunately I managed to lose this syslog file.
When eve booted up it copied that existing, full syslog
file to the card and started a new syslog as expected.
My problem may have occured when doing the move_metppf script
on the panasonic when I abnormally exited using a ctrl-c.  It appears
somewhere along the line an old version syslog file was copied and
tagged for the 6/17 dated version.

The EVE file-system problem was recovered by the power cycle as expected.

Keep an eye on Sta-1.  This has happened before in my recollection
and appears due to a timing conflict during operations of a 'fast-save'
station which is heavily loaded and doing the regular 'metppf' saves
to ramdisk.

EVE TO-DO NOTE:
This type of file-system error should initiate a 'reboot' assuming
that'll recover this problem.

Replaced EVE prom EVE 4/21/02 6047 with IHOP02 5/14 0xA7BA

Gordon swapped the prom in EVE.  He replaced EVE 4/19/02 4FC6 with
IHOP02 5/14/02 0xA7BA.

He also checked the clock, which had only a 1 sec error compared to
the GPS time.

Rebooted at 10:27 CDT.

Power variables look okay.

To be sure new SDR configuration is working, we swapped microdrives
at 8.  Josh was out there and did it for us:

Removed #1, inserted #5.

The SDR (serial data recorder) is in the EVE VME cage, the card
with a yellow button and 4 LEDs:

top LED: fault
         received data
         recording
bottom:  power

Eject button is right above the microdrive.

When you arrive, the power and recording lights should be solid on,
and the received data flickering.

1. Press the yellow button, recording light should go out.  
2. Press black eject button. Remove microdrive
3. Insert empty microdrive until eject button is flush with side of
   card. Note number (1-6) of microdrive.
4. Press yellow button. Recording light should go on <-- important.
5. Close and secure the EVE door. Don't pinch any cables
   when you close it.
 

That's it!  You've done your work for today.  Drive 200 miles back
to Wichita.

If you have the panasonic notebook, you can check the contents of
the drive you removed:

  put microdrive in PCMCIA adaptor
  insert adaptor in either PCMCIA slot
  do the following from a shell window
	mount /mnt/pccard
	ls -l /mnt/pccard
	umount /mnt/pccard




          

Swapped microdrives at 8.

Removed #6 XHA78295 99B
Inserted #1 XHAV9598 09D

Left #5 (XHA76012 99B) in eve box.  It is empty and ready to use.

File sizes at site 8:  14.1 Mbyte/8 hours =  42.3 Mbyte/day

This is like station 1 (see entry #71), a 358 Mbyte
microdrive should take a week's data.

See entry #80 about the barometer & SDR. The SDR at station 8 is
not being commanded from EVE and will be doing its own file
opening/closing, with the period set to 8 hours.

Serializer and SlowAD were checked against a 'dial-a-volt' reference
for consistency.

Reference: 2.3995  (based on 4.5 digit dvm.  output varied slightly
		    between boards from about 2.3993 to 2.3998)

A/D Values reported are nominal of what was observed.
A/D Consistency: The 4 channels on each pmb-ad board were consistent within a
		few counts between the channels and between readings.
Serializer Values: The measurement was that reported by EVE after scaling of
		the raw readings.  These include the divide-by-2 cable used
		to scale the krypton voltage and represent mV.
		Note that for every unit, the nominal values shown had
		noise of about "1" mV total during the brief test.
Serial Numbers:	Dash numbers represent the residual reference voltage reading
		taken by Steve Oncley used in adjusting the output.
		For example"-07" represents 2.5007, "-93" represents 2.4993.
		PMB boards were tagged "A1" and "S1", etc.


Sta	s/nAD	s/nSerial	AD0	AD1	Serializer
1	A1-93	S1-07		23988	23987	2385		Note 1)
2	A2-95	S2-02		23976	23962	2390
3	A3-	S3-		23987	23988	2390
4	A4-04	S4-96		23988	23987	2392
5	A5-88	S5-94		23985	23983	2391
6	A6-91	S6-09		23985	23981	2392
7	A7-06	S7-88		23984	23980	2392
8	A8-94	S8-99		23976	23976	2395
9	A9-97	S9-86		23985	23987	2391

Note 1) Station 1 was sitting in the cal-lab adjacent to the running
	co2 system.  There was quite a bit of noise on the a/d counts
	while taking the readings inside to box there.  The readings were
	roughly 23740/23770 and were changing by up to 100.  The readings
	shown above were taken in a different box in a different location
	and the a/d counts were extremely consistent.  I did not repeat
	the testing to determine whether the noise was endemic to station
	1's box or not.   Note that the readings for station 8 were taken
	in the same location.