CHATS: hot films Messages: 34 Entries..

Replaced eastern-most hot film on array, sonic 7t.  Probe 8 was replaced by 3
appropriate switches were dipped.  Gordon verified proper operation.  
Operational at 11:15 am.

June 6, 10:50 LT
hfarray:  removed ptec#1, installed ptec#2


12:00 LT
hftower: removed ptec#11, installed ptec#6

Array hotfilm looks good.  Last dummy channel is > 3 V as it should be:

data_dump -d 6 -s 11 -p sock:localhost

2007 06 06 01:30:16.1009  0.0005      24      1.826      1.865      1.865      1.951      2.391      3.235
2007 06 06 01:30:16.1014  0.0005      24      1.808      1.864      1.864      1.953      2.391      3.225
2007 06 06 01:30:16.1019  0.0005      24      1.828      1.865      1.862      1.953      2.391      3.231
2007 06 06 01:30:16.1024  0.0005      24      1.839      1.865      1.862      1.953      2.391      3.231
2007 06 06 01:30:16.1029  0.0005      24      1.825      1.865       1.86      1.953      2.391      3.239
2007 06 06 01:30:16.1034  0.0005      24      1.808      1.866      1.859      1.953      2.391      3.229
2007 06 06 01:30:16.1039  0.0005      24      1.829      1.866      1.859      1.951      2.391      3.247
2007 06 06 01:30:16.1044  0.0005      24       1.84      1.867      1.858       1.95      2.391      3.253


hftower:

2007 06 06 01:34:41.4657  0.0005      36      2.186      2.168      2.104      2.388      2.407       2.25      2.477      2.414      2.379
2007 06 06 01:34:41.4662  0.0005      36      2.186      2.168      2.103      2.386      2.407       2.25      2.478      2.415      2.379
2007 06 06 01:34:41.4667  0.0005      36      2.186      2.168      2.103      2.385      2.405      2.246      2.478      2.416      2.378
2007 06 06 01:34:41.4672  0.0005      36      2.186      2.168      2.103      2.384      2.405      2.245      2.479      2.417      2.379

1jun07  10:40-12:45  Tower Climbed

HF tower pocketec #7 removed, #11 installed

The triple films were installed as before #1 at 6m, #4 at 10m and #3 at 14m 
between 1115LT and 1200LT. hftower was started with the switch and did not
come up. adn/aup was unsuccessful in mounting the Ptec.
Hard cycle was unsuccesful in mounting the Ptec.
adn/aup was successful in starting the data system and mounting the Ptec.
DAQ at 1944:46UTC.

Along with lowering the wide array to the 'low' position the single-films were
installed and reconnected (now that the spraying is done). Data began at
1903:22LT.

23may07  15:15-16:10  tower climbed

hftower turned off
Ptec#6 removed,  Ptec#7 installed
Removed 3-axis hotfilms and covered connector.

23may07 16:30-18:30  hfarray stopped

Lowered array to remove hot-films.
Ptec #1 removed,  Ptec#11 installed.
We decided to raise the array back to mid position and keep the central tower
at that height because the winds look good for overnight into tomorrow.
On Friday we'll need to restart and lower the array to 'low.'

data_dump of ~ 10 hfarray...dat and hftower...dat files on Aster resulting
in finding values 'in-range' for all films (SFs 4,5,2,7,8 on array and TFs
1,4,3 on tower). The voltages are good.

The 'zeroed' channel (#6) has values over 3V.

18may07  hftower work

15:45 - 16:15 Tower climbed

Removed ptec#7.
Took data system off tower given ongoing
flakey behaviour to change out the interface panel
(its corrosion may contribute to the flaky behavior
after 'water flooding' earlier).
Did not swap viper.   Cleaned power/analog panel.

17:30 - 18:00 Tower climbed
Reinstalled box.   Ptec #6 (which had Gordon's full r/w fsck so
it should be aok).
Bootup had exactly same behavior as before.   Didn't work, no
pocketec, on hard bootup but subsequent 'adn' and 'aup' did
mount it properly and things worked fine.   Thus i don't expect
this run to stay up either.

17may07 16:15 16:35

hftower media swap
The pocketec storage was down.
Powered down (Greg shut down daq)
removed #11
install #6	later ran copy_usbdisk.sh (5/15,18:30 - 5/17,5:00
Power up
no-go on pocketec from hard bootup, not mounted.
adn
aup
and everything worked.
Perhaps adam startup sequence needs additional delay to allow pocketec to init?

The hfarray data system came up but the Pocketec did not during the array 
lowering process (from high to middle position). hfarray was halted at 0909LT
after operating from 1219LT 11 May until 0909LT 17 May (continuous, all full
sized files).

Came online of its own at 1403LT 17 May after array was repositioned in the
middle configuration. 

15may07  11:10 - 11:45  On-Tower

Hftower adam

Swapped pocketec
removed #6
installed #11
swapped cable also

Pocketec had died last night
after working ok for ~4-hours.
Physically the connection appeared OK, and it was powered up,
however, the data system did not have-it mounted.
A 'adn' took down the daq 'normally', and an 'aup' brought
everything back up normally after installing #11.   At that time
new data was being written normally.

Continues to operate as of 1500LT.

As part of the ongoing analysis of the data-system/hftower/3-d hotfilm problems
we are planning to change out the 'good' Pocketecs for the 'flaky' pocketecs
today. As a result we have cleared the data off of them.

We cleared PTEC#11 of 2 partial files it had gathered from 11 May before
flaking.

We cleared PTEC#2 of ~ 24hrs of files it had gathered from 12-13 May before
flaking.

The 3-d hot-film system has been down since 13 May 1612LT. A large amount of 
time has been spent since ~10LT this morning in trying to resurrect the 
measurements from the three 3-d hotfilms. 

John finally got it going by inserting a USB stick in place of the Pocketec.
This seemed to clear out the system somehow, then allowing the Pocketec
to mount.

Hotfilm is up as of 1650LT 14 May.

The winds swung into the south at ~1630LT (within the +-30degree band of primary
interest).

hftower TF4 switches in miniCTA adjusted to proper position 1445LT.
hftower usbdisk replaced. After a complex aup/adn, reboot sequence it is up
   as of 1530LT.

hftower installed and flakey (incomplete pings 10-40% data loss) - apparently
this is how it behaved earlier, was troubleshot then and deemed sufficient. 
Sensors ordered TF1 bottom (6m), TF4 (spare, 10m) and TF3 (14m). 
Switches not correct  for TF4. Data not being taken yet.

hfarray data system install completed. Order from west to east is SF4, SF5, SF2
replacing SF6), SF7, SF8. All BNCs sealed. Data system
functional. Data started at 1218LT.

Installed single hot-films on array 1625-1745LT.
Order SF4, SF5, SF2 (replacing SF6), SF7, SF8 west to east (as before)
   - SF2 switches confirmed correct

3-d hotfilms installed on the 30m tower between 1245 and 1345LT. Data system
hard-booted with a new Pocketec in place of the one that had not been working
(we are suspicious that it was the soaking it endured during the break). It
was placed in the USB2 plug. Chris earlier had noted some condensation in the 
data system box (from the Pocketec cloth cover that had absorbed much water)
so we air dried it for a bit.

TF4 - the spare 3-d hotfilm, now installed at 10 m in place of TF2, had its 
switches adjusted to the proper configuration. 

hftower functional as of 1530LT with 130Meg files per half hour being 
written to the Pocketec. 

ssh to root@hftower is slow (be patient). 

On removal, the hot-film probes were:
ha
3t: 4
4t: 5
5t: 2 (replacing 6 that broke and is being repaired)
6t: 7
7t: 8
vt
6m: 1
10m: 2 (but at least one wire is now broken and being repaired)
14m: 3

Since bridge settings are probe-specific, the probes should be reinstalled 
at the beginning of Phase II in this order.  Note that 10m.vt will now be
the spare (4) and will require changing the bridges settings.

These probes are in boxes on the shelf in the computer room of the base, to the
left of the printer.

Also note that some method of protecting the connector joint on at least the
triple probes from rain should be devised.  (This was perhaps the reason
for an apparently dead signal from the 10m probe.)

To avoid filling up the Pocketec with useless files, I stopped data collection
with adn.

From about 1430-1500 removed the triple probes in order: 6m (1), 10m (2), and
14m (3).  Both 6m and 10m were sliders, so sonic data probably are okay.
14m was a tilt-up, that was up from ~1453-1457.

I had enabled the 6th A/D channel on hfarray either last night or the night
before.  It is shorted with a shorting plug.  As of just now, we are getting
"nan" (which seems to be what the software reports for 0) in the 6th
column -- exactly where it should be.

The single hot films for the east, west, and center sections where reinstalled after the saturday tree spraying.

After the tree spraying Saturday morning, the triple wire Hot Films where reinstalled on the 30 meter tower starting the lowest and working upward.

P.S.
I  this was the time when we installed the new data acquistion code via
the spare Pocketec to avoid the channel slip problem.

To avoid damage from the spraying last night, we removed all hotfilms yesterday
afternoon.

~1430-1500 removed triple probes from 30m tower in the order bottom to top.
~1530-1700 removed single probes from horiz. array in the order west to east.

Gordon may have been playing with the adams hftower and hfarray during this
time, so the data during these removals may not have been recorded.

We also bagged some of the stuff on the ground and turned the guywire lights on.
Don and I drove by at night and confirmed that the lights are quite visible.

We also pulled the Pocketecs out of hftower and hfarray about 1800.  These have
now been copied to the trailer USB disks.

The probe we removed from the central tower yesterday (SF6) did appear to be 
broken when I poked at it this morning.  Tom is taking it to Boulder to be
sent for repair.

We installed a new probe (SF2) at about 1140, but had the wrong DIP switch
settings.  Settings were corrected 1239 and the signal looks okay.

**** WARNING ****
The probe appeared to be slightly rotated, so I reoriented it more vertically.
I estimate that it was rotated ~5 degrees, with the bottom more Easterly than
the top.  I realize that the calibration procedure may have to be altered.

P.S. 4/10/07:
Under microscope inspection of SF6, Greg and SteveS found a minute separation
between one end of the hot film and its support.

~1147: tested hotfilm channels by shorting each input sequentially for 10s each.
The order was: (frontpanel input channel) 0,1,2,3,4.

I'll look at the data to see if this ended up being correct.

Note that the central hotfilm (ch 2) was reporting (incorrect, but fluctuating)
voltages at about 2V.

[THE RESULTS!!! (in GMT, using: 
data_dump -d 6 -s 11 -p hfarray_20070405_183000.dat | more)]

This file just happened to have a data dropout example, that shifted the
data from being reported correctly (broken hotfilm voltage of 2.6V was on
[0-based] ch2), to being shifted by 2 columns.

The shorts of frontpanel inputs 0,1,2,3,4 appeared in data columns 3,4,0,1,2
(during the period 18:48:37-18:50:18).  Note that the shorts appear as "nan"
and that disconnected inputs float to >3V.


-- loss of data.  Note the negative deltaT! --
2007 04 05 18:30:00.2490  0.0005      20      1.884      1.757      2.638      1.814      1.878
2007 04 05 18:30:00.2495  0.0005      20      1.884      1.755      2.635      1.815      1.878
2007 04 05 18:30:00.1999 -0.0496      20      1.885      1.754      2.632      1.815      1.878
2007 04 05 18:30:00.2004  0.0005      20      1.883      1.753      2.631      1.816      1.846
2007 04 05 18:30:00.2009  0.0005      20      1.846      1.846      1.846      1.846      1.846
-- data recovers, shifted by 2 columns [note 2.6V] --
2007 04 05 18:30:00.2479  0.0005      20      1.669      1.669      1.669      1.669      1.669
2007 04 05 18:30:00.2484  0.0005      20      1.669      1.669      1.884       2.57       2.57
2007 04 05 18:30:00.2489  0.0005      20       2.57       2.57       2.57       2.57       2.57
2007 04 05 18:30:00.2494  0.0005      20       2.57       2.57       2.57       2.57       2.57
2007 04 05 18:30:00.2499  0.0005      20       2.57       2.57       2.57       2.57       2.57
2007 04 05 18:30:00.2504  0.0005      20       2.57       2.57       2.57       2.57       2.57
2007 04 05 18:30:00.2514   0.001      20       2.63      1.817      1.879      1.883       1.75
2007 04 05 18:30:00.2519  0.0005      20      2.627      1.817      1.879      1.883       1.75
2007 04 05 18:30:00.2524  0.0005      20      2.627      1.817       1.88      1.883       1.75

Here is a sample data dump from hfarray.  It appears that the last channel 
may be bad.  It is highter than the rest and more constant with  time.

[After talking with Gordon, this Viper was rebooted ~1000?? and the channel 
order changed!  Now that we've replaced the probe, it isn't easy to dump these
data.] 

ORIGINAL:

2007 04 05 00:39:48.4527  0.0005      20      1.691      1.886      1.771      1.629      2.572
2007 04 05 00:39:48.4532  0.0005      20      1.692      1.885       1.77      1.628      2.572
2007 04 05 00:39:48.4537  0.0005      20      1.691      1.885       1.77      1.629      2.572
2007 04 05 00:39:48.4542  0.0005      20      1.691      1.885       1.77      1.629      2.572
2007 04 05 00:39:48.4547  0.0005      20      1.691      1.885      1.771      1.628      2.572
2007 04 05 00:39:48.4552  0.0005      20      1.691      1.884      1.771      1.628      2.572
2007 04 05 00:39:48.4557  0.0005      20      1.692      1.885      1.771      1.628      2.573
2007 04 05 00:39:48.4562  0.0005      20      1.692      1.885      1.772      1.627      2.572
2007 04 05 00:39:48.4567  0.0005      20      1.692      1.884      1.771      1.627      2.572
2007 04 05 00:39:48.4572  0.0005      20      1.693      1.883      1.772      1.627      2.572
2007 04 05 00:39:48.4577  0.0005      20      1.694      1.885      1.772      1.628      2.572
2007 04 05 00:39:48.4582  0.0005      20      1.694      1.884      1.772      1.628      2.572
2007 04 05 00:39:48.4587  0.0005      20      1.696      1.884      1.772      1.627      2.572
2007 04 05 00:39:48.4592  0.0005      20      1.697      1.884      1.773      1.627      2.572
2007 04 05 00:39:48.4597  0.0005      20      1.698      1.884      1.773      1.627      2.572
2007 04 05 00:39:48.4602  0.0005      20        1.7      1.884      1.773      1.627      2.573
2007 04 05 00:39:48.4607  0.0005      20      1.702      1.884      1.773      1.626      2.573
2007 04 05 00:39:48.4612  0.0005      20      1.704      1.884      1.772      1.634      2.572
2007 04 05 00:39:48.4617  0.0005      20      1.705      1.884      1.774      1.627      2.572
2007 04 05 00:39:48.4622  0.0005      20      1.708      1.884      1.773      1.627      2.572
2007 04 05 00:39:48.4627  0.0005      20       1.71      1.883      1.773      1.626      2.572
2007 04 05 00:39:48.4632  0.0005      20      1.713      1.884      1.774      1.627      2.573
2007 04 05 00:39:48.4637  0.0005      20      1.715      1.884      1.773      1.626      2.573
2007 04 05 00:39:48.4642  0.0005      20      1.718      1.884      1.775      1.627      2.573
2007 04 05 00:39:48.4647  0.0005      20       1.72      1.884      1.774      1.627      2.572
2007 04 05 00:39:48.4652  0.0005      20      1.723      1.885      1.774      1.627      2.572
2007 04 05 00:39:48.4657  0.0005      20      1.725      1.884      1.774      1.627      2.573
2007 04 05 00:39:48.4662  0.0005      20      1.726      1.884      1.774      1.626      2.572
2007 04 05 00:39:48.4667  0.0005      20      1.729      1.885      1.774      1.626      2.572
2007 04 05 00:39:48.4672  0.0005      20       1.73      1.884      1.775      1.626      2.572
2007 04 05 00:39:48.4677  0.0005      20      1.731      1.884      1.774      1.627      2.572
2007 04 05 00:39:48.4682  0.0005      20      1.733      1.884      1.775      1.626      2.572
2007 04 05 00:39:48.4687  0.0005      20      1.734      1.884      1.775      1.626      2.573
2007 04 05 00:39:48.4692  0.0005      20      1.734      1.884      1.775      1.626      2.572
2007 04 05 00:39:48.4697  0.0005      20      1.735      1.884      1.776      1.626      2.572
2007 04 05 00:39:48.4702  0.0005      20      1.736      1.884      1.776      1.626      2.572
2007 04 05 00:39:48.4707  0.0005      20      1.736      1.884      1.776      1.627      2.572
2007 04 05 00:39:48.4712  0.0005      20      1.737      1.884      1.776      1.626      2.573
2007 04 05 00:39:48.4717  0.0005      20      1.737      1.884      1.776      1.625      2.572
2007 04 05 00:39:48.4722  0.0005      20      1.737      1.884      1.776      1.626      2.572
2007 04 05 00:39:48.4727  0.0005      20      1.738      1.884      1.776      1.626      2.573
2007 04 05 00:39:48.4732  0.0005      20      1.738      1.884      1.777      1.625      2.572
2007 04 05 00:39:48.4737  0.0005      20      1.738      1.884      1.776      1.625      2.573
2007 04 05 00:39:48.4742  0.0005      20      1.738      1.885      1.778      1.625      2.572
2007 04 05 00:39:48.4747  0.0005      20      1.739      1.885      1.778      1.626      2.572
2007 04 05 00:39:48.4752  0.0005      20      1.739      1.886      1.778      1.625      2.572
2007 04 05 00:39:48.4757  0.0005      20      1.739      1.886      1.778      1.625      2.572
2007 04 05 00:39:48.4762  0.0005      20       1.74      1.886      1.777      1.625      2.573
2007 04 05 00:39:48.4767  0.0005      20      1.741      1.887      1.779      1.625      2.572
2007 04 05 00:39:48.4772  0.0005      20      1.742      1.888      1.779      1.625      2.573
2007 04 05 00:39:48.4777  0.0005      20      1.744      1.888      1.779      1.625      2.572
2007 04 05 00:39:48.4782  0.0005      20      1.745      1.889       1.78      1.626      2.572
2007 04 05 00:39:48.4787  0.0005      20      1.746      1.889      1.779      1.625      2.572
2007 04 05 00:39:48.4792  0.0005      20      1.747       1.89      1.781      1.625      2.572
2007 04 05 00:39:48.4797  0.0005      20      1.747       1.89      1.779      1.625      2.573
2007 04 05 00:39:48.4802  0.0005      20      1.749      1.891      1.781      1.625      2.572
2007 04 05 00:39:48.4807  0.0005      20       1.75      1.891       1.78      1.624      2.572
2007 04 05 00:39:48.4812  0.0005      20      1.751      1.891      1.781      1.624      2.572

[AFTER restart]

2007 04 05 18:30:00.0019  0.0005      20      1.767      1.689      2.568       1.76      1.911
2007 04 05 18:30:00.0024  0.0005      20      1.766      1.689       2.57      1.761      1.912
2007 04 05 18:30:00.0029  0.0005      20      1.765      1.689      2.572      1.761      1.912
2007 04 05 18:30:00.0034  0.0005      20      1.764      1.689      2.573      1.762      1.913
2007 04 05 18:30:00.0039  0.0005      20      1.765      1.689      2.572      1.762      1.914
2007 04 05 18:30:00.0044  0.0005      20      1.764      1.689      2.572      1.763      1.914
2007 04 05 18:30:00.0049  0.0005      20      1.764      1.689      2.572      1.764      1.915
2007 04 05 18:30:00.0054  0.0005      20      1.763      1.689      2.573      1.763      1.916
2007 04 05 18:30:00.0059  0.0005      20      1.764      1.689       2.57      1.763      1.915
2007 04 05 18:30:00.0064  0.0005      20      1.764      1.689      2.571      1.764      1.916
2007 04 05 18:30:00.0069  0.0005      20      1.765      1.689      2.571      1.764      1.917
2007 04 05 18:30:00.0074  0.0005      20      1.766      1.689      2.571      1.765      1.917
2007 04 05 18:30:00.0079  0.0005      20      1.766      1.689       2.57      1.765      1.917
2007 04 05 18:30:00.0084  0.0005      20      1.766      1.689       2.57      1.766      1.918
2007 04 05 18:30:00.0089  0.0005      20      1.768      1.689       2.57      1.765      1.918
2

Use the data_dump commands from aster laptop for the array or tower:

data_dump -d 5 -s 11 -p sock:hftower

data_dump -d 6 -s 11 -p sock:hfarray

2000 Hz data will fly by, do ctrl-C to quit.

Hot films on horizontal array started around 12:40 PDT, April 1.

Installed hot film sensors on vertical tower.  Started hftower adam at
12:53.

Deployed hot films late Saturday afternoon, 3/25 07:07 GMT.

10/11/07, TWH:  This time makes no sense.  It would be midnight PDT!
                Looking at the hot film data file, this time should be
                00:07:07 GMT

Stopped data collection 3/26 17:52 GMT.
ssh root@hftower
password: ...

root@hftower root# adn
Stopping adam ...
kill -HUP 875 876 878 879 880 882 883 ...
kill -HUP 876 ...
kill -HUP 876 ...
done
Unmounting /var/tmp/usbdisk ... done

Removed hot films and retrieved pocketec, ~11:10 - 11:30 PDT