CuPIDO: LOG Messages: 47 Entries..

AJ and I went to station 3 to get it back on the air. The station was running
and data was being recorded on the USB. The logger communicaiton was down,here was also a bad cable from the rain gage.
  We were able to get all of the problems fixed except for GOES. The weather
turned bad so we had to get out of there. On the ride back we had to wait at
a wash due to flooding.
We were at the site from ~11:00 to 12:30. Any rain tips in this period
should be ignored.

details:
rain - the rain gage cable was cut. This may have been due to the cable
running across some sharp rocks on the ground and somebody may have stepped on
the cable.
logger - the logger output was working fine. The problem was in port 5. It was
a mystery. I tried reversing the TRH and Logger ports. Both sensors worked fine.
When the logger was reconnected to port 5 there was no data transfer. The
problem is either the esrial interface panel or port 5 on the Emerald. In the
end we changed the config file and moved the logger to port 8.
GOES - due to the weather we did not have any time to determine the problem
with GOES. The response from the GOES command gave one the impression it was
communicating but none of the parameters looked correct.
 

A.J.I. and Steve S. arrived ~11:40 and left at 12:30

Activities:
- were limited because of lightning
- Took TRIME measurements and gravimetric samples

A.J.I. and Steve S. at around 13:00, but were unable to do much because of 
lightning.

Actvities:
- Replaced desiccant in the Rnet
- Swapped usb to look for lost data on Ausgust 12th, 2006.

A.J.I. and Steve S. arrived at 13:09 and left 14:05

Activities:
- Mast was lowered from 13:09-13:28
- Cleaned kh20 at ~13:15, voltage went from .14 to .22
- cleaned radiometers at 13:50
- TRIME measurements taken
- All false tips from ~13:55-13:59

A.J.I. and Steve S. arrived at 15:40 and left at 15:55

Activities:
- Rnet was clean
- 1 test tip at 15:44

A.J.I. and Steve S. arrived at 14:50 and left at 15:10

Activities:
- Cleaned Rnet at 14:51, appeared clean
- 1 test tip at 14:53

A.J.I. and Steve S. arrived 20:15 and left at ~20:45

Activities:
- went to investigate hightened Tsfc compared Tsfc.pyrg, determined that the 
sensor was facing a huge dead spot of sand while the rest of area is pretty 
well vegetated. So its seems ok.
- Downloaded program from datalogger which shut operations of logger from 13:21-
13:40
- Re-routed electric fence wire just to the top wire, it appears as though the 
bottom wire was shorted due to vegetation growth.

A.J.I. and Steve S. arrived at 11:32 and left at ~12:00

Activities:
- Changed battery for electric fence
- Replaced desiccant on the Rnet, appeared clean
- 1 test tip at 11:36

A.J.I. and Steve S. arrived at 1:28 and left at 2:15

Activities:
- Mast down from 1:40-1:53
- Krypton cleaned at 1:45, voltage was at .4
- 1 test tip at 1:56
- Radiometers cleaned at 2:01
- TRIME and gravimetric samples

Gave a tour of site 10 to Constance Brown (Indiana Univ.), Karyn Sawyer, and
Jose Meitin yseterday.  Grabbed the USB data as long as I was there, but no
other service.   Everything seemed okay.  On site about 11:00-11:40.

A.J.I. and Steve O. arrived at 12:29 MST and left at 12:57 MST

Actvities:
- The cable connector going from the solar panel to the charge controller was 
charred on the ground side. Steve simply cleaned the connection and that 
established continiuity.
- The power was off from 12:29-12:45
- Cleaned Rnet at 12:50 MST
- One test tip was performed on the rain guage at 12:53

A.J.I and Steve O. arrived at the site at 8:47 MST and left at 9:14

Activities:
- Cleaned Rnet at 8:48 MST
- 1 test tip done at 8:52 MST
- Removed Rnet dome to remove condensation buildup, screw were in through the 
bottom, put them through the top. This was done during 9:00-9:11

A.J.I. and Steve O. arrived at 2:50, and left at 3:30

Actvities:
- Cleaned radiometers at 2:50
- 1 test tip of rain gauge at 3:02
- Steve O. on tower from 3:02-3:13
- Cleaned Kh2o at 3:07, voltage went from .15 to .23
- TRIME measurements are as follows taken at 3:20:
1st: 11.3%, TDR=53
2nd: 11.3%, TDR=49 (ten minutes later)
3rd: 11.2%, TDR=50
4th: 11.2%, TDR=50
- Gravimetric samples taken at 3:25 (Note: sample 3-6 not good for bulk density.

A.J.I. and Steve O. arrived at 1:03, and left at 1:11

Activities:
- Cleaned Rnet at 1:05
- 1 test tip at 1:07

A.J.I. and Steve O. arrived at 10:27 MST, and left at 10:48

Activities:
- Replace broken rain gauge at 10:28 MST
- 10 Test tips at 10:30
- 1 test tip with water at 10:31
- No need to replace the TRH fan, it sounds good
- Cleaned Rnet at 10:32 
- Grounded the datalogger Box

A.J.I. and Steve O. arrived at 8:40 MST and left at 9:45 MST:

Activities:
- Mast lowered at 8:47-9:09
- Cleaned Kh2o at 9:00, voltage went from .02 to .04 (first time), and then up 
to .05 after the second cleaning, finished cleaning at 9:03 MST.
- TRIME measurement probe in at 9:15, then measuremnt done at ~9:20 with 
readings as follows:
1st: 6.7%, TDR=56
2nd: 7.6%, TDR=? (After Stomp of soil)
3rd: 7.7%, TDR=55
4th: 7.6%, TDR=55
- Gravimetric measurements at 9:25 MST
- Cleaned radiometers at 9:25-9:32

A.J.I. and Steve O. arrived at 2:01 MST at station 8 on August 11,2006

activities:
- Changed Rnet nighttime calibration coefficient from -10.84 to 10.84 at 
2:02 MST 

A.J.I. and Steve Oncley arrived at station 1 at 10:55 on August 11,2006

Activities:
-Datalogger hooked up at 11:02 MST
-Hooked up TRH at 11:00 MST, then dismounted at 11:47 check peculiar fan 
sound, put back on at 12:19, but fan still sounds wimpy.
-Rnet was hooked up at 11:34, was wired backwards for a few minutes before that,
the Rnet coefficient was changed from 9.66 to 9.00 and 11.81 to 11.18 for new 
sensor at 12:24 
-Rain gauge has failed from electric shock

visit to site 9 from about 0120-0140Z

Activities:
- swap usb stick
- install new software 
- fix electric fence (trashed again!)

visited site 8 from about 0000-0050Z

Activities:
- swap usb
- update software (including change of config /ttyS5 -> /ttyS8
- ~0010: change sign of nighttime Rnet coefficient. Apparently failed to do 
  this.  Will try again tomorrow. 
- ~0035: Replaced Rnet upper dome. (This one was abraded on the top when I installed
  it 2 days ago, but I wasn't able to fix it then.)  Unplugged logger during 
  this operation.
- orientation for AJ

Arrived Aug 9, 13:15,  Oncley, Maclean

This station has not been reporting since last night.

System OK, sensors OK, archive disk OK and complete
but statsproc processing not running.

ck_goes could not talk to goes transmitter - timeouts on wakeup.

Swapped in SE 1200 (didn't get serial number. This unit had
been at site 1 earlier in the project).

Installed new software. Swapped USB disk.

13:35 Gravimetric soil samples taken.

13:46 Trime:

vol: 26.7%  26.8% 29.9%
TDR   39    39     40

13:50 cleaned radiometers, all OK.


13:58 mast down to clean krypton

krypton voltage increased from 0.1V to 0.25 V on cleaning.

14:08 mast up.

Left 14:17

Arrived Aug 9 10:17 MST,  Oncley, Maclean

Station in bad shape after last nights storm.

Net radiometer head blown out from apparent lightning strike.

Blown fuse on charge controller board.  Swapped in fuse from
second charge circuit - immediately blew again.

Data box #9 interface panels show electrical arc-ing damage on 
ttyS3 (goes) and ttyS5 (logger) ports.  Putting power to this box 
blows fuses on the charge controller.


Removed box 9, swapped in spare box 8.

Barometer OK.  Prop vane OK.

TRH sent bad data for a short period: 
TRH08  with a T=39.94, RH=1.05, V=0 (wrong)
then reported the values typical of a A2D lockup: 
T=210, RH=257

Eventually would report the following after being plugged in:

TRH System Initialization Complete
A/D not responding

and then nothing.  Removed it for repair.

Installed 7.5 A fuse on load circuit of charge controller.

Serial connector on logger box looks fried.  Voltmeter
reported shorts between ground and most other pins.
Removed logger for repair.

GOES transceiver not responding.  SE 110, SN 1007.

Installed GOES SE 120, SN 3515.

Someone helped themselves to the step ladder which was
left beside the road while Steve drove
to Oracle for fuses and Gordon was at the site.

Station status:  station transmitting and archiving,
  prop vane OK, barometer OK.
  no T, RH, Vbatt, Rnet or Rainr data while sensors are repaired.

Left 12:45.

Later found that the usbdisk from this station is not
readable.  The light on the drive does not come on
when it is inserted. The last data from the last
disk swap was Aug 4 22:33, so it looks like data
about 5 days of archive data has been lost from this station.

Tonight was a huge storm over Tucson, with continual lightning from about 
8:00-10:30.  Apparently, another is on the north side of Mt Lemmon, since
site 2 reported 15mm of rain and site 1 19mm of rain before each died.

Site 2 died about 8:30 and site 1 died about 9:00.  Clearly, we are going to
have some serious station visits tomorrow. :(

Visited site for routine maintenance.  Arrived 13;35,  
Oncley, Maclean, Susanne Grossman-Clarke from ASU.

Net radiometer dome is punctured and cracked, looks like a bird peck.

Did not have small phillips to replace dome, so swapped
in spare unit:  Q94125,  +8.75, -10.84.  Entered coefs
into CR10.


Downloaded new software, swapped usb.

Test rain tip at 13:58.

Photos.


Left 14:00

Routine site visit. Arrived 10:45 MST.  Oncley, Maclean. 

Visitor: Susanne Grossman-Clarke from ASU.

Installed new software, but ntpq was not installed due to script
goofup.

Top wire of electric fence was largely off the posts - cows have
frequented this site, but no damage other than a tilted rain gauge.

Guyed corner posts of fence.  Swapped e-fence battery, voltage was
11.7V.   New battery: 12.6V.


Cleaned net radiometer. Small dimple in dome.

2 test tips of rain gauge.

Swapped usbdisk.

This is data box #12, goes model 120.

Left 11:15.

Just for fun (and to show me for the first time), Gordon and I did a standard
visit to site 6 from about 11:20-11:45 this morning.

Activities:
- swapped usb
- installed new software
- restarted data system -- noted no logger data [stupid me!]
- manually changed logger /ttys5 to /ttys6 in /root/.../cupido.xml
- restarted data system -- now all okay
- poured 1 tip of water into gauge at 11:40
- inspected Rnet domes and decided not to clean
- attached a ground strap to a T-stake at the logger
- took photos

I did a solo visit to 5 from about 0840-0940 this morning to fix the TRH.
It ended up being just a blown fuse on /ttyS2, though I found blown fuses also
on (unconnected) /ttyS9 and /ttyS10 (which I was using to test).  I replaced
all 3 of these fuses.  Since the fuse for /ttyS2 is close to the internal
ethernet connector (which is grounded), I had to shut down power while doing
this.

I went ahead and swapped the TRH (704, I think), with the spare (007), since
704 had one outage earlier in the program and may have a bad fan which caused
the fuse to blow in the first place.  We'll check it in the base now that we're
back.

Also, I swapped the 4W solar panel on the rad stand with a 10W, so our logger
power should start getting better.

I just came back from a solo visit to 10 from ~1700-1750 (there was time both
before and after hauling stuff up and down the hill).

Actions:
- check fence -- seemed fine
- take photos
- swap USB
- upgrade software
- check "ntpq -p" -- gave -52 offset, 84 jitter for *GPS
- clean radiometers
- test rain gauge
- klean krypton (mast down 1725, up 1734)
- take TRIME and gravimetric sample
- tighten CR10 screws

Just visited site 5 from 10:08-10:43 to diagnose rad stand power.
Found that this is using a small solar panel on the rad stand, but didn't
have the spare with us, so left it unchanged.  (Should swap panels on next
visit.)

We HAD just purchased a new sealed battery (at the local BatteriesPlus).  Since
it was charged, the voltage went up significantly (old ~11.3, new ~12.8V).
It also seems to be being charged.  Thus we made no other changes.

Our fallback option was to cable this stand with copper, since we now have
spare data cables (that probably would reach) and the change would only
require flipping one switch in the logger box.

We'll keep an eye on this.

P.S. while there, also labeled the Pocketec drive so it will work automagically
with Gordon's code to download.

Visit to 1, ~15:30-16:00 MST, 4 Aug.

Purpose: Show Steve the site

Actions:
- repair electric fence.  cows got inside, but no station damage
- exchanged usb drive
- upgrade software
- clean radiometer
- test rain gauge
- take photos

Visit to 3, ~11:20-12:25 MST, 4 Aug.

Purpose: [first visit in a while, due to road closure] fix intermittent 
Rpile.out data gaps

Actions:
- repair electric fence.  appeared that a horse stepped through site, but
  no station damage
- exchanged usb drive
- upgrade software
- fix Rlw connection in logger [blue wire to H1 in block 3 on AM416 was 
  pretty loose]
- clean radiometers
- test rain gauge
- soil samples
- klean krypton [0.08 -> 0.2V] (requires mast down + ladder)
- take photos

Notes:
- box 4
- GOES 120

Visit to 5, ~08:50-09:40 MST, 4 Aug.

Purpose: [first visit in a while, due to road closure] to download data

Actions:
- exchanged pocketec drives with spare I brought from Boulder
- upgrade software
- test rain gauge
- check electric fence (okay)
- clean radiometers
- klean krypton [0.7 -> 1.2V]
- soil samples
- take photos

Notes:
- box 6
- GOES 120

Visited site 7 last night, just at sunset ~19:50-20:30 (avoid the rebar spikes!!)

Purpose: determine why no GOES data.

Actions:
- /var/log/messages showed "SOH (start-of-header)" error messages
- swapped USB stick
- upgraded software (now has GOES status messages)
- executed "ck goes" -- still SOH errors
- cycled power on GOES by disconnecting cable at battery box
- executed "ck goes" -- now okay!
- restarted data system
- checked all values (including wind direction) okay
- found new way out: following Catalina (north) => Catalina Shadow 

Notes:
- box 13
- GOES 110
- surge supressor tied to ground stake

Battery voltage plot of site 4 has indicated problems in the
charging circuit for the past two days. 

Arrived at site, 10:30 MST

Cleaned net radiometer.

Battery voltage 12.4 V
Measured 18 V from solar panel at charge controller.

Eventually pulled controller out of box:

Pro-Star 30, has a digital display:

Batt : 12.5 V
Array   0.0 A
Load    0.3 A

Found a loose (blackened) ground connection to charger board,
and tightened screw:

Batt  12.9 V
Array  3.0 A
Load   0.3 A

Swapped usb disk.

Installed ntpq program.  Shows offset 4.376 msec, jitter 88.9 msec
(GPS without PPS)

Installed latest nidas code.

Other info:  GOES 110, not grounded.

Data box #14

Shored up electric fence.

Tested rain gauge: 2 tips

Left 12:00

Arrived site 10, 1420 MST, July 29

Upper fence wire off SE stake, repaired; fence tested okay
Swapped USB disk, cleaned rain gauge tube, took soil samples, cleaned
radiometers

Departed 1540

Arrived 17:25.

Primary purpose: to do a soil sample.

Also installed new nidas software, and swapped usb disk.

Cleaned radiometer domes.

Left 17:50

Arrived 15;40

GOES data recovery at this site has been bad since 02:00 MST, Jul 25.

statsproc running.

This site has a GOES 1200, software installed 10/14/2004 16:08:26.
Also has a surge arrestor.

Installed new nidas software.

Swapped usb disk.

GOES tranmission power measured: 9.8 Watts

Swapped in model 110.  Tranmission power 12 Watts.

Plan to install 1200 at a site nearer Tucson, like 7.

Cleaned Rnet.

Left 16:53 MST

Logger variables (Rnet, rain, Vbatt.rad) have been out since 00:00 MST
7/28 at this site.

Arrived 12:30 MST

More than 1 inch of standing water in rain gauge.
Drain tube was plugged with small piece of blue plastic.
This may have effected all rain data from this site up
to this date.

Logger getting power (14.5 V), multiplexer ticking.
Some water inside logger.  Taped over a bulkhead amp
connector that had open holes, with no pins.

Determined that port 0 on the emerald serial card (ttyS5)
was bad. Switched logger to ttyS6.

Considered grounding the logger, but decided
it would create a ground loop between the data system
ground and the logger ground.

Rnet clean - did not wipe.

Swapped USB disk, installed new nidas software (has
GOES status).

Rain test, 3 tips at 13:15.

Left 13:35

Site visit to station 3 on July 26

Arrived 9:56 am MST
Departed 11:14 am MST

Fixed electric fence grounding on prickly pear

Swapped USB disk

Mast down 10:46 am MST
Mast up   11:02 am MST

Arrived 15:08 MST

Krypton voltage at 615 mV. Decided a cleaning wasn't necessary.

Radiometers clean and net radiometer level.

Swapped usbdisk.

Taped lid of GOES cooler.

Inspected and tipped rain gauge.

Sampled soil, see next entry.

Left  15:55 MST

Arrived 14:20 MST.

Net radiometer was level and clean.

Inspected rain-gauge by raising cylinder, and 
doing a tip, saw tip in logger data.

Top wire of electric fence had been knocked off a post.
Straightened out wire and posts. Hand-tested electric
fence.

Swapped usbdisk.

Left 14:40 MST

Arrived 10:50 MST

Cover was off the electric fencer. Weighted it with rock. Cows nearby.

Taped lid on GOES cooler.

11:15 lowered mast, cleaned krypton (see next entry), raised mast at
11:23 MST.

Radiometers looked clean.  Re-leveled Rnet.

Swapped USB disk.

Attempted to take soil sample - soil very loose, only succeeded
in 0-3cm sample, in tub #2.

Trime unit was discharged.  Have to some back.

Left 12:30 MST

Station 4 – Web plots down – rebel cattle incursion and destructive activities

Purpose: With the web plots down we sought to bring the station back up

Time: 1445pm arrival 1745 departure.

Upon inspection we noted that cattle had wrought havoc with the electric fence 
(taking down about 1/5th of the fencing, had knocked over the GOES transmitter 
cooler, and had chewed through the green cable to the logger and the GOES, 
as well as the gray cable to the precip and net rad sensors! In addition, 
with exposed wires and rains the previous day, the fuses had blown to both 
GOES and the logger. John took pictures.

The electric fence was reset on the posts and tightened successfully. Initially
it did not work because the fence wire was touching cactus with good contact. 
After eliminating that contact the fence began working rather well and will 
hopefully prevent future incursions by the rebel cattle outlaws of Lone Hill.

The only data lost was that from the logger-based rain gauge and net radiation 
sensors. We should be able to discern whether the weather or the cattle hit 
the system first by looking at the locally archived data.

Fuses were replaced for ttyS5 and ttyS3 (GOES in port 13)

Cables were replaced with spares or mended. The GOES system was moved out from 
under the solar panel and duct taped to prevent future water entry 
(… duct tape on the remaining coolers at other stations is advised …). 
This was done because the replacement green cable was not long enough to reach 
the previous position and the spare cabling (left over gray cabling from the 
hot-film at T-REX did not have the CTS wire).

The Net Rad sensors was changed out completed with a spare (the cable to the 
earlier NRad will need to be fixed).

Q94125 – Cal values entered in logger from plate on sensor – 
positive 8.75, neg 10.84.

The old hot-film special cable was used to power the logger (two Bulgins were 
taped at their junction to reach).

The precip cable was re-worked to good cable (it had been chewed near the gauge and had enough length to keep using).

On data drive exchange the usb-drive did not properly unmount, or upon reboot 
it give errors in mount. However, the data system and data archiving was fine 
despite these errors. A future visit might seek to fix this.

16Jul06 11:30 - 16:00

Rain Gauge: checked and had slight level adjustment....very minor.

Electric Fence Checked:
	Battery is at 13.1 now so moved PV charging it back onto the
	main station array.
	The fencer is working but the zap is weak....probably because
	this 'soil' has many rocks and not highly conductive plus very
	Finally saw a cow 'in the area' roughly 1-2 miles away!

GOES:	PVC elbows replaced cable feedthroughs to provide a small amount
	of ventillation.
	Checked SigStr: 11W; good
	With the wattmeter installed the nesdis quality parameters were
	good, but disconnected it dropped back to 38ish dbm: too low.

	Removed surge supressor from inline to antenna.  After waiting
	the results from nesdis were back up to 47ish: good.
	No spares at this point for the antenna surge arrestor, but will
	leave it off.  The antenna is close to the ground so hopefully
	won't take a hit.  The arrestors are a safety precaution we haven't
	used before.

Data Uploaded:
	Disk was at roughly 30% since 5jul.

Licor Cleaned: 12:00
	voltage raised from 350mV to 700mV

Rad Cleaned: 12:30

Site visit: 1432 - 1605

The net radiometer was cleaned between 1433 - and 1438.
PVC was inserted into the GOES cooler box to allow heat to escape.
An electric fence was installed.
The precip gauge was checked.

1250-1349.
The data drive was exchanged during this visit.
PVC heat outlet holes were incorporated into the GOES cooler.
The radiometer was cleaned (it had accumulated soil stains visible to the
naked eye).

initial log file /usr/local/isff/aster/projects/CuPIDO/logbook/tklog.log created by USER aster