At 615am local time it was observed that Aster at the Base trailer was off. The author had never seen Aster shut down the entire experiment and had not shut it down when departing the trailer at approximately 2000 25 Apr.Approximately 2 minutes later the sounding team, Bob Street, Stanford and Serena Chew, DRI, showed up for the morning sounding during the current IOP of T-REX. They noted that lightning had occured overnight, near MAPR, after Midnight local.
An unsuccessful attempt was made to turn on the system at ~0620 after checking power cords and connectors (noting that the power was on to all drives, the monitor, the modems, the Terabeam, the lights, the Leeds sounding computer and various surge protectors. Thus, it appeared that either Aster or the computer port had experienced a catastrophic failure of some kind. Further investigation revealed that
1) the Aster machine would not turn on with battery power (it had drained after power to it had failed),
2) the computer Docking Station would not turn on with when powered from any plug, including known functional power
3) that there was no access to a fuse on the Docking Station through any panels etc.
4) the original 90W power cable for the Aster (Dell Latitude D800) was not in the trailer (two Poulos searches, one Chew search),
5) the power cord for Poulos' Dell Latitude D410 would apply power to Aster, but was of 65W and therefore subject to lesser performance
6) internet access was down in the trailer,
7) Agee reported from online in Bishop that the data plots had been down since before midnight (~2300 local), which when combined with the fact that no other electronic equipment was down, eliminated to a significant degree the 'lightning' theory,
7) #5 allowed some power to the battery and Aster could be functional - leading to the conclusion that indeed the Docking Station had a catastrophic failure.
After lengthy trouble shooting and Semmer looking up the viability of running Aster on 65W power, we decided to bring up Aster on 65W power. The only peripherals plugged in were the 12.0 and 14.0 connections to limit power usage.
Fortunately, this worked after a 2nd hardboot. The GPS time unit was subsequently plugged in as a crucial element of timekeeping after consultation with Gordon. This created no power or performance issues on sub-optimally-powered-Aster.
While no data had been written to Aster over the wireless, pings, file listings and data_stats of the 3 towers indicated that indeed the local data storage was complete overnight during the Aster outage. No data had been lost and soon thereafter files were filling on Aster with on-line data plots updating.
Subsequently, it was decided after discussions with Gordon/Steve S. to one-by-one attach various peripherals to Aster in its weak power state in hopes of retaining near normal functionality. An attempt was first made to enable the use of the 19" computer screen rather than the Aster laptop screen - this was unsuccessful - with electronic gibberish being the onscreen result. We thus are using Aster without the big screen until Golubieski arrives with the 90W power supply from Boulder (none were available at Schat.net in Bishop when Agee arrived first thing this morning and it would be 4-5 days to ship).
Agee also used a multi-meter to assess the circuits on the Docking station power supply. He found 'beeps' (circuit completion) when the meter was attached to the cord itself plugged in, but not when attached to the male pins at the back of the Docking Station. Our preliminary conclusion is that the internal power supply has failed or a fuse has blown within the Docking Station. Once that occured, apparently randomly, battery power allowed Aster to function temporarily while the battery ran down to zilch. Thus the state at 0615 26 Apr.
Gordon believes that since exthd7 and 9 are powered separately that their draw will be limited from the USB when plugged. Steve S. has suggested doing this in half-hour stages to ensure Aster functionality at each step.
We also agreed that since the data acquisition is quite solid onsite, an IOP is underway and the Central tower wireless still potentially shaky after yesterday's tests, that off-loading local storage is not advisable at this time to fill in the online data plot gaps.