**** BEGIN LOGGING AT Sat Dec 6 08:08:52 2003 MST Dec 06 08:08:52 --> blanc-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 08:09:05 --- blanc-c130 is now known as daniels-c130 Dec 06 08:10:42 --> hercules-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 08:11:36 --- hercules-c130 is now known as ADS-C130 Dec 06 08:12:21 John, so far only normal one second data going down the satcomm link... Dec 06 08:15:02 There have been two transmissions to a computer called sabl-185.atdd.ucar.edu on the ground, though. Dec 06 08:20:10 System okay. No C-midgets since it was crashing the forward dsm during taxi. Dec 06 08:20:10 I will play with it after we complete research. Dec 06 08:20:30 --> Brumbies (~Brumbies@vpn12.ucar.edu) has joined #C130Q Dec 06 08:27:00 For some reason, I can't ssh down to machines on the ground. Still investigating this. Dec 06 08:28:41 From hercules, it reports "ssh: ale.atd.ucar.edu: Temporary failure in name resolution" when I try to ssh in. Dec 06 08:28:41 Using the IP address explicitly is no better. Dec 06 08:29:36 082821: Communications down to lenado.atd.ucar.edu are happening every so often. Wonder which machine is doing those? Dec 06 08:31:16 <-- ADS-C130 has quit (Ping timeout) Dec 06 08:31:33 hercules' IP address appears to be 128.117.68.68 Dec 06 08:31:58 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 08:32:48 John, did you quit chat? Dec 06 08:35:19 Nope I was still looking at it. Dec 06 08:36:09 The ppp link went down Dec 06 08:37:08 I think I need to log some ppp events. BTW, John, I'm just logging stuff here in chat so we have some notes when we look into it further Dec 06 08:41:29 Some messages in hercules:/var/log/messages that indicate the PPP link went down. "Dec 6 08:30:20 hercules pppd[7616]: Serial link appears to be disconnected." Dec 06 08:41:55 <-- ADS-C130 has quit (Ping timeout) Dec 06 08:41:56 084022: PPP link went down. Went up and checked it this time, the phone connection had dropped. Dec 06 08:42:43 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 08:47:39 John's chat session timed out, yet the other client aboard the 130 stayed up. Need to look into the timeout settings on herclues chat. Dec 06 08:48:19 Yeah, I'm back. Dec 06 08:51:47 checking the different settings in blanc's xchat versus hercules' xchat... Dec 06 08:52:08 I would say all chat clients Mike. If you look at the Ping times I bet they are huge. Dec 06 08:54:25 hercules xchat is version 1.8.1, blanc's is 1.8.11 Dec 06 08:54:25 John, yes I think we should start chat on all the c130 clients to see if they also drop out Dec 06 08:56:58 <-- ADS-C130 has quit (Ping timeout) Dec 06 09:00:13 <-- daniels-c130 has quit (Ping timeout) Dec 06 09:00:36 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:01:23 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:01:36 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:02:11 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:02:53 Hey, Hey I'm back again .... Dec 06 09:02:54 phone disconnected Dec 06 09:02:54 finally, the clients are no also attched Dec 06 09:03:13 Heading down to 1000 feet. Dec 06 09:03:14 "now also attached"! Dec 06 09:05:15 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:05:47 i want to see if grendel drops out the same time as hercules Dec 06 09:06:38 090350 lots of "hosts unreachable" being logged over the ppp link. filename is /jnet/tmp/ppp3.log Dec 06 09:06:38 clyde finally joined, i started xchat about 7 minutes ago... Dec 06 09:06:46 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:09:27 xchat on clyde, that is... Dec 06 09:09:48 <-- ADS-C130 has quit (Ping timeout) Dec 06 09:10:13 <-- agave-c130 has quit (Ping timeout) Dec 06 09:11:10 <-- clyde-c130 has quit (Ping timeout) Dec 06 09:11:32 090637 some machine is trying to go to www.rhns.redhat.com.atd.ucar.edu -- with a name like that, it's no wonder it can't be found ;-) Dec 06 09:11:32 i must have started to xchats on clyde by mistake Dec 06 09:11:45 <-- grendel-c130 has quit (Ping timeout) Dec 06 09:11:58 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:12:05 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:12:30 another phone dropout Dec 06 09:12:36 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:12:45 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:16:27 Mike, I think mine was the only one to disconnect that time. The rest of you seemed to stay up. Dec 06 09:17:00 when the phone drops out this machine (blanc) stays "connected" while other machines disconnect Dec 06 09:17:00 so far, just normal 1 second UDPs out the PPP link Dec 06 09:17:55 we are very low, john how many feet above the water? Dec 06 09:19:28 <-- agave-c130 has quit (Ping timeout) Dec 06 09:19:43 <-- grendel-c130 has quit (Ping timeout) Dec 06 09:20:53 since John was not connected, he did not see the disconnect messages that I saw for the other clients. Dec 06 09:20:53 all the clients (except this one) disconnected Dec 06 09:20:58 this one is running the newest version of xchat, could it be related to that? Dec 06 09:20:58 perhaps the release notes may tell us what was fixed in this version Dec 06 09:21:07 another phone dropout Dec 06 09:21:12 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:21:19 <-- ADS-C130 has quit (Ping timeout) Dec 06 09:21:23 this time it appears that none of the clients disconnected Dec 06 09:22:24 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:22:24 sort of a delayed disconnect message there, the link has been up for a few minutes now Dec 06 09:26:07 John notices that sometime, actracker will indicate the link is down (with a yellow highlighted button), but the phone will look like it's up Dec 06 09:26:18 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:26:45 however, during every one of these times so far, the ppp link went down and had to be re-established Dec 06 09:31:45 so far, there have been 6 phone dropouts Dec 06 09:32:38 our IP address, at least, seems to consistently go to 128.117.68.68 Dec 06 09:35:26 I can't telnet, ssh or rlogin to ale.atd.ucar.edu from up here. Why is that? Dec 06 09:39:22 <-- daniels-c130 has quit (Ping timeout) Dec 06 09:39:58 <-- agave-c130 has quit (Ping timeout) Dec 06 09:42:43 Woooo Hooo. Turning for home. Dec 06 09:43:37 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:43:58 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:44:06 <-- grendel-c130 has quit (Ping timeout) Dec 06 09:48:45 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 09:51:42 grendel was the only client that dies that time. i noticed it is connecting to 128.117.78.2 Dec 06 09:52:02 Heading up to 22K for ferry home. Dec 06 09:52:02 Cleared the last cloud deck. Dec 06 09:52:35 notice a few connections from 128.117.78.2 to 128.117.84.29 are coming across the PPP link Dec 06 09:53:28 they are ircd traffic Dec 06 09:53:28 nice and smooth here Dec 06 09:57:54 Cold up here. Air temp is -34C Dec 06 10:01:05 <-- daniels-c130 has quit (Ping timeout) Dec 06 10:01:06 <-- clyde-c130 has quit (Ping timeout) Dec 06 10:01:43 <-- agave-c130 has quit (Ping timeout) Dec 06 10:01:46 <-- ADS-C130 has quit (Ping timeout) Dec 06 10:02:24 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:02:52 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:03:08 another ppp drop Dec 06 10:03:24 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:04:24 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:04:36 John reports that the phone didn't appear to disconnect on this one (i.e. the display was not blinking) Dec 06 10:04:36 It has happened this way before Dec 06 10:06:27 this time blanc also disconnected from chat Dec 06 10:07:23 hercules took quite a while to re-join chat Dec 06 10:08:19 i don't think grendel is back on yet Dec 06 10:10:15 <-- ADS-C130 has quit (Ping timeout) Dec 06 10:10:22 oops, yes it is. I think it never disconnected Dec 06 10:11:00 grendel never disconnected Dec 06 10:11:05 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:11:33 another ppp dropout Dec 06 10:15:17 101345, before and after -- lots of DNS stuff going across PPP Dec 06 10:16:57 Do a 'grep "10:13:" ppp8.log' Dec 06 10:22:03 <-- ADS-C130 has quit (Ping timeout) Dec 06 10:25:58 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:27:11 took ta while for hercules to reconnec Dec 06 10:27:47 the "Ping timeout" message came well after the ppp link dropped Dec 06 10:32:06 Just a few notes about the dropouts. No indication that anything unusual is happening on the PPP link when it drops Dec 06 10:32:06 these drops happen when flying straight, at 22K Dec 06 10:32:57 during that most recent drop, hercules appeared to be the only one that disconnected Dec 06 10:46:53 <-- agave-c130 has quit (Ping timeout) Dec 06 10:47:37 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:47:47 <-- ADS-C130 has quit (Ping timeout) Dec 06 10:48:59 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 10:50:13 I accidentally wrote over the ppp9.log file Oops. Dec 06 10:51:03 Both agave and hercules disconnected during that dropout Dec 06 10:56:55 ETA now 1415 L or 3:17 Left in the flight. Dec 06 11:00:05 Checking through /var/log/messages*, it appears that the PPP link dropped quite often. Just search for "Serial link appears to be disconnected" in those files... Dec 06 11:10:51 <-- clyde-c130 has quit (Ping timeout) Dec 06 11:11:01 <-- grendel-c130 has quit (Ping timeout) Dec 06 11:11:05 <-- agave-c130 has quit (Ping timeout) Dec 06 11:11:22 <-- daniels-c130 has quit (Ping timeout) Dec 06 11:11:38 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:12:34 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:13:37 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:13:43 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:17:03 all the clients disconnected that time, John reports that the phone display also blinked Dec 06 11:17:03 Is there anyone chatting from the ground? Dec 06 11:21:00 <-- daniels-c130 has quit (Ping timeout) Dec 06 11:25:31 <-- ADS-C130 has quit (Ping timeout) Dec 06 11:25:52 <-- agave-c130 has quit (Ping timeout) Dec 06 11:26:22 <-- grendel-c130 has quit (Ping timeout) Dec 06 11:26:39 <-- clyde-c130 has quit (Ping timeout) Dec 06 11:35:50 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:36:41 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:37:57 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:38:02 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:38:27 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:39:42 Back up now. Big phone problem. Dec 06 11:39:59 A pretty serious dropout occured that time. John couldn't get the link to reconnect and had to restart ACtracker and power cycle the phone Dec 06 11:42:21 <-- agave-c130 has quit (Ping timeout) Dec 06 11:42:37 <-- grendel-c130 has quit (Ping timeout) Dec 06 11:42:42 <-- clyde-c130 has quit (Ping timeout) Dec 06 11:42:43 <-- ADS-C130 has quit (Ping timeout) Dec 06 11:43:00 <-- daniels-c130 has quit (Ping timeout) Dec 06 11:43:02 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:43:24 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:44:48 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:45:03 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:46:15 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 11:51:56 I think it would be useful to take a close look at the logs from the Iridium to see how many separate phone calls were made for this flight Dec 06 11:51:56 does the Iridium connection actually stay up during these PPP dropouts? If so, that would point to a problem with the serial link from hercules to the phone Dec 06 12:09:01 ETA now 1412L about 2 hours from now. Dec 06 12:11:43 <-- agave-c130 has quit (Ping timeout) Dec 06 12:15:57 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:16:58 <-- clyde-c130 has quit (Ping timeout) Dec 06 12:16:59 <-- ADS-C130 has quit (Ping timeout) Dec 06 12:17:16 <-- daniels-c130 has quit (Ping timeout) Dec 06 12:18:54 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:19:59 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:20:07 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:22:14 <-- grendel-c130 has quit (Ping timeout) Dec 06 12:26:47 <-- daniels-c130 has quit (Ping timeout) Dec 06 12:26:49 <-- ADS-C130 has quit (Ping timeout) Dec 06 12:27:30 <-- agave-c130 has quit (Ping timeout) Dec 06 12:29:19 --> ADS-C130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:29:29 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:32:15 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:33:01 We are crossing into Nebraska. Dec 06 12:33:01 Only 2 MORE states to go until we get to HOME . Dec 06 12:37:56 <-- ADS-C130 has quit (Ping timeout) Dec 06 12:39:59 <-- agave-c130 has quit (Ping timeout) Dec 06 12:42:59 <-- daniels-c130 has quit (Ping timeout) Dec 06 12:46:42 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 12:51:55 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:01:18 !! Dec 06 13:01:46 --> hercules-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:02:25 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:02:33 Had to restart this chat client to get it back up. Dec 06 13:03:23 It locked up when it tried to restart and there was a phone dropout in the middle of it's login. Dec 06 13:03:43 grendel needed to be restarted as well... Dec 06 13:04:31 We can see the ground for the first time this flight. Dec 06 13:10:45 So far, 20 phone dropouts on this flight Dec 06 13:10:55 <-- agave-c130 has quit (Ping timeout) Dec 06 13:12:14 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:17:00 /tmp/pppd.log shows connections of 29.5, 9.3, 18.8, 9.9, 8.6, 3.3, 35.5, 9.3, 11.1, 23.8, 22.4, 14.6, 5.5, 33.8, 10.3, 15.8, 2.6, 4.6, 8.6, 8.8 minutes each Dec 06 13:22:42 <-- clyde-c130 has quit (Ping timeout) Dec 06 13:22:46 <-- daniels-c130 has quit (Ping timeout) Dec 06 13:22:51 <-- agave-c130 has quit (Ping timeout) Dec 06 13:24:29 <-- grendel-c130 has quit (Ping timeout) Dec 06 13:24:52 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:25:09 <-- hercules-c130 has quit (Ping timeout) Dec 06 13:26:06 --> hercules-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:27:09 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:27:14 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:27:20 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:36:46 <-- clyde-c130 has quit (Ping timeout) Dec 06 13:37:28 <-- grendel-c130 has quit (Ping timeout) Dec 06 13:37:28 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:37:31 <-- hercules-c130 has quit (Ping timeout) Dec 06 13:38:12 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:38:42 --> hercules-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:42:46 We are in colorado!!!!!!!!!!!!!!!!!!!! Dec 06 13:46:40 We are starting down, leaving 22K for 17K. Dec 06 13:46:47 <-- daniels-c130 has quit (Ping timeout) Dec 06 13:49:48 --> daniels-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 13:51:00 <-- daniels-c130 has quit (Client Exiting) Dec 06 13:59:44 <-- clyde-c130 has quit (Ping timeout) Dec 06 13:59:55 <-- agave-c130 has quit (Ping timeout) Dec 06 14:00:27 <-- grendel-c130 has quit (Ping timeout) Dec 06 14:01:17 --> blanc-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 14:01:30 --> clyde-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 14:01:32 We are now direct to Jeffco. Dec 06 14:01:45 --> agave-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 14:02:29 --> grendel-c130 (~ads@ras68.ucar.edu) has joined #C130Q Dec 06 14:07:40 Going down to 12K. Light chop. Dec 06 14:12:22 Now talking to approach. Dec 06 14:13:16 cleared for visual approach. Dec 06 14:13:16 Talking to Jeffco tower. Dec 06 14:14:10 Gear going down. Dec 06 14:16:37 <-- Brumbies has quit (Leaving) Dec 06 14:16:40 <-- grendel-c130 has quit (Ping timeout) Dec 06 14:16:40 <-- blanc-c130 has quit (Ping timeout) Dec 06 14:16:55 <-- agave-c130 has quit (Ping timeout) Dec 06 14:17:11 <-- hercules-c130 has quit (Ping timeout) Dec 06 14:17:23 <-- clyde-c130 has quit (Ping timeout) Dec 07 04:03:38 <-- anstett has quit (CGI:IRC (EOF)) **** END LOGGING AT Sun Dec 7 04:03:38 2003 MST