FLAT90: Logbook Entries

FLAT90: SOFTWARE Messages: 25 Entries..

Return to Logbook Contents Page
Entry Date Title Site Author #Graphics
17 Thu 26-Jul-1990std_io beats raw io! nonemartin
18 Thu 26-Jul-1990std io improves again nonemartin
21 Sun 09-Sep-1990ingest bug noneclm
24 Sun 09-Sep-1990ingest bug revisted noneclm
39 Wed 12-Sep-1990CHARLIE: fix logbook nonetwh
78 Thu 20-Sep-1990ingest modified noneclm
81 Fri 21-Sep-1990Created directory results/profiles nonetwh
125 Tue 25-Sep-1990change to logbook script noneclm
127 Tue 25-Sep-1990meta-log moved to /home/aster/apps/meta noneclm
130 Tue 25-Sep-1990PREPROCESS NEEDS FIX noneclm
135 Wed 26-Sep-1990how to change meta-log types noneclm
167 Fri 28-Sep-1990A bug in Preprocess nonesrs
171 Fri 28-Sep-1990windavg, psycavg, radavg changes nonesrs
172 Fri 28-Sep-1990Graph routines nonesrs
179 Sat 29-Sep-1990More graph routines nonesrs
201 Tue 02-Oct-1990logbook has -v and -x options noneclm
202 Tue 02-Oct-1990campbell calibration function noneclm
269 Mon 08-Oct-1990CHARLIE or STEVE S.: nonetwh
300 Fri 12-Oct-1990tape initialize fails noneclm
320 Sun 14-Oct-1990how to crash an ingestor noneclm
328 Mon 15-Oct-1990hot wire data goes to disk noneclm
432 Tue 23-Oct-1990marigold ingest crashed with SIGSEGV noneclm
450 Thu 25-Oct-1990ingestor shuts down noneclm
570 Thu 08-Nov-1990PSYC - wet bulb bug nonesrs
647 Fri 10-Nov-1995Revised fun.baro.configtwh


17: SOFTWARE , Site none, Thu 26-Jul-1990 16:55:36 GMT, std_io beats raw io!
Timing tests on archive program: 
 
results were obtained prom the profiler, using 
gprof. Number of calls to readExact determined  
the number of samples. 
 
adam      n samples   cpu secs    samples/second 
 
raw io: 
 
ragwort     1808        0.74      2443 
marigold   35236       12.10      2912 
 
 
fwrite std_io: 
 
ragwort     3504        0.77      4550      1.86 
marigold   51072        8.47      6029      2.07 
marigold  823177      131.70      6250 
 
 
fwrite and fread std_io: 
marigold   56276        1.05     53596 
 
 
__________________________________________________ 
 
Timing tests on preprocessor. 
 
To determine: 
   1) What is the overhead from formatted output (-p versus -b) 
   2) What improvement can be expected from buffered (std_io) output   
   3) what overhead is imposed by the profiler 
 
These results are from the command: 
time preprocess -a cosmos -f data  -p or -b  > temp 
 
The default preprocess configuration has r=1.0 for 
all cosmos channels. The data file has two hours of 1 sec. 
values from 4 propvanes 
 
raw io: 
                                                                                (-b)/(-p)  std/raw 
37.7u 19.2s 3:47 25% 0+208k 49+100io 60pf+0w  (-p option, no profiling )  59.6 
20.2u 53.7s 3:36 34% 0+192k 56+95io 60pf+0w   (-b option, no profiling )  73.9    124% 
 
std_io: 
                                                                                
39.3u 19.9s 3:50 25% 0+360k 48+100io 44pf+0w  (-p option, no profiling)    59.2              101%  
19.6u 19.1s 1:52 34% 0+208k 51+88io 44pf+0w   (-b option, no profiling)    38.7    65%        52% 
 
62.2u 23.5s 5:47 24% 0+352k 47+116io 39pf+0w  (-p option, with profiling)  85.7     
33.1u 20.9s 2:25 37% 0+336k 49+99io 44pf+0w   (-b option, with profiling)  54.0    63% 
 
Big difference bewtween the -p and -b options that the format conversion 
routines are being called for -p.  
 
Answers: 
1) for raw i/o, -b is 24% slower, probably because more  
   bytes are written. 
   for std_io, -b is 35% faster than -p. 
2) for std_io output, there is no improvement with the -p option. 
   for std_io output, there is a 50% improvement when using the -b option. 
3) profiling adds about 40% overhead to this process. 
 
 
 
18: SOFTWARE , Site none, Thu 26-Jul-1990 17:20:19 GMT, std io improves again
TEST ARE EXTENDED TO STD IO INPUT: 
 
Timing tests on preprocessor. 
 
To determine: 
   1) What is the overhead from formatted output (-p versus -b) 
   2) What improvement can be expected from buffered (std_io) output   
   3) what overhead is imposed by the profiler 
 
These results are from the command: 
time preprocess -a cosmos -f data  -p or -b  > temp 
 
The default preprocess configuration has r=1.0 for 
all cosmos channels. The data file has two hours of 1 sec. 
values from 4 propvanes 
 
 
raw io: 
                                                                                (-b)/(-p)  std/raw 
37.7u 19.2s 3:47 25% 0+208k 49+100io 60pf+0w  (-p option, no profiling )  59.6 
20.2u 53.7s 3:36 34% 0+192k 56+95io 60pf+0w   (-b option, no profiling )  73.9    124% 
 
std_io on writes only: 
                                                                                
39.3u 19.9s 3:50 25% 0+360k 48+100io 44pf+0w  (-p option, no profiling)    59.2              101%  
19.6u 19.1s 1:52 34% 0+208k 51+88io 44pf+0w   (-b option, no profiling)    38.7    65%        52% 
 
62.2u 23.5s 5:47 24% 0+352k 47+116io 39pf+0w  (-p option, with profiling)  85.7     
33.1u 20.9s 2:25 37% 0+336k 49+99io 44pf+0w   (-b option, with profiling)  54.0    63% 
 
std_io on reads and writes: 
 
18.7u 1.1s 0:56 35% 0+216k 49+84io 44pf+0w    (-b option, no profiling)    19.8               27% 
32.7u 1.1s 1:37 34% 0+352k 53+98io 46pf+0w    (-b option, with profiling)  33.8 
 
Big difference bewtween the -p and -b options that the format conversion 
routines are being called for -p.  
 
Answers: 
1) for raw i/o, -b is 24% slower, probably because more  
   bytes are written. 
   for std_io, -b is 35% faster than -p. 
2) for std_io output, there is no improvement with the -p option. 
   for std_io output, there is a 50% improvement when using the -b option. 
   for std_io input and output, under -b option, there is a 4x improvement! 
3) profiling adds about 40% overhead to this process. 
 
 
21: SOFTWARE , Site none, Sun 09-Sep-1990 18:38:52 GMT, ingest bug
ingest bug 
 
ingestor kept seeing errno 10 on file writes 
This error is ECHILD, don't know why this 
should happen on an fwrite ( or fflush). 
Rebooted marigold. 
 
 
24: SOFTWARE , Site none, Sun 09-Sep-1990 21:04:06 GMT, ingest bug revisted
ingest bug revisted 
 
Problem seems to be related to the SIGPIPE signal. Found 
that when a socket connection is broken, the ECHILD error 
can be obtained before the SIGPIPE signal is received. After this, 
SIGPIPE is received, and an EPIPE error is received during the next  
fwrite, and clean_up_broken_channel then closes the socket. 
 
In earlier cases, SIGPIPE was not received, and ECHILD 
was encountered continuously on fwrites, since clean_up... 
was not closing the socket. Perhaps clean_up... should close the 
socket on ECHILD as well as EPIPE? 
 
39: SOFTWARE , Site none, Wed 12-Sep-1990 20:15:02 GMT, CHARLIE: fix logbook
Charlie:  I still get emacs with 'logbook -v' 
 
78: SOFTWARE , Site none, Thu 20-Sep-1990 23:42:51 GMT, ingest modified
ingestor modified to be able to decode into up to 10  
variables. Motivated by the need to handle the noaa ati sonic. 
 
81: SOFTWARE , Site none, Fri 21-Sep-1990 01:57:35 GMT, Created directory results/profiles
I created the directory ~aster/projects/FLAT90/results 
and a subdirectory profiles.  Within the directory 
profiles, I placed the programs windavg and psycavg 
for creating daily 5-min-averaged wind and psychrometer 
data files.  I also created the subdirectory .Data  
used by S+ and put the functions fun.winds and fun.wdir 
to enable plotting the wind speed and direction for  
each day.  See the file README within profiles for 
instructions on accomplishing these tasks. 
 
125: SOFTWARE , Site none, Tue 25-Sep-1990 14:57:03 GMT, change to logbook script
have added a lock to the logbook script which 
will prevent mu;tiple users from running logbook. 
It works by creating a file called logbook.lock upon 
entry and removing it on exit. If the file exists on 
entry, one assumes that logbook is in use. 
 
If the script is crashed, then logbook.lock 
will not be removed properly. Manually remove logbook.lock 
to get logbook running again. 
 
127: SOFTWARE , Site none, Tue 25-Sep-1990 15:28:42 GMT, meta-log moved to /home/aster/apps/meta
meta-log moved to /home/aster/apps/meta-log 
 
130: SOFTWARE , Site none, Tue 25-Sep-1990 16:46:01 GMT, PREPROCESS NEEDS FIX
preprocess needs to be modified to take advantage 
of the calibration package enhancements. IT CURRENTLY 
CALLS sonic_uw_1_init DIRECTLY. 
 
135: SOFTWARE , Site none, Wed 26-Sep-1990 00:33:05 GMT, how to change meta-log types
to make changes to the TYPEs availible in 
meta-log, enter the hidden odify command. 
It will ask for a password, which 
is currently a carraige return.  
 
Be careful not to change this password, which 
is done witin the modify menu. 
 
167: SOFTWARE , Site none, Fri 28-Sep-1990 00:49:33 GMT, A bug in Preprocess
A bug in Preprocess 
 
	While entering the calibration coefficient 
for the ussr net radiation, discovered that a 
coefficient of absolute magnitude less than 1 must 
have a '0.' preceeding it. If not the magnitude of 
the output data is scaled VERY high, ie, 10^9. 
Charlie might want to verify this condition. 
 
171: SOFTWARE , Site none, Fri 28-Sep-1990 19:16:46 GMT, windavg, psycavg, radavg changes
windavg, psycavg, radavg changes 
 
	Modified code in files psycavg, radavg, and 
windavg (look in directory projects/FLAT90/results, 
subdirectories profiles and rad). It now prompts you 
if you want to delete the 5-minute average file, if it 
already exists. The option exists to save to old file as 
a '.old' file. 
 
172: SOFTWARE , Site none, Fri 28-Sep-1990 20:23:24 GMT, Graph routines
Graph routines 
 
	Below is a list of some graphing routines 
which can work on preprocessed data or raw data. 
 
graph_rad: 
 
	resides in /home/aster/projects/FALT90/results/rad 
	usage: graph_rad  
	purpose: graphical output of standard radiation sensors 
 
graph_net: 
 
	resides in /home/aster/projects/FALT90/results/rad 
	usage: graph_net  
	purpose: graphical output of ncar and ussr net radiatio 
		 sensors 
	 
compsyc: 
 
	resides in /home/aster/projects/FALT90/results/profiles 
	usage: compsyc  <-d -w -b> 
	purpose: graphical output of psychrometers at all heights 
		 flags: -d   dry bulb temperatures 
			-w   wet bulb temperatures 
			-a   both temperatures 
 
graph_psy: 
 
	resides in /home/aster/projects/FALT90/results/profiles 
	usage: graph_psy  <-1 -2 -4 -7 -10 -15) 
	purpose: graphical output of individual psychrometers 
		 flags: <-1 -2 -4 -7 -10 -15> represent the 
			psyc. to be plotted based on height. 
 
 
179: SOFTWARE , Site none, Sat 29-Sep-1990 23:18:14 GMT, More graph routines
More graph routines 
 
	A few more graph routines to play with. 
 
graph_ts: 
	 
	resides in /home/aster/projects/FLAT90/results/rad 
	usage: graph_ts  
	purpose: provide graph of tsoils and tsurface via xgraph 
 
 
 
graph_wind: 
 
	resides in /home/aster/projects/FLAT90/results/profiles 
	usage: graph_wind  
	purpose: graph wind data for 10m sensor via Splus 
		 Splus is executed in a batch mode, no plot 
		 appears on screen. 
 
compwind: 
 
	resides in /home/aster/projects/FLAT90/results/profiles 
	usage: compwind  <-s -d> 
	purpose: graph comparison of speed or direction data 
		 for all anenometers. 
		 flags:   -s     speed 
			  -d     direction 
 
201: SOFTWARE , Site none, Tue 02-Oct-1990 19:47:36 GMT, logbook has -v and -x options
logbook has -v and -x options 
 
-v for vi editor 
 
-x for xedit 
 
 
202: SOFTWARE , Site none, Tue 02-Oct-1990 19:54:55 GMT, campbell calibration function
added calibration function for the Campbell 
krypton hygrometer: 
 
q_kh20_1.c 
 
requires two parameters: 
 
Kw(gain factor) in m^3 g-1 cm 
 
ln(offset voltag V0(mV)) 
 
returns: 
q - g/m^3 
 
The routine must be recompile, or 
cloned, to change the pathlength, A/D gain 
and inut offset voltage. 
 
269: SOFTWARE , Site none, Mon 08-Oct-1990 00:22:19 GMT, CHARLIE or STEVE S.:
CHARLIE or STEVE S.:  
  How about calculating 5-min averages 
in background, say at the end of each 
2 hour archival period?  This would 
save a lot of time in the evening and 
would make the current plots easily 
available during the day. 
 
 
300: SOFTWARE , Site none, Fri 12-Oct-1990 14:46:28 GMT, tape initialize fails
tape initialize fails 
 
Sometimes, when running the bkfiles script for 
the morning backup, the tape initialization 
procedure will fail. In this case just run bkfiles 
again. The script will repeat the mv sequence, and then 
try the tape initialize again. 
 
IF you can't get bkfiles to do the initialization at 
all, manually initialize the tape with: 
 
tape_init 
 
 
320: SOFTWARE , Site none, Sun 14-Oct-1990 21:18:47 GMT, how to crash an ingestor
how to crash an ingestor 
 
found that by running preprocess (and have 
it attach to an ingestor), and then very quickly 
interrupting preprocess (with a ctrl^c)  
will cause an ingestor to crash with a segmentation 
fault. Undoubtedly it comes from killing 
the preprocess in the middle of an RPC transaction. 
 
328: SOFTWARE , Site none, Mon 15-Oct-1990 21:01:35 GMT, hot wire data goes to disk
fast_analog modified to write a block of 
data to disk every so often. Currently it 
will write every 600th block. The interval is 
determined by a define in fast_analog. 
 
These files are going into 
/home/aster/raw_data/hot_wire. 
 
 
432: SOFTWARE , Site none, Tue 23-Oct-1990 17:42:32 GMT, marigold ingest crashed with SIGSEGV
marigold ingest crashed with SIGSEGV 
 
I think it was due to receiving two 
connect requests simultaneously. I will 
try to find a core file and check it out. 
 
450: SOFTWARE , Site none, Thu 25-Oct-1990 00:27:52 GMT, ingestor shuts down
ingestor shuts down 
 
Have seen several ingestor failures today - 
loking at the syslog, we can see that it 
happens right after a connect request is made. 
The RPC call to ingest must be interfering in  
some way, and causes a SIGSEGV: 
Oct 25 00:02:18 aster INGEST[12356]: ingest marigold>connect request received from  aster, 4554 for 9 channels 
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 4 requested bytes 
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 1 requested bytes 
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 1 requested bytes 
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 12 requested bytes 
Oct 25 00:02:19 aster INGEST[6944]: INGEST [12356] has stopped, w_stopval:8b, w_termsig:SIGSEGV, w_retcode:0 
Oct 25 00:02:20 aster ARCHIVE[12441]: INGEST_SOCKET_ZERO_READ for marigold 
 
570: SOFTWARE , Site none, Thu 08-Nov-1990 16:14:59 GMT, PSYC - wet bulb bug
PSYC - wet bulb bug 
 
	The parsing code for the psyc. message has a 
bug in it. The wet bulb temperature is parsed to 4 characters. 
This works fine unless we get temperatures below -10 C then we 
lose the last digit and data becomes a factor of 10 less. We'll 
try a fix to the code and hope it gets cold again. 
 
647: SOFTWARE , Site , Fri 10-Nov-1995 18:10:27 GMT, Revised fun.baro.config
I have revised fun.baro.config to include the p_new pressure sensors.
Rather than revising the existing version, I started from the latest
template in apps/sfun/config.