- 17: SOFTWARE , Site none, Thu 26-Jul-1990 16:55:36 GMT, std_io beats raw io!
Timing tests on archive program:
results were obtained prom the profiler, using
gprof. Number of calls to readExact determined
the number of samples.
adam n samples cpu secs samples/second
raw io:
ragwort 1808 0.74 2443
marigold 35236 12.10 2912
fwrite std_io:
ragwort 3504 0.77 4550 1.86
marigold 51072 8.47 6029 2.07
marigold 823177 131.70 6250
fwrite and fread std_io:
marigold 56276 1.05 53596
__________________________________________________
Timing tests on preprocessor.
To determine:
1) What is the overhead from formatted output (-p versus -b)
2) What improvement can be expected from buffered (std_io) output
3) what overhead is imposed by the profiler
These results are from the command:
time preprocess -a cosmos -f data -p or -b > temp
The default preprocess configuration has r=1.0 for
all cosmos channels. The data file has two hours of 1 sec.
values from 4 propvanes
raw io:
(-b)/(-p) std/raw
37.7u 19.2s 3:47 25% 0+208k 49+100io 60pf+0w (-p option, no profiling ) 59.6
20.2u 53.7s 3:36 34% 0+192k 56+95io 60pf+0w (-b option, no profiling ) 73.9 124%
std_io:
39.3u 19.9s 3:50 25% 0+360k 48+100io 44pf+0w (-p option, no profiling) 59.2 101%
19.6u 19.1s 1:52 34% 0+208k 51+88io 44pf+0w (-b option, no profiling) 38.7 65% 52%
62.2u 23.5s 5:47 24% 0+352k 47+116io 39pf+0w (-p option, with profiling) 85.7
33.1u 20.9s 2:25 37% 0+336k 49+99io 44pf+0w (-b option, with profiling) 54.0 63%
Big difference bewtween the -p and -b options that the format conversion
routines are being called for -p.
Answers:
1) for raw i/o, -b is 24% slower, probably because more
bytes are written.
for std_io, -b is 35% faster than -p.
2) for std_io output, there is no improvement with the -p option.
for std_io output, there is a 50% improvement when using the -b option.
3) profiling adds about 40% overhead to this process.
- 18: SOFTWARE , Site none, Thu 26-Jul-1990 17:20:19 GMT, std io improves again
TEST ARE EXTENDED TO STD IO INPUT:
Timing tests on preprocessor.
To determine:
1) What is the overhead from formatted output (-p versus -b)
2) What improvement can be expected from buffered (std_io) output
3) what overhead is imposed by the profiler
These results are from the command:
time preprocess -a cosmos -f data -p or -b > temp
The default preprocess configuration has r=1.0 for
all cosmos channels. The data file has two hours of 1 sec.
values from 4 propvanes
raw io:
(-b)/(-p) std/raw
37.7u 19.2s 3:47 25% 0+208k 49+100io 60pf+0w (-p option, no profiling ) 59.6
20.2u 53.7s 3:36 34% 0+192k 56+95io 60pf+0w (-b option, no profiling ) 73.9 124%
std_io on writes only:
39.3u 19.9s 3:50 25% 0+360k 48+100io 44pf+0w (-p option, no profiling) 59.2 101%
19.6u 19.1s 1:52 34% 0+208k 51+88io 44pf+0w (-b option, no profiling) 38.7 65% 52%
62.2u 23.5s 5:47 24% 0+352k 47+116io 39pf+0w (-p option, with profiling) 85.7
33.1u 20.9s 2:25 37% 0+336k 49+99io 44pf+0w (-b option, with profiling) 54.0 63%
std_io on reads and writes:
18.7u 1.1s 0:56 35% 0+216k 49+84io 44pf+0w (-b option, no profiling) 19.8 27%
32.7u 1.1s 1:37 34% 0+352k 53+98io 46pf+0w (-b option, with profiling) 33.8
Big difference bewtween the -p and -b options that the format conversion
routines are being called for -p.
Answers:
1) for raw i/o, -b is 24% slower, probably because more
bytes are written.
for std_io, -b is 35% faster than -p.
2) for std_io output, there is no improvement with the -p option.
for std_io output, there is a 50% improvement when using the -b option.
for std_io input and output, under -b option, there is a 4x improvement!
3) profiling adds about 40% overhead to this process.
- 21: SOFTWARE , Site none, Sun 09-Sep-1990 18:38:52 GMT, ingest bug
ingest bug
ingestor kept seeing errno 10 on file writes
This error is ECHILD, don't know why this
should happen on an fwrite ( or fflush).
Rebooted marigold.
- 24: SOFTWARE , Site none, Sun 09-Sep-1990 21:04:06 GMT, ingest bug revisted
ingest bug revisted
Problem seems to be related to the SIGPIPE signal. Found
that when a socket connection is broken, the ECHILD error
can be obtained before the SIGPIPE signal is received. After this,
SIGPIPE is received, and an EPIPE error is received during the next
fwrite, and clean_up_broken_channel then closes the socket.
In earlier cases, SIGPIPE was not received, and ECHILD
was encountered continuously on fwrites, since clean_up...
was not closing the socket. Perhaps clean_up... should close the
socket on ECHILD as well as EPIPE?
- 39: SOFTWARE , Site none, Wed 12-Sep-1990 20:15:02 GMT, CHARLIE: fix logbook
Charlie: I still get emacs with 'logbook -v'
- 78: SOFTWARE , Site none, Thu 20-Sep-1990 23:42:51 GMT, ingest modified
ingestor modified to be able to decode into up to 10
variables. Motivated by the need to handle the noaa ati sonic.
- 81: SOFTWARE , Site none, Fri 21-Sep-1990 01:57:35 GMT, Created directory results/profiles
I created the directory ~aster/projects/FLAT90/results
and a subdirectory profiles. Within the directory
profiles, I placed the programs windavg and psycavg
for creating daily 5-min-averaged wind and psychrometer
data files. I also created the subdirectory .Data
used by S+ and put the functions fun.winds and fun.wdir
to enable plotting the wind speed and direction for
each day. See the file README within profiles for
instructions on accomplishing these tasks.
- 125: SOFTWARE , Site none, Tue 25-Sep-1990 14:57:03 GMT, change to logbook script
have added a lock to the logbook script which
will prevent mu;tiple users from running logbook.
It works by creating a file called logbook.lock upon
entry and removing it on exit. If the file exists on
entry, one assumes that logbook is in use.
If the script is crashed, then logbook.lock
will not be removed properly. Manually remove logbook.lock
to get logbook running again.
- 127: SOFTWARE , Site none, Tue 25-Sep-1990 15:28:42 GMT, meta-log moved to /home/aster/apps/meta
meta-log moved to /home/aster/apps/meta-log
- 130: SOFTWARE , Site none, Tue 25-Sep-1990 16:46:01 GMT, PREPROCESS NEEDS FIX
preprocess needs to be modified to take advantage
of the calibration package enhancements. IT CURRENTLY
CALLS sonic_uw_1_init DIRECTLY.
- 135: SOFTWARE , Site none, Wed 26-Sep-1990 00:33:05 GMT, how to change meta-log types
to make changes to the TYPEs availible in
meta-log, enter the hidden odify command.
It will ask for a password, which
is currently a carraige return.
Be careful not to change this password, which
is done witin the modify menu.
- 167: SOFTWARE , Site none, Fri 28-Sep-1990 00:49:33 GMT, A bug in Preprocess
A bug in Preprocess
While entering the calibration coefficient
for the ussr net radiation, discovered that a
coefficient of absolute magnitude less than 1 must
have a '0.' preceeding it. If not the magnitude of
the output data is scaled VERY high, ie, 10^9.
Charlie might want to verify this condition.
- 171: SOFTWARE , Site none, Fri 28-Sep-1990 19:16:46 GMT, windavg, psycavg, radavg changes
windavg, psycavg, radavg changes
Modified code in files psycavg, radavg, and
windavg (look in directory projects/FLAT90/results,
subdirectories profiles and rad). It now prompts you
if you want to delete the 5-minute average file, if it
already exists. The option exists to save to old file as
a '.old' file.
- 172: SOFTWARE , Site none, Fri 28-Sep-1990 20:23:24 GMT, Graph routines
Graph routines
Below is a list of some graphing routines
which can work on preprocessed data or raw data.
graph_rad:
resides in /home/aster/projects/FALT90/results/rad
usage: graph_rad
purpose: graphical output of standard radiation sensors
graph_net:
resides in /home/aster/projects/FALT90/results/rad
usage: graph_net
purpose: graphical output of ncar and ussr net radiatio
sensors
compsyc:
resides in /home/aster/projects/FALT90/results/profiles
usage: compsyc <-d -w -b>
purpose: graphical output of psychrometers at all heights
flags: -d dry bulb temperatures
-w wet bulb temperatures
-a both temperatures
graph_psy:
resides in /home/aster/projects/FALT90/results/profiles
usage: graph_psy <-1 -2 -4 -7 -10 -15)
purpose: graphical output of individual psychrometers
flags: <-1 -2 -4 -7 -10 -15> represent the
psyc. to be plotted based on height.
- 179: SOFTWARE , Site none, Sat 29-Sep-1990 23:18:14 GMT, More graph routines
More graph routines
A few more graph routines to play with.
graph_ts:
resides in /home/aster/projects/FLAT90/results/rad
usage: graph_ts
purpose: provide graph of tsoils and tsurface via xgraph
graph_wind:
resides in /home/aster/projects/FLAT90/results/profiles
usage: graph_wind
purpose: graph wind data for 10m sensor via Splus
Splus is executed in a batch mode, no plot
appears on screen.
compwind:
resides in /home/aster/projects/FLAT90/results/profiles
usage: compwind <-s -d>
purpose: graph comparison of speed or direction data
for all anenometers.
flags: -s speed
-d direction
- 201: SOFTWARE , Site none, Tue 02-Oct-1990 19:47:36 GMT, logbook has -v and -x options
logbook has -v and -x options
-v for vi editor
-x for xedit
- 202: SOFTWARE , Site none, Tue 02-Oct-1990 19:54:55 GMT, campbell calibration function
added calibration function for the Campbell
krypton hygrometer:
q_kh20_1.c
requires two parameters:
Kw(gain factor) in m^3 g-1 cm
ln(offset voltag V0(mV))
returns:
q - g/m^3
The routine must be recompile, or
cloned, to change the pathlength, A/D gain
and inut offset voltage.
- 269: SOFTWARE , Site none, Mon 08-Oct-1990 00:22:19 GMT, CHARLIE or STEVE S.:
CHARLIE or STEVE S.:
How about calculating 5-min averages
in background, say at the end of each
2 hour archival period? This would
save a lot of time in the evening and
would make the current plots easily
available during the day.
- 300: SOFTWARE , Site none, Fri 12-Oct-1990 14:46:28 GMT, tape initialize fails
tape initialize fails
Sometimes, when running the bkfiles script for
the morning backup, the tape initialization
procedure will fail. In this case just run bkfiles
again. The script will repeat the mv sequence, and then
try the tape initialize again.
IF you can't get bkfiles to do the initialization at
all, manually initialize the tape with:
tape_init
- 320: SOFTWARE , Site none, Sun 14-Oct-1990 21:18:47 GMT, how to crash an ingestor
how to crash an ingestor
found that by running preprocess (and have
it attach to an ingestor), and then very quickly
interrupting preprocess (with a ctrl^c)
will cause an ingestor to crash with a segmentation
fault. Undoubtedly it comes from killing
the preprocess in the middle of an RPC transaction.
- 328: SOFTWARE , Site none, Mon 15-Oct-1990 21:01:35 GMT, hot wire data goes to disk
fast_analog modified to write a block of
data to disk every so often. Currently it
will write every 600th block. The interval is
determined by a define in fast_analog.
These files are going into
/home/aster/raw_data/hot_wire.
- 432: SOFTWARE , Site none, Tue 23-Oct-1990 17:42:32 GMT, marigold ingest crashed with SIGSEGV
marigold ingest crashed with SIGSEGV
I think it was due to receiving two
connect requests simultaneously. I will
try to find a core file and check it out.
- 450: SOFTWARE , Site none, Thu 25-Oct-1990 00:27:52 GMT, ingestor shuts down
ingestor shuts down
Have seen several ingestor failures today -
loking at the syslog, we can see that it
happens right after a connect request is made.
The RPC call to ingest must be interfering in
some way, and causes a SIGSEGV:
Oct 25 00:02:18 aster INGEST[12356]: ingest marigold>connect request received from aster, 4554 for 9 channels
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 4 requested bytes
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 1 requested bytes
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 1 requested bytes
Oct 25 00:02:19 aster ARCHIVE[12441]: fread only returned 0 of 12 requested bytes
Oct 25 00:02:19 aster INGEST[6944]: INGEST [12356] has stopped, w_stopval:8b, w_termsig:SIGSEGV, w_retcode:0
Oct 25 00:02:20 aster ARCHIVE[12441]: INGEST_SOCKET_ZERO_READ for marigold
- 570: SOFTWARE , Site none, Thu 08-Nov-1990 16:14:59 GMT, PSYC - wet bulb bug
PSYC - wet bulb bug
The parsing code for the psyc. message has a
bug in it. The wet bulb temperature is parsed to 4 characters.
This works fine unless we get temperatures below -10 C then we
lose the last digit and data becomes a factor of 10 less. We'll
try a fix to the code and hope it gets cold again.
- 647: SOFTWARE , Site , Fri 10-Nov-1995 18:10:27 GMT, Revised fun.baro.config
I have revised fun.baro.config to include the p_new pressure sensors.
Rather than revising the existing version, I started from the latest
template in apps/sfun/config.