Project #1999-102 INDOEX
NSF/NCAR EC-130Q Hercules (N130AR)
Data Quality Report
This summary has been written to outline basic instrumentation problems affecting the data set and is not intended to point out every bit of questionable data. It is hoped that this information will facilitate use of the data as the research concentrates on specific flights and times.
The following report covers only the RAF-supplied instrumentation and is organized into two sections. Section I lists recurring problems, general limitations, and systematic biases in the standard RAF measurements. Section II lists isolated problems occurring on a flight-by-flight basis. The LRT (low-rate) data set was analyzed for this report, but the information also should apply, in general, to the HRT (high-rate) data set. The Ferry Flights had very limited analysis.
A discussion of the performance of the SABL lidar, AIMR, MCR, MASP and dropsonde systems will be provided separately, as will their respective data sets.
In December 2001 differences were discovered between CONCF_LPI, FSSP-100 Concentration (all cells), and CFSSP_LPI, FSSP-100 Concentration (per cell). One should get the same value for the former and the total of the latter. Such is not the case in this data set, because a correction equation from Darrel Baumgardner was omitted from the CONCF_LPI calculation. To obtain correct values for CONCF_LPI, one needs to sum up all the values in CFSSP_LPI and use that value instead.
Aircraft data were taken during the 5 Test flights (TF01-TF05), 11 Ferry flights (FF01-FF11) and 18 Research flights (RF01-RF18). Data from the Test flights have not been processed for Production output. Data from the Ferry flights have been processed at LRT but with a limited number of measurements. (See the example netCDF File Header for the measurement list.) A basic HRT Production data set has been produced but it has not been thoroughly analyzed. The analysis of the LRT data also should apply, in general, to the HRT data.
The LRT data for the Research Flights include refinements and additions to some of the measurements. (These were calculated separately and merged into the Production LRT data set.)
Some minor problems with the RAF airborne data system (ADS) occurred during the project. In the final data set, these problems primarily manifested themselves in two ways. In some circumstances the entire onboard system crashed resulting in a complete loss of data for intervals lasting 10 to 20 minutes. In those instances the data gaps have been filled in the netCDF data files with the missing data code of -32767. More frequently, data transfers from one of the distributed data sampling modules (DSMs) were interrupted for a short period of time (generally a few seconds, but some intervals were longer). During these intervals, a false signal was recorded in each of the channels being sampled by that DSM. The resulting outputs are typically uncharacteristic of or physically impossible to obtain in a normal atmosphere. However, this may not be true for certain of the affected measurements. Some good variables to track to monitor such occurrences are: radar altitude (HGM232 = 0), static pressure (PSXC = 50) and ambient temperature (ATX = -66.36). During such intervals, all data should be considered suspect.
The wind data for this project were derived from measurements taken with the radome wind gust package. As is normally the case with all wind gust systems, the ambient wind calculations can be adversely affected by either sharp changes in the aircraft's flight attitude or excessive drift in the on-board Inertial Reference System (IRS). Turns, and more importantly, climbing turns are particularly disruptive to this type of measurement technique. Wind data reported for these conditions should be used with caution.
Special sets of in-flight calibration maneuvers were conducted on one test flight (TF03) and one research flight (RF13) to aid in the performance analysis of the wind gust measurements. The calibration data identified a systematic bias in the pitch and sideslip. These offsets have been removed from the final data set. The time interval for each type of maneuver has been documented in both the Flight-by-Flight Summary below and on the individual Research Flight Report prepared for RF13. Late in the program excessive drift in the position output from the IRS resulted in some errors in the basic wind data. Such cases also are noted in the Flight-by-Flight Summary.
As an additional enhancement to this data set, a second-pass correction was applied to the wind data using position and ground speed updates provided by the Global Positioning System (GPS). Both the GPS-corrected and basic uncorrected values are included in the final data set. RAF strongly recommends that the GPS-corrected winds be used for all research efforts.
Two sets of vertical winds also were calculated (WI, XWIC). They are calculated using different aircraft vertical velocities. During the straight-and-level flux legs, the XWIC value will be significantly better.
A Trimble Global Positioning System (GPS) was used as a more-accurate position reference during the program. The system performed extremely well, and the GPS position values should be used for all research efforts (GLAT, GLON).
RAF uses redundant sensors to assure data quality. Performance characteristics differ from sensor to sensor with certain units being more susceptible to various thermal and dynamic effects than others. Good comparisons were typically obtained between the two static pressures (PSFDC, PSFC), the three standard temperatures (ATRL, ATRR, ATWH), two dynamic pressures (QCRC, QCFC), three liquid water sensors (PLWCC, PLWCC1, XGLWC), and the two dew pointers (DPT, DPB). Exceptions are noted in the flight-by-flight summary. The digital static pressure (PSFDC) was selected as the reference pressure (PSXC) used in all of the derived measurements. The two remote surface temperature sensors (RSTB, RSTB1) generally functioned well, but there were times when the two measurements differ significantly.
Temperature measurements were made using the standard heated (ATWH) and unheated (ATRR, ATRL) Rosemount temperature sensors along with a near-field remote-temperature-sensing device made by the Ophir Corporation (OAT). All of the standard temperature sensors performed reasonably well, encountering the usual problems with sensor wetting during cloud passes. A comparison of the data sets yielded good correlation in instrument signatures and only small differences in absolute value (±0.2 °C) throughout the program. A comparison of pre- and post-project calibrations indicated that all units maintained stable and independent calibrations. ATRR was selected as the reference value (ATX) used in calculating the derived measurements.
The absolute accuracy of the Ophir sensor is not good due to thermal drift in its black-body reference. However, the unit is not sensitive to the presence of liquid water and is very useful in looking at temperature signatures during cloud passes. In the final data set, the long-term drift in the data has been taken out by linking the absolute value of the unit to the reference temperature. Use of the OAT data should be limited to in-cloud periods only.
Humidity measurements were made using two collocated thermoelectric dew point sensors and two Lyman-alpha fast-response hygrometers. Generally, the humidity sensors performed well. As is typically the case, the two dew point sensors (DPBC, DPTC) were set up differently to provide the best coverage under the widest range of ambient conditions. DPBC was set up for fast response, but its dynamic range was more limited. DPTC was a little slower but had the capability of measuring greater dew point depressions. A comparison of the data sets yielded good correlation in instrument signatures during the largest portions of the flights when both instruments were functioning normally. However, some problems with water ingestion occurred which resulted in some sensor drift. Each flight was evaluated on a case-by-case basis to see which dew point sensor was functioning better on that particular flight. The selection of a reference humidity sensor (DPXC) for use in calculating all of the derived moisture measurements was made accordingly.
Lyman-alpha hygrometers are susceptible to in-flight drift in the instrument's bias voltage. Due to this problem, RAF uses a special data-processing technique to remove the bias drift by referencing the long-term humidity values to one of the more stable thermoelectric dew point sensors. Measurements from the two systems remained well correlated for clear-air sampling but showed significant differences during cloud penetrations. The two Lyman-alpha hygrometers used different housing types. The stub unit (MRLA, RHOLA) tends to be slightly faster but is more susceptible to in-cloud wetting and thermal drift. The cross-flow unit (MRLA1, RHOLA1) is therefore recommended as the reference sensor for basic analysis. The Lyman-Alphas have fast response (> 10Hz), and their primary function is for use as a water vapor flux measurement. Typically during analysis, the mean component is removed before calculating fluxes, and the slow drift is not a factor. For flux calculations, MRLA1 should be used.
A set of standard upward- and downward-facing radiometers were used to measure shortwave, ultraviolet, and infrared irradiance. It should be noted that all units are hard mounted, and that none of the raw data have been corrected for changes in the aircraft's flight attitude. RAF recently added a new set of derived irradiance variables which have been corrected for aircraft attitude and relative sun angle. A description of this correction process is provided in RAF Bulletin No. 25.
Thermal temperature chamber experiments have indicated that the Heimann sensors used to remotely measure the surface temperature (RSTB, RSTB1) are sensitive to some thermal drift. In an attempt to deal with these problems, the units were equipped with a temperature-control heater system. Generally, the heater system stabilized the signals fairly well. Some drift is still apparent in the data set. RSTB1 seemed to be the more stable of the two units and exhibited better accuracy in the regular, single-point, precision tests made prior to each flight. Therefore, RSTB1 is recommended as the reference system for this measurement.
In addition to their thermal sensitivity, the output of each sensor is dependent upon the total amount of water in the sensing path. In such a moist, tropical atmosphere, this sensitivity appears as an altitude dependence in the raw surface temperature. Using a compilation of vertical soundings from all of the research flights, RAF was able to empirically generate two mean-moisture, vertical profiles that could be used to characterize the conditions encountered during the research flights. To aid in the analysis of the data, RAF has added a special calculation of sea-surface temperature (XTSURF) which makes a rough attempt to account for this dependency.
The altitude of the aircraft was measured in several ways. The primary measurement (PALT, PALTF) is derived from the static pressure using the hydrostatic equation and the U.S. Standard Atmosphere, which assumes a constant surface pressure of 1013 mbar and a mean surface temperature of 280C. For this project, the mean surface temperature has been increased to 300C to provide a more accurate representation of a tropical atmosphere.
The Inertial Reference System (IRS) outputs a measurement of altitude (ALT) by combining static pressure measurements with vertical accelerations. Because these data come directly from the IRS, RAF was unable to apply the same correction (300C) to the data for operation in a tropical atmosphere. To avoid confusion by the data users, this measurement has been removed from the final data set.
A radar altimeter (HGM232) was aboard the aircraft for the project. This unit worked well and, due to the fact that most of the research was conducted over a water surface, showed an excellent correlation with the calculated pressure altitude.
The Global Positioning System (GPS) also provides an altitude readout (GALT). The GALT signal has been "de-tuned" by the military ("selective availability") and exhibits erratic oscillations of ±100 M. To avoid confusion, this measurement has been removed from the final data set.
Two hot-wire liquid water sensors (PMS King Probes, PLWCC, PLWCC1) were used during the program and worked extremely well. A comparison of the two units yielded a good correlation in instrument signatures and only small differences in absolute value. Special note should be made that both of these instruments are calibrated for a specific range of aircraft speeds. Small changes in the baseline are apparent with speed changes, as are small zero offsets. Each cloud penetration will require a baseline adjustment with the relative change providing the sampled liquid water content. Due to the nature of this sampling technique, it should be clear that water contained in ice particles will not be observed. This fact should be taken into account when comparing data from these sensors with the calculated liquid water content obtained from the optical particle probes.
A Gerber Model PVM-100 liquid water probe was included in the research instrumentation package. The unit responded well to the presence of liquid water but exhibited a large baseline offset that was strongly dependent upon changes in the aircraft's true airspeed. Calibration for this unit comes from the manufacturer. The unit was returned for a full post-program evaluation and calibration. During final data processing, RAF's loose-couple technique was used to remove some of the remaining baseline drift.
The calculation of CN-sized aerosol particle concentrations (CONCN) is dependent upon total particle counts (CNTS) and the measurement of sample flow (FCN, FCNC).
Note: the internal sample flow (FCN) has been corrected (FCNC) to ambient conditions only, not to STP, for the calculation of particle concentration.Droplet shattering during cloud penetrations sharply increases the number of counts and can falsely increase indicated CN concentrations by several orders of magnitude. If there are any questions about segments of the CONCN data, the flows should be examined to determine if the pump malfunctioned or if an obstruction in the inlet limited particle sampling.
Six PMS particle probes (FSSP-100, FSSP-300, 2D-C, 2D-P, PCASP, 260X) were used on the project at different times. Some specific details about each of the probes are summarized below:
The PCASP aerosol probe used during the project was a unit borrowed from NASA and did not have the standard housekeeping outputs for "activity" or "sample flow." Both the quick-look and final data sets used fixed values of 0 and 1 cm3s-1 for these measurements, respectively. This action will cause a tendency to underestimate the total concentration of particles in the probe's sampling range (0.1 to 3 µm). On certain flights the output of the optical counter would drop to zero in a periodic cycle. Although this makes analysis difficult, a close examination of the data indicates that the data are good when the counts are present. This phenomenon occurred on research flights 1, 6, 7, 8 and 9.
Two FSSP-100 probes were used during the project. The left pod unit worked well during the first few flights, but the signal degraded over the course of the program. The right pod unit had a major optical failure during the initial ferry flights to the site and performed poorly during the early research missions. Once the right-side unit was repaired, it performed well. To avoid confusion over differences in the outputs from the two systems, only one set of FSSP-100 data is output for each flight.
Note: Two housekeeping parameters output by the FSSP-100 typically are used in calculating particle concentrations from this instrument (Activity = FACT; Beam Fraction = FBMFR). It has been determined through a detailed review of the data that the output from these two channels was questionable throughout the program. Per the instructions of the RAF PMS Probe expert, Darrel Baumgardner, the entire set of FSSP-100 data were processed using values of 0.0 for FACT and 0.6 for FBMFR, respectively. This process will serve to remove some of the "fine-scale" structure seen in the Quick Look data set provided in the field.
The FSSP-100 measurements have been reprocessed to reflect post-project size and sample-volume calibrations. The reprocessing also includes corrections for particle losses due to coincidence (more than a single particle in the beam) and deadtime (particles through the beam during the electronic reset period of the instrument). To compensate for the lack of housekeeping data, the following coincidence-correction algorithm was added to the per-cell concentration calculations.
ConcC[i] = ConcO[i] / (1.0 - 6.0e-4*ConcO[i])
where:
ConcC = concentration after correction algorithm (output to production netCDF file)
ConcO = original concentration (before applying correction algorithm)
The post-project size calibrations, coupled with new information about the channel thresholds of the FSSP-100 related to Mie scattering calculations, extend the size range of the FSSP-100 to 60 µm from the previously-assumed 47 µm. This new size calibration and particle-loss corrections have little effect on the derived concentrations, but liquid water contents are, on average, about 15% higher than the values originally calculated.
The FSSP-300 aerosol probe covers a range of particle sizes that bridges the gap between the true aerosols and the smaller droplets (0.3 to 20 µm). Like all 1-D optical probes, however, the FSSP-300 has no way to distinguish between aerosols, ice or water. In addition, the optical particle sizing of the FSSP-300 can be affected by heavy concentrations of smaller hygroscopic particles like those found in the primary INDOEX research area. An examination of the data set clearly shows this effect in the form of atypically-low concentrations in the larger size bins during cloud penetrations. Care should be used in making comparisons of size distributions between the normal missions north of the equator and those "clean" missions to the southern hemisphere.
Note: The bin sizes vary significantly in the particle-sizing routines for this probe.
The range of the 260X probe is specified to be 10 to 640 µm in 10 µm increments, but it has some problems sampling the smaller sizes when mounted on an aircraft. NCAR data processing uses the Baumgardner correction alogrithms to correct the 260X data for these problems. (See Reference below.) Effectively this changes the range of the unit to 50 to 640 µm when it is mounted on the EC-130Q. The unit functioned well throughout the program, but most cloud penetrations were limited to small cumulus clouds or scattered alto-stratus. This has resulted in a very small sampling of droplets large enough to trigger the new minimum threshold on this precipitation probe.
Both of the two-dimensional imaging probes (2D-C and 2D-P) performed extremely well throughout the program. Due to the special nature of the 2D data set, only the digital computations of droplet concentrations have been included in the production data set. The actual image data have been archived separately. Access to the images will require the use of specialized software routines that may necessitate direct assistance from the RAF staff. Most cloud penetrations were limited to small cumulus clouds or scattered alto-stratus. This has resulted in a very small sampling of droplets large enough to trigger the minimum threshold on the precipitation probe.
The TECO Model 49 ozone analyzer is a slow-response instrument which provides 10-second averaged values. The corrected output (TEO3C) has been adjusted for variations in sample pressure (TEP) and temperature (TET). Generally the instrument performed well. Intermittent spikes do occur but are fairly isolated and obvious to any user. More information on the performance of this sensor, and the modified TECO Model 48 carbon monoxide analyzer, appears in Appendix A below.
The Counterflow Virtual Impactor (CVI) was supported for the INDOEX Project by Dr. Cynthia Twohy. She operated the equipment in the field and was responsible for processing the data included in the production data set. Detailed information on the CVI data is provided in Appendix B below.
Data recording typically begins well in advance of the actual aircraft takeoff time. Virtually all measurements made on the aircraft require some sort of airspeed correction, or the systems simply do not become active while the aircraft remains on the ground. None of the data collected while the aircraft is on the ground should be considered as valid.
Reference
Baumgardner, D., Korolev, A., 1997: Airpseed Corrections for Optical Array Probe Sample Volumes. J. Atmos. Oceanic Tech., 14, 1224-1229.
Note: All times listed below are Coordinated Universal Time (UTC).
RF01
RF02
RF03
RF04
RF05
RF06
RF07
RF08
RF09
RF10
RF11
RF12
RF13
RF14
RF15
RF16
RF17
RF18
Section 1: Slow Ozone Measurements
Teresa Campos
The variable TEO3C contains corrected data from the TECO Model 49 UV ozone analyzer. The data have been corrected for pressure and temperature deviations from standard P and T using support airborne measurements (variables TEP and TET, unless noted below). Data taken at the beginning of the flights and during data system failures have been replaced by bad data flags. The TEP data contained a considerable amount of noise spikes. These data were treated with a despiking filter to greatly reduce this noise contribution. This filter removed single-point noise spikes which exceeded a 5 mbar noise threshold. Some noise spikes remained in the output; many of these were removed manually from the final data set.
Data gathered in clouds and in rapidly-changing humidity environments are suspect. Egregious humidity problems were identified by visual comparison of raw ozone (TEO3) data and the time derivative of the dew point (DPXC). Figure 1 and Figure 2 illustrate the implementation of this criterion. Data in problematic intervals were replaced by bad data flags (-32767). Some suspicious data remain in the files. It is left to the individual investigators to evaluate the remaining data using this and other indicators (e.g., PLWCC). Finally, in areas of high aerosol content, the TECO instrument can have a positive interference. The dual absorption cell instrument creates a blank by passing ambient air through an ozone destroyer. This component alters the aerosol number density in the absorption cell, causing less scattering extinction in the blank measurement.
Listed below are the flights in which either TEP or TET failed. In these cases, temperature and/or pressure corrections utilized CN support measurements, CNTEMP and PCN, respectively. Differences between ozone and CN temperatures will have negligible effects on data uncertainty. However, comparison between PCN and TEP during flights when both are present indicate a ±30 mbar offset between the two pressures. This implies an additional 3 % systematic uncertainty in TEO3C for those flights in which TEP was not functional. The sign of the bias cannot be determined.
RF04
RF05
RF06
RF09
RF14
RF16
RF17
Section 2: Carbon Monoxide Measurements
Teresa Campos
The carbon monoxide data have a one sigma precision of 30 ppbv for a 10-second average. The data have been further filtered with a 60-second running-average digital filter to improve precision as much as possible. The duty cycle consisted of 7 minutes of measurements followed by a 5 minute zero reading. Zero subtraction from measured voltages were performed using a spline-fitting routine. This required that each data file contain only data gathered for a particular front-panel zero setting. So each flight was separated into subset files, each containing data gathered for a particular zero offset. Because of this necessity, the processed CO data are being submitted to the INDOEX archive in netCDF files, but separately from the remainder of the RAF data set.
Comments by flight:
RF01RF02
RF03
RF17
RF18
Counterflow Virtual Impactor (CVI) Measurements
Cynthia Twohy
During INDOEX, the counterflow virtual impactor (CVI) was used to measure condensed water content and cloud droplet number (number of residual particles after evaporation), data that are included in the RAF data set. Condensed water content was measured with a modified Lyman-alpha hygrometer and particle number with a TSI 3760 condensation nucleus counter. Other measurements made behind the CVI included residual-particle size using a LAS-AIR particle counter, residual-particle critical super-saturation spectrum using Jim Hudson's CCN spectrometer, and residual-particle composition using Jim Anderson's electron microscope analysis. Contact Cynthia Twohy (twohy@oce.orst.edu) for more information about these measurements.
The following variables in the NCAR netCDF files relate to the CVI:
To correct for the lag time in the CVI plumbing and differences in timing between the ADS and the CVI data system, the CVI data were adjusted by comparison to the FSSP data. Data from takeoff and landing often were contaminated with exhaust particles and gases and are flagged as missing data, as are other "bad" data times.
The range of aerodynamic cloud particle sizes, which the CVI measures, is between the cut size CVRAD (about 4 µm radius) and about 25 µm radius. Larger droplets or ice crystals will impact on the bend downstream of the CVI inlet tip. The evaporated residue from these large particles will not be transmitted to the sensors measuring the aerosol, although the condensed water content for the larger particles will be detected by the Lyman-alpha (Twohy, et al., 1997). In the polluted INDOEX clouds, the mean droplet diameter was often at or below CVRAD, resulting in CVI number concentrations and water contents substantially lower than corresponding measurements from the FSSP.
Number concentration, CVN, may be artificially enhanced on any flight due to breakup of large (> 100 µm) droplets or crystals in the CVI inlet. This was generally only a problem in INDOEX during the convective cell penetrations. Use the 260X, 2D-C, or 2D-P data to determine when these are present, or compare CVN to number concentration from the FSSP (CONCF). If CVN > > CONCF, breakup is likely to be occurring, and the data are questionable.
Condensed water content was set to zero whenever the total or interstitial aerosol inlets (INFLAG 1 or 2) were used to obtain an ambient aerosol sample. Spikes in CVCWC may occur on either side of the ambient sample if the flag was set slightly after the sample began. Also, non-zero condensed water contents sometimes occurred immediately after ambient samples and dense cloud penetrations due to hysteresis in the sample line.
Flights with missing or problem data are given below:
RF01
RF02
RF06
RF11
RF12
RF17
RF18
Reference
Twohy, C.H., A.J. Schanot and W.A. Cooper, 1997: Measurement of condensed water content in liquid and ice clouds using an airborne counterflow virtual impactor. J. Atmos. Oceanic Tech., 14, 197-202.
