DATA REDUCTION FORMULAE TABLE
Reduced Data Items Computed
for
|
|
|
|
# Values |
|
|
|
|
Tag # |
Description |
Output |
Units |
Method of Computation |
|
|
|
|
|
|
|
|
|
|
|
101 |
Dynamic Pressure #1 |
(20 Hz) |
1 mb |
6.30452E–3 * Raw – 0.0489 |
||
|
102 |
Dynamic Pressure #2 |
(20 Hz) |
1 mb |
5.28371E–3 * Raw – 1.5768 |
|
|
|
103 |
Static Pressure #1 |
(20 Hz) |
1 mb |
1.5791E–2 * Raw + 530.37 |
|
|
|
104 |
Static Pressure #2 |
(20 Hz) |
1 mb |
1.0917E–2 * Raw + 691.92 |
|
|
|
105 |
Rate of Climb |
(20 Hz) |
1 m/s |
5.625E–4 * Raw, for Raw > = 0 |
|
|
|
|
|
|
|
5.287E–4 * Raw, for Raw < 0 |
|
|
|
106 |
Rosemount Temp |
|
1 deg C |
mach2 = 5*((l
+dyn_pr/stat_pr)**(2/7)-l) |
|
|
|
|
|
|
|
divisor = 1 + 0.195 * mach2 |
|
|
|
|
|
|
|
temp = (1.83105E-3 * Raw + 243.16)/divisor-273.16 |
|
|
|
107 |
Reverse Flow Temp |
|
1 deg C |
divisor = 1 + 0.1594 * mach2 |
|
|
|
|
|
|
|
temp = (3.02109E-3 * Raw + 222.06)/divisor-273.16 |
|
|
|
108 |
Manifold Pressure |
|
1 " Hg |
3.1098E-3 * Raw + 0.159275 |
|
|
|
109 |
Acceleration |
(20 Hz) |
1 g's |
6.25E–5 * Raw [prior to |
|
|
|
|
|
|
|
6.25E–5 * Raw + 1.0 [after |
|
|
|
110 |
Pitch |
(20 Hz) |
1 deg |
–3.05175E–3 * Raw + 50 |
|
|
|
111 |
Roll |
(20 Hz) |
1 deg |
3.05175E–3 * Raw – 50 |
|
|
|
116 |
Voltage Regulator |
|
1 volts |
1.5258789E–4 * Raw |
|
|
|
118 |
NCAR true air speed |
|
1 m/s |
3.96744E–3 * Raw |
|
|
|
119 |
PMS End Element 1 |
|
1 volts |
|
|
|
|
120 |
PMS End Element 2 |
|
1 volts |
|
|
|
|
121 |
Interior Temp (computer) |
|
1 deg C |
3.05175E–2 * Raw |
|
|
|
123 |
High Voltage Current |
|
1 mA |
1.5258789E-4 * Raw |
|
|
|
124 |
Heater current |
|
1 amp |
3.05175E–3 * Raw |
|
|
|
130 |
Event Code bits |
|
1 flags |
bit 0 = 1 --> system running |
|
|
|
|
|
|
|
bit 1 = 0 -- > in cloud |
|
|
|
|
|
|
|
bit 2 = 0 -- > foil on |
|
|
|
131 |
GPS warning codes |
|
1 flags |
11 bit codes |
|
|
|
140 |
FSSP counts |
|
15 number |
Raw |
|
|
|
|
FSSP total counts |
|
1 number |
Sum of tag 140s (tot_count) |
|
|
|
142 |
FSSP ave diameter |
|
1 mm |
sum of diams / (tot_count) |
*2 |
|
|
143 |
FSSP concentration |
|
cm3 |
vol = 0.22275 * tas (volume
sampling rate (m3/s)) |
*2 |
|
|
|
|
|
|
denom = 1 – .55 * activ / 100 (adjustment for probe activity) |
|
|
|
|
|
|
1 #/cm3 |
conc = tot_count /vol / denom |
|
|
|
|
FSSP total mass |
|
g |
mass = sum of counts * volumes |
*2 |
|
|
144 |
FSSP water |
|
1 g/m3 |
water = mass / vol / denom * 1.E6 |
|
|
|
145 |
Probe Activity |
|
1 % |
Raw / 10 |
|
|
|
147 |
2DC Total Shadow Or |
|
1 #/s |
Raw |
|
|
|
148 |
FSSP equivalent diameter |
|
1 mm |
Ratio of sum of diam **3 to sum of diam **2 |
*2 |
|
|
149 |
FSSP equivalent diameter variance |
|
1 mm |
Consult listing or Appendix B (Data Dictionary) |
*2 |
|
|
150 |
Hail size counts |
|
1 #/s per channel |
Raw |
|
|
|
151 |
Slow Particle |
|
1 #/s |
Raw |
|
|
|
152 |
Hail total counts |
|
1 number |
Sum of tag 150s (tot_counts) |
|
|
|
153 |
Hail ave diameter |
|
1 cm |
sum of diams / tot_counts |
|
|
|
154 |
Hail concentration |
|
1 #/m3 |
conc = tot_counts / (0. 1 * tas) 0.1 m2 = sampling area (0.1 * tas) = volume sampling rate (m3/s) |
|
|
|
|
Hail total mass |
|
g |
mass = Si counti * volumei * 0.9 (0.9 g/cm3 = hail density) |
|
|
|
155 |
Hail water |
|
1 g/m3 |
water = mass / (0. 1 * tas) |
|
|
|
160 |
Top field mill, low res |
(20 Hz) |
1 kV/m |
-2.1117E-2 * Raw + 0.094 |
*3 |
|
|
161 |
Bottom field mill, low res |
(20 Hz) |
1 kV/m |
-9.915E-2 * Raw + 6.4869 |
*3 |
|
|
162 |
Left field mill, low res |
(20 Hz) |
1 kV/m |
-9.6722E-2 * Raw – 0.9751 |
*3 |
|
|
163 |
Right field mill, low res |
(20 Hz) |
1 kV/m |
-9.7641E-2 * Raw – 2.0759 |
*3 |
|
|
168 |
Fifth field mill, low res |
(20 Hz) |
1 kV/m |
-2.0911E-2 * Raw + 0.1814 |
*3 |
|
|
169 |
Sixth field mill, low res |
(20 Hz) |
1 kV/m |
-3.8462E-2 * Raw – 0.16678 |
*3 |
|
|
170 |
HVPS Total Shadow Or |
|
1 counts |
Raw |
|
|
|
171 |
HVPS Housekeeping Data |
|
oC oC oC oC oC V V V |
hvps1 = Raw(0) *0.001 - 23.1 hvps2 = Raw(1)*0.001 - 23.1 hvps3 = Raw(2)*0.001 - 23.1 hvps4 = Raw(3)*0.001 - 23.1 hvps5 = Raw(4)*0.001 - 23.1 hvps6 = Raw(5)*0.0001 - 0.0 hvps7 = Raw(6)*0.0001 - 0.0 hvps8 = Raw(7)*0.0006 - 0.0 |
|
|
|
172 |
GPS latitude |
|
1 deg |
degree + (minute + hundredths/100)/60 |
|
|
|
173 |
GPS longitude |
|
1 deg |
degree + (minute + hundredths/100)/60 |
|
|
|
174 |
GPS groundspeed |
|
1 m/s |
1852 / 36000 * Raw |
|
|
|
175 |
GPS grnd track angle (mag N) |
|
1 deg |
Raw / 10 |
|
|
|
176 |
GPS magnetic deviation |
|
1 deg |
Raw / 10 (Raw is 32-bits, not 16) |
|
|
|
177 |
GPS time since solution |
|
1 s |
(Raw + 6.0) /10 |
||
|
|
GPS altitude |
|
1 m |
Raw / 10 / .3048 (Raw is 32-bits, not 16) |
|
|
|
179 |
GPS time |
|
UT |
Time recorded from T-28 GPS receiver. |
|
|
|
185 |
GPS ROC |
|
1 m/s |
10ths of ft/min to m/s |
|
|
|
186 |
DMT LW |
|
|
p = 20 * Raw * 1.5258789E-4 (p = Power) tlfm = (twk + tk)/2 (twk = Wire Temp; tk = Air Temp) |
|
|
|
|
Thermal Conductivity |
|
|
cnd = 5.8E-5 *
(398 / (125 + tflm)) * (tflm/273)^1.5 |
|
|
|
|
Viscosity |
|
|
visc = 1.718E–4
* (393/(120 + tflm)) * (tflm/273) ^1.5 |
|
|
|
|
Density |
|
|
dens = pres / (2870.5 * tflm) (pres = Pressure) |
|
|
|
|
|
|
|
fct = p * l * cnd * (twk – tk) (l = Wire Length) |
|
|
|
|
Reynold's Number |
|
|
re = 100 * dens * tas * d/visc (tas = True Air Speed) |
|
|
|
|
Prandtl Numbers |
|
|
prf = 0.24 * visc/cnd prw = 0.24 * vscw/cndw |
|
|
|
|
Dry Air Loss |
|
|
dryp = 0.26 * re ^ 0.6 * prf ^ 0.37 * (prf/prw) ^ 0.25 *
fct/0.239 |
*5 |
|
|
|
LWC |
|
1 g/m3 |
lwc = (p–dryp) * fact |
|
|
|
187 |
DMT Slave |
|
1 V |
Raw |
|
|
|
190 |
FSSP gated strobes |
|
1 number |
Raw |
|
|
|
191 |
FSSP total strobes (div by 10) |
|
1 number |
Raw |
|
|
|
192 |
FSSP reference voltage |
|
1 volts |
Raw / 25.5 |
|
|
|
193 |
Magnetic Heading |
|
1 deg |
(Raw*5.49316E-3 – 7.0) + 8.5 – 121.9 |
|
|
|
205 |
Altitude |
|
1 m |
4.43077E4 * (1 –(stat_pr/l013.3027)**.190284) (stat_pr = Static Pressure) |
|
|
|
206 |
qe |
|
1 K |
|
|
|
|
|
Saturation Vapor Pressure |
|
mb |
svp = 6.1078 * exp(17.26939 * rft/(tempk–35.86)) |
|
|
|
|
Saturation Mixing Ratio |
|
kg/kg |
smr = svp / (stat_pr – svp) * 0.622 |
|
|
|
|
|
|
K |
ts = tempk * (1000/stat_pr) **0.286 |
|
|
|
|
qe |
|
K |
thetae = ts * exp(597.3*smr)/(0.24*tempk)) |
|
|
|
207 |
Saturation mixing ratio |
|
1 kg/kg |
smr from above |
|
|
|
208 |
Point dz/dt |
|
1 m/s |
(alt - prev_alt (2 secs previous)) / 2 |
|
|
|
209 |
Indicated airspeed |
|
1 m/s |
c = 1 + dyn_pr / 1013.3027 |
|
|
|
|
|
|
|
ias =
sqrt(5.79E5*(c**(2/7)–l)) |
|
|
|
210 |
Updraft (uncorrected) |
|
1 m/s |
ul = change in alt ((i + l) – (i–1))/2 |
|
|
|
|
|
|
|
u2 = (27 - man_pr) * 92 |
|
|
|
|
|
|
|
u3 = (1.94254 * ias - 140) * 17.7 |
|
|
|
|
|
|
|
updr = ul + (u2 + u3) * 0.00508 |
|
|
|
211 |
Calculated TAS |
|
1 m/s |
sqrt(rftuc * mach2 * 401.856/divisor) |
|
|
|
212 |
Updraft correction factor |
|
1 m/s |
gps_gs1 * (gps_gs0 – gps_gs2)/2/9.775 (gps_gs = GPS Groundspeed) 0) current 1) 1 sec previous 2) 2 secs previous |
|
|
|
213 |
Cooper Updraft |
|
1 m/s |
updraft + updraft correction factor |
|
|
|
214 |
Kopp Updraft |
|
1 m/s |
dens = 0.34838 * stat_pr /tempk |
|
|
|
|
|
|
|
ang = pitch * 0.0174533 |
|
|
|
|
|
|
|
Kopp = ul + (62.12/.65) * accel * 9.775/(dens * calc_tas) |
*6 |
|
|
|
|
|
|
–((0.02028/.65)+ang) * calc_tas |
|
|
|
216 |
Turbulence |
|
1 cm2/3/s |
Weighted sum of calculated TAS power spectrum |
|
|
|
|
|
|
|
Static and dynamic pressure values, along with RFTs, are fed into a fast Fourier transform routine. Consult program listing. |
|
|
|
217 |
Air density |
|
1 kg/m3 |
0.34838 * stat_pr / tempk |
|
|
|
219 |
FSSP mixing ratio |
|
1 g/kg |
FSSP_water / density |
|
|
|
220 |
Hail mixing ratio |
|
1 g/kg |
hail_water / density |
|
|
|
221 |
RFT uncorrected |
|
1 deg C |
temp = (3.02109E-3 * Raw + 222.06)-273.16 |
|
|
|
244 |
JW equiv water |
|
1 g/m3 |
mass = sum of counts * volumes, where diams > 30 |
|
|
|
|
|
|
|
microns are treated as equal to 30 |
|
|
|
|
|
|
|
water = mass/vol/denom * 1.E6 |
|
|
|
260 |
Ambient vert EF |
|
1 kV/m |
(tfm - 2 * bfm) / 11.2 |
|
|
|
261 |
Plane vert EF |
|
1 kV/m |
(tfm + 2 * bfm) / 11.2 |
|
|
|
262 |
Ambient lateral EF |
|
1 kV/m |
(lfm – rfm) / 44.8 |
|
|
|
263 |
Plane lateral EF |
|
1 kV/m |
(rfm + lfm) / 32.48 |
|
|
|
264 |
Ambient vert EF (with roll) |
|
1 kV/m |
cosr = cos(roll_rad) |
|
|
|
|
|
|
|
sinr = sin(roll_rad) |
|
|
|
|
|
|
|
t264 = t260 *
cosr + t262 * sinr |
|
|
|
265 |
Ambient lat EF (with roll) |
|
1 kV/m |
t265 = -t260 *
sinr + t262 * cosr |
|
|
|
266 |
Exq –- Earth |
|
1 kV/m |
sinpitch = sin(pitch_rad) cospitch = cos(pitch_rad) |
*3 |
|
|
|
|
|
|
grd_trk_ang_deg = fmod(grd_trk_ang_deg + mag_dev, 360.) grd_trk_ang_rad = 0.017453 * grd_trk_ang_deg |
|
|
|
|
|
|
|
sinhead = sin(grd_trk_ang_rad) coshead = cos(grd_trk_ang_rad) |
|
|
|
|
|
|
|
Ex = (fm6 – 1.1*fm5)/4.5 |
|
|
|
|
|
|
|
Ey = (lfm-rfm)/44.5 |
|
|
|
|
|
|
|
Ez = ((3.6821 * lfm + rfm) * 0.132 – fm5) * 0.1735 |
|
|
|
|
|
|
|
cpExsh = cospitch * Ex * sinhead Ezcrshsp = Ez * cosroll * sinhead * sinpitch |
|
|
|
|
|
|
|
Ezchsr = Ez * coshead * cosroll Eychcr = Ey * coshead * cosroll |
|
|
|
|
|
|
|
Eyshspsr = Ey * sinhead * sinpitch * sinroll |
|
|
|
|
|
|
|
Exq = cpExsh – Ezcrshsp + Ezchsr – Eychcr - Eyshspsr |
|
|
|
|
Ey --- Earth |
|
1 kV/m |
Exchcp = Ex * coshead * cospitch Ezchcrsp = Ez * coshead * cosroll * sinpitch |
*3 |
|
|
|
|
|
|
Ezshsr = Ez * sinhead * sinroll Eycrsh = Ey * cosroll * sinhead |
|
|
|
|
|
|
|
Eychspsr = Ey * coshead * sinpitch * sinroll |
|
|
|
|
|
|
|
Ey = Exchcp - Ezchcrsp - Ezshsr + Eycrsh - Eychspsr |
|
|
|
268 |
Ez – Earth |
|
1 kV/m |
Ezcpcr = Ez * cospitch * cosroll Exsp = Ex * sinpitch |
*3 |
|
|
|
|
|
|
Eycpsr = Ey * cospitch * sinroll |
|
|
|
|
|
|
|
Ez = Ezcpcr + Exsp + Eycpsr |
|
|
|
269 |
Ey (267) - Ey (265) |
|
1 kV/m |
t267 - t265 (Difference between old and new methods) |
|
|
|
270 |
Ez (268) - Ez (264) |
|
1 kV/m |
t268 - t264 (Difference between old and new methods) |
|
|
|
272 |
GPS deg lat |
|
1 deg |
integer portion
of tag 172 (t172) |
|
|
|
273 |
GPS min lat |
|
1 min |
fractional part of t172 * 60 |
|
|
|
274 |
GPS deg long |
|
1 deg |
integer portion
of tag 173 (t173) |
|
|
|
275 |
GPS min long |
|
1 min |
fractional part of t173 * 60 |
|
|
|
276 |
GPS true bearing |
|
1 deg |
mod(t175 + t176 + 360,360) |
|
|
|
290 |
Accelerometer (x) |
(20 Hz) |
1 g's |
(Raw * 1.5258789E-4 – 2.511) / 0.492 |
|
|
|
291 |
Accelerometer (y) |
(20 Hz) |
1 g's |
(Raw * 1.5258789E-4 – 2.49) / 0.5 |
|
|
|
292 |
Accelerometer (z) |
(20 Hz) |
1 g's |
(Raw * 1.5258789E-4 – 2.513) / 0.503 |
|
|
*1 In some cases the equation variables are averages. Consult the listing for exact details. All quantities are recorded at 1 Hz unless otherwise noted.
*2 Calibration of the FSSP (via
bead test) led to new channel assignments in file FSSP.CHN for 2002.
*3 The raw count for the electric field mills are multiplied with “-1”, so that a positive voltage out of the meter yielded a negative value of the electric field it was sensing. (valid for 2002 project)
*4 Data recorded and processed as UTC.
*5
DMT LW formula corrected after
STEPS project.
*6
Kopp updraft formula adjusted
during STEPS.