DATA REDUCTION FORMULA TABLE

Tag #

Description

Output

Units

Method of Computation

101

Dynamic Pressure #1

(20 Hz)

1       mb

6.30452E–3 * Raw – 0.0489

102

Dynamic Pressure #2

(20 Hz)

1       mb

5.28371E–3 * Raw – 1.5768

103

Static Pressure #1

(20 Hz)

1       mb

1.5791E–2 * Raw + 530.37

104

Static Pressure #2

(20 Hz)

1       mb

1.0917E–2 * Raw + 691.92

105

Rate of Climb

(20 Hz)

1       m/s

5.625E–4 * Raw, for Raw > = 0

5.287E–4 * Raw, for Raw < 0

106

Rosemount Temp

1       deg C

mach2 = 5*((l +dyn_pr/stat_pr)**(2/7)-l)

divisor = 1 + 0.195 * mach2

temp = (1.83105E-3 * Raw + 243.16)/divisor-273.16

107

Reverse Flow Temp

1       deg C

divisor = 1 + 0.1594 * mach2

temp = (3.02109E-3 * Raw + 222.06)/divisor-273.16

108

Manifold Pressure

1       " Hg

3.1098E-3 * Raw + 0.159275

109

Acceleration

(20 Hz)

1       g's

6.25E–5 * Raw [prior to 7/14/94]

6.25E–5 * Raw + 1.0 [after 7/14/94]

110

Pitch

(20 Hz)

1       deg

–3.05175E–3 * Raw + 50

111

Roll

(20 Hz)

1       deg

3.05175E–3 * Raw – 50

116

Voltage Regulator

1       volts

1.5258789E–4 * Raw

118

NCAR true air speed

1       m/s

3.96744E–3 * Raw

119

PMS End Element 1

1      volts

120

PMS End Element 2

1      volts

121

Interior Temp (computer)

1       deg C

3.05175E–2 * Raw

123

High Voltage Current

1      mA

1.5258789E-4 * Raw

124

Heater current

1       amp

3.05175E–3 * Raw

130

Event Code bits

1       flags

bit 0 = 1 --> system running

bit 1 = 0 -- > in cloud

bit 2 = 0 -- > foil on

131

GPS warning codes

1       flags

11 bit codes

140

FSSP counts

15     number

Raw

141

FSSP total counts

1       number

Sum of tag 140s (tot_count)

142

FSSP ave diameter

1       mm

sum of diams / (tot_count)

*2

143

FSSP concentration

         cm³

vol = 0.22275 * tas  (volume sampling rate (m³/s))
(.22275 mm² = sampling area)

*2

denom = 1 – .55 * activ / 100     (adjustment for probe activity)

1       #/cm³

conc = tot_count /vol / denom

FSSP total mass

         g

mass = sum of counts * volumes

*2

144

FSSP water

1       g/m³

water = mass / vol / denom * 1.E6

145

Probe Activity

1       %

Raw / 10

147

2DC Total Shadow Or

1      #/s

Raw

148

FSSP equivalent diameter

1       mm

Ratio of sum of diam **3 to sum of diam  **2

*2

149

FSSP equivalent diameter variance

1       mm

Consult listing or Appendix B (Data Dictionary)

*2

150

Hail size counts

1   #/s per channel

Raw

151

Slow Particle

1   #/s

Raw

152

Hail total counts

1       number

Sum of tag 150s (tot_counts)

153

Hail ave diameter

1       cm

sum of diams / tot_counts

154

Hail concentration

1       #/m³

conc = tot_counts / (0. 1 * tas) 

0.1 m² = sampling area 

(0.1 * tas) = volume sampling rate (m³/s)

Hail total mass     

         g

mass = S_i count_i * volume_i * 0.9 (0.9 g/cm³ = hail density)

155

Hail water

1       g/m³

water = mass / (0. 1 * tas)

160

Top field mill, low res

(20 Hz)

1       kV/m

-2.1117E-2 * Raw + 0.094

*3

161

Bottom field mill, low res

(20 Hz)

1       kV/m

-9.915E-2 * Raw + 6.4869

*3

162

Left field mill, low res

(20 Hz)

1       kV/m

-9.6722E-2 * Raw – 0.9751

*3

163

Right field mill, low res

(20 Hz)

1       kV/m

-9.7641E-2 * Raw – 2.0759

*3

168

Fifth field mill, low res

(20 Hz)

1       kV/m

-2.0911E-2 * Raw + 0.1814

*3

169

Sixth field mill, low res

(20 Hz)

1       kV/m

-3.8462E-2 * Raw – 0.16678

*3

170

HVPS Total Shadow Or

1      counts

Raw

171

HVPS Housekeeping Data

^oC

^oC

^oC

^oC

^oC

       V

V

^V

 hvps1 = Raw(0) *0.001 - 23.1

 hvps2 = Raw(1)*0.001 - 23.1             

 hvps3 = Raw(2)*0.001 - 23.1             

 hvps4 = Raw(3)*0.001 - 23.1             

 hvps5 = Raw(4)*0.001 - 23.1

 hvps6 = Raw(5)*0.0001 - 0.0             

 hvps7 = Raw(6)*0.0001 - 0.0             

 hvps8 = Raw(7)*0.0006 - 0.0

172

GPS latitude

1       deg

degree + (minute + hundredths/100)/60

173

GPS longitude

1       deg

degree + (minute + hundredths/100)/60

174

GPS groundspeed

1       m/s

1852 / 36000 * Raw

175

GPS grnd track angle (mag N)

1       deg

Raw / 10

176

GPS magnetic deviation

1       deg

Raw / 10 (Raw is 32-bits, not 16)

177

GPS time since solution

1       s

(Raw + 6.0) /10

*4

178

GPS altitude

1       m

Raw / 10 / .3048 (Raw is 32-bits, not 16)

179

GPS time

UT

Time recorded from T-28 GPS receiver.

185

GPS ROC

1       m/s

10ths of ft/min to m/s

186

DMT LW

p = 20 * Raw * 1.5258789E-4 (p = Power)

tlfm = (twk + tk)/2    (twk = Wire  Temp; tk = Air Temp)

Thermal Conductivity

cnd = 5.8E-5 * (398 / (125 + tflm)) * (tflm/273)^1.5
cndw = 5.8E–5 * (398/(125 + twk))* (twk/273)^ 1.5

Viscosity

visc = 1.718E–4 * (393/(120 + tflm)) * (tflm/273) ^1.5
vscw = 1.718E–4 * (393/(120 + twk)) * (twk/273)^1.5

Density

dens = pres / (2870.5 * tflm)   (pres = Pressure)

fct = p * l * cnd * (twk – tk) (l = Wire Length)

Reynold's Number

re = 100 * dens * tas * d/visc  (tas = True Air Speed)

Prandtl Numbers

prf = 0.24 * visc/cnd

prw = 0.24 * vscw/cndw

Dry Air Loss

dryp = 0.26 * re ^ 0.6 * prf ^ 0.37 * (prf/prw) ^ 0.25 * fct/0.239
fact = 1.238E6 * 0.239/((2.0*0.19)*(tas * 100 * (597.3 + 373.16 – tk)))

*5

LWC

1       g/m³

lwc = (p–dryp) * fact

187

DMT Slave

1      V

Raw

190

FSSP gated strobes

1       number

Raw

191

FSSP total strobes (div by 10)

1       number

Raw

192

FSSP reference voltage

1       volts

Raw / 25.5

193

Heading

1      deg

(Raw*5.49316E-3 – 7.0) + 8.5 – 121.9

205

Altitude

1       m

4.43077E4 * (1 –(stat_pr/l013.3027)**.190284) (stat_pr = Static Pressure)

206

q_e

1       K

Saturation Vapor Pressure

         mb

svp = 6.1078 * exp(17.26939 * rft/(tempk–35.86))  
(tempk = RFT in K)

Saturation Mixing Ratio

         kg/kg

smr = svp / (stat_pr – svp) * 0.622

         K

ts = tempk * (1000/stat_pr) **0.286

q_e

         K

thetae = ts * exp(597.3*smr)/(0.24*tempk))

207

Saturation mixing ratio

1       kg/kg

smr from above

208

Point dz/dt

1       m/s

(alt - prev_alt (2 secs previous)) / 2

209

Indicated airspeed

1       m/s

c = 1 + dyn_pr / 1013.3027

ias = sqrt(5.79E5*(c**(2/7)–l))

210

Updraft (uncorrected)

1       m/s

ul = change in alt ((i + l) – (i–1))/2

u2 = (27 - man_pr) * 92

u3 = (1.94254 * ias - 140) * 17.7

updr = ul + (u2 + u3) * 0.00508

211

Calculated TAS

1       m/s

sqrt(rftuc * mach2 * 401.856/divisor)

212

Updraft correction factor

1       m/s

gps_gs₁ * (gps_gs₀ – gps­_gs₂)/2/9.775 (gps_gs = GPS Groundspeed) 0) current   1) 1 sec previous   2) 2 secs previous

213

Cooper Updraft

1       m/s

updraft + updraft correction factor

214

Kopp Updraft

1       m/s

dens = 0.34838 * stat_pr /tempk

ang = pitch * 0.0174533

Kopp = ul + (62.12/.65) * accel * 9.775/(dens * calc_tas)

*6

–((0.02028/.65)+ang) * calc_tas

216

Turbulence

1      cm^2/3/s

Weighted sum of calculated TAS power spectrum

Static and dynamic pressure values, along with RFTs, are fed into a fast Fourier transform routine. Consult program listing.

217

Air density

1       kg/m³

0.34838 * stat_pr / tempk

219

FSSP mixing ratio

1       g/kg

FSSP_water / density

220

Hail mixing ratio

1       g/kg

hail_water / density

221

RFT uncorrected

1       deg C

temp = (3.02109E-3 * Raw + 222.06)-273.16

244

JW equiv water

1       g/m³

mass = sum of counts * volumes, where diams > 30

microns are treated as equal to 30

water = mass/vol/denom * 1.E6

260

Ambient vert EF

1       kV/m

(tfm - 2 * bfm) / 11.2

261

Plane vert EF

1       kV/m

(tfm + 2 * bfm) / 11.2

262

Ambient lateral EF

1       kV/m

 (lfm – rfm) / 44.8

263

Plane lateral EF

1       kV/m

(rfm + lfm) / 32.48

264

Ambient vert EF (with roll)

1       kV/m

cosr = cos(roll_rad)

sinr = sin(roll_rad)

t264 = t260 * cosr + t262 * sinr

265

Ambient lat EF (with roll)

1       kV/m

t265 = -t260 * sinr + t262 * cosr

266

E_xq –- Earth

1       kV/m

sinpitch = sin(pitch_rad)

cospitch = cos(pitch_rad)

*3

grd_trk_ang_deg = fmod(grd_trk_ang_deg + mag_dev, 360.)

 grd_trk_ang_rad = 0.017453 * grd_trk_ang_deg

sinhead = sin(grd_trk_ang_rad)

coshead = cos(grd_trk_ang_rad)

Ex = (fm6 – 1.1*fm5)/4.5

Ey = (lfm-rfm)/44.5

Ez = ((3.6821 * lfm + rfm) * 0.132 – fm5) * 0.1735

cpExsh = cospitch * Ex * sinhead

Ezcrshsp = Ez * cosroll * sinhead * sinpitch

Ezchsr = Ez * coshead * cosroll

Eychcr = Ey * coshead * cosroll

Eyshspsr = Ey * sinhead * sinpitch * sinroll

E_xq = cpExsh – Ezcrshsp + Ezchsr – Eychcr - Eyshspsr

267

E_y  --- Earth

1       kV/m

Exchcp = Ex * coshead * cospitch

Ezchcrsp = Ez * coshead * cosroll * sinpitch

*3

Ezshsr = Ez * sinhead * sinroll

Eycrsh = Ey * cosroll * sinhead

Eychspsr = Ey * coshead * sinpitch * sinroll

Ey = Exchcp - Ezchcrsp - Ezshsr + Eycrsh - Eychspsr

268

E_z – Earth

1       kV/m

Ezcpcr = Ez * cospitch * cosroll

Exsp = Ex * sinpitch

*3

Eycpsr = Ey * cospitch * sinroll

Ez = Ezcpcr + Exsp + Eycpsr

269

E_y (267) - E_y (265)

1       kV/m

t267 - t265  (Difference between old and new methods)

270

E_z (268) - E_z (264)

1       kV/m

t268 - t264  (Difference between old and new methods)

272

GPS deg lat

1       deg

integer portion of tag 172 (t172)

273

GPS min lat

1       min

fractional part of t172 * 60

274

GPS deg long

1       deg

integer portion of tag 173 (t173)

275

GPS min long

1       min

fractional part of t173 * 60

276

GPS true bearing

1       deg

mod(t175 + t176 + 360,360)

290

Accelerometer (x)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.511) / 0.492

291

Accelerometer (y)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.49) / 0.5

292

Accelerometer (z)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.513) / 0.503