DATA REDUCTION FORMULAE TABLE

 

Reduced Data Items Computed for Norman Project

Norman, Oklahoma,  May 13 - June 11, 2003 (see footnote *1)

 

 

 

# Values

 

 

Tag #

Description

Output

Units

Method of Computation

 

 

 

 

 

 

 

101

Dynamic Pressure #1

(20 Hz)

1       mb

6.30452E–3 * Raw – 0.0489

102

Dynamic Pressure #2

(20 Hz)

1       mb

5.28371E–3 * Raw – 1.5768

 

103

Static Pressure #1

(20 Hz)

1       mb

1.5791E–2 * Raw + 530.37

 

104

Static Pressure #2

(20 Hz)

1       mb

1.0917E–2 * Raw + 691.92

 

105

Rate of Climb

(20 Hz)

1       m/s

5.625E–4 * Raw, for Raw > = 0

 

 

 

 

 

5.287E–4 * Raw, for Raw < 0

 

106

Rosemount Temp

 

1       deg C

mach2 = 5*((l +dyn_pr/stat_pr)**(2/7)-l)

 

 

 

 

 

divisor = 1 + 0.195 * mach2

 

 

 

 

 

temp = (1.83105E-3 * Raw + 243.16)/divisor-273.16

 

107

Reverse Flow Temp

 

1       deg C

divisor = 1 + 0.1594 * mach2

 

 

 

 

 

temp = (3.02109E-3 * Raw + 222.06)/divisor-273.16

 

108

Manifold Pressure

 

1       " Hg

3.1098E-3 * Raw + 0.159275

 

109

Acceleration

(20 Hz)

1       g's

6.25E–5 * Raw [prior to 7/14/94]

 

 

 

 

 

6.25E–5 * Raw + 1.0 [after 7/14/94]

 

110

Pitch

(20 Hz)

1       deg

–3.05175E–3 * Raw + 50

 

111

Roll

(20 Hz)

1       deg

3.05175E–3 * Raw – 50

 

116

Voltage Regulator

 

1       volts

1.5258789E–4 * Raw

 

118

NCAR true air speed

 

1       m/s

3.96744E–3 * Raw

 

119

PMS End Element 1

 

1      volts

 

 

120

PMS End Element 2

 

1      volts

 

 

121

Interior Temp (computer)

 

1       deg C

3.05175E–2 * Raw

 

123

High Voltage Current

 

1      mA

1.5258789E-4 * Raw

 

124

Heater current

 

1       amp

3.05175E–3 * Raw

 

130

Event Code bits

 

1       flags

bit 0 = 1 --> system running

 

 

 

 

 

bit 1 = 0 -- > in cloud

 

 

 

 

 

bit 2 = 0 -- > foil on

 

131

GPS warning codes

 

1       flags

11 bit codes

 

140

FSSP counts

 

15     number

Raw

 

Text Box: 41141

FSSP total counts

 

1       number

Sum of tag 140s (tot_count)

 

142

FSSP ave diameter

 

1       mm

sum of diams / (tot_count)

*2

143

FSSP concentration

 

         cm3

vol = 0.22275 * tas  (volume sampling rate (m3/s))
(.22275 mm2 = sampling area)

*2

 

 

 

 

denom = 1 – .55 * activ / 100     (adjustment for probe activity)

 

 

 

 

1       #/cm3

conc = tot_count /vol / denom

 

 

FSSP total mass

 

         g

mass = sum of counts * volumes

*2

144

FSSP water

 

1       g/m3

water = mass / vol / denom * 1.E6

 

145

Probe Activity

 

1       %

Raw / 10

 

147

2DC Total Shadow Or

 

1      #/s

Raw

 

148

FSSP equivalent diameter

 

1       mm

Ratio of sum of diam **3 to sum of diam  **2

*2

149

FSSP equivalent diameter variance

 

1       mm

Consult listing or Appendix B (Data Dictionary)

*2

150

Hail size counts

 

1   #/s per channel

Raw

 

151

Slow Particle

 

1   #/s

Raw

 

152

Hail total counts

 

1       number

Sum of tag 150s (tot_counts)

 

153

Hail ave diameter

 

1       cm

sum of diams / tot_counts

 

154

Hail concentration

 

1       #/m3

conc = tot_counts / (0. 1 * tas) 

0.1 m2 = sampling area 

(0.1 * tas) = volume sampling rate (m3/s)

 

 

Hail total mass     

 

         g

mass = Si counti * volumei * 0.9 (0.9 g/cm3 = hail density)

 

155

Hail water

 

1       g/m3

water = mass / (0. 1 * tas)

 

160

Top field mill, low res

(20 Hz)

1       kV/m

-2.1117E-2 * Raw + 0.094

*3

161

Bottom field mill, low res

(20 Hz)

1       kV/m

-9.915E-2 * Raw + 6.4869

*3

162

Left field mill, low res

(20 Hz)

1       kV/m

-9.6722E-2 * Raw – 0.9751

*3

163

Right field mill, low res

(20 Hz)

1       kV/m

-9.7641E-2 * Raw – 2.0759

*3

168

Fifth field mill, low res

(20 Hz)

1       kV/m

-2.0911E-2 * Raw + 0.1814

*3

169

Sixth field mill, low res

(20 Hz)

1       kV/m

-3.8462E-2 * Raw – 0.16678

*3

170

HVPS Total Shadow Or

 

1      counts

Raw

 

171

HVPS Housekeeping Data

 

oC

oC

oC

oC

oC

       V

V

V

 hvps1 = Raw(0) *0.001 - 23.1

 hvps2 = Raw(1)*0.001 - 23.1             

 hvps3 = Raw(2)*0.001 - 23.1             

 hvps4 = Raw(3)*0.001 - 23.1             

 hvps5 = Raw(4)*0.001 - 23.1

 hvps6 = Raw(5)*0.0001 - 0.0             

 hvps7 = Raw(6)*0.0001 - 0.0             

 hvps8 = Raw(7)*0.0006 - 0.0

 

172

GPS latitude

 

1       deg

degree + (minute + hundredths/100)/60

 

173

GPS longitude

 

1       deg

degree + (minute + hundredths/100)/60

 

174

GPS groundspeed

 

1       m/s

1852 / 36000 * Raw

 

175

GPS grnd track angle (mag N)

 

1       deg

Raw / 10

 

176

GPS magnetic deviation

 

1       deg

Raw / 10 (Raw is 32-bits, not 16)

 

177

GPS time since solution

 

1       s

(Raw + 6.0) /10

*4

178

GPS altitude

 

1       m

Raw / 10 / .3048 (Raw is 32-bits, not 16)

 

179

GPS time

 

UT

Time recorded from T-28 GPS receiver.

 

185

GPS ROC

 

1       m/s

10ths of ft/min to m/s

 

186

DMT LW

 

        

p = 20 * Raw * 1.5258789E-4 (p = Power)

tlfm = (twk + tk)/2    (twk = Wire  Temp; tk = Air Temp)

 

 

Thermal Conductivity

 

 

cnd = 5.8E-5 * (398 / (125 + tflm)) * (tflm/273)^1.5
cndw = 5.8E–5 * (398/(125 + twk))* (twk/273)^ 1.5

 

 

Viscosity

 

 

visc = 1.718E–4 * (393/(120 + tflm)) * (tflm/273) ^1.5
vscw = 1.718E–4 * (393/(120 + twk)) * (twk/273)^1.5

 

 

Density

 

 

dens = pres / (2870.5 * tflm)   (pres = Pressure)

 

 

 

 

 

fct = p * l * cnd * (twk – tk) (l = Wire Length)

 

 

Reynold's Number

 

 

re = 100 * dens * tas * d/visc  (tas = True Air Speed)

 

 

Prandtl Numbers

 

 

prf = 0.24 * visc/cnd

prw = 0.24 * vscw/cndw

 

 

Dry Air Loss

 

 

dryp = 0.26 * re ^ 0.6 * prf ^ 0.37 * (prf/prw) ^ 0.25 * fct/0.239
fact = 1.238E6 * 0.239/((2.0*0.19)*(tas * 100 * (597.3 + 373.16 – tk)))

*5

 

LWC

 

1       g/m3

lwc = (p–dryp) * fact

 

187

DMT Slave

 

1      V

Raw

 

190

FSSP gated strobes

 

1       number

Raw

 

191

FSSP total strobes (div by 10)

 

1       number

Raw

 

192

FSSP reference voltage

 

1       volts

Raw / 25.5

 

193

Heading

 

1      deg

(Raw*5.49316E-3 – 7.0) + 8.5 – 121.9

 

205

Altitude

 

1       m

4.43077E4 * (1 –(stat_pr/l013.3027)**.190284) (stat_pr = Static Pressure)

 

206

qe

 

1       K

 

 

 

Saturation Vapor Pressure

 

         mb

svp = 6.1078 * exp(17.26939 * rft/(tempk–35.86))  
(tempk = RFT in K)

 

 

Saturation Mixing Ratio

 

         kg/kg

smr = svp / (stat_pr – svp) * 0.622

 

 

 

 

         K

ts = tempk * (1000/stat_pr) **0.286

 

 

qe

 

         K

thetae = ts * exp(597.3*smr)/(0.24*tempk))

 

207

Saturation mixing ratio

 

1       kg/kg

smr from above

 

208

Point dz/dt

 

1       m/s

(alt - prev_alt (2 secs previous)) / 2

 

209

Indicated airspeed

 

1       m/s

c = 1 + dyn_pr / 1013.3027

 

 

 

 

ias = sqrt(5.79E5*(c**(2/7)–l))

 

210

Updraft (uncorrected)

 

1       m/s

ul = change in alt ((i + l) – (i–1))/2

 

 

 

 

 

u2 = (27 - man_pr) * 92

 

 

 

 

 

u3 = (1.94254 * ias - 140) * 17.7

 

 

 

 

 

updr = ul + (u2 + u3) * 0.00508

 

211

Calculated TAS

 

1       m/s

sqrt(rftuc * mach2 * 401.856/divisor)

 

212

Updraft correction factor

 

1       m/s

gps_gs1 * (gps_gs0 – gps­_gs2)/2/9.775 (gps_gs = GPS Groundspeed) 0) current   1) 1 sec previous   2) 2 secs previous

 

213

Cooper Updraft

 

1       m/s

updraft + updraft correction factor

 

214

Kopp Updraft

 

1       m/s

dens = 0.34838 * stat_pr /tempk

 

 

 

 

 

ang = pitch * 0.0174533

 

 

 

 

 

Kopp = ul + (62.12/.65) * accel * 9.775/(dens * calc_tas)

*6

 

 

 

 

–((0.02028/.65)+ang) * calc_tas

 

216

Turbulence

 

1      cm2/3/s

Weighted sum of calculated TAS power spectrum

 

 

 

 

 

Static and dynamic pressure values, along with RFTs, are fed into a fast Fourier transform routine. Consult program listing.

 

217

Air density

 

1       kg/m3

0.34838 * stat_pr / tempk

 

219

FSSP mixing ratio

 

1       g/kg

FSSP_water / density

 

220

Hail mixing ratio

 

1       g/kg

hail_water / density

 

221

RFT uncorrected

 

1       deg C

temp = (3.02109E-3 * Raw + 222.06)-273.16

 

244

JW equiv water

 

1       g/m3

mass = sum of counts * volumes, where diams > 30

 

 

 

 

 

microns are treated as equal to 30

 

 

 

 

 

water = mass/vol/denom * 1.E6

 

260

Ambient vert EF

 

1       kV/m

(tfm - 2 * bfm) / 11.2

 

261

Plane vert EF

 

1       kV/m

(tfm + 2 * bfm) / 11.2

 

262

Ambient lateral EF

 

1       kV/m

 (lfm – rfm) / 44.8

 

263

Plane lateral EF

 

1       kV/m

(rfm + lfm) / 32.48

 

264

Ambient vert EF (with roll)

 

1       kV/m

cosr = cos(roll_rad)

 

 

 

 

 

sinr = sin(roll_rad)

 

 

 

 

 

t264 = t260 * cosr + t262 * sinr

 

265

Ambient lat EF (with roll)

 

1       kV/m

t265 = -t260 * sinr + t262 * cosr

 

266

Exq –- Earth

 

1       kV/m

sinpitch = sin(pitch_rad)

cospitch = cos(pitch_rad)

*3

 

 

 

 

grd_trk_ang_deg = fmod(grd_trk_ang_deg + mag_dev, 360.)

 grd_trk_ang_rad = 0.017453 * grd_trk_ang_deg

 

 

 

 

 

sinhead = sin(grd_trk_ang_rad)

coshead = cos(grd_trk_ang_rad)

 

 

 

 

 

Ex = (fm6 – 1.1*fm5)/4.5

 

 

 

 

 

Ey = (lfm-rfm)/44.5

 

 

 

 

 

Ez = ((3.6821 * lfm + rfm) * 0.132 – fm5) * 0.1735

 

 

 

 

 

cpExsh = cospitch * Ex * sinhead

Ezcrshsp = Ez * cosroll * sinhead * sinpitch

 

 

 

 

 

Ezchsr = Ez * coshead * cosroll

Eychcr = Ey * coshead * cosroll

 

 

 

 

 

Eyshspsr = Ey * sinhead * sinpitch * sinroll

 

 

 

 

 

Exq = cpExsh – Ezcrshsp + Ezchsr – Eychcr - Eyshspsr

 

267

Ey  --- Earth

 

1       kV/m

Exchcp = Ex * coshead * cospitch

Ezchcrsp = Ez * coshead * cosroll * sinpitch

*3

 

 

 

 

Ezshsr = Ez * sinhead * sinroll

Eycrsh = Ey * cosroll * sinhead

 

 

 

 

 

Eychspsr = Ey * coshead * sinpitch * sinroll

 

 

 

 

 

Ey = Exchcp - Ezchcrsp - Ezshsr + Eycrsh - Eychspsr

 

268

Ez – Earth

 

1       kV/m

Ezcpcr = Ez * cospitch * cosroll

Exsp = Ex * sinpitch

*3

 

 

 

 

Eycpsr = Ey * cospitch * sinroll

 

 

 

 

 

Ez = Ezcpcr + Exsp + Eycpsr

 

269

Ey (267) - Ey (265)

 

1       kV/m

t267 - t265  (Difference between old and new methods)

 

270

Ez (268) - Ez (264)

 

1       kV/m

t268 - t264  (Difference between old and new methods)

 

272

GPS deg lat

 

1       deg

integer portion of tag 172 (t172)

 

273

GPS min lat

 

1       min

fractional part of t172 * 60

 

274

GPS deg long

 

1       deg

integer portion of tag 173 (t173)

 

275

GPS min long

 

1       min

fractional part of t173 * 60

 

276

GPS true bearing

 

1       deg

mod(t175 + t176 + 360,360)

 

290

Accelerometer (x)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.511) / 0.492

 

291

Accelerometer (y)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.49) / 0.5

 

292

Accelerometer (z)

(20 Hz)

1       g's

(Raw * 1.5258789E-4 – 2.513) / 0.503

 

*1          In some cases the equation variables are averages.  Consult the listing for exact details.  All quantities are recorded at 1 Hz unless otherwise noted.


*2          Calibration of the FSSP (via bead test) led to new channel assignments in file FSSP.CHN for 2003.

       *3         The raw count for the electric field mills are multiplied with “-1”, so that a positive voltage out of the meter yielded a negative value of the electric field it was sensing. (valid for 2003 project)

*4         Data recorded and processed as UTC.

*5         DMT LW formula corrected after Norman project.

*6         Kopp updraft formula adjusted during Norman project.