-
Notifications
You must be signed in to change notification settings - Fork 1
Examples using R
To use the tests decide which tests are relevant to the analysis and are to be used for masking data from the analysis. For this example, all tests with an assessment of "Bad" are chosen, except for test 4 which is not in use. The tests used are [1, 2, 3, 5]. Test 6 is described as "Indeterminate" and will not be part of the mask applied to the data.
For this example, assume the data and companion QC fields are ten element vectors and equal to the values
direct_normal_broadband <- c(10, 9, -1, -0.1, 11, 20, 50, 110, 12200, 134)
qc_direct_normal_broadband <- c(16, 16, 3, 1, 0, 0, 0, 0, 4, 32)
Create a vector of tests to use in the mask to exclude values
tests <- c(1, 2, 3, 5)
Create a single integer mask combining all the tests of interest from the 'tests' vector.
mask_test <- sum(bitwShiftL(1, tests-1))
print(intToBits(mask_test))
# Note the integer to bits function returns the binary representation from left to
# right vs. the standard notation of right to left.
[1] 01 01 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Search the companion QC field for at least one of the tests to be tripped, and replace those values with IEEE NaN value to exclude from further analysis. Could also use R 'NA' value.
mask <- bitwAnd(qc_direct_normal_broadband, mask_test) != 0
print(mask)
[1] TRUE TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE
direct_normal_broadband[mask] <- NaN
print(direct_normal_broadband)
[1] NaN NaN NaN NaN 11 20 50 110 NaN 134
There you go, four lines of code and you're done! If a different set of tests were desired the only part that needs to change is 'tests', the rest of the code is independent of datastream or analysis. This will also work with multi-dimensional data variables.
If you're interested in a description using the bits directly you can find a description here