HI all:
I've been getting a mix of NANs showing up in the TOA5 files of a weather buoy with a CR1000. Here is an example from a file:
"TOA5","MOB","CR1000","47897","CR1000.Std.32.03","CPU:MOB.CR1","5126","Met"
"TIMESTAMP","RECORD","AirTemp","AirTemp_Avg","AirTempTRH","AirTempTRH_Avg","RH","RH_Avg","WaterTemp","WaterTemp_Avg","BP","BP_Avg","SolarRad","SolarRad_Avg","BattVoltage"
"TS","RN","decC","decC","degC","degC","%","%","degC","degC","hPa","hPa","w/m^2","w/m^2","vdc"
"","","Smp:Instantaneous air temperature (CS109)","Avg:Average air temperature (CS109)","Smp:Instantaneous air temperature (TRH probe)","Avg:Average air temperature (TRH probe)","Smp:Instantaneous relative humidity","Avg:Average relative humidity","Smp:Instantaneous water temperature","Avg:Average water temperature","Smp:Instantaneous atmospheric pressure","Avg:Average atmospheric pressure","Smp:Instantaneous solar radiation","Avg:Average solar radiation","Min:Minimum battery voltage"
"2020-07-23 16:15:00",8238,NAN,NAN,111.4,30.58,NAN,NAN,10.26,10.96,"NAN","NAN","NAN","NAN",NAN
"2020-07-23 16:30:00",8239,9.39,NAN,10.05,73.12,NAN,NAN,11.42,10.74,999.0234,"NAN",133.579,"NAN",NAN
"2020-07-23 16:45:00",8240,10.96,9.96,11.63,10.73,NAN,NAN,11.4,11.59,999.996,999.5905,195.9608,158.206,12.88
"2020-07-23 17:00:00",8241,11.4,11.16,11.91,11.65,NAN,NAN,11.42,11.52,999.9138,999.8487,199.374,215.003,12.95
The NANs without quotes didn't present a problem, but the "NAN"s did, my processing routines read those as text instead of numeric, which broke everything. I've written a workaround but am curious to know if anyone knows why some are being written as text and some are not. When the sensors are working the data is all numeric, so I'm not sure where the "text" NANs are coming from.
Thanks, Rob
Values of data type string will show up in your data table with the quotes around them. Columns that are saved as a IEEE4 data type can be NAN without the quotes. Check your data tables in your datalogger program to see which data types you have selected.
That's why I was a bit confused, if you look at the 2nd to last column (Average solar radiation) you can see it's a mix of numerics and NaNs as strings. Whereas the last column (Battery voltage) it's "numeric NaN" (i.e. no quotes) and also numerics. That got a bit ugly from the formatting, here's the lines:
"NAN",NAN
"NAN",NAN
158.206,12.88
215.003,12.95
I checked the table, the datatypes are all IEEE4 and FP2. It looks the the IEEE4 datatype corresponds to the "NAN" while FP2 is NAN. So for whatever reason, IEEE4 is throwing the string NANs.
In my experience, NANs in TOA5 always come wrapped in double-quotes. What version OS is producing these files?
This is probably a regression since FP2 used to be represented as -7999
OS is 32.03, so pretty current.
I have not seen double-quoted NANs in my TOA5s previously, I noticed these ones when they broke my processing routines which were expected numbers.
I'm not sure when the transition began. I'm perusing data files and see double-quoted NANs in IEEE4 fields in:
* CR3000.Std.32.05
* CR3000.Std.31.09
* CR3000.Std.28.02
* CR1000.Std.31.08
* CR1000.Std.32.05
* CR6.Std.07
I collect data files using LoggerNet fyi. Hopefully a Campbell rep can reply with more details.
I am running Loggernet 4.6.2 (the latest version). All NANs have quotes around them in the TOA5 file format. I see the same results on both a CR1000 and CR300.
"TOA5","CR300_Jacob","CR300","8253","CR300.Std.10.02","CPU:Test.CR300","17725","Test"
"TIMESTAMP","RECORD","FloatVal","FP2Val","LongVal","StringVal"
"TS","RN","","","",""
"","","Smp","Smp","Smp","Smp"
"2020-07-30 09:43:30",0,"NAN","NAN",-2147483648,"NAN"
Viewing a table with the Table Monitor on the Connect screen, the string field won't show the quotes. The quotes are present in the data file.
To not see the quotes you must be using TableFile in the datalogger.
I am seeing that FP2 formatted fields do not get the quotes, but the other types do. I will look into this more, because it is inconsistent with what Loggernet does.
The logger's program doesn't use TableFile, the file that I shared was sent out via FTPClient.
The program does save locally to a CF card. That subroutine uses GetRecord and FileWrite. I pulled the file from that day off the logger and had a look, none of the NaNs have quotes in that file.
When I look at the tables in Loggernet Connect I don't see any quotes on NaNs. All of the text fields (i.e. Strings) don't have quotes either though.
So perhaps an FTPClient thing.
GetRecord and FTPClient file streaming options use the same internal routine as TableFile for creating formatted data.
An operating system fix is now in the works. IEEE4 and strings work correctly. The problem is just data fields using FP2 format.
Glad to see this resolved so quickly 👍 Good timing too, I was just about to start using FTPClient - I'll just wait until a release comes out.