missing data range when valid data present

Errors and unexpected results
Post Reply
User avatar
kbedka
Posts: 46
Joined: Fri Jan 04, 2013 8:53 pm

missing data range when valid data present

Post by kbedka »

I've put a file on the SSEC ftp server: pub/incoming/GOES_Overshooting_Cloud_Top_Detection_Daily_2015182.nc

In this file, I display a parameter Visible Texture Rating. It's a big array so I recommend subsetting over the U.S. to conserve memory. I see that the data range appears as From: missing To: missing. I found this to be quite odd, so I did an ncdump of this parameter and found many non-missing values. Here's the ncdump of the parameter metadata:

ubyte ot_rating_visible(time, grid_lat, grid_lon) ;
ot_rating_visible:long_name = "Visible Texture Detection Rating" ;
ot_rating_visible:units = "unitless" ;
ot_rating_visible:valid_range = 1UB, 255UB ;
ot_rating_visible:coordinates = "time grid_lat grid_lon" ;
ot_rating_visible:_FillValue = 0UB ;

I know that Mc-V has some issues with handling unsigned byte values greater than 128 but, when I dump out the data, I see lots of values between 0 and 128 which shouldn't be a problem. Any idea what's happening here?
User avatar
jayh
Posts: 424
Joined: Thu Jan 15, 2009 10:34 pm

Re: missing data range when valid data present

Post by jayh »

Hi Kris-

I'm able to replicate the missing values for "Visible Texture Detection Rating" from your file in -V and the IDV. I tried a couple of other variables and saw the same missing values with "Overshooting Top IR Pattern Recognition Rating." I was able to display "Overshooting Top Probability" and "IR Brightness Temperature Image."

I have written up inquiry 2439 for evaluation. If any programmer has a thought on why those variables are behaving differently than others, I'll pass that information along.

Thanks, Jay
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: missing data range when valid data present

Post by bobc »

Hi Kris -

This inquiry was revisited when looking into a similar problem with one of your Himawari netCDF files. As with your Himawari file, we can work around this issue with the GOES_Overshooting_Cloud_Top_Detection_Daily_2015182.nc with a NcML wrapper. So it is available to you and other users who look at this forum post, here is a wrapper file:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location="GOES_Overshooting_Cloud_Top_Detection_Daily_2015182.nc">
  <variable name="ot_rating_visible" type="byte">
    <attribute name="valid_range" type="short" value="1 255" />
  </variable>
</netcdf>

A few things to note:
  1. This wrapper file just sets the valid_range attribute of the ot_rating_visible variable to 1-255 as a short, which allows the range to be respected properly by McIDAS-V.
  2. This wrapper file should be loaded the same way as the original netCDF file, as a gridded data source through the General>Files/Directories chooser.
  3. The way this file is written, it is required to be in the same directory as the original netCDF file. If you want this NcML file in a separate directory, you can specify a full path to the netCDF file in location= (the 2nd line of the file).
  4. There are likely other variables in this file with the same issue. If you come across them, you can add them to the wrapper file and use the ot_rating_visible variable as a template.

I'll keep you updated on any status updates to Inquiry 2439.

Thanks -
Bob Carp
McIDAS Help Desk
User avatar
ghansham
Posts: 175
Joined: Thu Nov 11, 2010 5:40 pm

Re: missing data range when valid data present

Post by ghansham »

Post this file
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: missing data range when valid data present

Post by bobc »

Hi Ghansham,

The original file is here:
ftp://ftp.ssec.wisc.edu/pub/incoming/GOES_Overshooting_Cloud_Top_Detection_Daily_2015182.nc

The NcML file that allows the "Visible Texture Detection Rating" to be displayed is here:
ftp://ftp.ssec.wisc.edu/pub/incoming/GOES_Overshooting_Cloud_Top_Detection_Daily_2015182.ncml

This NcML file just sets the valid_range attribute of the ot_rating_visible value to a short with a range of 0 to 255. Note that the way location= is specified in the NcML it is expected that the NcML and netCDF files will be in the same directory in order for it to work.

These files will be on ftp for about a week until they are automatically removed. Please let me know if you have any questions.

Thanks,
Bob
Post Reply