Page 1 of 1

Decoding Snow/Ice Surface from VIIRS EDR CLOUD Mask HDF file

Posted: Thu Jan 24, 2019 3:41 pm
by leonidas
I decoded Snow/Ice Surface Pixel bits from npp-VIIRS EDR Cloud Mask (in QF1_VIIRSCMEDR dataset). I am working with CLASS/NOAA hdf files.
The results looks reasonable, snow on peaks of mountainous areas.
But I also get some strange results. I get some snow/ice near coastline (blue circles). The image is for March 24 2017 and there is no obvious reason for snow or ice on those areas for that season in Greece.

Image available here.

I'm using Python/numpy to decode the bits:

Code: Select all

 SnowIce_mask=(mynpArray & 32)>>5 

Bits for QF1_VIIRSCMEDR are available here.

McIDAS does not contain those areas near to coastlines. But it also returns a more generalized result excluding a lot of small areas (also excluding some areas on mountains).

Image available here.

How McIDAS extracts Snow/Ice Surface?
Does it perform any Cloud Mask Quality test? Does it apply any filter for noise reduction?

Re: Decoding Snow/Ice Surface from VIIRS EDR CLOUD Mask HDF file

Posted: Fri Jan 25, 2019 9:01 pm
by barryr
Hi leonidas,

McIDAS-V does not do any extra processing or filtering of "bad" pixels. It should just display the field you selected as-is.

I discussed your post with a colleague. We think what you're seeing is due to the UnderDevelopment>Imagery-JPSS chooser displaying a lower resolution by default. If you request full resolution we think it will match up with the plot you did in Python/numpy.

For example, the image below shows the preview image in the Region tab of the Field Selector with a VIIRS-CM-EDR_All/QF1_SnowIce_Surface_Pixel image we loaded.

If you just click Create Display, the data will be displayed at a reduced resolution. For example, if I do that and then zoom into a region of interest I get the following:

To get the full resolution of the data, you can hold down the Shift key and do Left-Click+Drag in the Region tab (completely within the bounds of the data). Everything within the green box you draw will be displayed at full resolution. For example, I opened a new tab then followed that procedure to draw a green box containing all of the data. After clicking Create Display I zoomed into that same region of interest to get the following (which displays many more points because it was done at full resolution):

If you try the above procedure and are not getting the results you want, can you please post the file to our anonymous ftp site? Here are instructions for doing that:

user: anonymous
password: your_email_address
cd pub/incoming
put filename

Once you post the data, please let us know the filename and we'll pick it up. Note that if you have the data (VICMO) and geolocation (GMODO) in separate files, please post both of them.


Re: Decoding Snow/Ice Surface from VIIRS EDR CLOUD Mask HDF file

Posted: Sat Jan 26, 2019 8:45 pm
by leonidas
Thank you! I follow your tip. It was McIDAs display issue. Now I'm getting the same results with numpy.
How accurate is the snow/ice surface detection in VIIRS?
It's really strange to see snow or ice in such areas (blue circles) in South Greece that season...
Is there any confidence flag for snow/ice surface pixels?

Re: Decoding Snow/Ice Surface from VIIRS EDR CLOUD Mask HDF file

Posted: Tue Jan 29, 2019 5:49 pm
by bobc

I'm glad you are able to replicate your Python display in McIDAS-V now. I looked through some older (May 2015) cloud mask EDRs I have and they seem to have the same issue as your data. Looking at some documentation from NOAA/NESDIS STAR on this data (see link here), there are a variety of tests done to determine if each data point is snow/ice. It seems like they've improved how well the snow/ice mask works over time. I ordered some data from yesterday and it looks to be more realistic than my older data. One issue that looks to remain in the data is that some pixels are classified as snow/ice when they are also classified as being "Confidently Cloudy" from the "QF1_Cloud_Detection_and_Confidence_Pixel" variable. You can use a formula in McIDAS-V to mask these pixels out from the display. Here are some steps:

  1. Load your data into the Field Selector.
  2. From the Field Selector, choose "Formulas" from the left panel.
  3. Select the "Imagery > Mask Value" formula and the "Imagery > Image Display" display type.
  4. Click Create Display.
  5. In the Select Input window, enter the values in this image and click OK:

    Here, we're defining to only display values less than 3. 3 is the value from the "QF1_Cloud_Detection_and_Confidence_Pixel" for "Confidently Cloudy". This will plot snow/ice pixels only where the cloud detection field is "Confidently Clear" or "Probably Clear".
  6. From the Field Selector, in the left panel (inputFieldForMask), choose the "QF1_Cloud_Detection_and_Confidence_Pixel". This is the field we are passing through the mask to only include pixels with values less than 3. In the right panel (displayFieldToBeMasked), choose the QF1_SnowIce_Surface_Pixel field. This is the field that will be displayed where the "QF1_Cloud_Detection_and_Confidence_Pixel" value is less than 3.
  7. Click OK.

Now, you should only see the snow/ice pixels where the the pixels aren't considered to be "Confidently Cloudy". There is an overall quality flag variable in the file. You can select the "QF1_Cloud_Mask_Quality_Pixel" field from the Field Selector and click Create Display (after any subsetting in the Region tab). Probing the display by holding down the middle mouse button will print out text output at each pixel location. For example, the pixels could say "High", "Medium", or "Low". From what I've seen, there doesn't appear to be a quality flag field specifically for snow/ice.

Bob Carp
McIDAS Help Desk

Re: Decoding Snow/Ice Surface from VIIRS EDR CLOUD Mask HDF file

Posted: Mon Feb 11, 2019 7:13 pm
by leonidas
Bob, thank you for the detailed instructions!
I also noticed that some fire pixels are also detected as clouds.
In any case, this is not a real problem because I 'll use all of them (fire,clouds,snow) as a mask for my research.