McIDAS-V Support Forums

Support and discussion of the McIDAS-V software package
Forum HomeMcIDAS-V Home
It is currently 13 Jul 2020, 00:52

All times are UTC




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: problems loading GOES ABI L2 files from local data server
PostPosted: 21 May 2020, 21:50 
User avatar

Joined: 11 Feb 2015, 23:36
Posts: 2
I've never been able to successfully get a local ADDE server to load files for GOES ABI L2 files downloaded from the NOAA CLASS server. I can create the local data server and choose the GOES ABI format and connect to it. However, once I try to select the Image Type to load the data in the Data Explorer window, I am given this java error:

Encountered a problem (server: 'localhost:8112'):

Error reading times:

java.io.EOFException

I've tried every machine I have (all Macs running OS 10.14.6) and can't seem to get this data to load. I can manually open the individual netcdf files via General->Files/Directories, but this is tedious for a lot of files and can't find a straightforward way to create a loop after opening them individually.

If anyone has encountered this error and has a fix, I would appreciate it. I've found a workaround for creating a loop, but it's not a very elegant solution.

Thanks!


Top
 Profile  
 
 Post subject: Re: problems loading GOES ABI L2 files from local data server
PostPosted: 26 May 2020, 20:26 
User avatar

Joined: 08 Jan 2009, 17:42
Posts: 211
Hello arapp,

I consulted a colleague who has used various CLASS data in McIDAS-V. He said that he downloaded some cloud optical depth (OR_ABI-L2-CODC*.nc) files from CLASS and they worked fine through ADDE servers in McIDAS-V 1.8 on his macOS laptop.

We have three questions and a couple requests for you:

  1. What version of McIDAS-V are you running?
  2. What CLASS products are you working with?
  3. Do any local servers work for you? For example, do they work with standard ABI L1b files?
  4. Can you please provide an "ls -l" listing of your data directory?
  5. Can you please provide either a copy of your RESOLV.SRV file or a screenshot of the Edit Local Dataset window you see when you go to the Local Data tab of the ADDE Data Manager and double-click on the dataset.

Here's an example screenshot of the window requested in #4:

Attachment:
mcv_edit_local_dataset.jpg
mcv_edit_local_dataset.jpg [ 26.39 KiB | Viewed 37 times ]

Thanks,
Barry


Top
 Profile  
 
 Post subject: Re: problems loading GOES ABI L2 files from local data server
PostPosted: 26 May 2020, 22:14 
User avatar

Joined: 11 Feb 2015, 23:36
Posts: 2
Hi Barry,

Thanks for the response! The answers to your questions are below.

Thanks,
Anita

1. What version of McIDAS-V are you running?

V1.8

2. What CLASS products are you working with?

ABI L2 Cloud and Moisture Imagery product (OR_ABI-L2-CMIPC)

3. Do any local servers work for you? For example, do they work with standard ABI L1b files?

Yes, the local servers work fine with other ABI L2 retrieved datasets (such as optical depth) and ABI L1b radiances (as well as many other local servers I've set up for MODIS, AMSR-E, etc.). It is only the ABI L2 Cloud and Moisture Imagery product (OR_ABI-L2-CMIPC) that I get these EOF errors.

4. Can you please provide an "ls -l" listing of your data directory?

Sure. Here are the most recent files I tested, but I've started with a single file in an otherwise new, empty directory in a variety of different paths and the error is the same.

[~/Desktop/GOESR] arapp% ls -lh
total 204M
-rw------- 1 arapp staff 69M May 21 16:03 OR_ABI-L2-CMIPC-M6C02_G16_s20201222201117_e20201222203489_c20201222204018.nc
-rw------- 1 arapp staff 68M May 21 16:02 OR_ABI-L2-CMIPC-M6C02_G16_s20201222206116_e20201222208489_c20201222209007.nc
-rw------- 1 arapp staff 68M May 21 16:03 OR_ABI-L2-CMIPC-M6C02_G16_s20201222211117_e20201222213489_c20201222214024.nc


5. Can you please provide either a copy of your RESOLV.SRV file or a screenshot of the Edit Local Dataset window you see when you go to the Local Data tab of the ADDE Data Manager and double-click on the dataset.
Attachment:
localdataset.png
localdataset.png [ 71.34 KiB | Viewed 38 times ]


Top
 Profile  
 
 Post subject: Re: problems loading GOES ABI L2 files from local data server
PostPosted: 27 May 2020, 22:03 
User avatar

Joined: 08 Jan 2009, 17:42
Posts: 211
Hi Anita,

Thanks for reporting this issue. We confirmed that the ABI L2 Cloud and Moisture Imagery product (OR_ABI-L2-CMIPC) is not working in the local ADDE servers. I will write it up as an issue/bug in our inquiry system.

In the meantime, the same colleague provided a workaround using an NcML file that allows you to load multiple timesteps of CMIPC files using the General>Files/Directories chooser. This method may not be quite as easy and convenient as using local ADDE servers, but we think it will make the CMIPC data usable for you.

Here is the workaround from my colleague:

Quote:
The attached NcML file (agg.ncml) allows you to easily create a single data source that contains all of the timesteps. The way this script is written, the user could work with the file this way:

  1. Put the agg.ncml file in the same directory as the netCDF files.
  2. Through the General>Files/Directories chooser, navigate to the directory containing the netCDF and NcML files.
  3. Select the NcML file and click Add Source.
  4. From the Field Selector, work with the data as you normally would with a loop of data.

A few points about the NcML file:

  • The NcML file uses dateFormatMark extract the date/time info from the filename and creates a new "time" dimension from it. This is necessary because the ABI files don't contain a time dimension if you read them in individually. dateFormatMark is defined as "OR_ABI-L1b-RadC-M6C02_G16_s#yyyyDDDHHmmss". Note that everything before the "#" is considered to be a wildcard and could be replaced by any character. The important part is making sure that you have the right number of characters before the date/time characters in the filename.
  • I have "scan location" set to an empty string ("") since I have the NcML file in the same directory as the netCDF files. If you have the NcML file in a separate directory, you can replace this empty string with a path to the netCDF files.
  • I chose to aggregate both CMI (Cloud and Moisture Imagery" and DQF (Data Quality Flags) so loops could be created from both variables.
  • I'm not entirely sure why, but the "t" variable (for time) must be removed in order for the data to load properly.

This forum post contains some more information about aggregating ABI files together into a data source:
viewtopic.php?f=14&t=1879

Different variants of NcML aggregation, including more details on dateFormatMark can be found here:
https://www.unidata.ucar.edu/software/n ... ation.html

Barry


Attachments:
agg.ncml [362 Bytes]
Downloaded 33 times
Top
 Profile  
 
 Post subject: Re: problems loading GOES ABI L2 files from local data server
PostPosted: 22 Jun 2020, 21:02 
User avatar

Joined: 18 Jun 2009, 17:05
Posts: 66
Barry

Quick correction - The CMIP files are not supported currently by the ABI server at this time.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group