Data probe/Time series for netCDF and HDF files

How do I...?
Post Reply
User avatar
kaba
Posts: 51
Joined: Tue Mar 24, 2009 2:52 pm

Data probe/Time series for netCDF and HDF files

Post by kaba »

Hello Guys,
Following along the lines of what Joleen is doing with AREA files,
I tried doing the same (time series) with 2D gridded netCDF files and could not.
I just had a conversion with Tom Rink on this issue and he has some great ideas
on how to make this work, but I wanted to share it with the rest of the group.

As it stands right now, for McIDAS-V to understand the "time" concept in a netCDF file, we had to adjust our
2D grided netCDF files (x,y) to something more like a 3D (time, x,y). i.e "time" as a dimension. I have tied this
last year with help from Tom Rink and we were able to get it to work.
However, this can be very inconvenient since one would basically have to re-write all her/his netCDF files for each variable.
Also I doubt this is how must people out their write out a 2D gridded netCDF file.

The other option, would be to teach McIDAS-V recognize "time" as a variable or an attribute, eg time(0)= 2215UTC etc.
Tom beliefs this can be done and I belief he will be looking into it for us.

If anyone has some ideas or suggestion to add on to it, that will be great.
User avatar
tomw
Posts: 296
Joined: Tue Dec 23, 2008 3:40 pm

Re: Data probe/Time series for netCDF and HDF files

Post by tomw »

Kris --

If you have one "time" per file, can you use NcML's aggregation to create a "logical time sequence"? I recall having done this once before for you (eons ago....).
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: Data probe/Time series for netCDF and HDF files

Post by joleenf »

Tom,

I have not used NCML extensively. I thought it was only a tool to fix netCDF. I will soon be working with hdf files which do not have a time domain. One option would be to request a change in the data format provided, but that could be a problem for everyone else who has used the data for years. A while ago, I brought in point data and there is a gui which allows the user to assign fields. Could there be a similar option for assigning a time dimension to netCDF and hdf when they are brought into McV? The time dimension is needed for looping and time-series, but perhaps unnecessary and a waste of memory for single image displays.

Joleen
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: Data probe/Time series for netCDF and HDF files

Post by bobc »

Hi Kaba and Tom R.,

I have written an inquiry for evaluation (inquiry 1002 http://mcidas.ssec.wisc.edu/inquiry-v/?inquiry=1002) to teach McIDAS-V to recognize "time" as a variable or an attribute.

Thanks,
Bob and Jay
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: Data probe/Time series for netCDF and HDF files

Post by joleenf »

Thank-You to Tommy Jasmin for helping me with this ncml wrapper to create a time series from a group a files which only contain date/time in the filename. There is a detailed tutorial at UNIDATA: http://www.unidata.ucar.edu/software/ne ... ation.html

I thought it might be helpful for V users to have a snippet of code posted here as well.

I have series of netCDF with filenames of


geocatL2.GOES-13.2011221.173200.hdf.ci.hdf
geocatL2.GOES-13.2011221.174500.hdf.ci.hdf
geocatL2.GOES-13.2011221.181500.hdf.ci.hdf
geocatL2.GOES-13.2011221.183200.hdf.ci.hdf
geocatL2.GOES-13.2011221.184500.hdf.ci.hdf
geocatL2.GOES-13.2011221.190200.hdf.ci.hdf
geocatL2.GOES-13.2011221.191500.hdf.ci.hdf
geocatL2.GOES-13.2011221.193200.hdf.ci.hdf
geocatL2.GOES-13.2011221.194500.hdf.ci.hdf
geocatL2.GOES-13.2011221.200200.hdf.ci.hdf
geocatL2.GOES-13.2011221.201500.hdf.ci.hdf

The fields appear in the field selector as
Screen Shot 2012-02-10 at 9.48.42 AM.png


There are not date/time dimensions in the file, so the Time tab does not appear in the bottom right panel under displays.

I wish to create a time sequence of the field box_average_11um_ctc.

I use the following ncml wrapper. In this case, aggregate the desired field (<variableAgg name="box_average_11um_ctc"/>) together into a time series (<aggregation dimName="time" type="joinNew">) based on the new time coordinate variables assigned to the files (coordValue="yyyy-mm-dd HH:mmZ")

Code: Select all

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
   <aggregation dimName="time" type="joinNew">
        <variableAgg name="box_average_11um_ctc"/>
        <netcdf location="geocatL2.GOES-13.2011221.173200.hdf.ci.hdf" coordValue="2011-08-09 17:32Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.174500.hdf.ci.hdf" coordValue="2011-08-09 17:45Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.181500.hdf.ci.hdf" coordValue="2011-08-09 18:15Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.183200.hdf.ci.hdf" coordValue="2011-08-09 18:32Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.184500.hdf.ci.hdf" coordValue="2011-08-09 18:45Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.191200.hdf.ci.hdf" coordValue="2011-08-09 19:12Z"/>
        <netcdf location="geocatL2.GOES-13.2011221.193200.hdf.ci.hdf" coordValue="2011-08-09 19:32Z"/>
   </aggregation>
</netcdf>


LoopGeocat.ncml
(888 Bytes) Downloaded 383 times


Next, I load the ncml wrapper (LoopGeocat.ncml) and select Aggregate Grids by Time as the data type.
Screen Shot 2012-02-10 at 9.47.55 AM.png


Notice that when "LoopGeocat.ncml" is selected as the Data Source, the new aggregated field appears at the end of the field list and has a time sequence associated with the data.
When I select LoopGeocat.ncml under the Data Sources, the aggregated field not has a time sequence which can be displayed as a loop/movie.
When I select LoopGeocat.ncml under the Data Sources, the aggregated field not has a time sequence which can be displayed as a loop/movie.


Joleen
Post Reply