CERES hdf data display

[kbedka1]

NASA Langley is a major user of data and developer of algorithms for the CERES instrument:
http://science.larc.nasa.gov/ceres/

Data from this instrument will have significant uses toward the better understanding of climate change as it has been flying since 1997 and will continue to fly on future platforms. Folks here have strong interest in using McIDAS-V to view this data. This data is traditionally in HDF format, but I don't see a way via Hydra or any other tool in V for us to view this data. Could someone take a look at this sample file and give me an idea on the feasibility for including this instrument's data in Mc-V?

ftp://ftp.ssec.wisc.edu/pub/incoming/CE ... 2007091023

[jessica.braun]

Hi Kris,

We've grabbed the file and have written up an inquiry requesting this enhancement:

http://mcidas.ssec.wisc.edu/inquiry-v/?inquiry=679

~Jess

[jayh]

Hi Kris-

Similar to the CALIPSO data, the programmers noted the same information with this CERES data:

From what I can tell about the layout, the data are contained in 1-dimensional
arrays. This means the data must be handled as point, not as a 2-D grid as most
of the HDF files we've seen so far. As far as I know, the current version of the
netCDF 4 library can only HDF data as 2-D grids.

We have an ADDE HDF point server in McIDAS-X which was not able to read this
file.

Therefore, there will be a sizable effort required to read this type of data.
In the short-term, you may consider using an HDF utility to dump the data into
a text file, then reading that into McV.

If you wish to provide funding to do this work sooner, please let us know.

Thanks, Jay

[kbedka1]

From my past experiences in dealing with lightning data as an ascii point dataset, Mc-V performance is not optimal for point data especially if the symbol used to display the data at the point is composed of many pixels (i.e. a circle). Based upon what the developers see in my example CERES hdf file, could one of the programmers suggest the best solution for getting my data to display most efficiently display in Mc-V? With regards to funding for development, we'd like to see how Mc-V can display this CERES data and benefit our research before making financial commitments.

[kbedka1]

as an example of the sub-optimal performance of point data in Mc-V, I prepared a zipped bundle and placed it at:

ftp://ftp.ssec.wisc.edu/pub/incoming/MSG_pointdata.mcvz

Though the satellite image is quite large, when this was the only dataset in the display, it animated fine when I clicked back and forth between the frames. As soon as I introduced the ascii text point data (overshoots_HHMM.txt), Mc-V started performing very poorly and lagged significantly when I panned around the image or moved between frames. The ascii text files have maybe 60 points total. My memory usage is far below the max available (431/3271 MB) , so this is not a problem for me (for a change :->).

If I were to treat the CERES data as a 1-D series of ascii point data, I am pretty sure that Mc-V would have serious issues.

[tdrink]

Hi Kris,

I'm wondering if you could use the CERES Gridded TOA (SFC) level3 product at least
initially to start working with this data in McV? In the meantime, we could investigate
some options with the footprint level2 data - perhaps we could re-use parts of the
calipso/cloudsat work.

Tom