Page 1 of 1

Mc-V hanging on image display

Posted: Wed Oct 19, 2016 7:02 pm
by kbedka
I am trying to aggregate some GOES-13 hemispheric scan data for display. I have put a file with some data at pub/incoming/GOES13.tar I first aggregate the files with NH in the name and display IR Brightness Temperature Image. Then I move onto the SH files. When I display the SH IR Brightness Temperature Image, Mc-V hangs and I can't do anything else within the software. I see the message Time 9/16 at the bottom of the display window. Do you have any idea on why it is behaving this way? I would expect to see the timesteps for the NH scans to appear, and then the SH timesteps to be added to the timeline.

Re: Mc-V hanging on image display

Posted: Wed Oct 19, 2016 8:48 pm
by jayh
Hi Kris-

I didn't see your tar file in pub/incoming on our ftp server. Can you upload it again please?

Thanks, Jay

Re: Mc-V hanging on image display

Posted: Fri Oct 21, 2016 3:42 pm
by kbedka
sorry it's there now. Should have said "I plan to put the file on your ftp" :-)

Re: Mc-V hanging on image display

Posted: Mon Oct 24, 2016 1:57 pm
by bobc
Hi Kris -

I'm replicating your results, and this looks to be related to Inquiry 1252, where Aggregate Grids By Time data type doesn't work if the navigation/dimensions aren't the same between all of the files. As an FYI, this appears to be a 'limitation' with NetCDF itself. Looking at Unidata's NcML Aggregation page, there's a note under JoinExisting (which is what we are doing here, aggregating the files around an existing time dimension):
...All the other variables, dimensions, and attributes are taken from the first file that declares them.

I wrote up a NcML wrapper file using joinNew and the behavior matches the Aggregate Grids By Time data type. Doing a ncdump on the NH files, there are 4 different sizes:

nlines x pixels.....times
2452 x 3263.......1, 2, 3, 4, 5, 7, 10, 11, 13, 14, 15, 16
2714 x 3600.......6, 12
1606 x 3267.......8
892 x 3160.........9

When Aggregate Grids By Time is used, the dimensions from time 1 are applied to all of the timesteps. If you just create a loop of the aggregated NH times, there are a few things you will notice:
  1. Time steps 1, 2, 3, 4, 5, 7, 10, 11, 13, 14, 15, and 16 will be correct.
  2. Time steps 6 and 12 will display, but their nav will be shifted.
  3. Time steps 8 and 9 won't display at all

This is all related to the dimensions of time step 1 being applied across the board to all time steps. One workaround I've found thus far is to load each file individually (or select all of them and load with the 'Grid files (netCDF/GRIB/OPeNDAP/GEMPAK)' data type). This will create individual data sources for each time step. At this point, you can use the 'Miscellaneous > Make a time sequence from single time grids/images'. This formula does require you to individually select the field you want to display in each file so it is a bit cumbersome, but it does create a good loop where everything displays and there's no incorrect navigation shift.

Thanks -
Bob Carp

Re: Mc-V hanging on image display

Posted: Mon Oct 24, 2016 2:43 pm
by bobc
Hi again Kris -

Another potential workaround is to use a script. Here's a sample script you can use:

Code: Select all

# imports
from os.path import basename
from glob import glob

# define location of NH files
nhDatapath = '/Users/rcarp/Data/NH/'
nhFiles = glob(nhDatapath+'*.nc')

# define location of SH files
shDatapath = '/Users/rcarp/Data/SH/'
shFiles = glob(shDatapath+'*.nc')

# load in all of the NH files in a loop referenced as 'nhLoop'
nhLoop=[]
for filename in nhFiles:
    nhParams = dict(
        filename=filename,
        field='ir_brightness_temperature',
        )

    nhData=loadGrid(**nhParams)
    nhLoop.append(nhData)

# load in all of the SH files in a loop referenced as 'shLoop'
shLoop=[]
for filename in shFiles:
    shParams = dict(
        filename=filename,
        field='ir_brightness_temperature',
        )

    shData=loadGrid(**shParams)
    shLoop.append(shData)

# build a window referenced as 'panel'
panel = buildWindow(800,800)

# display both the NH and SH loops in the window
nhLayer = panel[0].createLayer('Color-Shaded Plan View', nhLoop)
shLayer = panel[0].createLayer('Color-Shaded Plan View', shLoop)

# set an enhancement to the NH and SH layers with a range
# that matches data values contained in both loops
nhLayer.setEnhancement('Inverse Gray Scale', range=(188,330))
shLayer.setEnhancement('Inverse Gray Scale', range=(188,330))

This script adds 31 individual data sources (one for each NH and SH file) to the Field Selector, and creates a loop in a way similar to the 'Make a time sequence from single time grids/images' formula. Note that this script does appear to use up a lot of memory, probably because of all of the data sources being used.

If you give this script a try and have any problems/questions, please let me know.

Thanks -
Bob