maximum across time series

[kbedka1]

I have aggregated a set of netcdf files encompassing 2 hours in time with 10 min time steps. I want to create a single plot showing the maximum value of a certain parameter at each grid point across the 2 hrs. I tried the formula Grids -> Time Steps -> Max over time steps or Merge time sequences, but neither seem to create the plot I desire. For example the Max over time steps seems to provide an animation, not a "timeless" single plot of the max across the 2 hours.

Is there a way to use existing Mc-V functionality to create the plot I desire?

[jayh]

Hi Kris-

I am having success using the formula Grids -> Time Steps -> Max over time steps to produce a display or value plot of the max value of a time series. Here are the steps I'm using. Perhaps you can apply them to your data.

- I have a set of 4 netCDF files, 1 minute apart.
- In the Data Sources tab I load the files in the General -> Files/Directories chooser. For Data Type: I use "Aggregate Grids by Time" and highlight the files in the file chooser and click Add Source.
- In the Field Selector I choose Formulas -> Grids -> Time Steps -> Max over time steps. For Display, I choose Image Display and then click Create Display
- In the formula's Field Selector window I chose a variable to display and click OK.

For me, this produced one display with the max value at each point for the series of files. If this process doesn't work for your files...

1. What version of the software are you running? We aren't aware of any beta versions that would have broken this functionality, but you never know.
2. Could you post your files somewhere for us to take a look at them? Possibly something in the files is preventing this process from working.

Thanks, Jay

[kbedka1]

When I do this, I only see valid data over mostly the water, when in fact later in the time series there are many high values over land, see below. I have put 9 files at SSEC ftp server at pub/incoming. For the example I show in the attached plot, I aggregated files from 0[2-8]0000.nc

It seems as if it's only showing data from the first timestep

[kbedka1]

BTW, I'm displaying Visible Texture Detection Rating

[jayh]

Hi Kris-

I'm seeing the same behavior. I'm going to hypothesize that this is due to Visible Texture Detection Rating being an unsigned byte array (like in your previous thread - viewtopic.php?f=24&t=1702). Perhaps when it's going through the max over time steps formula there is an error. IR Brightness Temperature Image works correctly with the formula.

I'll talk with Bob and do more investigation tomorrow.

Thanks, Jay

[jayh]

Hi Kris-

Bob found inq. 1935 that deals with the max over time steps formula not handling missing data or 'nan' values correctly. We believe this is the problem (not the unsigned bytes). Bob has been unable thus far to get this working with NcML or through a formula, but here's a script that produces the image you are looking for:

Code: Select all

    # script to work around bug written as inquiry 1935

    # import  glob module
    from glob import glob

    # generate list of files
    datapath = '/Users/rcarp/Data/kris3/'
    fileList = glob(datapath+'*.nc')

    # define the loop
    loop=[]

    # loop through files in fileList and append together into the loop
    for filename in fileList:
        params = dict(
            filename=filename,
            field='ot_rating_visible',
            )
        data=loadGrid(**params)

        # The missing values of this variable is set by 'nan'
        # The nan values need to be replaced otherwise if a nan
        # is found in any of the files the final result is a nan
        changeMissingData = replaceMissing(data, 0)
        loop.append(changeMissingData)

    # maxOverTime does not accept lists, which is what 'loop' is
    # so we must change the loop to a flat field via makeFlatFieldSequence
    flatFieldLoop = makeFlatFieldSequence(loop)
    maxVals = maxOverTime(flatFieldLoop)

    # create a display of the loop in the currently active window
    layer=activeDisplay().createLayer('Image Display', maxVals)
    layer.setEnhancement('Gray scale', range=(0, 86))

To get this to work, change is the datapath variable near the beginning of the script to point at your data. We've attempted some NcML and UI formulas, but we haven't gotten anywhere. At this time we think this script is the best workaround.

Thanks, Jay