script saving an image sequence does not work correctly

[hproe]

Hi -

I have a large chunk of code that saves an image sequence, including some fancy items like a precise image width/height and viewpoint (i.e. going 'full screen') and two optional mattes - one for a label and one for a colour bar. Since beta2 the saving does not work correctly anymore, like (with the delay activated prior to saving, see below):

Windows 8.1
1st saved image contains 1st frame of sequence
2nd saved image contains 1st frame of sequence
3rd saved image contains 2nd frame of seuqnece
...

Windows 7
1st saved image contains last frame of sequence
2nd saved image contains 1st frame of sequence
3rd saved image contains 2nd frame of seuqnece
...

I.e., irrespective of the Windows version, the 1st image is wrong and the successive images lag by one time slot. The lag is a serious problem when I include the slot time in the label matte (what I usualy do!).

Find below the code reduced to those lines that reproduce the irregularities. Under Windows 7 I have tested it both in beta2 and beta1. It works fine in beta1!

You will note an outcommented delay in the code prior to the start of image saving. In beta2 with the delay activated you get what is reported above. In beta2 with the delay de-activated the first image is empty in both Window7 and 8.1, while in beta1 and Windows 7 the sequence is saved correctly (not tested with Windows 8.1). Looks like, since beta2, there is a lack of synchronisation between my code and the underlying machinery.

cheers, HP

Code: Select all

def t(relPath):

# derive absolute save path
 homeDir=os.path.expanduser('~') # OS-independent root of save path
 imPath=homeDir+relPath

# if path does not exist, create it
 if not os.path.exists(imPath):
  os.makedirs(imPath)
 print "image(s) saved in: ",imPath
 
# get display controls
 imDisplay=activeDisplay()

# get parameters of image sequence
 imAnim=imDisplay.getAnimation()
 imTimes=imAnim.getTimes()
 imSteps=imAnim.getNumSteps()

# start saving from beginning of sequence
 imAnim.setCurrent(0)

# not needed before beta2
# time.sleep(5)
 
# go through sequence and save frames
 for image in range(imSteps):
  idx=imAnim.getCurrent()
  strTime=str(imTimes[image]) # compose date-time string for filename
  HH,mm,SS=imTimes[image].timeString().split(":")
  yyyy,MM,dd=imTimes[image].dateString().split("-")
  dString=yyyy+MM+dd+HH+mm
  fileName=imPath+dString+".png"
  print idx,'   ',fileName
  writeImageAtIndex(fileName,idx)
#  imDisplay.captureImage(fileName)
  imAnim.takeStep()  # step on

 return



def writeImageAtIndex(fname, idx, params='', quality=1.0):
    """Captures a particular animation step from the active display.
    Args:
        fname: Filename for the captured image.
        idx: Zero-based index of the desired animation step.
        params: IDV ISL directives. Default value is "".
        quality: Image quality (for formats like JPEG). Values may be 
                 between 0.0 and 1.0. Default value is 1.0.      
    Returns:
        Nothing.
    """
    from ucar.unidata.idv.ui import ImageGenerator
    isl = ImageGenerator.makeXmlFromString(params)
    xml = '<image file="%s" animation_index="%s" quality="%s">%s</image>' % \
        (fname, idx, quality, isl)
    elem = islInterpreter.makeElement(xml)
    macros = islInterpreter.applyMacros(fname)
    islInterpreter.captureImage(macros, elem)

[bobc]

Hi HP -

Thanks for reporting this and also for giving a script. I ran a variety of tests (w/ and w/o sleep()) in both 1.5 beta1 and beta2, and I am seeing odd results in beta2. I haven't been able to replicate your exact issue of the first image being incorrect or missing, but I am seeing problems dependent on how I display my data. In beta2, if I display 5 satellite images by clicking through the GUI and then run your script, I am seeing cases where all of the actual images are correct, but the layer label at the bottom of the images can have the incorrect time. Also, there are cases where images are repeated in the sequence of images created from the script, which places the incorrect image in the file.

If I add and display images in the writeImageAtIndex script, then everything seems to work fine. Here is my script where everything is working fine. Are you displaying your data in a different way?

Code: Select all

import time
relPath = '\\HP\\'
homeDir=os.path.expanduser('~') # OS-independent root of save path
imPath=homeDir+relPath

# if path does not exist, create it
if not os.path.exists(imPath):
  os.makedirs(imPath)
print "image(s) saved in: ",imPath

panel = buildWindow()

addeParms= dict(
    server='adde.ucar.edu',
    dataset='RTIMAGES',
    descriptor='GE-IR',
    unit='TEMP',
    coordinateSystem=LATLON,
    location=(40,-85),
    )

print 'getADDEImage output'
myLoop=[]
for pos in range(-4,1):
    metadata,data=getADDEImage(position=(pos),**addeParms)
    myLoop.append(data)

layer = panel[0].createLayer('Image Sequence Display', myLoop)

imDisplay = activeDisplay()
imAnim=imDisplay.getAnimation()
imTimes=imAnim.getTimes()
imSteps=imAnim.getNumSteps()
imAnim.setCurrent(0)
time.sleep(5)

for image in range(imSteps):
    idx=imAnim.getCurrent()
    strTime=str(imTimes[image]) # compose date-time string for filename
    HH,mm,SS=imTimes[image].timeString().split(":")
    yyyy,MM,dd=imTimes[image].dateString().split("-")
    dString=yyyy+MM+dd+HH+mm
    fileName=imPath+dString+".png"
    print idx,'   ',fileName
    writeImageAtIndex(fileName,idx)
#    imDisplay.captureImage(fileName)
    imAnim.takeStep()  # step on

... I tried with both the sleep() commented out and also included and both are working for me. I wrote up the different problems we are seeing in Inquiry 1866. Since we're seeing different issues with writeImageAtIndex, I am wondering if there is something different in our script. Does it look like you are doing anything different than I am? If you try running my script above, do you see any problems with the first image?

Thanks -
Bob Carp
McIDAS Help Desk

[hproe]

Hi Bob -

Many thanks for your script. I am sorry to say that it creates (irrespective of the delay) a completely confused sequence under both W7 and W8.1, but not the same. Actually, I usually create my sequences with scripts very similar to the first part of your script (the 'loading' and 'saving' part are seperated to allow for ad-hoc adjustments before saving).

I note that your code is looping through the image sequence after having finished loading it. My code just jumps to the first image of the sequence after the final load. Maybe the looping is creating even more confusion to the saving part. How can the looping be stopped?

Anyway, there is one thing I cannot undestand at all. It is evident that during the saving the images are regularly displayed one after the other when the code goes excutes the saving. Assuming that the images are saved from the active display (or is this wrong?), why is the code then picking up images from other time slots (and how can it do this)?

HP

[bobc]

Hi HP -

Just as a test, can you try adding a pause() to your for loop at the end of your script before the imAnim.takeStep() command?

Code: Select all

for image in range(imSteps):
    idx=imAnim.getCurrent()
    strTime=str(imTimes[image]) # compose date-time string for filename
    HH,mm,SS=imTimes[image].timeString().split(":")
    yyyy,MM,dd=imTimes[image].dateString().split("-")
    dString=yyyy+MM+dd+HH+mm
    fileName=imPath+dString+".png"
    print idx,'   ',fileName
    writeImageAtIndex(fileName,idx)
#    imDisplay.captureImage(fileName)
    pause()
    imAnim.takeStep()  # step on

Please give this a try and let me know if you see better results.

Thanks -
Bob

[hproe]

Hi Bob -

Thank you for mentioning the pause() command. Actually, I have it in my full script but it got lost in the reduced version (method t) included in my first email of this thread. It really is needed (see also below). It improves things a little bit when saving with my t method and your script, but does not solve the problem.

Rather serendipitously, I might have found the root cause of the problem: the layer label. After deleting it manually in the properties widget of the layer, the t method and the full script work just fine. The pause() is needed when I add mattes for a label at the bottom and the colour bar at the top of the image frames. When I delete the layer label in the full script, what I did until now like

Code: Select all

imControls=imDisplay.getControls()
if switch > 0:
 imControls(1).setShowInDisplayList(0) # label off
else:
 imControls(1).setShowInDisplayList(1) # label on

I get the same irregularities as before.

As I always save the images without layer label (adding it in a matte) I am fine again with my script, with the caveat that I have to remember always to delete the layer label before activating it.

I hope this is helpful to track down the culprit, HP

[bobc]

Hi HP -

After speaking with a programmer, here are some comments/things for you to try:

- The example snippet from HP's post doesn't work--the result of imDisplay.getControls() is a list. The snippet should be something like the following instead:

Code: Select all

imControls = imDisplay.getControls()
if switch > 0:
    imControls[1].setShowInDisplayList(False) # label off
else:
    imControls[1].setShowInDisplayList(True) # label on

- Try verifying that the state of the layer label is consistent with the *expected* value. For example, if the previous snippet resulted in the layer label being turned off, try checking the layer's properties manually and/or running the following:

Code: Select all

print imControls[1].getShowInDisplayList()

Put another way…if we ran "imControls[1].setShowInDisplayList(False)" we'd expect the result of "print imControls[1].getShowInDisplayList()" to be False *and* that the layer label should not appear within the GUI.

- It might be worth inserting (yet another...) pause() after the "imControls = imDisplay.getControls()" line.

- If this is a potential threading problem, I'm curious about what would happen if HP ran something like:

Code: Select all

pause()
imControls[1].notifyViewManagersOfChange()

... or ...

Code: Select all

pause()
imDisplay.displayControlChanged(imControls[1])

Thanks -
Bob

[hproe]

Hi Bob -

Sorry for the delay. On one hand to many todos, on the other hand when turning off the layer label manually the script works. Only when saving a very long sequence (>100, say) there might a couple of hiccups (i.e. repeated frames), but the time series remains intact. I have then just to re-caputre the pertinent slots individually.

Anyway, I have tried the proposed code snippets in the t script. The label switch on/off works (it worked already before). However, the print command and the imControls... do not work - for both I get 'object not callable'. The last of the proposed code line works, but I still get as before
second frame = first frame
third frame = second frame
...
As suggested I also added another pause().

HP

[bobc]

Hi HP -

Sorry for the delayed response. We are still having trouble replicating your results. Our programmer would like you to run the following two commands and send us the output:

Code: Select all

print type(imControls)
print type(imControls[1])

To figure out exactly what is happening, we may need your entire script. If you can, please post your script to the forum or send it to the McIDAS Help Desk email.

Thanks -
Bob

[hproe]

Hi Bob -

Here the out put from the two print commands:

<type 'java.util.ArrayList'>
<type 'ucar.unidata.idv.control.ImageControl'>

I have attached the entire script, including also the two print commands. I have also enabled again the switch on/off of the layer label, i.e. the first 2 saved images show both the first frame, and from the third one onward the frames are shifted backward by 1 time slot.

HP

[bobc]

Hi HP -

Thanks for your script. I was able to replicate both of the problems you mentioned (the blank first image without the sleep(5) and the mismatched images where the first image would repeat and the subsequent images would be off by one). As you reported, this is not a problem in beta1. One of our programmers mentioned earlier that this might be a threading problem (see the post from 06 Nov, 21:55 above). This does appear to be the case, as adding in the following code to your svImageSeq function helped:

Code: Select all

pause()
imControls[1].notifyViewManagersOfChange()

I added these two lines right near the end of the loop in your function (after the 'imAnim.takeStep()' line). Once I added this, when I ran the svImageSeq function in the Jython Shell the images were saved in the correct order. After adding the two lines of code, I was also able to remove the 'time.sleep(2)' near the end of your function.

You can do the same thing with the 'time.sleep(5)' earlier on in the script, where you can remove this sleep and substitute in the two lines of code above.

I'll show this to the programmer and write up an inquiry for this threading problem. Hopefully the workaround of adding the code above instead of the sleep() lines would work for you for now.

Thanks -
Bob