encoding not working completely

Errors and unexpected results
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

encoding not working completely

Post by joleenf »

Hi,

Something is wrong with the encoding of symbols in scripting. It seems that when I initially type something like

sensorDesc = String('Suomi NPP VIIRS 0.672 µm Reflectance')

in the jython shell, it is okay, but after I restart McIDAS-V and recall the old scripting lines, characters are added:

Screen Shot 2014-09-17 at 5.30.47 PM.png


In addition, I was not able to get the greek characters to work in writeImage, and I am certain that had been fixed at one point.

1.5beta2 2014-09-24

Joleen
Last edited by joleenf on Tue Jan 05, 2016 2:33 pm, edited 1 time in total.
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: encoding not working completely

Post by bobc »

Hi Joleen -

Thanks for reporting this. I tested out the Jython Shell history problem you mentioned, and I was able to get some characters to work, like the non-italicized mu character, but other characters gave me the same output that you saw. I wrote this up as Inquiry 1831.

In testing of writeImage, I also ran into some problems with different characters. Again, I was able to get the non-italicized mu character to work, but not other characters. When a character didn't work correctly, it would be replaced by a question mark in the image. Is this what you were seeing?

Here is a writeImage command that worked for me, creating an image with a mu character:

Code: Select all

writeImage('C:/Users/rcarp/mu.png', 'overlay text=Suomi NPP VIIRS 0.672 µm Reflectance anchor=UM place=UM,5,10 color=green fontsize=32 fontface=Arial')

I pulled this from the unicodedata cp437 forum post and just substituted in your text.

I also ran some tests with our _testCaptureImage, which is bringing in some of the writeImage functionality, and I ran into problems with special characters here, as well. I wrote this up as Inquiry 1832.

Thanks again for reporting these problems -
Bob Carp
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: encoding not working completely

Post by joleenf »

Hi Bob,

All three of the following version produced the same result, even the one that worked for you...

Code: Select all

directory="/Users/joleenf/test/"
outFile=directory+"test1Label.png"

labelParse = (String('0.64 '+u'\u00B5'+'m/'+'11.45 '+u'\u00B5'+'m')).toString()

formatLabelTemplate='matte background=black left=5;matte background=black right=5;matte background=black top=5;matte background=black bottom=70;overlay text=%s anchor=LM place=LM,0,-12 color=Yellow fontsize=24 fontface=Arial;\
      overlay text=%s place=LM,0,-37 anchor=LM color=Yellow fontsize=24 fontface=MSReferenceSansSerif'

finalLabels=formatLabelTemplate % ('2014-08-18 18:43-18:48 UTC', labelParse)

writeImage(outFile, finalLabels)

print 'Done'



outFile2=directory+"test2Label.png"

formatLabelTemplate='matte background=black left=5;matte background=black right=5;matte background=black top=5;matte background=black bottom=70;overlay text=%s anchor=LM place=LM,0,-12 color=Yellow fontsize=24 fontface=Arial;\
      overlay text=%s place=LM,0,-37 anchor=LM color=Yellow fontsize=24 fontface=MSReferenceSansSerif'

finalLabels=formatLabelTemplate % ('2014-08-18 18:43-18:48 UTC', '3.9µm')

writeImage(outFile2, finalLabels)

outFile3=directory+"test3Label.png"


writeImage(outFile3, 'overlay text=Suomi NPP VIIRS 0.672 µm Reflectance anchor=UM place=UM,5,10 color=green fontsize=32 fontface=Arial')


I know that all three versions above worked in earlier versions of McV.
Attachments
test3Label.png
test2Label.png
test1Label.png
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: encoding not working completely

Post by bobc »

Hi Joleen -

Thanks for the script. I ran the script on 3 platforms, and 2 of them showed the problem (Mac and Linux-64). On the other hand, all 3 of the images look fine on a Windows 7-64bit machine (see the attached images). I added this information to Inquiry 1832.

Thanks again -
Bob
Attachments
test3Label.png
test2Label.png
test1Label.png
User avatar
hproe
Posts: 504
Joined: Sat Nov 27, 2010 3:46 pm

Re: encoding not working completely

Post by hproe »

Hi-

I have been running Joleen's 1st script. mu and slash were correctly reproduced. Now I would very much like to write an ampersand like this:

Code: Select all

(...):
[quote]labelParse = (String(u'\u0026')).toString()
(...)

However, this produces the follwing error message:
java.lang.IllegalStateException: Error parsing xml: The entity name must immediately follow the '&' in the entity reference.
The entity name must immediately follow the '&' in the entity reference. line:2 column:351

cheers, HP
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: encoding not working completely

Post by bobc »

Hi HP -

I was able to replicate your ampersand problem with writeImage. From looking online, there are a variety of ways in XML to escape ampersand characters, but none of them seem to work well with writeImage. I added some specifics to Inquiry 1832.

A possible alternative for now would be to use annotate(). I was able to get an ampersand symbol to write out from annotate() without having to enter any unicode text... just simply:

Code: Select all

panel = buildWindow()
panel[0].annotate('Testing the & symbol',line=0,element=0,\
    font='Arial',size=24,alignment=("right","bottom"))

Thanks -
Bob Carp
User avatar
hproe
Posts: 504
Joined: Sat Nov 27, 2010 3:46 pm

Re: encoding not working completely

Post by hproe »

Hi Bob -

Thanks for pointing me to annotate. However, I would like to use the ampersand in a banner. But to my knowledge banners can only be created by writeImage, or is there another method available? In any case, a common approach to the handling of special characters in McIDAS-V would be very welcome.

HP
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: encoding not working completely

Post by bobc »

Hi HP -

Unfortunately, it doesn't seem to be possible to do the ampersand character with writeImage in its current state, and I'm not sure of any other way to add banners to images, either. The XML encoding will have to be updated to UTF-8 in order for this to work. A couple of weeks ago, we tried updating to UTF-8, but we ran into some problems so we had to back out of the change. This is something that we're still looking into.

Here are a couple of the relevant inquiries for this:

Once this functionality becomes available, I will post to this forum to let you know.

Thanks -
Bob
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: encoding not working completely

Post by joleenf »

Hi,

In version 1.5, editFile does not recall special encoding correctly.

Joleen
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: encoding not working completely

Post by bobc »

Hi Joleen -

Thanks for bringing this to our attention. I'm able to replicate the problem. After speaking with a programmer, this is a result of the Jython Shell not working with UTF-8 encoding. I wrote this up as Inquiry 2040. We believe that this is a separate issue from the writeImage and captureImage character problems written up in Inquiry 1832.

Thanks again -
Bob Carp
Post Reply