how to load data locally from scrip, without using bundle

Cool displays
User avatar
glez_b
Posts: 87
Joined: Thu May 05, 2011 9:19 pm
Contact:

Identifying bright values ??in the pixels of an infrared ima

Post by glez_b »

Hello, again giving trouble.
doing some deployments in McIDAS-V, I note that the values ??of lines and elements which correspond to Mexico are the values ??determined by the AREA Coordinates (see AREA.jpg in attached),that is, the values ??associated with the image displayed on screen. This is true?
Assuming that is correct and After several changes, now I have a script that identifies pixels whose value is 200 for the region of Mexico. The code is as follows:

[cod
def scm(directory):
import os;
fs = os.listdir(directory);

# now fs will be listcm(directory):
import os;
fs = os.listdir(directory);

# now fs will be a list of all the files in directory

from edu.wisc.ssec.mcidas import AreaFile;
for name in fs:

print "Reading in:",name
af = AreaFile(directory+"/"+name);
ad = af.getAreaDirectory();
count = 0;
data = af.getFloatData();

# now look y count pixels
for i in xrange(ad.getLines()):
for j in xrange(ad.getElements()):
if 504 < i < 912 and 2 < j < 562: # find values ??for the region of Mexico
if (data[0][i][j]) == 200:
print i, j, data[0][i][j] ;

scm("/home/mcidas/Documents/datos/IR")
[/code]
when I run the scrip get the following (see mcv in attached)
I following have trouble in my image to identify the pixels whose value of 200 is maintained for 3 hours or more. can guide me to selecting only data that meets the condition referred up?
Attachments
mcv.txt
(3.75 KiB) Downloaded 492 times
Area.jpg
Boris_MCS
User avatar
glez_b
Posts: 87
Joined: Thu May 05, 2011 9:19 pm
Contact:

Re: how to load data locally from scrip, without using bundle

Post by glez_b »

correcting errors in the code written in the previous message:

Code: Select all

def scm(directory):
  import os;
  fs = os.listdir(directory);

  # now fs will be a list of all the files in directory

  from edu.wisc.ssec.mcidas import AreaFile;
  for name in fs:

      print "Reading in:",name
      af = AreaFile(directory+"/"+name);
      ad = af.getAreaDirectory();
      count = 0;
      data = af.getFloatData();

      # now look through the first band y count pixels
     

      for i in xrange(ad.getLines()):
          for j in xrange(ad.getElements()):
              if 504 < i < 912 and 2 < j < 562:
                  if (data[0][i][j]) == 200:
                      print i, j, data[0][i][j];

           

scm("/home/mcidas/Documents/datos/IR")

I commented that the data, I have every hour
Sorry
Boris_MCS
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: how to load data locally from scrip, without using bundle

Post by joleenf »

You can convert values to lat/lon, I am not sure if that will help. Below is some code that I used once, so it has not been heavily tested.
theAreaFile in the code below corresponds to you "af" in your code. As far as following a convective feature through time, this is a difficult subject. Dr. Valliappa Lakshmanan (National Severe Storms Laboratory & University of Oklahoma) has done a large amount of work developing object tracking methods for radar (http://www.wdssii.org/). This has been applied to convective objects in satellites images, but not to the point of decay. Here a the link to Dr. Lakshmanan's page which lists his publications and upcoming book: http://www.cimms.ou.edu/~lakshman/
His book is in press. I understand that the java code which does the object tracking will be released with the book. For references on satellite object tracking:

http://library.ssec.wisc.edu/publicatio ... ,%20Justin
http://library.ssec.wisc.edu/publicatio ... 25&tab=all

A paper which has been submitted for application may be of interest to you
"Development and Application of a Satellite-based Convective Cloud Object-Tracking Methodology: A Multipurpose Data Fusion Tool" (Sieglaff et. al). I am not sure of the status for this paper, so keep checking the link on Dr. Lakshmanan's site or the Schwerdtfeger Library publications list for the release of the publication. (http://library.ssec.wisc.edu)


I hope this helps!
Joleen

Code: Select all

  navblock=theAreaFile.getNav()           # get the nav block
  nav = theAreaFile.getNavigation()    # get the preset navigation from directory block
  nav2 = nav.makeAreaNav(navblock)      # set the navigation based on the nav block
  data = theAreaFile.getFloatData()

  ad=theAreaFile.getAreaDirectory()       # from the AreaFile, get the directory block
  areaDir=ad.getDirectoryBlock()
  nlines = ad.getLines()
  nelems = ad.getElements()

  nav2.setImageStart(areaDir[5], areaDir[6])
  nav2.setRes(areaDir[11], areaDir[12])
  nav2.setStart(0,0)
  nav2.setMag(1,1)

  line = 588.0                           # these are the "File Line/Elem" numbers
  for elem in range(891.0,4761.0):
      pt = nav2.toLatLon(((elem,),(line,)))
User avatar
glez_b
Posts: 87
Joined: Thu May 05, 2011 9:19 pm
Contact:

Re: how to load data locally from scrip, without using bundle

Post by glez_b »

I do not understand that part of the code. I add this to my original code?
Boris_MCS
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: how to load data locally from scrip, without using bundle

Post by joleenf »

Hi Boris,

Sorry for the delayed response, I wanted to check part of the code briefly before replying.

Yes, I was thinking that you could get the necessary information to convert to lat/lon values:

Code: Select all

navblock=af.getNav()           # get the nav block
  nav = af.getNavigation()    # get the preset navigation from directory block
  nav2 = nav.makeAreaNav(navblock)      # set the navigation based on the nav block
 
  ad=af.getAreaDirectory()       # from the AreaFile, get the directory block
  areaDir=ad.getDirectoryBlock()
  nlines = ad.getLines()
  nelems = ad.getElements()

  nav2.setImageStart(areaDir[5], areaDir[6])
  nav2.setRes(areaDir[11], areaDir[12])
  nav2.setStart(0,0)
  nav2.setMag(1,1)


and then in your loop, add the lat/lon conversion and report:

Code: Select all

for i in xrange(ad.getLines()):
          for j in xrange(ad.getElements()):
              if 504 < i < 912 and 2 < j < 562:
                  if (data[0][i][j]) == 200:
                      pt = nav2.toLatLon(((j,),(i,)))
                      lat= pt[0][0]
                      lon= pt[1][0]
                      print i, j, data[0][i][j], lat, lon;


This is an outline, and you will need to check that your output matches reality. I have not worked much with the lat/lon conversions except in a few cases, it was a quick proof of concept, you will want to do more detailed checking.
User avatar
glez_b
Posts: 87
Joined: Thu May 05, 2011 9:19 pm
Contact:

Re: how to load data locally from scrip, without using bundle

Post by glez_b »

Hello Joleenf:

Apologize for late reply. I commented that I did a test with the code you sent me and it runs successfully. The adapted code is as follows:

Code: Select all


# Call data from the directory(/home/mcidasv/Documentos/SSEC/20040601)

def scm(directory):
  import os;
  fs = os.listdir(directory);

  # now fs will be a list of all the files in directory

  from edu.wisc.ssec.mcidas import AreaFile;
  for name in fs:

      # Coordinates are converted to Lat / Lon
 
      print "Reading in:",name
      af = AreaFile(directory+"/"+name);
      navblock = af.getNav();
      nav = af.getNavigation();
      nav2 = nav.makeAreaNav(navblock);
         
      ad = af.getAreaDirectory();
      areaDir = ad.getDirectoryBlock();
      nlines = ad.getLines();
      nelems = ad.getElements();
         
      nav2.setImageStart(areaDir[5], areaDir[6]);
      nav2.setRes(areaDir[11], areaDir[12]);
      nav2.setStart(0,0);
      nav2.setMag(1,1);
      count = 0;
      data = af.getFloatData();

 
      # now look through the first band y count pixels
      # MCS detected when his temperature infrared (TIR) is < 208
     
      for i in xrange(ad.getLines()):
          for j in xrange(ad.getElements()):
              if 489 < i < 927 and 14 < j < 562:
                  if (data[0][i][j]) > 207 and (data[0][i][j]) < 209:
                      pt = nav2.toLatLon(((j,),(i,)))
                      lat= pt[0][0]
                      lon= pt[1][0]
                      print i, j, data[0][i][j], lat, lon;
                                       
scm("/home/mcidas/datos-tesis/SSEC/abril-03-2004")
 




Now I would like to ask some idea:

1. How I can localize, track or target only the brightness values ??that are equal to or greater than 208 and that this region of brightness remain for three hours.

2. Once regionalized the corresponding brightness values??, it is possible display them in McIDAS-V?
Boris_MCS
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: how to load data locally from scrip, without using bundle

Post by joleenf »

Boris,

I responded to your other post which contained the same question:

viewtopic.php?f=27&t=1102

Joleen
User avatar
joleenf
Posts: 1123
Joined: Mon Jan 19, 2009 7:16 pm

Re: how to load data locally from scrip, without using bundle

Post by joleenf »

Hi Boris,

New methods were created and are available within the McIDAS-V 1.2 release. There was a major change in loading ADDE data into McIDAS-V via a script, the old methods will work, but the code outline below uses the new methods. This is not complete code, it is intended to help you get started.

First, some of the methods which will be useful are

methods that load the data, get the lat/lons and find data within a certain range:
getADDEImage: loads either locally or remotely served ADDE data
GridUtil.getLatLon(domain): will return a set of lat/lons for the given domain.
findWithinRange(data,min,max): returns the indices where data falls within the min/max values given

and methods which return area information and perform calculations:
createAreaField: provide a flat field, area information will be calculated for each point
computeSum: provide the result of createAreaField and pixel indices to compute area of those pixels.

findWithinRange, createAreaField, and computeSum are all from JPythonMethods http://www.ssec.wisc.edu/visad-docs/jav ... thods.html

Next, there is documentation for getADDEImage which can be found here
http://www.ssec.wisc.edu/mcidas/doc/mcv ... pting.html
a basic scripting tutorial here
http://www.ssec.wisc.edu/mcidas/softwar ... ation.html
and one of multiple posts which detail loading local data on the forum, such as:
viewtopic.php?f=14&t=1122&hilit=getDescriptor

Here a code outline to help you get started with this work:


Code: Select all

import java.lang.Float as Float
import jarray

dataSet = 'AVIATION'
imageType = 'GOES'

desc = None
# get a list of local adde server entries
localEntries = _mcv.getServerManager().getLocalEntries().toArray()         
for entry in localEntries:
   if entry.getName() == imageType and entry.getGroup() == dataSet:   
      desc = str(entry.getDescriptor()).upper()                                                               

# notify if no matching descriptor found
if (type(desc).__name__ == 'NoneType'):
   raise Exception("A matching entry was not found in the local server manager")

# get the local data, with some constraints
params = dict(
   mag=(1,1),
   size=(600,900),
   location=(31,-84.2),
   place=CENTER,
   coordinateSystem=LATLON,
   time=("18:32:00","18:32:00"),
)

# return the data, will be -1 if the getADDEImage request fails
metaData, band2Temp=getADDEImage(descriptor='FD', band=2, **addeParms)

#Find out how many data points are in the line (number of elements in the satellite image)
numAreaElements = metaData['directory-block'][9]

# find the indices where the values match a criteria (in this case between a TEMP of 130 to 135)
indices=findWithinRange(band2Temp,130,135)
# get the size of the array so that an array of zeros matching the size can be constructed
numIndices = indices.__len__()

# Set up an array of zeros the same size as the number of indices found (zero will indicate a pixel that has not been grouped)
assigned = []
assigned.append(jarray.zeros(numIndices,'i'))

# get the lat/lon values for the entire domain of the data.
domain = band2Temp.getDomainSet()
fieldLatLonValues = GridUtil.getLatLon(domain)

#loop through the indices, check for valid lat/lon values, group neighboring pixels
#note:  indices are returned in a 1D array.
groupNo = 1
for currentPixel in indices:
   if (assigned[currentPixel] = 0):
      # check for valid lat/lon values:  flag as non-navigated if lat/lons are not valid (-1)
      if (Float.isNaN(fieldLatLonValues[0][currentPixel]) or Float.isNaN(fieldLatLonValues[1][currentPixel])):
         assigned[currentPixel]=-1
      # else assign a group.
      else:
         assigned[currentPixel] = groupNo
         # check neighbors within radius (assign them a group number if they are in the list)
            ...


When the desired pixels are flagged, use the createAreaField, computeSum and computeAverage (if needed) to find out information about the size of region affected:

# createAreaField returns a flat field of computed areas, supply the NavigatedImage returned from getADDEImage
myAreaField = createAreaField(band2Temp)

# pretend selectedIndices is a subset of the indices returned from findWithinRange. Find out the total area which falls within the range of 130-135 K.
areaAffected = computeSum(myAreaField, selectedIndices)

Joleen
Post Reply