Make Data Source Local

How do I...?
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Make Data Source Local

Post by rriosalido »

Hi:
I am trying to generate a local dataset for GOES images, I am following the instructions:

1. I load my data into the Field Selector: GE-IR - GOES-East 10.7
2.I display the band/calibration: temperature
3. I right-click on the data source and select "Make Data Source Local".

and nothing happened, also the McIDAS-V session gets stuck and I have to manually abort it.

Previously I have created a LOCAL DATASET with McIDAS AREA format, the port 8112 is open and local servers are running
.
I'm using Mac OX and McIDAS-V 1.8

I don't know if this is a configuration problem or if something is missing.

Than you
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: Make Data Source Local

Post by bobc »

Hello,

This seems to work on Windows 10, but I replicated your error on macOS. I wrote this up as inquiry 2995. I did find a way to get this working though. Here are the steps:
  1. Load your data source as normal.
  2. Display your data as normal.
  3. In the Layer Controls tab of the Data Explorer, select "File > Save > Save As Local Data Source". Select a directory (just click the name of the directory, don't double-click to open it) and click Save. Click OK in the next window where the band/calibration to write to the AREA are set. This will write out an AREA.
  4. Now, any attempts at writing a local AREA from the Field Selector with "Make Data Source Local" should work as McIDAS-V will remember the directory specified in step 3 above. This should work in the current and any future sessions.
Thanks for reporting the problem and I hop the workaround works for you.

Also, just an FYI, the "adde.ucar.edu/RTIMAGES" dataset was created to emulate GOES-13 (GVAR) data. If you are interested in more bands and higher resolution data, you could use GOES-16 data with "adde.ucar.edu/RTGOESR" or GOES-17 data with "adde.ucar.edu/RTGOESS".

Thanks,
Bob Carp
McIDAS User Services
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Re: Make Data Source Local

Post by rriosalido »

Thank you very much Bob, the workaround works fine.

Ricardo
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Re: Make Data Source Local

Post by rriosalido »

Another question Bob:

I already have a local dataset with the last 5 GOES images. Is there any way to have it automatically updated with the latest image available, just like on a remote server.
The pulling option obviously does not work because the images are not refreshed.

Thank you

Ricardo
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: Make Data Source Local

Post by bobc »

Hello,

There is no simple way of doing what you are suggesting. Is there any reason why you need the AREAs locally and can't work directly off of the remote server/dataset?

Please note that the advice below might be useful if you have familiarity with Python scripting, especially in McIDAS-V.

If I understand correctly, you're looking to automatically write a new AREA file when a new image comes in. This would allow polling to work as the display would be updated as new AREAs were written to the directory. It may be possible to run background scripts using a scheduler (external to McIDAS-V) that would execute a McIDAS-V script every x number of minutes. You could use the VisAD AreaFile class to write up AREAs.

Here's a sample template you could use. You would have to update the dataDir directory to specify a directory on your machine to write the AREAs to. You would also have to point to the server/dataset you want to be using:

Code: Select all

# import VisAD AreaFile class
from edu.wisc.ssec.mcidas import AreaFile as AF

# dictionary to pass through loadADDEImage
addeParms = dict(
    server = 'adde.ucar.edu',
    dataset = 'RTGOESR',
    descriptor = 'CONUS',
    band = 13,
    unit = 'TEMP',
    size = 'ALL',
    )

# grab the data with loadADDEImage
data = loadADDEImage(**addeParms)

# grab a reference to the ADDE URL and time of the data object
dataUrl = data['url']
dataTime = data['start-time']
dataTime2 = str(dataTime).replace(' ', '_').replace(':','')

# define data directory and set path/filename of AREA file
dataDir = 'C:/Users/rcarp'
dataFile = 'area_%s.area' % dataTime2
fullPath = '%s/%s' % (dataDir,dataFile)

# create an Area File
remoteArea = AF(dataUrl)
remoteArea.save(fullPath,1)
print('done')
You could save the above text to a Python file (e.g. areaWriter.py) and run it from the McIDAS-V-System directory with: runMcV -script /path/to/areaWriter.py

If you choose to go this route, I'd encourage you to look for scripting documentation in the User's Guide as well as tutorials on the McIDAS-V Documentation webpage.

Thanks,
Bob
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Re: Make Data Source Local

Post by rriosalido »

Thanks a lot Bob,

My intention was that in a configuration with several machines running McIDAS only one would access the remote servers, create a local dataset and the rest of the machines would access this local dataset, all this in order to minimize internet traffic.

There are probably more efficient ways to do this.
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: Make Data Source Local

Post by bobc »

Thanks for the explanation, that sounds like a good justification for writing the AREAs locally.

I think my suggestion of running background scripts to write the AREAs might be the easiest solution for this, especially if you want things to run without much user interaction of if you plan on updating displays often.

Another option to consider would be writing zipped bundles. Bundles are described in the User's Guide. In short, bundles save the state of your session. You would be interested in the zipped version of the bundle (*.mcvz) which actually writes out AREA files and saves them into the bundle. You could have one machine where you display the data and save the bundle (File > Save in the Main Display window). After the zipped bundle is saved, you could move it somewhere the other machines could access it. Users on those machines would then go to File>Open from the Main Display and load the zipped bundle.

I'll let you know if I think of any other ideas to get your task accomplished.

Thanks,
Bob
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Re: Make Data Source Local

Post by rriosalido »

Hi Bob,

I have implemented your suggestion of running scripts in background through the cron and it works perfectly, I think it is a very good solution.

Thank you very much again

Ricardo
User avatar
bobc
Posts: 988
Joined: Mon Nov 15, 2010 5:57 pm

Re: Make Data Source Local

Post by bobc »

Great! I'm glad to hear that my suggestion appears to be working well. Please let me know if you have any further questions.

Thanks,
Bob
User avatar
rriosalido
Posts: 28
Joined: Wed May 01, 2019 12:40 pm

Re: Make Data Source Local

Post by rriosalido »

Hello again Bob:

I am having problems with using cron. The script works perfectly from the jython shell and from the terminal, but when running from cron I always get the following error:

"Exception in thread "main" java.lang.UnsupportedOperationException: Unable to open DISPLAY"

This happens to me on both Mac OS and Ubuntu and whether I have a McIDAS session open or not.

Thanks
Post Reply