Travel

This last week was fairly quiet on the travel front. Found the Labyrinth park out west, that was a nice trip, and a very peaceful park. Finally went inside a couple of the Gaudi buildings - the Casa Battlo was my favorite - quite the organic building. Sadly everything inside - each piece of baseboard, door etc is custom made. You'd never be able to make that kind of house today without spending a large fortune!

Saw the inside of the Sagrad Familia - beautiful church - only under construction now for close to 100 years, but "soon" it will be done!

Went to Monserrat Sunday - another nice area. The ride up the mountain on the train was real fun! the Hike to Sant Jordi mountain was a blast. (only 5km or so). We walked back down what is known as the pilgrims trail - and unless you really iike climbing stairs down is the way to go here!

Pictures

Loaded 1400 photos to the computer - now to start the cataloging 8)

 

Report

Accomplishments

Scripting and running WRF v2.2.1 is now working on my account. I'm running through the installs and data in Javier's setup from last year. It turns out the XML versioning had changed and that was gumming up some of the script attempts but it took a couple of folks to figure out what was wrong. Also the WRF executable has multiple exe's in the directories, most of which are symbolic links to something other than what you would think is the main wrf executable.

So as of right now I am running a 15km simulation with WRF v2.2.1, if it finishes in time I'll try the V3 then grab the traces for comparison later. Currently this is running on 4 processor as 4 tasks - so a minimal job for this application. As of this writing the wall clock is 50 minutes out of 6 hours slotted for the run.

 I have the Amazon scripting all done - just waiting for a screencast of the final install for the head node, then I'll migrate wrf and the tools needed to the dynamic cluster. Once that is done dynamic wrf runs start.

Plans

Working on getting further access to MN. Access cuts off Friday.

Waiting for updates to EC2 install instructions

Reading cloud computing paper(s)

Seeing the south of spain then to Madrid and back to Barcelona.

Summaries/Critiques

Reshaping the Computation with Clouds: an Analysis on Opportunities and Issues of Cloud Computing

In general this paper attempted to provide an anaylsis of cloud computing, but onyl gave a cursory look at the issues and discussions surrounding cloud computing. The references provided were good for further reading, hoewever the paper eft much to be desired in terms of details and research.

Travel

A weekend of train rides around the countryside. Saturday was Tarragona. We visited teh old roman ampitheater, the circus a n old cathedral and more. This was a nice day of visiting an old city - it was amazing what they uncovered in the old Roman ruins, and even more amazing that any are still standing - sadly a bunch was blown up but still a significant amount are still around.

Sunday was the trip to Figueres to see Dali's museam, jewelry and more.  Quite a tour of the museum - at least 3 hours of walking around and looking at stuff. We also hiked to the largest fortification in Europe - a huge fort built in the 1700's to defend against the evil French. The cistern of the fort could hold 5 million liters of water! quite an amazing tour of the fort and view of the city!

 

Pictures

Same as last week, I shot a bunch of pictures this week - but I'm running behind in the catalog and naming function.

 

Report

Accomplishments

Finished the International Grid School in Sophia Antipolis, France and returned to Barcelona.

I have most of the ruby, rails and capistrano scripts up and running to load an image in amazon. Starting the dependency tracking to determine what I need to instrument a master and worker node with WRF and mpitrace to perform virtual tracing on the cluster. I have hit a snag in the deployment script dealing with passwords and admin rights to Amazon AMImachines. Turns out there is a bug in capistrano. I'm waiting on communications with the gentleman who is publishing the scripts to decide what to do next.

So the amazon addition to WRF is on temporary hold, but this consumed most of the week banging out scripts and elarning some ruby/rails.

I currently have the complete install of ruby, rails, gems, capistrano and more installed on the laptop - turns out there are interesting differences from the standard Leopard install and the current versions - some things are several cycles out of date, and stored in difference locations. Spend lots of time reading installation guides and instructions to figure things out. Even teh ec2onrails failed for unknown reasons, then started working when I shut the terminal window by accident - I guess it needed sourcing to work correctly.

After sending email to the support folks I finally have my access to Mare Nostrum - turns out the emails they were sending me were being eaten by unicorns or something.

I'm logging into BSC's machine and reviewing the installation, paraver and tracing instructions and data sets for the rest of the week.

Plans

Run traces on Mare Nostrum for a series of tests comparable to what I can do on Amazon - 4, 8 and 16 nodes with 1,2,4 processors for each.

Summaries/Critiques

None

Travel

Well this was one exciting travel event to France. Left the falt with plenty of time to catch the train to the airport, except I forgot my glasses, so.... I ran back grabbed them, and caught the metro to the train station (Sants Estacio). Missed the early train by 1 minute! Grrr. I still had plenty of time, so I waited for the next train (30 minute wait) and caught the next train to the airport. Upon arrival - there were signs indicating T1 and T2 for the different terminals. Not seeing my airline I asked the nice people which way? The told me take the bus to the next terminal - T2. Well bouncing on the bus, the gentleman beside me asked me where I was going, after telling him - he laughed and said - wrong way - you need T1! grrr. Now here, it's a 25 minute buss ride to the new terminal (on the other side of the airport), and a 15 minute ride back to the correct terminal. Walking swiftly I went to the ticket counter - they checked me in and said to go to M5 to catch the flight. Right now its 825 for a 900 flight - so woosh off I go. Needless to say it was teh wrong terminal. And by wrong I mean the other end of the airport M0 was where I was to be. So at 837 I scurry to the other end, hop in the boarding line and whew mad the flight to Nice. Rather too much excitement for my morning I'd say.

Arriving in Nice the Grid School had lots of folks helping - gathering up the bunches of international travelers - stuck us in a cab for the 30 minute and $65 euro cab ride (ouch) to the houseing for the week. Not too bad - suite style dorms with a nice view, but sadly no AC and it was a bit warm (cool compared to Miami though)

Thus commence a week of 8am to 8pm sessions learning all kinds of grid technologies, then practicing with them in preparation for the end of school exercise.

 

Pictures

Well I shot a bunch of pictures this week - but I'm running behind in the catalog and naming function - the school has kept us quite busy this week. When I post them they'll be in the gallery with the rest!

 

Report

Accomplishments

Finished week one of the International Grid School in Sophia Antipolis, France. Technologies and discussions included:

• Distributed System Principles
• Distributed system architectures and scenarios - Both this and the previous discussion by Miron Livny asked some very interesting basic questions. In computer science many terms are used such as scalability, yet are very loosly defined and hard to quantify. What do we need to do as computer scientists to bring more rigor to the discussion? A cleaner definition of terms and the basis of how to measure them would be a start.
• t-Infrastructure and Security
• Unicore - Principles of job submission and execution management, Architecture and components, The basic and rich client interfaces: A rather nice syste for both the command line and their rich client interface into the grid. The rich client includes a workflow manager and the ability to easily see your data storage of results when finished. You can monitor your jobs very nicely with this system
• High throughput computing - A good discussion on aggregation of resources from anywhere to do work. The basic theory is to do something now rather than later - especially given the demand for the HPC resources today.
• Condor - the basics of grid submission, followed by more advanced applications
• Principles of service oriented architecture
• Globus - overviews and advanced work including tutorials and a demonstration of advanced applications on Globus. A good days work with a rather interesting tool set.
• Glite - we covered a significant amount of this toolset, from the beginning to the advanced usage and an industrial appication demo.
• ARC - An interesting tool since both HPC and HTC resources tend to be co-located where this tool was developed. It was rather nice in how it deployed to both architectures and how it interfaced to other technologies.

Plans

Continue the Grid school and technologies

Summaries/Critiques

No papers were read.

Happy fourth of July everyone! I'll miss the fireworks this year, although from the Miami weather report it looks like they'll be rained on anyway. I'll just delay my fireworks extraveganza until the 14th - Bastille day in France.

This week was a calm week - not a lot of touring around, revisited some places thast weekend and posted new pictures, plan on watching the Fira Montjuic again tonight - just have to get there early, there is a fashion show going on, and a fair bit of the street and area around the fountain is gated off.

Gabriel discovered a new market near us (well about 6 or more blocks, but closer than the Boqueria) that I plan to go to tonight before the fountain. Supposed to be nice and cheap.

Report

Accomplishments

Judit gave me a couple of hours of training in Paraver - quite the complex tool set. Spending some quality time with a small trace of WRF I have from the tutorial. Still waiting for Access to learn how to run my own traces.

Installed WRF on the VM that I built - that was a pain. Turns out if you are installing WRF in Ubuntu linux you need a scripting utility called m4 installed - or the whole thing bombs. It will give you file not found errors - and in the compile logs does not relly give you an error about that not being present. It is actually buried in the log file! what a pain. That took a couple of days to track down - I thought I was missing a file. I can now compile and run WRF and have run the first real test in the test suite. 

AFter that I installed WPS - another useful WRF tool, that went well.

Sadly the visualization tools, while free, require signing up to get a login - I'm not sure why- but without the free sign up, and the admins approval of your login, you can't the tools. A tad bit paranoid I'd say. So now I wait to try the visualization end of things. Also looking at the output format from wrf as a side adventure.

Downloaded the WRF Portal program from wrfportal.org, so far so good - at least it installed clean (after installing ant) and seems to be working, I need to do more configuration though.

Plans

Grid School next week at Sofia Antipolis. Turns out the tour d'France is going through there the day I arrive - I expect a it of chaos. We even have word the airport is expecting to have significant delays - fortunately after I get there (I hope). 

So the plans can be summed as learn grid tools, enjoy Nice and the surrounding area.

Summaries/Critiques

Cloud Computing and Grid Computing 360-Degree Compared by  
Ian Foster, Yong Zhao, Ioan Raicu, Shiyong Lu

I found the paper to be a well written overview of both grid and cloud and a reasonable comparison between them. He identified the strengths of each and the weaknesses too. Some of the observations he made on the uses of clouds vs grids were accurate from a business point of view, although I think that he assumed equal access to both a grid and a "cloud grid" in his comparisons. For those without access to a grid HPC on a cloud grid may be acceptable. I did rather like his observation on the future of the desktop cpu, where will computing be when you have 1000 cores on a chip and a 8 cpu system - what then for the grid and cloud? He did point out that the grid is a much more mature organization with the appropriate tools and more. From a business point of view at what point do you release control completely to an external agency? He hinted at some of the data tracking and provenance issues - with new US rules on controls, clouds just aren't able to meet those requirements.
 
All in all a great overview paper (and quite the reference paper list to read)

Ontological Directory and Directory Load-Balancing for Large-Scale Grids

 

Juan Li

This paper is a reasonable discussion of overlay networks and the problems involved in resource discovery and routing.

The Weeks Activities

This week was an interesting week - the city celebrated Saint Johns day on Tuesday night. From just before dark until easily 9am people all over the city were shooting fireworks, not just little ones but mortars! quite the time. We actuallt went down to the beach with what seemed like half the city! There were drummers marching, bands playing on the beach - it was quite the celebration! The day after (Wednesday) was a holiday and most of the city was closed down. AFter getting up late I went to Parc Guell and had a great time wandering through the Gaudi park admiring all the tile work. Quite an unusual place.

Last week we were wandering through a plaza and there was a contest? that had groups of people stacked up then walking through the square. Kind of cool - look for the Stacking People in the gallery.

Of course other things were found during the past two weeks, the chocolate museum, chocolate con churros - mmmmm good, a new market that is big and cheap and 2 blocks from the house and a myriad other things spotted in the city. Time is flying by.

Added some panoramic shots to the panorama gallery - enjoy!

Pictures are posted of course at http://gallery.me.com/codpilot

Report

Accomplishments

Installed a Linux VM machine to run paraver and dimemas.

Paraver is working as of today, dimemas not so much. So far I have run through the first two paraver tutorials.

Issues/Problems

Plans

After I get Mare Nostrum access I plan to run both wrf 2.0 and 3.1 to do a short comparison of the traces.

Complete the installation of dimemas

Review other portential tools for analysis of parallel computation

Read a dissertation on timings and statistical analysis of parallel applications.

Summaries/Critiques

Read and critiqued a paper for International Journal of Computing & Information Technology.

Last week was a great week for ICAC'09 - autonomous computing all the way. I had the pleasure to meet many of the researchers whose papers I've been reading over the last several years.

Some of the photo's have been posted in the gallery as GMAC'09 ( a great one day workshop) the rest will be posted soon under ICAC'09.

The banquet that Wednesday night was at a restaurant at the top of a cable car tower - so we had a complete view of the city as the sun set over the mountains. For those who haven't been to Spain, dinner started at 8:30PM and ended at 12:30PM - WOW. Then I had a nice walk back through the city (the Metro was closed by this time), but Barcelona at night is rather pretty.

All in all last week was a load of fun.

Had a wonderful weekend touring around the city. Did a walk through the market off of Las Ramblas, the Gothic Quarter, the walkway to the top of the hill above the Font Monjuic at the Modern Art Museum. Also toured through the Joan Miro park, nice area with all kinds of pretty flowers and people playing.

I also found out the fish we had for dinner (Gabriel did some great cooking!) called Sepia here - is actually in English is a Cuttle fish. It was fantastic! almost a Calamari texture - but better. That, bread, Olive oil, Garlic, Bottle of White wine and it was a great dinner!

Monday was the start of ICAC '09, so spent the day in talks based on Topics in Autonomic Computing. I finally got to meet the folks whose papers I have been reading. The rest of the week will be spent in lectures and discussion events - Rather fun!

Of course for more pictures (I loaded a bunch of galleries) go to http://gallery.me.com/codpilot

I even added some stiched photos for a panoramic of parts of the city. Enjoy!

Activities

Covered a great deal of papers on the Superensemple creation, methods and data processing. Covered the plan for Florida and the future of the Portal system. 

Acomplishments

Have a rough outline of potential projects including the propogation of WRF/AWR to the BCS center and the Open grid (depending on current deployments) and how to integrate the whole thing.

Issues/Problems

None current

Plans

Next week is the ICAC 2009 conference here in Barcelona, I will be attending that.

More readings today:

Multi-model Ensemble for Short-Range Predictability, JOSE A. GARCIA-MOYA*, A. CALLADO, C. SANTOS, D. SANTOS-MUÑOZ, J. SIMARRO., Spanish Meteorological Institute (INM). 28040 Madrid. Spain. Brief paper on shortrange predictions, not a lot of model detail here, just another use of the meso-scale models and the ensemble forcasting method.

Evaluation of the FSU synthetic superensemble performance for seasonal forecasts over the Euro-Mediterranean region
B y SEVINC SIRDAS, ROBERT S. ROSS, T. N. KRISHNAMURT and A. CHAKRABORT - rather a good article, it introduced to me the concept of Synthetic Data sets, kind of a pre-ensemble creation data set for each model used in the ensemble then those are used for input to the superensemble model. Rather interesting. Some papers had hinted at post processing of the data to remove known bias and errors produced by the ensemble, but this was the first pre-load created data set.

Improving Tropical Precipitation Forecasts from a Mulitanalysis Superensemble, T Krishnamurti, C Kishtawal, D Shin, C Williford, Journal of Climate - a good coverage of some of the long range forcasting issues and rain and how the superensemble works with them, additional coverage of some basic data set preperation is covered too for large scale regions.

Yesterday - with all the reading and teh conference call at 9pm was a rather long day here in Barcelona. I found out they lock the doors at some time! I though I'd be sleeping in the hallway until I found a set of doors with an out button! Didn't get out of the lab until at least 10:30pm. All in all a good day.

Tooks some night shots on the way back to the flat. I need to do wom processing work on them but the Nexus Two building is quite amazing at night. You can see the night shots here:

http://gallery.me.com/codpilot#100023&bgcolor=black&v

I also took another shot of Gabriel and his videography - quite good actually. You can also see a second picture of the metro signs. They tend to show you the last station on route, in this case L3 and the way to the univerisity. Helps to keep from getting lost 8)    See them here: http://gallery.me.com/codpilot#100030&bgcolor=black&v

Back to work!