Kasturi Chatterjee, FIU PhD Candidate

Using Graph Similarity for Social Network Analysis

Ms. Chatterjee was awarded First Place for her work in creating and presenting this poster. 

 

Ricardo Koller, FIU PhD Student

APOS: An Application Level Power Oracle for Virtualized Servers

Mr. Koller was awarded Second Place for his work in creating and presenting this poster.

 

Juan Bernal, FAU Bachelor’s Student

Data Mining and Machine Learning Cross Validation over Distributed Networks

 

Justin Beyer, FAU Bachelor’s Student

Gene Selection for Cancer Classification

 

Marlon Bright, FIU Bachelor’s Student and Javier Delgado, FIU PhD Student

Application Profiling and Prediction in the Grid Environment (Research)

Application Profiling and Prediction in the Grid Environment (Experience)

 

Ingrid Buckley, FAU Master’s Student

Web Services Reliability Patterns

 

Paula A. Carrillo, FAU Master’s Student

Data Mining for Video Encoding (Low complexity H.264)

Christopher Holder, FAU Master’s Student

Data Mining for Video Transcoding and Encoding

 

Selim Kalayci, FIU PhD Candidate

Pattern Based Fault-Tolerance at Workflow Management Systems

 

Tariq King, FIU PhD Candidate

Leveraging Traceability Recovery in Test Planning and Optimization

 

Alex Kotlarchyk, FAU PhD Student

BioInformatics Algorithms for High Performance Computing

 

Allison Lanager, FIU Master’s Student and Sean Leslie, FIU Master’s Student

A Web-Based Portal for Hurricane Mitigation

 

Jan Mangs, FAU Master’s Student

Accessing the Grid from Mobile Devices

 

Lina Ortega, FAU Bachelor’s Student

Text, Image, and Audio/Video Datasets

 

Simone Pasmore, FAU Master’s Student

Web Accessibility for the Hearing Impaired

 

Camilo Silva, FIU Bachelor’s Student and Michael Robinson, FIU PhD Student

Finding Repeats and Signatures in DNA Sequences Using MPI Clusters

 

 

[File does not exist]

  

WEEKLY STATUS REPORT
Camilo A. Silva
July 21-27

 

ACTIVITIES:
Time has gone fast. This was my last week in Guadalajara. But, let me tell you—what a great time I had! I am happy because I had a great opportunity to live. To go to another country and get to familiarize with it is a memorable experience. In my case, I was able to excel in my studies. But, also I was able to meet new Mexican friends.

 

Ok. Going back to my activities, during this past week I had the meeting with my team members on Monday, where we discussed the progress of our work—and guess what? We completed it. All the parallelization of our project is completed. Only one thing was left: the error handling of the program. Thus, both Gary and I decided to work together in this task.

 

My friend Gary did a great job in getting this running promptly, while it took me a bit longer to complete mine. Therefore, we were able to run the first set of data during the weekend (Michael kindly asked GCB users to let us the cluster). The first set of data results were completed with no errors so far. There was one complication, though—this will be commented below in the “ISSUES/PROBLEMS” sections.

 

All in all, I was able to complete my program. I was able to fulfill my goal. My program is a parallel program that is capable of managing MPI communication errors only.

 

On Wednesday, I had my last meeting with Dr. Duran. During that meeting I shared with him the progress of my job and the steps to follow afterwards. He was of great help through out my visit.

 

Another important thought that I want to emphasize is that I never expected to have such a great working experience with my friends from the Bioinformatics group. I was amazed at how well we were able to “virtually” work together from different parts of the globe: China, Mexico, and USA. Truly, I was also glad by the great leadership and companionship from Mr. Michael Robinson. He was always there to help me, full of patience, and good guidance. My partner Gary is a great member as well—a hard worker and very knowledgeable about programming. I sincerely feel that I am with the best team members.

 

Looking back to the wonderful time that I have spent in Guadalajara, I am proud to say that I do not regret anything at all. I am sincerely grateful with FIU CIS and the PIRE program for giving me this prestigious opportunity.  

 

One important thing, through out my PIRE experience I had the chance to work with many important people that helped me solved issues. I never had the opportunity to thank them publicly so I want to take the time to thank them all:

1. I want to thank God for giving the opportunity to participate in this research experience
2. All the FIU students that helped me: David, Juan Carlos, both Javier Delgado and Javier Figueroa, and Michael Robinson
3. Big thanks to Michael Robinson because he was a great team leader I am both proud and happy to have partnered with him and Gary
4. Many thanks to Gary because he gave me tons of insight in my programs
5. Special thanks to all Professors and Administrators in charge of PIRE: Dr. Sadjadi, Dr. Graham, Ms. Carbajo, Dr. Hector Duran, Dean Yi Deng, and every single person that helped out with the PIRE program
6. Lastly, special thanks to both of my Professors and Advisors in Mexico and USA: Dr. Duran, and Dr. S. Masoud Sadjadi

ACCOMPLISHMENTS:
I am proud to say that I completed my proposed plan for the Summer. I completed my parallelized program with the capability of self-healing whenever an MPI error message is detected at the time the master node sends a message to the slave nodes.

ISSUES/PROBLEMS:
Just as I mentioned in the “ACTIVITIES” section, there was a little problem that we had during the runtime of our project. Yesterday night during our meeting, Michael shared with us that the problem dealt with something known as “memory leakage.” To be honest, I am not quite sure what the cause of the problem is and how it should be resolved. This is something that as a group we will find out and that Michael decided to look on.

PLANS:
The big plan now is to write the technical paper and complete my PIRE DVD on time.

 

SUMMARIES/CRITIQUES OF PAPERS:

  

FIRST READINGS: “MPI Error Handling”

REFERENCES:

http://beige.ucs.indiana.edu/I590/node85.html

 

http://www.mpi-forum.org/docs/mpi-11-html/node148.html

 

http://www.dartmouth.edu/~rc/classes/intro_mpi/mpi_error_functio

 

http://www.hlrs.de/people/gabriel/benchmarking/ft-vsuite-s

 

http://www.netlib.org/utk/people/JackDongarra/PAPERS/ft-mpi-l

 

The basic theory that I learned from all those articles about error handling is that the MPI communicator is more than just a group of processes that belong to it. Amongst some of the items that belong to the communicator and that it hides inside its body is the error handler. It is important to point out that whether an error message is printed or not, it depends on its implementation.

 

MPI error(s) arise whenever there are messages that are incorrectly constructed, addressed, set or received. Please note that MPI does not provide mechanisms for dealing with failures in the communication system. What MPI does is that it provides mechanisms to solve recoverable errors. Simply meaning that the default error handler of aborting an MPI program will be replaced with an appropriate error handler. Thus, in order for the application to identify the error code, the MPI_Error_class routine converts any error code into one of a small set of standard errors codes known as error classes. Furthermore, MPI provides only two types of predefined error handlers: MPI_ERRORS_ARE_FATAL which is the default that causes the MPI program to abort whenever an error is found. And, the other is MPI_ERRORS_RETURN which causes MPI to return an error value instead of aboting.

 

Since I read a whole lot of different papers or documents, I could generalize and comment that all of them were helpful. Some of the documents were heavy in a lot of theory which was a bit monotonic, while others were very simple in the definitions and provided great examples. All of them were easy to comprehend.

  

This topic was extremely important for the last part of my project because it helped me a lot in establishing a self-healing system to my project.

  

SECOND READINGS: “MPI debugging”

RESOURCES:

 

http://cw.squyres.com/columns/2004-12-CW-MPI-Mechanic.pdf

 

http://www.clustermonkey.net/index.php?option=com_search&searchw

 

http://www.hlrs.de/organization/amt/services/tools/debugge

 

http://www.cs.utah.edu/research/techreports/2007/pdf/UUCS-07-0

 

http://www.nacad.ufrj.br/sgi/007-3687-010/sgi_html/ch04.html

 

I wanted to learn what strategies were present in the debugging process of MPI. And I was able to be exposed to techniques that are used nowadays to debug parallel programs. The first technique that I learned was “printf()” debugging. This technique is not that effective in parallel programs because of the multiplicative effect meaning that many nodes will be printing the same thing unless there is something in the print out that could identify them. Also “printf()” techniques can only display a limited subset of the process state.

 

Other types of debugging techniques are to use the serial debuggers in parallel. Although serial debuggers were not developed to be used in parallel programs, they might provide some insight in finding certain bugs.

 

Memory Checking debuggers look for erroneous patterns such as accessing memory outside of an array or the local stack using heap memory that was already freed. One of the advantages of using such is that they report all errors in a file and with line numbers. The downside is that it cannot be used interactively and cannot be attached to already-running processes.

 

The last category of techniques for debugging is known as the parallel debuggers. Besides all the common functionality known from all debuggers, these type of debuggers are capable of setting breakpoints, examining variables, stepping through code, and also individually monitor and control all processes in a running MPI job.

 

This topic was very interesting. And all the authors did a great job in explaining it. Although most of the information could be found by reading only one of the five articles that I read.

 

This information was important to me because it helped me in understanding what the different types of debugging techniques are that could be used in a parallel MPI program.

     

 

 

WEEKLY STATUS REPORT
Sean Leslie
July 28, 2008

 

ACTIVITIES:
This past week we have been trying to fix various errors while working with spotty internet access. We also have been getting ready to go home and started getting information together to wrap up the project in general. A few changes to the namelist variable input page were made last week as well. While adding drop-down boxes to some variable with a fixed number of values was doable having them inside of if/else blocks was causing an issue we were unable to identify. A solution was chosen just to have regular input fields replace the combo boxes. Work was also done on creating java code that would parse a namelist file and take the values provided in the file and place them in the database. This required some critical thinking as the data fields did not have to appear in any particular order, or even appear at all. The string parsing works but requires testing on tomcat in order to be incorporated into the site.

 

We met with Dr. Duran and showed him what we had done so far. I also got to sight and have a 1 to 1 question answer session with him. We discussed various aspects of our project as well as WRF and grid-enabling it in general. It was a pleasure working with Dr. Duran he was helpful and responded to our questions in an honest and timely manner. If he could suggest a solution to a problem he would provide us with some web resources to look into the solution further, and if he could not help us he would tell us upfront. He proved to be a valued resource as we worked through problems while working in UdG.

ACCOMPLISHMENTS:
The namelist variable edit page can recall data from the database and add it to its fields, as well as submit data from its fields to the database.

 
ISSUES/PROBLEMS:

• I have been trying and failing to get winSCP working on my laptop. I have given up since we should be able to get this working from campus no problem.
• Having only one machine with winSCP means that only one of us could upload and debug at a time. It would have been great to have all of these tools installed and tested prior to getting to Mexico.

 

PLANS:
Short Term:

• Finish putting some aspects of the site together. Incorporating the work Seychelles and Elias have done with our own.
• Implement interaction with JFM
• Visualization

Long term:

• Finish up project documentation.
• Write a paper.

 

 

Today is my last day in China and last day of my work. It has been a memorable 10 weeks so far. There was work, fun, excitements and frustrations. Sometimes felt really tired of this different life and wanted to return to my comfort zone but at the same time on other sunnier days, life was a bliss here and some part of me wanted to start over with all the unknown adventures yet again.

The work was great. Accomplished quite a lot without feeling too overworked. The hours at office were really productive. I am happy that I could complete the work I planned to in the way I wanted to. Both my group and me knew from the very beginning that 10 weeks is really a very short period of time as far as research is concerned specially, since I strated a complete new project right from scratch. But we were optimistic that we would be able to do enough progress to have a clear roadmap for the future direction. And today I am satisfied that I could narrow down the broad problem, solved part of it and know how to progress to improve it. I will continue working on it once I am back to school and collaborate with my group here at IBM-CRL. We hope to have good results and come up with  paper for the upcoming pacific visualization conference.

I want to take this oppurtunity to thank the INFOVIS group at IBM-CRL and especially to my manager Dr. Shixia Liu. She was helpful and attentive to my requirements all through. The entire PIRE team deserves a special thank from me to make all this happen. Its one of the most wonderful memories I would have with me for a long time.

 25 days from today, in June I had my birthday celebrated here in Beijing. There was a nice surprise party at IBM-CRL as well as a wonderful dinner arranged by Chris, Paula, Leena, Jan Chris and Tariq at a nearby Indian Restaurant. The day was wonderful right from the start and I never got a chance to feel the pang of homesickness, thanks to everyone.

                                                                            

 

 

Also, here are few snaps of me working at IBM-CRL, having group meetings and just having nice time with the group. It was really a tough job getting permission to take these snaps and a lot of paperworks need to be completed to achieve it.

                                                    

 

Today as a farewell, my group is taking me to bowling followed by dinner. I still need to pack and then tomorrow I am off to India for a two weeks break.

I am very excited to be able to see my family after a long time and hope to spend some quality time with them. But, I am feeling a little sad that this journey is over, something I had anticipated for months. In my quite trouble free life at USA, I will miss seeing so many people on the roads, packed buses, taxi drivers each thinking themselves as cousins of Schumacher! I will miss the hard hard spartan bed at the dorm, the nihaos and the xie-xies.......and last but not the least... I will miss my thinkpad..I am sad I have to return it today! :(

Now that I have more time, I will post some of my experience here in China in details.

Till then..... 

 

 

Just wanted to take this time and let you all know that Colombia's Independence from Spain was declared on July 20 of 1810. For all of those that have Colombian friends its not too late to congratulete us for our Independence.

There is a website that I found that talks about the story of our Independence, check it out:
http://www.historyworld.net/wrldhis/PlainTextHistories.asp?historyid=ab81

  

       

Have a nice day my friends and God Bless you and my beautiful Nation, Colombia!!!

:)

  

WEEKLY STATUS REPORT
Camilo A. Silva
July 21

  

ACTIVITIES:
During last week, I participated on REU’s 2^nd meeting where I shared my project progress to my peers. I presented to them the details of the MPI program and all of its communication patterns.

  

On Monday and Wednesday, I had my weekly meetings with my group members where we shared the progress of the project so far and discussed issues and challenges to be solved.

 

ACCOMPLISHMENTS:
I am happy to report that the parallel program is running! The only thing is that it is only running successfully whenever there are less or equal number of tasks than nodes. Whenever there are more tasks than nodes there is an I/O file open error found. Such bug should be solved soon.

ISSUES/PROBLEMS:
The biggest challenge that I had this week dealt with the communication with my group members.  Specifically, we were dealing with a problem about a queue implementation for the parallelized project. However, I already had such implementation active in the parallel code so there was no need to do it again. I tried to explain that to my group members via EVO, but unfortunately the message was not well understood.

 

Fortunately, on our second meeting of the week, we went over my parallel code and I showed them the queue implementation which they completely understood. Furthermore, I presented to them a file I/O error that the sequential code was throwing in cases where there are more tasks submitted than the nodes present. I provided to my group members a print out of the error. One of my group members identified the problem or bug and agreed to help solve it.

 

Another challenge that I have deals in learning the procedure of writing a technical paper. Dr. Sadjadi provided with great insight in how to learn by reading sample technical papers and ask for help from my team members.  

  

PLANS:
First of all, the biggest plan right now is to have the parallelization program running in the cluster perfectly. There is only one bug to fix which deals with some file I/O of the sequential code of the application.

 

Secondly, my goal is to have an autonomic computing implementation ready for the parallel program as well.

 

Lastly, my goal is to start writing the technical paper and do my best to have it ready by the end of next week.

  SUMMARIES/CRITIQUES OF PAPERS:

N/A

 

ACTIVITIES:

This past week we worked on getting Aprof to function properly on MareNostrum.  We amended Aprof’s source code to fix the way that it inputs data.  We amended Amon’s source code to account for the different version of Linux that is installed on MareNostrum in comparison to GCB and Mind.  We developed a script to combine the Amon outputs from each node into one file and to properly configure the data into a Aprof readable format.  We, also, worked to generate predictions using Dimemas and received a demonstration from the Dimemas developer. Personally, I worked extensively on my 2^nd REU Progress presentation and a bit on my Linux studies as time allowed. 
 

ACCOMPLISHMENTS:
In this part, please only mention your significant accomplishments that you have not reported previously. Note that you do not have to have an accomplishment each week.

-Aprof source code successfully amended

-Aprof functioning properly-Amon source code successfully amended

-WRF v. 2.2 compiled on Mind
 

ISSUES/PROBLEMS:
In this part, please mention the minor and major problems that you are facing in your research.

-We are still faced with some decisions on what we are going to do about the different versions of WRF installed at the different sites, GCB, Mind, and MareNostrum.  This is an important issue if we want to be able to make comparisons between the results generated on each of the different clusters. Right now v. 2.2 is installed on Mind and GCB and v.2.2.1 is installed on MareNostrum 

PLANS:
In this part, please mention your short- and long-term plans for the weeks and months to com.  

At this point we have obtained objectives 1, 3a, and 3b, and we are continuing to work on objectives 2 & 4. As this is our final week, we have set out two special objectives to complete before going home or shortly thereafter.   

1.      Get Aprof query automation and comparison script up and functioning properly on MareNostrum.  Obtain a results set of Aprof predictions for the different scenarios we are simulating for.

2.      Have full knowledge of how to run Dimemas simulations and set up Dimemas simulations so that the project can get its first look at comparison of the two tools. 

These are our main objectives to close out our experience here at the BSC.  Once these are complete, we can reassess the specific goal of the research with the first results and prepare to arrive at our significant conclusions by the end of the summer.  With this we will keep in mind the prediction accuracy of Aprof and Dimemas and how they compare and contrast to each other.  This should allow us to improve our approach and present our significant follow-up findings.   

Personally, I will continue my studies in the Linux platform. 

Barcelona Research Objectives 

Overall Objective: Achieve accurate and timely performance prediction on compute cluster to be used in Grid Computing Environment Meta-Scheduling. 

1.      Get mpidtrace linking properly with WRF compiled on GCB, then Mind. GCB COMPLETED

2.      Use generated MPI tracefiles (Paraver and Dimemas) to do prediction between Mind and GCB IN PROGRESS

3.      a) Install Amon and Aprof on MareNostrum. COMPLETED

b) Run benchmarks on MareNostrum FIRST RUNS COMPLETED

4.      Experiment with how well Amon and Aprof scale to larger number of nodes IN PROGRESS

5.      Analyze how Amon and Aprof relate to/could possibly combined with Dimemas.

6.      Work with  Marc to see how we can optimize the gridification of WRF 

SUMMARIES/CRITIQUES OF PAPERS: 

Readings temporarily satisfied.

So after my last trip to Lisbon, Portugal, work resumed and tests continued on the virtual machines system. Different tests were done testing mainly Weka-Grid. It was frustrating seeing how this tools would process the requested task. In part it truly parallelize the model building (at least it claims it does based on documentation), plus it also parallelizes the Cross-validation tasks. However, its performance in regards to time taken to complete the task was really not impressive at all. It has a huge overhead before it presents the results, the actual times of building the models are ok but from the moment once sends the command to execute the task to the moment it presents the result it can take from 1 to 3 minutes. This compared to the same task executed locally with regular Weka is very disappointing. In any case even when executing a task locally with Weka-grid it would take a long time to present the results and in many cases it just won;t presetn a result and freeze (maybe was the network system between the virtual machines but could not find any particual cause for this behavior). Weka-grid actually evaluates if the servers have enough memory available then test the connections and rank each server, the first three process the data mining task, and the Cross-validation is executed on how many servers/host one indicates. Besides this fact, another annoying occurrence with Weka-grid was that when performing the RandomForest algorithm tasks, Weka-grid would never complete its initial testing and ranking of servers. No matter how many times I tried it could never display results either on parallel or localhost mode. In any case many insightful results were obtained and the results still have to be evaluated properly. In another area, a little meeting with the collaborators at UPC was scheduled to present some of the insights and results of the experiments. To this meeting Dr. Ricard Gavalda, Dr. Jordi Guitart, Ferran Julia, and Íñigo Goiri were present. I thank them endlessly for their help especially to Ferran and Íñigo since they both are in charge of the virtual machines I use for the experiments ( which continusly ran into issues and questions). We had a picture session but I can't yet post them (but will in the next day or so).

After that meeting more work was done this time on Weka-parallel. I could only implement one of the three algorithms planned due to the fact that Weka-parallel was implemented over an old version of Weka that did not had RandomForest or MultilayerPerceptron implementations. Regardless, Weka-parallel only parallelized the cross-validation portion of the data mining task. I ran with some issues trying to execute it from the command line which made me use the GUI explorer interface of it. but overall the results were gathered. Doing these tests I have learned a lot not just about data mining but also about the virtual systems and UNIX. Also about the many requirements that tools like Weka4ws need to execute properly. 

In any case the International PIRE experience is coming to an end but at the same time is the opening door to my ongoing desire to learn and give to the research community new tools and developments on the parallelization of data mining tools. This task I feel is way bigger than what I am capable of doing at the moment but it has open my eyes to appreciate each and every one of the experiences and knowledge I have and knowledge I need to acquire to accomplish my final goal of creating a plug-in parallelization tool for Weka and the data mining task. For the moment the PIRE experience is the start of a long journey into my research tasks. I just hope now that I am going back to Florida, I can manage my working life with my research life.

Dr. Gavalda and myself, last day in Barcelona:

Moving on into details about living and leaving Barcelona, Spain, last two days I was trying to cancel he HSI service I contracted, and well there was a problem with the last payment. They charge more than what I was expecting and the bank didn’t have enough funds (I opened a bank account here in Barcelona so I could order the HSI service). In any case I deposited enough to cover and I saw today they charged it so this is good since I won’t be leaving debts, at least not a big one. I say at least because they will cancel the service in 10 days so the service will be charged, but no matter how much I fought on the phone with the company they won’t cancel sooner…so I guess there is going to be a balance to pay and they do not accept over the phone or via internet payments. So I I will do a direct deposit from the U.S. as I see possible. But then maybe I don’t need too…I have to wait. In another area of the living experiences in Barcelona, I went souvenir hunting and I found out that one can ask to buy something for less than what is advertised…Yes, one has to kind of fight with the seller and try to convince him to deduct 1 to 5 Euros of an item but is doable, so a tip on saving money on souvenirs is to haggle. Now is a matter of catching some sleep before we go to the airport to take the flight back home.

But before that I just want to thank each and every one of the people that have made this research experience possible including the NSF itself and all the people at FIU and FAU and also the people at UPC and the BSC that in one or another way have collaborated to the PIRE program. I enjoyed knowing people in the field I am interested to develop my study and work, as well as acquiring knowledge and experience in new fields before unknown to me. I am glad I took this opportunity to get to know what I am capable of doing by myself with some help. The living abroad experience was the best I have had but after all I miss Florida and all the big advantages we can enjoy in the United States of America.

With all that's been going on (and going so fast!) I figured I should stop and make some time to share some of my more..err..interesting experiences this past week.  When we first got here it was insanely hot, spiking up into the 90's and with us having no air conditioner and a fan that sounded like a jet engine just about to stall, it wasn't pleasant in the dorm.  However, within the first week we began to realize with AC just isn't that necessary here.

When the night starts to roll in, about a third of the time so do the clouds.  And not just any clouds, but rainclouds.  No thunder or lightning, but just a soft drizzle that blocks out the sun and does amazing things for the temperature.  It keeps it around a nice 75 degrees in the evening, and if the clouds stick around the next day it can get downright chilly.  This is a fairly normal occurance and we've gotten used to it, but the past week it has begun to rain in earnest almost every night.  Catching the bus back in the rain is kind of unpleasant, but doable. 

The problem with it raining every day is the creepy crawlies have decided they're tired of drowning and would like to live in the nice warm safety of our dorm.  I can deal with ants (after all, I'm the proud owner of an industrial-sized can of ant spray) and roaches are kind of gross but you just get rid of them.  What I'm having difficulity with are the spiders.  In the past three days I've encountered six.  And not little bitty tiny things.  Things that are bigger around than a quarter.  And there was one in the hall, nicely flattened by some other resident, that looks suspiciously like the mexican equivalent of a brown recluse.  It's made me go from: "Awww, our trip's about to end."  To: "Eep, next flight please?"  My fear of them isn't really rational, especially since four of them were daddy-long-legs, but that doesn't make me any less skittish.

Some of the weather's side-effects are quite interesting, though.   The night before last we actually got a hail storm so strong it shoved the front doors open and caused little bits of hail to fly under our doorjam.  We normally leave the windows outside open for the breeze and close them when they start to rain, and when we went to react to the ice storm the main hallway outside was already flooded.  And freezing!  Almost immediately after we got everything closed up again the power went out.....and didn't come back on until the next night.  We cooked up what we could out of the groceries we had in the fridge, but in the end it was just a major cleanup process. 

I suppose we needed something to encourage us to go home, besides getting to see family again, because once you settle into Guadalajara it's hard to imagine leaving.