The second cohort of PIRE (Partnership for International Research and Education) students joined FIU's School of Computing and Information Science, FAU and LA Grid partners in a presentation of their accomplishments at the Seventh LA Grid Summit, October 22-23, 2009. This event included the second annual poster contest, which showcased the students' collaborative research, as well as panel discussions that focused on research collaboration.

PIRE is an NSF sponsored program intended to advance global partnerships in cyberinfrastructure research, innovation, and education. FIU-SCIS and FAU continue to nurture and expand this consortium of researchers and educators. It is exciting to see these collaborations serve our students and research partners so effectively. The summer 2009 cohort engaged in research with industry and universities across five nations. The program's multicultural, professional working experience will provide the student participants with experiences and a collaborative network that will benefit them throughout their careers.

Dr. S. Masoud Sadjadi, SCIS Assistant Professor and PIRE PI, led a panel
discussion among collaborators and PIRE students to explore the
"Opportunities and Challenges of International Research Collaborations".

Dr. Jainendra K. Navlakha, SCIS Professor and Interim Director, moderated a
complementary panel discussion focused on "Research Directions and
Applications".

Eric Meyer, FIU PhD Candidate
Enhanced Grid Enabled Weather Research and Forecasting (WRF)/Superensemble Portal
Mr. Meyer was awarded First Place for his work in creating and presenting this poster.

Scott Roepnack, FAU Master’s Student
Enhancing Item-Based Collaborative Filtering by Incorporating Tags
Mr. Roepnack was awarded Second Place for his work in creating and presenting this poster.

Andrew Allen, FIU PhD Candidate
Automatic Behavioral Aspects for Model-Driven Communication Applications

Michael Armella, FIU PhD Candidate
Multi-Type Data Clustering across Multiple Feature Spaces

Javier Delgado, FIU PhD Candidate
Performance Prediction of Scientific Applications

Adriana Garcia, FAU Master’s Student
Mobile Web Services using Globus

Peter Greko, FIU Bachelor’s Student
Automatic Installation of an IT Automation Software System

Yanelis Hernandez, FIU PhD Candidate
A case study to support testing of dynamic SPL

Seijiro Ikeda, FIU Bachelor’s Student
A Competitive Analysis of Some IT Automation Solution Providers:
Kaseya, LANDesk, Altiris, N-Able and SilverBack

Ricardo Koller, FIU PhD Candidate
Working Set: Model and Characterization

Gabriel Lizarraga, FIU Bachelor’s Student
Parallelized Analysis using Subdural Interictal EEG

Lester Melendez, FIU PhD Candidate
Computing Object Similarity using MapReduce

Alfonso Munive, FIU Bachelor’s Student
Meta-Monitoring: Proactive Monitoring of an IT Automation Monitoring Server

Jake Petersen, FAU Bachelor’s Student
A Comparison Framework of Rule Engines

Mark Rajan, FAU Master’s Student
Motion Estimation using NVIDIA CUDA

Jonathan Sanchez, FIU Bachelor’s Student
A Competitive Analysis of IT Automation Solution Providers:
Kaseya, Numara, KACE, LPI and LabTech

Yali Wu, FIU PhD Candidate
Engineering a Communication Modeling Language using Kermeta

Sajjad Zaidi, FIU Bachelor’s Candidate
A Competitive Analysis of IT Automation Solution Providers: Kaseya, LogMeln, ManageEngine, Zenith InfoTech and IT Control Suite

Activities

• Continuing the porting-process of the SWRAD modules
• Running benchmarks (of entire WRF simulations) on different resources
• Working on final presentation

   

Problems

• To get some results on different systems, Gabriel was able to run the simulations on his home computer, which has an nVidia 250 gt, and an Intel Core2Duo CPU. This configuration actually got worse performance for both the CPU and GPU versions of the test. This is a very bizarre outcome, considering that his computer is more powerful in every observable regard. The operation system distribution being used was the same as well. Furthermore, the GPU results were around 15% slower than the CPU results. He tried running a gprof analysis, but the execution failed.

   

• Another difficulty that I've found about working with cuda is that, even though blinking on the screen gives an indication that there is a problem with the code that is causing a runtime error, but the location of the runtime error is very vague. For exaple, while using our testing technique in which we only run certain sections of the code, commenting out certain portions causes this ``blink.'' However, this has shown some of the shortcomings of this porting approach that perhaps porting first to C would have helped with. For example, if I have a partial swrad kernel that simply sets one of the output variables to a static variable, the code runs fine. However, if I start modifying other variables, in a subroutine, even if I don't make any changes to the same output variable, the ``blinkage'' occurs and the variable's values are unset. If I set the variable's values to some function of the other variables, the code works fine. This makes the porting process difficult, since we need to modify small sections of code at a time to compare our results to the Fortran results.

   

Plans

• Continue with the porting
• Run tests on Teragrid

   

Some Final Pictures

Picture during my final presentation

Picture of Gabriel and I working at Esteban's Media Lab

Activities

• Continuing with port of WRF to CUDA
• Experimenting with the alternative solution of porting to regular C first.
• Experimenting with output verification script (for testing)

   

Accomplishments

• A rudimentary, partial port has been created, but we have not been able to test it.

Problems

• The GPU kernel templates at the NOAA site come with some testing scripts that compare the outputs of the variables on the the GPU and on the CPU. However, we have not gotten this to work. It seems to work fine and give good results. However, the problem is that it always gives good results. To test it, I used a stripped-down version of the WSM-5 module and compared the output to the full, CPU output, and the result from this program is the same. This leads me to believe there is a problem with the testing script.

Plans

• Figure out the problem with the testing script
• On Friday we converted the CUDA kernel to regular C. This required some effort due to the lack of automatically-generated variables that we had by virtue of using Michaelakes' scripts. However, it did reveal some bugs that had not revealed themselves in the CUDA implementation. Theoretically Michaelakes' script should make testing easier, but so far we have not been able to use it successfully. As a result, we are not sure which direction to go (i.e. whether to use regular C first or go straight to CUDA).

   

Things to Bring to China

• Bring or find out where to buy an egg crate for the bed as it is really hard. Either that or buy a comforter and use it like a sleeping bag so that it makes the bed softer and so you can use it as a blanket at the same time.
• Buy and bring with you one of those nice gushy gel seat covers for your bicycle. The seats are very uncomfortable and that will probably help out, but we didn’t think about that ahead of time.
• If you like to eat cereal in the morning bring some cereal/protein bars with you. That way if you want something quite to eat you have it. If not you can always buy cereal there some of the stores (although be prepared to pay the American equivalent in RMB for them)
• Bring Powered Gatorade (Gatorade is not easy to find in China and if you do find it will be in a small single serving bottle)
• Bring hand sanitizer or sanitizing wipes. You can bring a small bottle to carry with you and then bring a big one to use in the dorm room or to refill the small one.
• B ring one of the Tide to Go sticks as those can come in rather handy when you spill food/drink on your clothes
• Bring Peanut Butter or buy it there (the C-Store on campus has some though)
• Oatmeal (that way you can have something quick for breakfast that is easy to fix)
• Bring Peanut Butter Crackers as they make a quick snack or can be breakfast if you have to get somewhere fast
• Snacks as the snackfood in China is not what you are used to (The Chips Ahoy cookies taste rather weird). I brought two packages of Famous Amos Cookies and Cheez-Its, but I wish I had brought more.

• The Beijing Zoo
• The Beijing Aquarium (Inside the Zoo)
• The Temple of Heaven
• Summer Palace
• Olympic Sites (During the Day and at Night) – The Bird’s Nest, Watercube, and the rest of the Olympic Green (You can get off at the last exit on the Olympic Subway line and walk to the Bird’s Nest where you can get back on the Subway)
• The Beijing Planetarium
• The Great Wall (The Mutianyu Section is where we went)
• The Forbidden City & Jingshan Park
• Tian'amen Square (Mao’s Tomb)
• The Silk Market
• Houhai
• Tian Jin
• Happy Valley Amusement Park
• The Wangfujing Food Market (At Night)
• The Beijing Acrobat Show at the Chaoyang Theater

• Bell & Drum Towers
• Tang Dynasty Dinner & Show
• Xi’an City Wall (Ride around the wall)
• Terracotta Warrior Museum
• De Fe Chang Dumpling Restaurant (Top Floor)

Places to Eat in the Wu Dao Ku Area

• Grandma’s Kitchen (in the U-Center/Mall at Wu Dao Kou on the 5th Floor) – It has country western decorations and serves American food
• Lush – An American bar
• Pyro Pizza– Another American bar (same owners as Lush) that has good pizza and deals on their sandwiches on certain days
• La Bamba – A Mexican restaurant that had good tacos
• RBT (Green Rabbit Logo) – We referred to it as a Fusion Café and it has good sandwiches/drinks
• Issin – A good Japanese/Sushi restaurant
• Tafi – A fairly good Italian restaurant
• Charlie Brown Café – They serve good milkshakes
• New York Burgers – I can’t remember if this is the exact name but it is located near the 731 Bus stop in Wu Dao Ku (in the direction that takes you back to the dorm)
• Unknown Chinese Restaurant – Has excellent Chinese food. It is located behind the 731 Bus stop in Wu Dao Ku if you were coming from the dorms. There is usually an expensive car (Jaguar or something) parked outside the restaurant.
• Hot Pot – We tried two and they were both good, so just look around or ask the Tsinghua University students
• Café Near FIT Building – It’s on the top floor on the side of the building (sort of circular shaped). You walk up a long staircase to get there and it had good sandwiches.

Useful Websites

• http://chinabites.com/beijing/haidian/wudaokou/
• http://www.atthewu.com/
• http://www.tours.bj.cn/cbt6.htm
• http://bjcyjc.com:8081/aspx/enBookTickets.aspx?nodeid=34
• http://www.thebeijinger.com/
• Google Maps Using the Public Transit Feature - It can show you how to get places using public transportation

Thankfully the trip back home went smoothly. We got up Monday morning and checked out of the dorm. After that was all done we met up with Dennis and took two cabs to the airport. When we arrived we realized that they had switched the terminal, so we had to get on a tram and head over to Terminal 3 which was built recently for the Olympics. We got everything situated there and we noticed that they aren't as picky on the weight of your baggage as they are back in the United States. For instance both of my bags were a few pounds over the 50 lb weight limit, but they didn't seem to care and I didn't get charged any overweight fees. Once that was all said and done we thanked Dennis for all of his help and bid farewell.

While we were heading over to our gate we happened to run into someone we had met at the IMAX Theater when we went to go to the Harry Potter movie. It turns out he was headed back home to Chicago. It was rather strange running into a random person you met several weeks ago, especially considering how many people there are in China.

From there we went to go eat lunch and check our status at the gate. I happened to get lucky and got upgraded to First Class on the flight from Beijing to China. I thoroughly enjoyed the experience as I got served excellent food and got a lot more leg room. Oh and the best part was that the chair reclined, so I slept most of the flight.

The Thursday before we left we went to go see a Chinese acrobatic show at the Chaoyang Theater. We left after work and took the subway to get there and as it turned out it was right next to the subway exit. When we went to the counter to buy the tickets, the attendants told us that the ones for that price had sold out. Luckily we already had a backup plan in place, as the night before we went to the website for the show and booked/reserved tickets. All you had to do was fill out the form and click the submit button. You simply use this system to reserve the tickets (you don’t have to fill out any credit card info or anything) and then you just pay cash when you get there. Although there is no confirmation that it worked (you don’t get an email or anything), it seemed to have worked for us. All we had to do was mention that we had reserved tickets in advance and the name that we booked them under. They went back to the back, got the tickets and had us check our name off a sheet of paper.

After that we headed into the theater to go see the show. It turned out to be a much smaller theater than we had imagined, but the show was spectacular and I would definitely recommend it to the students that come to China next year.

Acrobatic Show Website: http://bjcyjc.com:8081/aspx/enBookTickets.aspx?nodeid=34

As for Saturday we just spend the day doing our last minute shopping. I had originally intended to do something else, which I’ll delve into further detail below.

So one of the things on my list of must see places was Happy Valley Amusement Park. Yeah I know what you are thinking, most people don’t even consider going to a theme park in a foreign country, least of all China,  since there is so much else to do. For those of you who know me though I love theme parks and couldn’t pass up this opportunity. It’s not every day you can go to an amusement park in another country. I had known about this before I had even got to China, I just hadn’t gotten the chance to go yet as we had been so busy.

My plan was to go to the amusement park on Saturday, but when I woke up it there was a huge thunderstorm in the area and it was pouring. Since it looked like it would last for at least several hours, if not the whole day, I decided I would go back to sleep and just do my shopping today. Turns out I should have gone to the theme park, since when I woke up later the sky was nice and clear. Anyway I figured it was no big deal, I could just go Sunday instead. I figured I could spend my last full day in China having a good time.

I woke up on Sunday morning and was at the subway station at around 8:00. The subway ride took around an hour and from there I had to get on a bus. The problem was figuring out where I could get on the bus. Luckily at all of the subway stations there is a map that shows most of the major bus stops. So I looked on the map and found bus 31. There were a few choices, so I picked one and exited the station. Due to the way the roads were laid out I ended up losing my sense of direction and wasn’t quite sure where to go. I wandered around for a few minutes until I saw the bus go by and then I just walked over to that road and started walking until I saw the sign for the bus stop. Sure enough I found it, I just wasn’t sure if it was going in the right direction. Turns out I got lucky and was going the right way, the only problem was that it was a lot farther away than I had thought, about 18 stops which ended up taking about an hour.

I finally made it to the park around 10:00, although I had hoped to be there a little earlier. I had brought my Tsinghua University student ID with me so I was able to buy a student ticket and then I headed into the park. It was much more massive, had better landscaping, and more elaborate theming than I had expected. The lines were all really long (too bad they didn’t have some sort of FastPass system, although even if they did I’m not sure I could have figured out how to use it or read what it said), but the rides that I rode were definitely worth the wait. The first thing I rode was Crystal Wings, which was one out of only several “Flying Coasters” in the entire world. I had been on one in Six Flags Georgia that was called Superman. Anyway you sit down like you would on a normal rollercoaster, except your feet are restrained as well. Then once everyone is ready the coaster cars rotate 90 degrees so that you feel as if you are flying, which is a whole lot of fun.

From there I rode a few other rides, once of which was the Snow-Covered Golden Wing rollercoaster. After that I ate lunch, which consisted of a Chinese version of a hotdog that was rather interesting. From there I went to go ride Titan Truck, which is sort of like a giant rotating and swinging pendulum (it’s the yellow sort of claw ride in the pictures). That’s when the day started to take a turn for the worse. The sky was getting really dark and I was just hoping to get on the ride before it started pouring, but no such luck. The worst part was the area I where I was standing was really low so all the water started to collect there since it was raining too hard for the drain to keep up. The area started flooding so bad that we had to stand on the benches just to keep our shoes from getting soaked, although it wouldn’t have matter that much since at this point I was already fairly wet as it was raining sideways.

Then some sort of announcement was made, but since it was in Chinese I had no idea what they were saying, although I gathered they probably were announcing that everything was temporary shutdown due to the inclement weather. After waiting for around 2 hours the rain finally let up and we ended up being able to get on the ride. The good news was the ride was totally worth the wait as it was my favorite ride in the entire park, even better than the roller coasters.

By then it was already late enough that they weren’t going to open up any of the rollercoasters or other big rides back up. So before I left I ended up going on this weird spinning thing where you lay down in a circle with other people and the floor spins in a circle trying to throw you off. It was rather odd, but fun at the same time. After that I left the park and had a little trouble finding the bus stop as the one going in the opposite direction wasn’t directly across from the spot I had got off at. Once I found it I made my way back to the subway station and met Scott at Wangfujing so we could go to the Nightly Food Market there. They have about everything you can imagine to eat there and some things you couldn’t even dream of eating. In addition, they also have tons of souvenirs and other stuff to buy as it functions as a market as well. Once we spent a little time there we headed back to the dorms to finish packing and get some much needed rest.

Overall I had a fantastic and fun filled day and I would definitely recommend the experience to future students who are interested in theme parks. Although, make sure to check the weather ahead of time and try to avoid the rain.

I have been back in Florida for a week now and I can tell you I'm still a bit jetlagged. My last week in China without my computer was spent trying to configure the applications and recovered code in the computer they lent me. I made some progress but not as much as I originally had planned. Anyway, things are looking up in that area. As soon as I came back to FL I mailed my laptop and made it just in time for the warranty to cover the repairs. I got it back today .

My last day of work in the lab was a bit sad. I was really sad to say goodbye to Yong Hou and Junwei Li. They both helped me so much and I consider them great friends. I wish them both the best in their endeavours and maybe one day in the near future we can meet again, this time in the US.

I was also very sad to say goodbye to Shadow, my Chinese tutor. I hope your dream of seeing America will come true one day sweety! You have a friend here already.

The weekend obviously was spent shopping but I couldn't miss out on the Lama temple and the Temple of Earth, topped by a litte visit to Wangfujing's food street. I cannot tell you tired I am after all this, my feet still hurt, even a week later.

The Lama Temple. If you decide to go you can see a Buddha 18m high! (sorry, no photos allowed there)

The temple of Earth is actually pretty similar to the temple of Heaven: "Heaven is round and the Earth is square". The park there is actually quite beautiful and peaceful. Definitely a good place to rest.

Wangfujing's Food Street. A definite must see. If you decide to have dinner there, that's up to you!!!

In summary, my trip to China was a  wonderful experience. I learned so much about people and culture, I made progress with my research and learned about other universities and ways of life. I ate things I'd never eaten and I made wonderful friends. This will be an experience I will never forget.

Last weekend, Gabriel invited me for lunch at his house on Saturday. There I met his maid and his menagerie of cats and dogs. After that, I went back to Rio for some more exploration and picture-taking. On Sunday, Esteban took me, Gabriel, and another temporary student who was about to leave, to the National Park of Tijuca Corcovado, where the statue of Christ the Redeemer, which I had been seeing from my balcony since day one, is located. The trip to the top is long and fun, as it involves a pretty long drive up the mountain, with several twists in between. The size and construction of the statue is very impressive. The view of Rio from up there is spectacular. After that we went to Praia do Pepe in Rio. This is another very nice beach with plenty of activities going on. There we went to a place with good açai. When we got back to Niteroi, we stopped by the Parque da Cidade in Niteroi, which also gives a very nice view of the city (this time, from the other side of the ocean). To close the afternoon in style, we went to the famous serbet place in Sao Francisco, which has very good sherbet.

Ipanema at dusk

Me in front of the Christ statue

Gabriel and I at the top of Tijuca Corcovado 

Praia do Pepe

View from the Parque da Cidade

This past Saturday, I went to Piratininga beach. My original intention was to go to Camboinhas, but there is no direct bus stop there and I did not want to venture. As it turns out, Piratininga is very nice. There are mountains and hills to walk up (and to dive down to the Beach, for those who dare). Apart from that, this was more of a relaxation beach. There weren't as many people playing sports as there are in Copacabana and Icarai.

Unfortunately, my cultural experience on Sunday involved going to a clinic. Apparently I lost my second battle with Brazilian cuisine, and got a bad virus. Gabriel was extremely helpful in this event, advising me on what to do and with translating for me at the clinic.

Activities

• Fixing CUDA installation problems
• Creating Skeleton of the SW-RAD physics module
• Working on documentation
• Understanding code

Accomplishments

• Working CUDA skeleton has been developed. All (40+) parameters that are passed into the Fortran version of the SWRAD module are being passed to the GPU. Some tests were performed to ensure that the values being passed in and sent back get modified.

   

• We have a better understanding now of the WSM-5 code, including the code that they wrote for helping automate the translation from Fortran to CUDA, as described in the paper.

   

Problems

• We still faced several system-related issues this week. We realized that CUDA does not always produce the same results, is sensitive to programming errors (but does not fail gracefully). For example, we spent an entire day installing different versions of Ubuntu with different versions of the CUDA driver, after realizing that different results were being generated for the same code. In the end, it turned out to be a programming error that did not always reveal itself. We believe there are bugs with certain versions, although we haven't been able to narrow any specific issue wee have had to a version issue.

   

• One of the issues that we recently found out about is that there is a limit of 256 megabytes of shared memory for arguments. On 32-bit architectures, the skeleton works. However, on 64-bit, an error is reported at compile-time, similar to the one in [1]. This is a problem so many arguments have to be passed to the kernel. We may be able to mitigate this by eliminiating unneeded parameters. Reference: [1] http://forums.nvidia.com/index.php?showtopic103440

   

Plans

• I have been looking into the I/O and the test program provided by Michaelakes. All of the I/O is written after the entire WSM5 calculations are done. Therefore, to perform an incremental port, I think the best solution is to break the existing fortran code into sections, and only do one section at a time. When optimizing, we will need to rearrange things, but that can be handled later.

   

• This week we will start the actual porting. We have not thought much about optimization, but I have been trying to keep in mind my goal of modeling the performance of GPU-enabled programs while working. I think one of the biggest challenges will be accounting for the placement of certain variables in memory. In Michaelakes' paper, he explains how he put certain variables in registers since they were used often. The transfer of memory from CPU to GPU in general will be an interesting problem.

   

Paper Review

John Michalakes, Manish Vachharajani: GPU acceleration of numerical weather prediction. IPDPS 2008: 1-7

This paper discusses the GPU implementation of the WRF WSM-5 module. The paper includes motivation for porting to CUDA as well as for the module that was ported. For the majority of the paper, the authors discuss the procedure taken to port the module from Fortran to C, some basic optimizations, and some results comparing execution time on a single CPU and on GPU. A good overview of how they did the port is provided. They describe the challenges faced when porting to C. For example, globally addressed arrays had to be addressed locally in C. They also show how they use special directives to provide some transparency in porting from C to CUDA. These directives allow them to specify which variables should go in registers, and they do automatic indexing of n-dimensional arrays going to and from the GPU. The GPU implementation resulted in a 17x speedup, and the results obtained were verified to be indistinguishable from the Fortran implementation.

Pictures

Wednesday we had our first weekly meeting. Gabriel and I gave an informal overview of what we have been working on, and shared some of the problems that we are facing. Before us, Marcelo (far left) gave a presentation on CUDA in relation to a problem with Petrobras. The gentleman to his left, who works is involved in the Petrobras-UFF alliance, was kind enough to attend both of our presentations and share his insight.

Gabriel and I working at the lab