Another week in Delft... Next week will be the last one! Upon realizing this, on how fast time has passed, I had an urge to finish everything I have started working on and focus on some issues I can complete before leaving. This is not easy as it seems, since --as I have explained before--, there are a lot of interesting things to do. First, I finished some work I had started with Dr. Iosup analyzing traces of a well known Web-based game. This is one of the areas this lab is specially strong in, and I didn't want to miss the opportunity to work on that.

The work is really interesting, and it involves analyzing a pcap file (Packet capture, usually collected with a program like wireshark) containing thousands of ethernet frames, and trying to reconstruct what was really going on: when are resources sent and received, which sizes and frequencies they have, etc. For me, the most interesting side of it is the opportunity to go the the lowest possible level of network communication and understand how things work there. It turned out to be more complex that I expected: ethernet frames encapsulate IP packets, which encapsulate TCP packets, which encapsulate HTTP data. We are interested in the HTTP part, and this can be split between different TCP packets, for example... in the end, this is like a giant puzzle that needs to be solved (which was a lot of fun!!).

I have also worked on other areas, considering different job allocation and VM provisioning policies, and I think we have a solid idea that we can try to publish in the future. Next week I'll focus on that, and try to define our future collaboration for the next months.

Another week ends, and I have only two more to go. In hindsight, the time I have been out has passed really fast, and I am happy with all I've done so far. I think that collaborating with different teams around the world gave me a much wider perspective about our research area and also about how research is conducted in different labs. The only downside is that I am too easily distracted by new exciting research challenges and loose focus on my own dissertation topic. If everything goes well, though, I think I'll be able to start redirecting all these efforts to contribute to my thesis :-)

Regarding my life outside of EWI, the department where I am at TU Delft, I haven't done much sightseeing this week. It was another gray and rainy one, so I didn't venture going out besides from the apartment to the university and back. Finally, during Sunday the sun appeared and the clouds cleared a bit, so perhaps it will be better next week. If the weather stays like that and I manage to accomplish some of my goals for next week, I'll try to go out in the weekend and visit Rotterdam or Amsterdam.

This has been a rainy week, so I have stayed inside most of the time. I have spent a lot of time in the university, and also in the apartment, and being used to the sun of Miami, seeing gray skies through the window for a week is... different. Luckily, last Saturday I could leave the apartment in the morning and walk around Delft for a bit. Since I was afraid it would start raining at any moment I decided to not try my luck, so I just went out for breakfast. Last week I visited a Bagels place, this week I decided to check an Starbucks-type one, which was huge for what I have seen so far, and very Starbucks like. In fact, besides the coffee itself it was difficult to tell them apart.

Work has been good, I have finished my first stage of refactoring of the Cmeter engine, and I can plug new policies to the system. During this week and the next one I'll start trying different workloads and see how they work in DAS-4, the grid/cloud system we have access to. Our cluster at FIU doesn't fare very well compared to it, so I'm taking advantage of it for the days I'll stay here :)

I also got the chance to upload some pictures from last week, when I visited the city square and the "new" church.

 The square

The "new" church

Inside the church 

I am happy that this week I have finally found an interesting research direction and I can start working towards it. While spending time trying to come up with ideas is fun, I think that the rush that comes with defining experiments, working in the implementation and preparing an article is much better.

We will work in adapting an existing cloud benchmarking framework, called Cmeter, so that it can use advanced policies for provisioning VMs and scheduling jobs to them. As one of our colleagues used to say: "policies are the fun part"... or so I expect. The interesting thing about them is that differently from the stage in which one is creating a large distributed system and most of the efforts are in just making it work (no matter if it is just for a brief period of time, just so that the required data can be collected), policies are developed once the system is more or less mature and supports some experimentation on top of it. When trying policies, a small change in how things are managed can produce completely different results, and therefore new ideas can be tested and analyzed quickly... we'll see if it is like that or not.

Besides that I tried to visit the old parts of Delft (still haven't left the city), and I was lucky the rain didn't get me... it started raining just after I arrived home. I visited what the locals call the new church, although it has been built for some centuries already (but there is an older one, that's why this is the new). I took some pictures, but I'll upload them later when I get them from the camera and fix them. The weekend has been rainy all the time, so I  stayed in the apartment most of the time. Hopefully the weather will improve and I'll finally visit some other places. 

A new week has passed, and this time I have been working with the new group at Delft. It has been a very nice experience, so far. I had the chance to meet Dr. Alexandru Iosup and his team, whose research focus is on Grid and Cloud computing, Peer to peer and Games. I was placed in a shared lab with other PhD and Masters students, and kind of started working from day one :-)

Since I hadn't had much communication with Dr. Iosup in the past, one of the toughest things has been trying to come up with some research direction that we both are interested in. Luckily, I found out we had many common interests and I think we'll be able to find something challenging to do. Anyway, I have spent most of the week reading papers about scheduling, inter-operation in Grids and Clouds and policies. Truth be told, even though some of my research has had the work scheduling in it since I almost started at FIU, I've never been exposed to the most difficult parts of it, namely the scheduling algorithms themselves. It has been good so far, since I'm usually more attracted to the systems part of the problem, but I feel this is going to be interesting. One of the challenges of coming up with good schedulers is that it is practically impossible to find the absolute best arrangement of tasks and resources, so every time somebody comes with a new contribution in the field it is by "cheating" in some aspect and simplifying the problem in a way it performs better for a certain scenario. This means any new attempt requires a good knowledge of what has been done previously (and believe me, scheduling is one of the oldest problems out there!)

Anyway, I have also tried to spend some time knowing the city of Delft, and even though there have been some gloomy days (I think I hadn't seen one of these for ages!) I did some "excursions". During the weekend I visited the center and walked around the canals, which are very nice. I haven't had much time to do anything else, but I guess once I get more used I'll visit some of the cities around, like Leiden, Rotterdam, the Hague or Amsterdam. I still have five more weeks to go, so I guess there's time...

By the way, I took some pictures of the canals:

What is the first thing a spaniard does when arriving to a new place?

Put the jamón in the fridge!

Ok, perhaps a spaniard wouldn't put it in the fridge. This must be my miamian self, which is afraid of any biological tissue rotting in front of my eyes when not refrigerated. Whatever...

So, the news are I am not in Spain anymore! I will miss the Complutense and the OpenNebula team, but a new stage of the travel has started with new collaborators and a new city to visit.

I worked until Thursday in Madrid, refactoring some of the code in the DEVA manager, specially the network provisioning and site-communication parts. When I implemented the prototype to run the last paper's experiments some months ago, most of the behavior worked only under certain conditions and for a particular setup (you would not believe some of the things one can do when under a deadline... talk about code smells!!) Thankfully, everything is better now, and the system is finally integrated with the latest version of OpenNebula. There is still a lot of work to do, and when Dr. Llorente proposed to make the project part of the OpenNebula ecosystem -that is, make it available to other people so that they can run it on their sites-, I realized that, although the idea seems great, there is a looong way until we can distribute the program for anyone to install and use. However, the current state is satisfactory for the months to come, even though I plan to keep working on the distribution and make the project available to others.

In any case my time at the Complutense arrived to its end on Thursday, and after saying my farewells to the group I had to prepare for the next day's plane for the Netherlands.

The trip was ok, although a bit stressful. I had to arrive to Delft before 5 p.m., since the housing agency offices closed at that time. More or less it went like this: I took the metro from my apartment in Madrid at 10 a.m. I arrived to the airport, where my plane left at 12 and landed in Amsterdam at around 3. Once I picked up my luggage I run to buy a train ticket. At 3:45 the train departed, and 45 minutes later I was at Delft. Then I run -again- to find a taxi to the housing agency, which luckily was still open.There I got the apartment keys, and finally arrived to my new house for the next month and a half.

I haven't officially started my touring around Delft, but I have visited the supermarket two or three times already. Delft is a beautiful city with a very large university campus. There are lots of students, and although I more or less knew about it, I was surprised by the number of bicycles you can see around. The computer science department is only 10 minutes away from my apartment, so I am still considering whether I should get one...

Here are some pictures I took during the weekend:

My apartment

Outside

That is all, by now. Tomorrow I'll visit the Computer Science department and start my collaboration with Dr. Iosup. I still have a lot of walking (and bicycling?) to do, and a whole new country to discover! :-) 

My second week in Madrid has been more straight-forward: I already had a place to work at the lab, and I was comfortable with the metro and the housing. I have spent most of the time there, working in the new version of our prototype, with some input from the OpenNebula team. I started to install the DEVA manager in my own laptop, which seemed to be a good idea since changing the code, re-deploying and testing is somewhat more complicated when done on a cluster over ssh. Unfortunately, my Linux distribution uses a more recent version of Xen (4.0.1), while we've been using 3.0.3 until now.

However, making OpenNebula work with my setup gave me an excuse to familiarize myself with the ticketing system and code repository. The guys at the Complutense have a really nice setup with Hudson and Git, which have a steep learning curve, but taking the effort to learn the system is worth it. I was really happy when I found a bug in the monitoring code and tracked it to the source :-) 

Finally, I got the problems solved and OpenNebula is working fine in my machine. I plan to do some refactoring to our network management code and have a demo running by next week (just before leaving for Delft).

Besides all of this, I didn't have a lot of free time during the beginning of the week. However, Thursday was a holiday here in Madrid and I spent the whole day at home. I tried to continue working, but the wi-fi started giving me some problems and I finally decided to do as the rest of people and take a break. It was kind of weird having a holiday on the 23rd (Corpus Christi), since in Catalonia we have it on the 24th (Nativity of Saint John the Baptist)

 For the weekend I had a trip to Toledo, around 80 Kms south of Madrid and in a different community, Castilla la Mancha (where don Quixote takes place). I had heard very good things about the city, but it still surprised me... Toledo is a really old city where people from different cultures have coexisted for long periods of time, and it shows in its streets, churches, mesquites, synagogues, city walls... I recommend anybody visiting Madrid to spend at least a day there. I took some pictures to show what I mean.

 Toledo streets

A courtyard

The outskirts

Synagogue

Celebration of Corpus Christi

Next week I'll try to have a demo with the new Manager and show how we can transparently create a virtual ensemble of VMs between FIU and the Complutense, and try to gather some help to write a journal version extending our previous work with new features. If everything works fine by the end of the week, I'll be able to start working on new problems with Dr. Alexandru Iosup in Delft by the beginning of July. I'm still deciding what it will be, but most likely we'll go for either performance evaluation or inter-site scheduling... these seem to be two very different things, but both are very important aspects of the DEVA manager

First post in the PIRE blog, as well as first entry from Madrid. I've been here for five days, roughly, and there have been quite a lot of things going on...

I arrived last Wednesday, and used the day to set things up, mainly. My first meeting with the team at the Universidad Complutense de Madrid  was scheduled for Thursday in the morning, so I had time to find my housing, get acquainted with the city (how the metro works, where things are located...) and visit around. Being in a new city, it is always a challenge to orient oneself, and if the city is one as big and full of history as Madrid, it is almost impossible to decide where to start from. I rode the metro to the Sol station, where the famous Puerta del Sol is located. This is the centermost square in Madrid, from where most of the interesting places in the city center can be reached. This is also where multiple demonstrators were protesting against the current political situation in Spain, something that is happening in many other cities in the country.

Thursday was a more work-oriented day, though. Fighting against the jet-lag,  I got up early to travel to the university. I arrived just in time, after walking for almost half an hour across the huge -and beautiful- campus. (Who would think School of Information Sciences means School of Journalism here? Yeah... I guess I should have known...). The school of Computer Science is located across a wonderful pine field, a relatively new building without much people (yup, spaniards are in finals). After finding the room where I was supposed to meet with Tino, one of the members of the OpenNebula team, I was welcomed by the awesome people in there.

OpenNebula is one of the hottest Infrastructure as a Service Cloud managers, developed by a group at this university. We extensively use it in our research, and meeting the individuals behind it is a great opportunity. The team is led by Drs. Ignacio Llorente and Ruben Montero, although I will be spending most of the time with the main developers, Javi, Tino, Jaime, Carlos and Dani. Luck of lucks, another researcher from University of Chicago and student of Dr. Ian Foster, Dr. Borja Sotomayor, is also visiting the group during this time.

Thursday and Friday have been quite busy, setting down and trying to find a good work direction. After discussing with the team, I decided to work on our prototype, the DEVA manager, to upgrade it to the latest version of OpenNebula (we use 1.4, while they are intensely working on version 3.0) and integrate it better with their core functionality.

Finally, I was invited to a get together today with the team (spaniards really like partying :) and had a blast; we played a strange variation of Risk where everybody had a different set of rules (ok, that happens in every Risk game) and ate Chicago chili dogs courtesy of Borja. I got home really tired, but looking forward to tomorrow for more hacking :)