Luckily Dr Gustavo Rossi, and his team have been very helpful in the progress of my thesis, development, design and evaluation. I was able to meet with Dr. Rossi again today, to show the prototype of the second project.

He suggested a few more accessibility mechanisms, and some planning strategies, in order to further progress the projects methodology.

The first project utilized speech recognition, which required voice training, and implemented a grease monkey plugin script in an attempt to create automated closed captioning, which sustained approximately 88% voice including misinterpretations.

here is a view of that first initial prototype

                                                            CaptionW.tif  

On a side note 

Here is a photo of lunch time in the lab, generally everyone eats together, its like one big happy family

My local Working Colleague (Matias Urbieta) and myself working on Internet Plugin for you tube

Myself and Loca Colleague Julian Grigera working on Web Site design

                                                         WebCam_20080627_1431.bmp

Our local translator for visa updating transactions Aldana Gomez

Apparently chorus shows, are easily found in Argentina. and even though they seem to be just a neighborhood chorus, believe me the voices were suitable for an opera!

After the show, I was invited to one of the performers homes' to enjoy some delivery of empanadas.

(it's a good thing we don't have empanada delivery in Florida, man I would be soooo fat)! Ofcourse we had to wait on the delivery so we had some mani (peanuts) to stave off the hunger. And, in true Argentinian style, Quilmes to wash it all down.

Yep this weeekend was a holiday weekend in Argentina. Even though it was cold, I braved it to the museum!

Needless to say, I stayed in for the remainder of the weekend

I know its pretty, but still it's just too cold

It was interesting to see how the mall side shops were set up on the ground!

several artists were demonstrating their work!

However it was really sad to see the kids begging, lying down in the cold

Ordering Water

Another interesting note about Argentina is ordering water with/ without gas!!!

The gas option tastes like soda/tonic water (not a preference of mine) but interesting.

Visa dilemma!

    The school requires that we change our status from visitors to students... So we took a trip to a few places and the final date should be on the 24th of June, even though we are scheduled to leave on the 13th of July, the requirement must be fulfilled.

So here is the proof, we are actually in the process

The Weather 

The temperature has gotten down to 0 degrees celsius. More than my body can handle even with the two heaters in the apartment.   

We really are working!!

So in the lab, they make a great deal about  birthdays! It's a real family atmosphere. The people in the lab planted balloons in one of the professor's cubicles, with a gift and a celebration. 

WEEKLY STATUS REPORT
Simone Pasmore
06.09.08

ACTIVITIES:

• Met with colleagues and updated server and client side requirements for software implementation.
• Read related papers
• Uploaded a video on youtube in order to implement/test the methodological synchronization of the acquired software.
• Investigated and located
  • An API to download video (Youtube-dl.py).
  • A media player that blocks (mplayer).
  • A Converter from flash video to mp3 format (works with mplayer).

ACCOMPLISHMENTS:

• Inserted caption bar on client side plugin.
• Created Batch file to appropriately synchronize the software for implementation.

ISSUES/PROBLEMS:

• The limitations of dictation software when transformed as an automatic closed captioning tool. (such as, punctuation, human ambiguity).
• Time length for processing of data is time-consuming.

PLANS:

• Update with colleagues.
• Research Time-stamp feasibility or pause insertion for newline entry or punctuation insertion.
• Event handlers for play, pause, and scroll.
• Investigate scroll for closed captioning synchronization.
• Conduct case studies based on error rate of software implementation.

SUMMARIES/CRITIQUES OF PAPERS:

Architecture, User Interface, and Enabling Technology in Windows Vista’s Speech Systems. Julian Odell and Kunal Mukerjee. IEEE Transactions on Computers, Vol.56, No. 9, September 2007.

This paper is written with the objective to encourage to the use of Windows Speech Recognition System, describe the overall architecture and the problems encountered with the software.

The speech recognition system provides the reserve to return to keyboard or speech input for clarity. The User Interface (UI) is to permit hands-free communication of the system. However, this is unattainable for the booting process, mainly for security purposes.

Unfortunately the paper does not discuss implementation of the API itself. Although the software has high accuracy rates for dictation, human ambiguity leaves for fall back correction, which has been a consistent a problem in Natural Language Processing (NLP).

The Windows Speech Recognition system is the software utilized to implement automatic closed captioning within youtube as a prototype for automated web closed captioning. This paper facilitates the digestion of architecture to further pursue improvements upon basic implementation of the prototype.

An Automatic Captioning System for Telemedicine Y. Zhao, X.Zhuang, R-S. Hu, J. Xue, X. Li, L. Che, R.Hu, L. Schopp.  2006 IEEE

                        This Paper presents the use of speech recognition for automatic captioning in the

                        Telemedicine environment. The system focuses on the health care

Providers voice solely as input, which is more adaptable for current speech recognition software. Implementing specific medical terms for captioning recognition.

The paper explains the necessity for caption rates to facilitate ease of reading for the deaf and hard of hearing community. As English is not their primary language, it is essential to have sufficient intervals to facilitate the absorption of materials presented.

The correlation is similar to my proposal, as the use of speech recognition software to facilitate automatic closed captioning. In order to adapt this idea in web format however, the necessity to accommodate multiple video posters is an essential area of investigation.

WEEKLY STATUS REPORT
Simone Pasmore

06.02.08

ACTIVITIES:

• Met with two fellow colleagues to discuss web pluggin feasibility in relation with the software and the integration  – decided on greaseMonkey, MonkeyGrease for client/server and server/client
• Read 3 Related Papers (based on Closed Captioning and Speech Recognition)
• Mixed audio for input so that audio may be captured directly from computer, rather than microphone.
• Downloaded sound software (soundForge “trial version”, Audacity), to research feasibility of voice extraction from music background, implementing pauses for dictation to be captures, and analyze voice patterns.
• Recorded several audio tracks from YouTube, news.bbc.co.uk, and CNN.com. to decipher specific voice regulations with dictation software.

ACCOMPLISHMENTS:

 ISSUES/PROBLEMS:

• Voice patterns are recognizable for single user only.
• Attaining API for all utilized software.

PLANS:

• Have a meeting with colleagues for a further update.
• Test the possibility of having multiple user recognition from the Windows Speech Recognition SoftWare.
• Test the feasibility of having Windows Speech Recognition validate the software for online web loaders automatically.
• Upload Video for testing purposes
• Integrate trial software process and investigate coding strategies to more efficiently, and effectively produce results.

SUMMARIES/CRITIQUES OF PAPERS:

Emotive Captioning Daniel G. Lee, Deborah I.Fels and John Patrick Udo Ryerson University, Toronto. ACM Computer in Entertainment, Vol 5, No. 2, Article xx. Publication date: August 2007

           Background music and noise is currently not renderable by traditional captioning. An alternative option for rendering emotions, for the deaf and hard of hearing populations, is to integrate the use of emoticons, such that the intent might be more accurately conveyed.

           The paper lacks an adequate case study. Such an implementation requires vast number of sectors, within the deaf community to have an accurate assessment of results.

           Additionally, this paper indicates that the deaf population was more comfortable with the conventional use of closed captioning, however the hard of hearing, younger segment was more comfortable with the emotive captioning.

           As research relates, age groups and degree of deafness, plays a significant role in closed captioning research, moreover, it is imperative to have the input of the deaf along the design process of any tool, used to enhance their lives.

Dancing with Words, Raisa Rashid, Quoc Vy, Richard Hunt, Deborah I Fels, University of Toronto, Ryerson University, York University, Ryerson University.

           Animation of closed Captioning to enhance the experience of the deaf population to demonstrate features of voice, including volume, speed, and pitch. The captions conveyed 5 basic emotions, anger, fear, happiness, sadness and disgust. Overall the introduction of animated captioning received a positive response.

           The Paper lacked substantial examples, and test cases for the purpose of research and further study.

           This paper reinforces that captioning improvement is not a lost cause, and can be enhanced to suit the deaf and hard of hearing communities.

Automatic Speech Recognition for Webcasts: How Good is Good Enough and What to Do When it Isn’t. Cosmin Munteanu, Gerald Penn, Ron Baecker, Yuecheng Zhang. University of Toronto, Canada

Speech recognition devices have significant Word Error Rates (WER’s). In order to overcome the deficiency, a collaborative tool has been developed in order to facilitate corrections, by users, in a wiki-like manner.

Applicable software and methods were only briefly discussed in the implementation of the tool. This left a huge gap between the theoretical and implementation aspects of the design.

The feasibility of overcoming obstacles may be attained with web 2.0 upgrades that have been recently integrated into the scheme of web services.

Groceries 

    There are 2 local grocery stores, Disco is definately the preffered choice (bigger with more options), Located on a diagonal, between 45/46 & 2/3. we were advised groceries were really inexpensive, however, I would definately advise otherwise. All supplies need to be purchased, paper products (toilet paper), dish washing liquid, spices for food...etc  (try to take everything into consideration).

fresh fruits and vegetables are available pretty much everywhere!!!! So they can be bought close to home on a daily basis (I guess thats the idea).

Laundry 

 One place to get your laundry done is 49 between 2 & 3 and the cost is pretty minimum $9.50 pesos for a weeks supply (give or take I guess). It is really close to the school, and therefore convinient. I was able to drop of my laundry at 8am, and be at school by 8:15, then the Laundry is ready for pick up at 6:00pm. Very important that you must have a bag to take your laundry to the laundry mat. (can you tell, I never thought of it). 

Going out in La Plata

Well, I was invited to a birthday party in La Plata, for one of the students in the lab. The time here is told in 24hour format, below is the slip I have. As you see, I have to write the 12hour format, so I can remember! (no memory)

Ha, and the partying really does not start till late in the night! so be prepared. 

Here are some pictures of the party for fun

        SideNote

             The men are not afraid to hug and kiss, get use to it!

Going to Church 

Well, I only have the Catholic information: This was hard and took a bit of perseverance and  a lot of patience!!!!

The Cathedral is beautiful (Tourist attraction) and is worth the trip but its far. You must apparently have more than one person to enter the tower tour (Thanks for the info Alex). Located 14/15 between  51/53. And Mass here is at 12 noon.

The closer alternative is  San Poinciana which is also a spectacle (beautiful but its not the cathedral). I actually prefer the mass at this location as they have more songs. Be aware that the mass is every hour 9am, 10am, 11am (at least these hours, not sure if it starts before, or when it ends). (dont walk in at 9:30 like me and think your late) just stand at the side.... thats what the people who came for the 11am Mass did. San Poinciana is located on 48 y 5 (another diagonal, but you can't miss it)

Bye the way

Men with girlfriends wear a simple small gold ban around there fingers in demonstration of their commitment, its so sweet.

 Well I saw people in the street playing drums in costumes. So far noone  has really explained why. However, drummers are very common here, and seeing drummers in the street is not out of the ordinary (so I am told)!

I am already pretty cold!!! be prepared that inside is (so far) very warm, and outside is pretty cold (bring t-shirts for inside) and lots of layers for outside (I pretty much only brought  clothes for the cold)! No clue how to prepare for this kind of weather.

 A local home gathering!!! 

 Alright well I know my pictures come out small, but you can enlarge to get the real thing!!

WEEKLY STATUS REPORT
Simone Pasmore
05.22.08

ACTIVITIES:

Ø  I read through the past papers of the W4A conference, in an aim to gain understanding of current and past studies in the field of accessibility (listed below).

Ø  Researched current tools, methodologies, and hindrances related to web accessibility for the hearing impaired.

Ø  Downloaded and tested speech recognition, transcribers, and dictation software for the purpose of attaining real time text for video materials.

Ø  Contacted the Rochester Institute of Technology for obtaining a possible API for sign language dictionary to be able to extract videos based on parsed text.

ACCOMPLISHMENTS:

Ø  Understanding of the current research trends for the Deaf Community

o    Closed Captioning not currently being implemented by end users

o    Tools for adding closed captioning given a video url include:

Bubleply + Veotag

Ø  Current limitations experienced based on current tool and methodological approaches.

o    The necessity of having closed captioning synchronized effectively with the video. (SMIL from W3C)

o    The problem of convincing users to add closed captioning to audio content.

Ø  Outlined the steps in my research to have a strategic plan for implementing browser plug-in.

Ø  Tested Bubleply + Veotag to experience ease of use and software application.

 
ISSUES/PROBLEMS:

Ø  Most transcriber software downloaded would not create the text file based on the trial version.

Ø  Finding an appropriate converter for any audio extension. (i.e not just .wav audio)

Ø  Appropriately training the speech recognition software in the lab, proved to be a difficult task.

Ø  Integration of dictation software with video materials, taking input from alternate source i.e(not directly from Microphone).

PLANS:

Ø  Utilize Windows speech recognition software (as an entry alternate)

Ø  Train the software

Ø  Attempt to integrate the dictation portion of the software with online videos, and measure clarity based on initial self trained voice and multiple videos.

Ø  Receive Videos/API from the RIT, might possibly need to appropriately tag these videos.

Ø  Work on browser plug-in for the dictation aspect of software

Ø  Explore JAVA widget as the possible plugin for browser.

SUMMARIES/CRITIQUES OF PAPERS:

Ø  Ajax live regions: chat as a case example
of software Peter Thiessen and Charles Chen [Bibtex entry for Thiessen et al.]

o    2.0 accessibility support with Java toolkits such as DOJO

Ø  Accessible image file formats: the need and the way (position paper) Sandeep R    Patil [Bibtex entry for Patil, S.]

o    Accessibility enhanced image formation technique with relevant modifications required in screen readers.

Ø  A preliminary usability evaluation of strategies for seeking online information with elderly people Sergio Sayago and Joseph Blat [Bibtex entry for Sayago et al.]

o    Comparing strategies for the elderly seeking information

                                                                           
Dr. Gustavo and his team, have been extreemely hospitable in accepting us here in La Plata. Some students, showed us to our apartment, office (lab), school, supermarket, even taking us for an Authentic Argentinan dinner last night.

So far it is amazing to me, just how much the Argentinians are welcoming and friendly, certainly refreshing !

Surprisingly, I noticed, many people smoke here. Both  the Kids and adults, they smoke everywhere in buildings in the streets, it does not matter, after all it is cold outside

We have only just begun ;)

Supervisor’s Name and Title at FIU/FAU:

Dr. Shihong Huang

Assistant Professor

Dept. of Computer Science & Engineering

Florida Atlantic University

Name of the PIRE International Partner’s Institution:

Research and Development in Advanced IT

National University of La Plata

Buenos Aires

Argentina

Supervisor’s Name and Title at the PIRE International Partner’s Institution:

Dr. Gustavo Rossi

Director

Lifia

Project Title:

Web Accessibility for the Hearing Impaired

Problem Statement: Briefly explain (in one paragraph) the research problem that you are going to address in this project.

The advancing of Internet technology provides an information super-highway for disseminating information on the web. The Web has become such an abundant resource for almost every single aspect of our lives, from disseminating information, research, entertaining, online business, to news update to just name a few. Multimedia features, such as embedded audio and video clips on the web, enrich the web contents and resources to a higher level. While many of people can fully embrace and enjoy these multimedia features on the web, some people who are hearing impaired are simply left out by these advanced features. This situation generates imbalanced web user experiences between different groups of people, and the situation needs to be addressed.

Motivation and Impact: Provide a brief explanation (in one paragraph) of the motivation of this work and its significance.

The motivation of this research is to provide the same level of web experiences for the deaf as that for hearing people, and to fully utilize the multimedia features on the web for the deaf people. By integrating existing technologies and develop new technology as needed, this research will provide a rich web accessibility for its contents that are in both text format and in multimedia format to the deaf people, and bridge the gap between the silent world and the multimedia content on the web so that browsing web for deaf people becomes a fully enjoyable experience.

Current Status: Provide a brief overview of the current status of the project. If this is an ongoing work, please describe preliminary results. In either case, what related work is going on in this area at FIU/FAU, the partner institution, or elsewhere?

The student who is in the project has been taking American Sign Language (ASL), and has been reading related papers of web accessibility during her first year of graduate study. Our partner institute, Lifia Research and development in Advanced IT in National University of La Plata has a strong research group on Web Engineering. Lifia’s Director Dr. Gustavo Rossi is a world-renowned scholar in web engineering field and he will be a great asset and adviser during the student’s visit in Lifia.

Research Roadmap: Provide an itemized list that shows your research roadmap/timeline and that lists expected outcomes of your research including a paper to be published, etc.

• Study literatures of work has been done in web accessibility for deaf people
• Compare and select audio to text software
• Compare and select text-to-ASL software, with either people or avatar
• Integration: develop prototype web site that integrate audio clip and sign language translation
• Validate the prototype using selected web site

The results and findings from this research will be published in peer-reviewed academic journal and conferences, such as Web Site Evolution (WSE 2008).

Relation to PIRE Core Research Projects: Briefly explain how the project fits into the PIRE theme of Cyberinfrastructure enablement. Which box in the table (our “9 box” figure at http://pire.fiu.edu/research.php) does your research fit into and why? If this project is not directly related to any of the boxes in the table, then provide a rationale on how this project is related to Cyberinfrastructure enablement.

Currently, this project doesn’t fit into any of these nine boxes listed in the PIRE Cyberinfrastructure enablement. Since web has become ubiquitous in the cyber space, improve web accessibility for hearing impaired people and provide the same level of web user experience for all the people are such a critical part of the whole cyberinfrastructure, and further advance cyberinfrastructure enablement.