Instructions for preparing proposal Part b for Integrated Projects in the ist priority

Task 2.1: Media fragments addressing (EURECOM)

The task WP2.1 aims at providing a URI-based mechanism for identifying, and retrieving the media objects detected in WP1. We will work under the assumption that an audio or a video resource has a single unified timeline. We will consider four dimensions for addressing fragments of image, audio and video resources namely: time, space, track and name. In liaison with the W3C Media Fragments Working Group, we will further specify the communication between user agents and web servers enhancing the whole web architecture for making video a first class citizen in the evolution of the world wide web. In this task, we will provide various implementations on client and serve side deploying the media fragments technology.

Task 2.2: Lightweight metadata models for hypervideo (EURECOM)

The task WP2.2 focuses on developing lightweight models for metadata interoperability. Media objects automatically detected in WP1 will be annotated using simple metadata schemas while being compatible with the numerous multimedia metadata standards. The starting point will be the COMM multimedia ontology developed within the FP6 EU project K-Space. We will adapt and simplify this model in order to create a Lite version according to the needs of the ConnectME use cases. We will implement an API and the necessary interfaces to be integrated in the ConnectME infrastructure. In liaison with the W3C Media Annotations Working Group, we will ensure that the metadata schema is largely interoperable with the numerous metadata standards, providing mappings between the description properties.

Task 2.3: Web mining for support of media annotation (UEP)

The goal of this task is to exploit web resources associated with the broadcast to assist the annotation process. It complements the work in WP1 (focusing on video analysis and on resources available in structured form). The methods will include wrapper-based extraction from structured tables and Named Entity Recognition over free text (scripts, subtitles, online reports to sports events etc.), or a mix of these (e.g. from programme guides). Novel methods of term expansion and disambiguation using WordNet, Wikipedia, and domain-specific databases will be developed, in order to arrive at uniquely identified concept needed for further processing. We will benchmark and compare numerous Information Extraction tools such as OpenCalais, SPROUT, GATE, KIM and evaluate novel disambiguation algorithms. The resulting multimedia annotations will then follow the linked data principles and contribute to extend the growing linked data cloud with additional interlinkage.

Task 2.4: Mining additional content from the web (UEP)

The goal of this task is to retrieve and pre-process external web resources to enrich the broadcasts with additional information about important concepts. We will investigate the use of advanced techniques for web mining that have not yet been considered in connection with multimedia concepts. First, large (manually pre-selected) collections of web resources, forming an 'information cloud' around individual broadcasts, will be examined at the level of individual document, using web spiders and document categorisers. Second, concrete documents selected via spidering and classification will be submitted to fine-grained information extraction (IE) tools.

As core IE techniques for this purpose we envisage rapid prototyping of rich extraction models combined with partial statistical model training (offline) and local wrapper induction (on the fly). The structure of the extraction models could borrow from the structure of Linked Data resources (such as DBPedia) in which the source concepts are anchored. The hybrid approach (rich hand-crafted extraction model + trained statistical model + local wrapper) is particularly suitable for collections of semi-structured, field-specific web resources, whose internal structure can only be roughly estimated.

Together with textual resources, structured resources with similar scope will be exploited. For example, extraction from free-text Wikipedia articles will be complemented with and/or triggered by processing of DBpedia. Furthermore, available social network metadata will be used. Text processing techniques such as co-occurrence analysis and named entity recognition will be applied to loose metadata structures such as folksonomy tags.

Work package 3: ConnectME interface and presentation

The work will be carried out in close cooperation with WP1 and WP2 on the underlying knowledge being made available, with the personalization and contextualization work in WP4, and with the front-end being developed in WP5 and the implementation of the scenarios in WP6.

The workpackage will construct hypervideo interaction interfaces for supporting information browsing, organization and presentation, and higher level tasks such as information gathering. It is based on the tasks implicit in the 3 scenarios and will generalize results to identify design patterns and guidelines.

The interface design will strive to hide the complexity of the ConnectME system operating in the background.  For example, the user should be unaware of the connection between the video pixels and the concept describing them, the relation between this and a thesaurus, and how these are enriched with other web-based information. At the same time, the user should be allowed access to the underlying information resources to enable them to check information sources when needed.  A metric for success for the ConnectME interfaces is that the user should not even notice “the interface”, but that they should be able to carry out their tasks with a minimum of effort.

Task 3.1 User requirements analysis based on the 3 scenarios.

A list of information tasks related to hypervideo will be developed in the initial stages of the project. These will be selected from tasks that occur in more than one of the scenarios, such as, e.g., highly visual information browsing (e.g. for children’s edutainment interfaces and cultural heritage), or information gathering for professional information users (e.g. environmental issues and cultural heritage).  From the list of identified tasks, a small number (3 or 4) will be selected for design, implementation and evaluation within the project.

The task will also collect requirements for the interfaces to be developed. Examples of such requirements are:

• 
  Users engaged in audiovisual content should not be overly distracted.
  
• 
  The role of the social context of the user should be taken into account.
  
• 
  The role of the personal interests of the user must not be neglected.