Home   >   Projects   >   MULTISUB

Multisub

Multilingual subtitling of classrooms and plenary sessions

Field
National
Date
01/01/2019 - 31/12/2021
Industry
  • Educación, tecnologías educativas, industria audiovisual, administraciones públicas
  • Otros
Budget
52393
Funded by

Spanish Ministry of Science, R&D&I Projects "Research Challenges" 2018

PROJECT INFORMATION

DESCRIPTION

The main aim of the project is to further improve the state of the art in Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT) to deal with the kind of audiovisual collections we are considering: lecture recordings and plenary sessions. Thus, different open research challenges in ASR and SMT will be addressed throughout the project: i) channel variability, noise and reverberation; (ii) far-field speech recognition; (iii) speaker diarization; (iv) multispeaker recognition; v) on-line speech recognition; (vi) neural machine translation; (vii) translation of sentences containing ASR errors.
The technology developed will be tested in two use cases. On the one hand, a repository of classroom video recordings called "Videoapunts" developed at the Universitat Politècnica de València. The classes are recorded using microphone arrays and the open source Opencast platform which supports the management of educational audio and video content. On the other hand, transparency portals of public administrations will be considered to create a repository of video recordings of plenary sessions.

The main aim of the project is to further improve the state of the art in Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT) to deal with the kind of audiovisual collections we are considering: lecture recordings and plenary sessions. Thus, different open research challenges in ASR and SMT will be addressed throughout the project: i) channel variability, noise and reverberation; (ii) far-field speech recognition; (iii) speaker diarization; (iv) multispeaker recognition; v) on-line speech recognition; (vi) neural machine translation; (vii) translation of sentences containing ASR errors.
The technology developed will be tested in two use cases. On the one hand, a repository of classroom video recordings called “Videoapunts” developed at the Universitat Politècnica de València. The classes are recorded using microphone arrays and the open source Opencast platform which supports the management of educational audio and video content. On the other hand, transparency portals of public administrations will be considered to create a repository of video recordings of plenary sessions.

Impact

The project will have a dramatic impact in OE and PO, overcoming communication barriers and enabling an uninhibited exchange of ideas across languages and cultures. The technologies developed will bring important advances in the hot topic research fields of ASR and SMT. The project is also aligned with market and user needs, providing plausible scenarios for products or services in which research results can be applied beyond the educational and parliamentary contexts.

Success stories

Commercial use of the MLLP Transcription and Translation Platform by third party companies such as the Hasso Plattner Institut (Germany), and by international organizations such as the European Organization for Nuclear Research (CERN): https://ttp.mllp.upv.es.

Entities

UPV

Contact information

Albert Sanchis Navarro
University Lecturer (Universitat Politècnica de València)

VRAIN

Technological capabilities

IA
Natural language processing and text mining