Inicio   >   Proyectos   >   MULTISUB

Multisub

Multilingual subtitling of classrooms and plenary sessions

Ámbito
Nacional
Fecha
01/01/2019 - 31/12/2021
Sector
  • Educación, tecnologías educativas, industria audiovisual, administraciones públicas
  • Otros
Presupesto
52393
Finanaciador

Ministerio de Ciencia de España, Proyectos I+D+i «Retos Investigación» 2018

INFORMACIÓN DEL PROYECTO

DESCRIPCIÓN

The main aim of the project is to further improve the state of the art in Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT) to deal with the kind of audiovisual collections we are considering: lecture recordings and plenary sessions. Thus, different open research challenges in ASR and SMT will be addressed throughout the project: i) channel variability, noise and reverberation; ii) far-field speech recognition; iii) speaker diarization; iv) multispeaker recognition; v) on-line speech recognition; vi) neural machine translation; vii) translation of sentences containing ASR errors.
The technology developed will be tested in two use cases. On the one hand, a repository of classroom video recordings called “Videoapunts” developed at the Universitat Politècnica de València. The classes are recorded using microphone arrays and the open source Opencast platform which supports the management of educational audio and video content. On the other hand, transparency portals of public administrations will be considered to create a a repository of video recordings of plenary sessions.

The main aim of the project is to further improve the state of the art in Automatic Speech Recognition (ASR) and Statistical Machine Translation (SMT) to deal with the kind of audiovisual collections we are considering: lecture recordings and plenary sessions. Thus, different open research challenges in ASR and SMT will be addressed throughout the project: i) channel variability, noise and reverberation; ii) far-field speech recognition; iii) speaker diarization; iv) multispeaker recognition; v) on-line speech recognition; vi) neural machine translation; vii) translation of sentences containing ASR errors.
The technology developed will be tested in two use cases. On the one hand, a repository of classroom video recordings called “Videoapunts” developed at the Universitat Politècnica de València. The classes are recorded using microphone arrays and the open source Opencast platform which supports the management of educational audio and video content. On the other hand, transparency portals of public administrations will be considered to create a a repository of video recordings of plenary sessions.

Impacto

The project will have a dramatic impact in OE and PO, overcoming communication barriers and enabling an uninhibited exchange of ideas across languages and cultures. The technologies developed will bring important advances in the hot topic research fields of ASR and SMT. The project is also aligned with market and user needs, providing plausible scenarios for products or services in which research results can be applied beyond the educational and parliamentary contexts.

Casos de Éxito

Uso comercial de la Plataforma MLLP de Transcripción y Traducción por parte de terceras empresas como el Hasso Plattner Institut (Alemania), y por parte de organismos internacionales como la Organización Europea para la Investigación Nuclear (CERN): https://ttp.mllp.upv.es

Entidades

UPV

DATOS DE CONTACTO

Albert Sanchis Navarro
Profesor Titular de Universidad (Universitat Politècnica de València)

VRAIN

CAPACIDADES TECNOLÓGICAS

IA
Natural language processing and text mining