Audio/Video Supervised Independent Vector Analysis through multimodal pilot dependent components
Loading...
Date
2017
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Independent Vector Analysis is a powerful tool for estimating the broadband acoustic transfer function between multiple sources and the microphones in the frequency domain. In this work, we consider an extended IVA model which adopts the concept of pilot dependent signals. Without imposing any constraint on the de-mixing system, pilot signals depending on the target source are injected into the model enforcing the permutation of outputs to be consistent over time. A neural network trained on acoustic data and a lip motion detection are jointly used to produce a multimodal pilot signal dependent on the target source. It is shown through experimental results that this structure allows the enhancement of a predefined target source in very difficult and ambiguous scenarios.
Description
Subject(s)
independent vector analysis, source separation, independent component analysis, speech enhancement, multi- modal processing
Citation
ISSN
ISBN
978-0-9928626-7-1