ACE Challenge

Researchers: James Eaton, Alastair H. Moore, Nikolay D. Gaubitch, Patrick Naylor

Several established parameters and metrics have been used to characterize the acoustics of a room. The most important are the Direct-To-Reverberant Ratio (DRR), the Reverberation Time (T60) and the reflection coefficient. The acoustic characteristics of a room based on such parameters can be used to predict the quality and intelligibility of speech signals in that room. Recently, several important methods in speech enhancement and speech recognition have been developed that show an increase in performance compared to the predecessors but do require knowledge of one or more fundamental acoustical parameters such as the T60. Traditionally, these parameters have been estimated using carefully measured Acoustic Impulse Responses (AIRs). However, in most applications it is not practical or even possible to measure the acoustic impulse response. Consequently, there is increasing research activity in the estimation of such parameters directly from speech and audio signals.

The ACE Challenge is part of the programme of Challenges organised by the IEEE Audio and Acoustic Signal Processing Technical Committee.

The aim of this challenge was to evaluate state-of-the-art algorithms for blind acoustic parameter estimation from speech and to promote the emerging area of research in this field. Participants evaluated their algorithms for T60 and DRR estimation against the ‘ground truth’ values provided with the data-sets. Furthermore, they were expected to present the results in a paper describing the method used.

  • Data:

    A data-set specifically designed for the challenge tasks was provided using anechoic speech convolved with AIRs measured from real rooms with additive noise recorded under the same conditions. This included speech from male and female talkers in different sized rooms and different noise conditions for a single microphone and for microphone arrays with two (laptop), three (mobile), five (cruciform), eight (linear), and thirty-two (spherical) microphones

  • Task 1:

    Single-microphone fullband T60 and DRR estimation

  • Task 2:

    Multi-microphone fullband T60 and DRR estimation

  • Task 3:

    Single-microphone T60 and DRR estimation in 1/3-octave ISO subbands

  • Task 4:

    Multi-microphone T60 and DRR estimation in 1/3-octave ISO subbands

  • Evaluation:

    The evaluation metrics were based on the ground truth values determined using established techniques across a range of dimensions in addition to T60 and DRR such as SNR, talker, and utterance length.

ACE Challenge Workshop

Challenge participants were invited to present their results at the ACE Challenge workshop, subject to paper review, which was held as a satellite event during WASPAA 2015 in New Paltz, NY, USA.

Important dates

Phase 1
  • 27th Jan 2015:

    Release of the development data-set

  • 2nd March 2015:

    Release of the evaluation data-set

  • 20th March 2nd April 2015 12:00 GMT:

    Submission of results by participants

  • 17th April 2015:

    WASPAA 2015 paper submission deadline

Phase 2
  • 3rd June 2015 11:00 GMT:

    Submission of results by participants

Closing phase
  • 3rd July 2015 12:00 GMT:

    Submission of camera-ready papers.

  • Early August, 2015:

    Notification of acceptance

  • 21st October 2015:

    ACE satellite workshop during WASPAA 2015

Organizing Committee

  • James Eaton
  • Nikolay D. Gaubitch
  • Alastair H. Moore
  • Patrick A. Naylor

Proceedings

The ACE Challenge proceedings are available here.

ACE Corpus

The ACE corpus is freely available under the Creative Commons Attribution-NoDerivatives 4.0 International License.

The corpus and the ACE Challenge are described in the following papers:

  • J. Eaton, N. D. Gaubitch, A. H. Moore, P. A. Naylor, “Estimation of room acoustic parameters: The ACE Challenge”. To appear in: IEEE Transactions on Audio, Speech, and Language Processing.
  • J. Eaton,N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “The ACE challenge – corpus description and performance evaluation,”Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct. 2015.

Please cite this whenever you use any part of the corpus (Bibtek reference available here).

The corpus comprises the following components:

Documentation and software
  • Corpus instructions including software operating instructions
  • Software to generate new datasets from the corpus materials (Matlab)
  • T60 and DRR measurements in fullband and ISO-266 preferred frequency bands
  • Room dimensions and approximate positions of microphones and sources
RIRs and noise by microphone configuration

Each archive below contains the set of fs=48 kHz 32-bit RIRs,
ambient, fan and babble noise .wav files for each room and microphone position for that microphone configuration, recorded in 7 different rooms in the Dept. of Electrical and Electronic Engineering at Imperial College London.

  • Single-channel (based on cruciform channel 1) 417 MB
  • 2-channel laptop 1.05 GB
  • 3-channel mobile 1.59 GB
  • 5-channel cruciform 2.84 GB
  • 8-channel linear 4.24 GB
  • 32-channel spherical 14.2 GB

The ACE corpus files including documentation and software are available here.

If you have already registered click here to obtain the corpus.

Creative Commons License

ACE Corpus by Imperial College London and University of Delft is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

Relevant publications:

Eaton, James; Gaubitch, Nikolay D.; Moore, Alastair H.; Naylor, Patrick A.

Acoustic Characterization of Environments (ACE) Challenge Results Technical Report (revised) Technical Report

Imperial College London 2017.

Abstract | Links | BibTeX

Eaton, J.; Gaubitch, N. D.; Moore, A. H.; Naylor, P. A.

Estimation of room acoustic parameters: The ACE Challenge Journal Article

IEEE Transactions on Audio, Speech, and Language Processing, 2016, ISSN: 2329-9290.

Abstract | Links | BibTeX

Eaton, J.; Gaubitch, N. D.; Moore, A. H.; Naylor, P. A.

The ACE Challenge - corpus description and performance evaluation Inproceedings

Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, 2015.

Abstract | Links | BibTeX

Eaton, J.; Naylor, P. A.

Reverberation time estimation on the ACE corpus using the SDD method Inproceedings

Proceedings of the ACE Challenge Workshop, a satellite event of IEEE-WASPAA, New Paltz, NY, USA, 2015.

Abstract | Links | BibTeX

Eaton, J.; Naylor, P. A.

Direct-to-Reverberant ratio estimation on the ACE corpus using a Two-channel beamformer Inproceedings

Proceedings of the ACE Challenge Workshop, a satellite event of IEEE-WASPAA, New Paltz, NY, USA, 2015.

Abstract | Links | BibTeX

Eaton, J.; Gaubitch, N. D.; Moore, A. H.; Naylor, P. A.

Proceedings of the ACE Challenge Workshop, a satellite event of IEEE-WASPAA Proceeding

New Paltz, NY, USA, 2015.

Abstract | Links | BibTeX

Parada, P. Peso; Sharma, D.; Waterschoot, T. van; Naylor, P. A.

Evaluating the Non-intrusive Room Acoustics Algorithm with the ACE Challenge Inproceedings

Proceedings of the ACE Challenge Workshop, a satellite event of IEEE-WASPAA, 2015.

Abstract | Links | BibTeX

Eaton, J.; Gaubitch, N. D.; Naylor, P. A.

Noise-robust reverberation time estimation using spectral decay distributions with reduced computational cost Inproceedings

Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, Canada, 2013.

Abstract | Links | BibTeX