MFA and Pretrained Acoustic Models

The Montreal Forced Aligner (MFA) is a third-party tool developed by Michael McAuliffe and others for time aligning orthographic and phonological forms from a pronunciation dictionary to orthographically transcribed audio files. It is open source software based on the Kaldi ASR toolkit.

LaBB-CAT includes a layer manager module called “MFA Manager” which integrates with MFA in order to facilitate forced alignment of LaBB-CAT corpus data.

The layer manager can work in two modes:

  • Train and Align - acoustic models are trained on the data you want to align, which can be in any language as long as you have a pronunciation dictionary for it.
  • Pre-trained Models/Dictionaries - pre-trained models and pronunciation dictionaries are supplied by the Montreal Forced Aligner and used for forced alignment. Languages for which dictionaries are available listed on the MFA website and include:
    • English
    • French
    • German
    • Brazilian Portuguese
    • Spanish
    • Catalan

These instructions assume that your corpus is in one of these languages, and uses the Pre-trained Models/Dictionaries approach…

MFA Installation

MFA is not included as part of LaBB-CAT, and so it must be installed on the server you have installed LaBB-CAT on before you can integrate LaBB-CAT with it.

If MFA has not been installed already, please follow the following steps, depending on the operatings system of your LaBB-CAT server:

To install the Montreal Forced Aligner on Linux systems for all users, so that your web server can access it if required:

  1. Download Miniconda:
    wget https://repo.anaconda.com/miniconda/Miniconda3-py38\_4.10.3-Linux-x86\_64.sh
  2. Start the installer:
    sudo bash Miniconda3-py38\_4.10.3-Linux-x86\_64.sh
  3. When asked the location to install Miniconda, use:
    /opt/conda
  4. When asked whether the installer should initialize Miniconda, this is unnecessary so you can respond no
  5. Change ownership of the conda files):
    sudo chown -R $USERNAME:$USERNAME /opt/conda
  6. Make conda accessible to all users (so you web server can access MFA):
    chmod -R go-w /opt/conda
    chmod -R go+rX /opt/conda
  7. Install the Montreal Forced Aligner
    /opt/conda/bin/conda create -n aligner -c conda-forge montreal-forced-aligner=2.2.17

To install the Montreal Forced Aligner on Windows systems for all users, so that your web server can access it if required:

  1. Download the Miniconda installer:   
    https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
  2. Start the installer by double-clicking it.
  3. When asked, select the “Install for all users” option. This will install conda somewhere like
    C:\ProgramData\Miniconda3
  4. When asked, tick the add to PATH option.
  5. Install the Montreal Forced Aligner by specifying a path to the environment
    conda create -c conda-forge -p C:\ProgramData\Miniconda3\envs\aligner montreal-forced-aligner=2.2.17

The 3rd party MFA software requires:

  • the possibility of running command-line programs during installation and forced alignement
  • the possibility that these programs can download data from the internet

On Windows, this can sometimes be complicated by the fact that Apache Tomcat and LaBB-CAT are installed as a ‘Windows Service’. Windows Services usually run using the permissions of a special anonymous login account called ‘Local System’, which in some environments has restricted permissions to access different resources.

If you install the MFA Manager LaBB-CAT integration module, but you find it returns errors when trying to interact with MFA, the problem may be that the Windows Service:

  • does not have permission to access the folder where MFA is installed, or
  • is not allowed to execute other programs, or
  • cannot access the internet.

Sometimes problems can be resolved by:

  • running the Apache Tomcat Windows Service as a different user other than ‘Local System’. (or if it was running as some other used, try setting it back to ‘Local System’), or
  • adjusting the permissions of the Windows Service users, or
  • adjusting the permissions of the folders where MFA is installed
  • configuring the service to use the local Internet Proxy settings to enable connecting to the internet.

PSexec is a tool that can be used to diagnose and solve problems on Windows.

PSexec

  1. Download PStool.zip from Microsoft:
    http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx
  2. Unzip it
  3. Put PSexec.exe into C:\Windows\System32
  4. Open cmd using “Run as Administrator”
  5. Run the command:
    Psexec.exe -i -s cmd.exe
    This opens a new command prompt for local system account
  6. In the new command prompt window, check you have the correct account type with the command:
    whomai

Then you can use the command prompt to run MFA commands to diagnose errors - e.g.:

  • conda activate montreal-forced-aligner - activates the MFA environment
  • mfa version - ensures MFA is installed and accessible, and confirms the version
  • mfa model download dictionary - ensures MFA can connect to the internet to get models etc.; this command should return a long list of language dictionaries, and not report errors.

Proxy Settings

To update proxy server settings:

  1. type inetcpl.cpl
  2. goto Connections tab
  3. click on the LAN Settings button
  4. Fill in the Proxy section with the correct details

If your LaBB-CAT server is installed in a Docker Container, it can download and install Miniconda and MFA itself, as part of the process of installing the MFA Manager LaBB-CAT module.

There is no need for a separate installation of the MFA software.

Layer Manager Installation

Once MFA has been installed, you have to install the MFA Manager, which is the LaBB-CAT module that provides MFA with all the data it needs, and then saves to alignments MFA produces back to your database.

  1. Select the layer managers menu option.
  2. Follow the List of layer managers that are not yet installed link.
  3. Find MFA Manager in the list, and press its Install button and then press Install again.
    As long as MFA has been installed for all users, you should see a box that’s already filled in with the location that MFA was installed to.
  4. Click Configure to continue the layer manager installation.
    You will see a window open with some information about integrating with MFA, including the information you’ve already read above.

Forced Alignment

Once you’ve

  1. Now you need to add a phrase layer for the HTK configuration:
    • Layer ID: mfa
    • Type: Text
    • Alignment: Intervals
    • Manager: MFA Manager
    • Generate: always
    • Description: MFA alignment time
  2. When you configure the layer, set the following options:
    • Dictionary Name: the dictionary language, e.g. english_uk_mfa
    • Pretrained Acoustic Models: the models language, e.g. english_mfa
    • The rest of the options can be left as their default values.
    • If you’re curious about what the configuration options do, hover your mouse over each option to see a `tool tip’ that describes what the option is for.
  3. Press Set Parameters
  4. Press Regenerate
    You will see a progress bar while LaBB-CAT force-aligns all the transcripts in the corpus, which may take a few minutes.
  5. When the layer manager has finished, you’ll see a message saying:
    Complete - words and phones from selected utterances are now aligned.
Tip

Not all MFA pre-trained acoustic models can be used with all dictionaries.

Apart from matching the language (e.g. English-trained acoustic models should be used only with English dictionaries), the phoneme symbol sets must also match.

MFA uses several symbol sets, including:

  • IPA - model and dictionary names ending in ..._mfa
  • ARPAbet - model and dictionary names ending in ..._arpa

So if you use an acoustic model ending in ...arpa, then the dictionary you choose must also end in ...arpa.

See the MFA documentation on models and dictionaries for more detailed information:
https://mfa-models.readthedocs.io

Reuse