MFA and Pretrained Acoustic Models
The Montreal Forced Aligner (MFA) is a third-party tool developed by Michael McAuliffe and others for time aligning orthographic and phonological forms from a pronunciation dictionary to orthographically transcribed audio files. It is open source software based on the Kaldi ASR toolkit.
LaBB-CAT includes a layer manager module called “MFA Manager” which integrates with MFA in order to facilitate forced alignment of LaBB-CAT corpus data.
The layer manager can work in two modes:
- Train and Align - acoustic models are trained on the data you want to align, which can be in any language as long as you have a pronunciation dictionary for it.
- Pre-trained Models/Dictionaries - pre-trained models and pronunciation dictionaries are supplied by the Montreal Forced Aligner and used for forced alignment. Languages for which dictionaries are available listed on the MFA website and include:
- English
- French
- German
- Brazilian Portuguese
- Spanish
- Catalan
These instructions assume that your corpus is in one of these languages, and uses the Pre-trained Models/Dictionaries approach…
MFA Installation
MFA is not included as part of LaBB-CAT, and so it must be installed on the server you have installed LaBB-CAT on before you can integrate LaBB-CAT with it.
If MFA has not been installed already, please follow the following steps, depending on the operatings system of your LaBB-CAT server:
To install the Montreal Forced Aligner on Linux systems for all users, so that your web server can access it if required:
- Download Miniconda:
wget https://repo.anaconda.com/miniconda/Miniconda3-py38\_4.10.3-Linux-x86\_64.sh
- Start the installer:
sudo bash Miniconda3-py38\_4.10.3-Linux-x86\_64.sh
- When asked the location to install Miniconda, use:
/opt/conda
- When asked whether the installer should initialize Miniconda, this is unnecessary so you can respond
no
- Change ownership of the conda files):
sudo chown -R $USERNAME:$USERNAME /opt/conda
- Make conda accessible to all users (so you web server can access MFA):
chmod -R go-w /opt/conda
chmod -R go+rX /opt/conda
- Install the Montreal Forced Aligner
/opt/conda/bin/conda create -n aligner -c conda-forge montreal-forced-aligner=2.2.17
To install the Montreal Forced Aligner on Windows systems for all users, so that your web server can access it if required:
- Download the Miniconda installer:
https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe - Start the installer by double-clicking it.
- When asked, select the “Install for all users” option. This will install conda somewhere like
C:\ProgramData\Miniconda3 - When asked, tick the add to PATH option.
- Install the Montreal Forced Aligner by specifying a path to the environment
conda create -c conda-forge -p C:\ProgramData\Miniconda3\envs\aligner montreal-forced-aligner=2.2.17
The 3rd party MFA software requires:
- the possibility of running command-line programs during installation and forced alignement
- the possibility that these programs can download data from the internet
On Windows, this can sometimes be complicated by the fact that Apache Tomcat and LaBB-CAT are installed as a ‘Windows Service’. Windows Services usually run using the permissions of a special anonymous login account called ‘Local System’, which in some environments has restricted permissions to access different resources.
If you install the MFA Manager LaBB-CAT integration module, but you find it returns errors when trying to interact with MFA, the problem may be that the Windows Service:
- does not have permission to access the folder where MFA is installed, or
- is not allowed to execute other programs, or
- cannot access the internet.
Sometimes problems can be resolved by:
- running the Apache Tomcat Windows Service as a different user other than ‘Local System’. (or if it was running as some other used, try setting it back to ‘Local System’), or
- adjusting the permissions of the Windows Service users, or
- adjusting the permissions of the folders where MFA is installed
- configuring the service to use the local Internet Proxy settings to enable connecting to the internet.
PSexec is a tool that can be used to diagnose and solve problems on Windows.
PSexec
- Download PStool.zip from Microsoft:
http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx - Unzip it
- Put PSexec.exe into
C:\Windows\System32
- Open
cmd
using “Run as Administrator” - Run the command:
Psexec.exe -i -s cmd.exe
This opens a new command prompt for local system account - In the new command prompt window, check you have the correct account type with the command:
whomai
Then you can use the command prompt to run MFA commands to diagnose errors - e.g.:
conda activate montreal-forced-aligner
- activates the MFA environmentmfa version
- ensures MFA is installed and accessible, and confirms the versionmfa model download dictionary
- ensures MFA can connect to the internet to get models etc.; this command should return a long list of language dictionaries, and not report errors.
Proxy Settings
To update proxy server settings:
- type
inetcpl.cpl
- goto Connections tab
- click on the LAN Settings button
- Fill in the Proxy section with the correct details
If your LaBB-CAT server is installed in a Docker Container, it can download and install Miniconda and MFA itself, as part of the process of installing the MFA Manager LaBB-CAT module.
There is no need for a separate installation of the MFA software.
Layer Manager Installation
Once MFA has been installed, you have to install the MFA Manager, which is the LaBB-CAT module that provides MFA with all the data it needs, and then saves to alignments MFA produces back to your database.
- Select the layer managers menu option.
- Follow the List of layer managers that are not yet installed link.
- Find MFA Manager in the list, and press its Install button and then press Install again.
As long as MFA has been installed for all users, you should see a box that’s already filled in with the location that MFA was installed to. - Click Configure to continue the layer manager installation.
You will see a window open with some information about integrating with MFA, including the information you’ve already read above.
Forced Alignment
Once you’ve
- Now you need to add a phrase layer for the HTK configuration:
- Layer ID:
mfa
- Type: Text
- Alignment: Intervals
- Manager: MFA Manager
- Generate: always
- Description: MFA alignment time
- Layer ID:
- When you configure the layer, set the following options:
- Dictionary Name: the dictionary language, e.g. english_uk_mfa
- Pretrained Acoustic Models: the models language, e.g. english_mfa
- The rest of the options can be left as their default values.
- If you’re curious about what the configuration options do, hover your mouse over each option to see a `tool tip’ that describes what the option is for.
- Press Set Parameters
- Press Regenerate
You will see a progress bar while LaBB-CAT force-aligns all the transcripts in the corpus, which may take a few minutes. - When the layer manager has finished, you’ll see a message saying:
Complete - words and phones from selected utterances are now aligned.
Not all MFA pre-trained acoustic models can be used with all dictionaries.
Apart from matching the language (e.g. English-trained acoustic models should be used only with English dictionaries), the phoneme symbol sets must also match.
MFA uses several symbol sets, including:
- IPA - model and dictionary names ending in
..._mfa
- ARPAbet - model and dictionary names ending in
..._arpa
So if you use an acoustic model ending in ...arpa
, then the dictionary you choose must also end in ...arpa
.
See the MFA documentation on models and dictionaries for more detailed information:
https://mfa-models.readthedocs.io