Facebook recently announced the global rollout of NMT (Neural Machine Translation). Switching from phrase based translation models to NMT has been rolled out for more than 2,000 translation directions and 4.5 billion translations per day. According to Facebook this provides an 11% increase in BLEU score, a well known scoring mechanism for translation scoring.
The engineers behind this blog post Juan Miguel Pino, Alexander Sidorov and Necip Fazil Ayan achieved a 2.5X boost in performance using Caffe2, the Machine Learning framework Facebook open sourced earlier this year.
The key to achieving this level of translation quality lies in using a recurrent neural network known as sequence-to-sequence LSTM (long short-term memory) with attention. A system like this takes into account the context of the source sentence and everything preceding it to generate more accurate translations. This allows for long-distance reordering in translation, solving a problem found amongst others between some pairs of translation like English-Turkish and English-Greek.
The attention mechanism of LSTM also helps when handling with unknown words in translations by soft aligning translations into a bilingual lexicon built from the training data. Vocabulary reduction is used to to reduce computation time significantly without notably affecting translation quality.
The FB Learner Flow framework was used to fine-tune parameters for each of the 2,000 translation pairs quickly and accurately. This alone contributed to a 3.7% increase in BLEU score for English-Spanish translation pairs.
As part of this project, Facebook contributed and open sourced Recurrent Neural Networks into Caffe2. Facebook’s Artificial Intelligence Research (FAIR) team has published its research on using convolutional neural networks (CNNs) for machine translation as part of this project as well.
Facebook’s announcement comes a few months after Google announced NMT for Google Translate back in 2016 and less than a month after Google announced its Multimodel Neural Network breakthrough.