Go back to publications View external

Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study

Ahmed Amine Aliane , Nasredine Semmar , Hassina Aliane August 2026 arXiv preprint
[NLP] [Machine Translation] [Vocabulary Pruning] [Model Compression] [Arabic]

Abstract

The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization, and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 while significantly outperforms the OPUS-MT baseline on the COMET metric (0.8730 vs 0.7911), revealing superior semantic adequacy and fluency.

Method

Our approach performs structural compaction of the pretrained model’s embedding space by retaining only parameters associated with tokens observed in target training data. The optimization framework operates through two sequential phases:

  1. Corpus-Driven Vocabulary Pruning: A five-stage pipeline that executes row-wise slicing operations on embedding matrices to discard inactive tokens while preserving functional integrity through mandatory special tokens and language tags.
  2. Targeted Fine-Tuning Protocol: Supervised fine-tuning on task-specific parallel corpora to recalibrate logits, adapt decoder subnetworks, and restore cross-entropy convergence within the reduced lower-dimensional categorical distribution.

Contribution

  • Architecture-Agnostic Compression: Introduces a minimalist optimization framework that slashes embedding layer parameter footprints by ~60% without architectural modifications or retraining from scratch.
  • Superior Semantic Performance: Demonstrates that pruned-and-tuned MNMT models (M2M100, NLLB-200, mBART-50) meet or exceed dedicated bilingual baselines on English-Arabic translation, obtaining significantly higher COMET and lower TER scores.
  • Domain Adaptation Insights: Establishes that combining formal and informal registers during fine-tuning provides crucial cross-domain generalization stability for pruned tokenizers on unseen benchmarks like FLORES-200.