Arabic Sentiment Analysis with Transformers

def prune_transformer(model, amount=0.3): for name, module in model.named_modules(): if isinstance(module, torch.nn.Linear): prune.l1_unstructured(module, name='weight', amount=amount) prune.remove(module, 'weight') return model # Efficient Multilingual NMT class LightTranslator(nn.Module): def __init__(self, vocab_size, d_model=512): super().__init__() self.encoder = TransformerEncoder(d_model) self.decoder = TransformerDecoder(d_model) def forward(self, src, tgt): memory = self.encoder(src) return self.decoder(tgt, memory)

Project Overview

Built high-accuracy sentiment classification models for Arabic text at CERIST, using transformer architectures (BERT, AraBERT, mBERT) via Hugging Face. The project targeted both Modern Standard Arabic and dialectal Arabic, which behave very differently under standard tokenization and required separate evaluation.

Technical Details

Architectures: BERT, AraBERT, mBERT, compared against classical ML baselines (SVM, logistic regression on TF-IDF features).
Frameworks: PyTorch, Hugging Face Transformers.
Data: Labeled Arabic text (tweets and reviews) across positive, negative, and neutral classes, covering both Modern Standard Arabic and dialectal varieties.
Evaluation: Accuracy and F1 compared across architectures and against traditional baselines, with error analysis on dialectal variation.