Abstract: Hate speech on social media causes real harm to real people. Yet automatic detectors still struggle with two problems: messy training data, and models that look good on paper but fail on new data. This paper presents a simple, data-first approach to hate speech detection. We clean the text, balance the classes, and add more training examples before we train any model. We then feed this clean data into HateBERT, a version of BERT trained further on abusive language, followed by a 2-layer bidirectional LSTM (BiLSTM). The model is trained with a standard loss function, with no extra tricks. We test this setup against twelve other combinations of embeddings (MiniLM, RoBERTa.....
Key Word: Hate speech detection, HateBERT, bidirectional LSTM, data-centric AI, noise reduction, Easy Data Augmentation, embedding comparison, model overfitting, Davidson dataset, Jigsaw Toxic Comments, social media text classification.
[1].
T. Davidson, D. Warmsley, M. Macy, And I. Weber, "Automated Hate Speech Detection And The Problem Of Offensive Language," In Proc. 11th Int. Aaai Conf. On Web And Social Media (Icwsm), 2017.
[2].
C. J. Adams, J. Sorensen, J. Elliott, L. Dixon, M. Mcdonald, N. Cukierski, And W. Cukierski, "Toxic Comment Classification Challenge," Kaggle / Jigsaw, 2018.
[3].
T. Caselli, V. Basile, J. Mitrović, And M. Granitzer, "Hatebert: Retraining Bert For Abusive Language Detection In English," In Proc. 5th Workshop On Online Abuse And Harms (Woah), Acl, 2021.
[4].
J. Wei And K. Zou, "Eda: Easy Data Augmentation Techniques For Boosting Performance On Text Classification Tasks," In Proc. Emnlp-Ijcnlp, 2019.
[5].
S. Hochreiter And J. Schmidhuber, "Long Short-Term Memory," Neural Computation, Vol. 9, No. 8, Pp. 1735–1780, 1997..
