Text Toxicity Level Detection using Deep Contextualized Embedding Models
Keywords:
Toxic Text, Deep Learning, Bert, Natural Language ProcessingSynopsis
This is a Chapter in:
Book:
Competitive Tools, Techniques, and Methods
Print ISBN 978-1-6692-0008-6
Online ISBN 978-1-6692-0007-9
Series:
Chronicle of Computing
Chapter Abstract:
Toxic text is a critical aspect of social media, particularly in today's digital landscape. With the spread of online communication, it has become increasingly easy for individuals to spread harmful or offensive content. Toxic texts include the spread of misinformation, the promotion of hate speech, bullying, and the erosion of trust in online communities. Text toxicity detection algorithms can help to identify and mitigate these negative effects by automatically flagging potentially harmful content. This allows social media platforms to intervene and take appropriate action, such as removing the content or warning the user. Usually, social media platforms offer a reporting strategy which acts after a human decision is made. However, social media now requires an automated system to do this task. In this work, we proposed a Deep learning Regression model to predict the toxicity level in text. Additionally, we fine-tuned multiple Bert models for this task. Our work was evaluated using Mean Square Error, Root Mean Square Error and Mean Absolute Error compared to the testing set of the data and we got for the base model MSE of 0.562, RMSE of 0.750 and MAE of 0.364 but for BERT we got MSE 0.403, RMSE 0.635 and MAE 0.232.
Cite this paper as:
Elgendya O., Nassifa A. B., Soudana B., (2024) Text Toxicity Level Detection using Deep Contextualized Embedding models. In: Tiako P.F. (ed) Competitive Tools, Techniques, and Methods. Chronicle of Computing. OkIP. CAIF24#9. https://doi.org/10.55432/978-1-6692-0007-9_3
Presented at:
The 2024 OkIP International Conference on Automated and Intelligent Systems (CAIS) in Oklahoma City, Oklahoma, USA, and Online, on October 3, 2024.
Contact:
Omar Elgendya
oelgendy@sharjah.ac.ae
References
Garlapati, A., Malisetty, N., & Narayanan, G. (2022). Classification of Toxicity in Comments using NLP and LSTM. 8th International Conference on Advanced Computing and Communication Systems (ICACCS), 16–21. https://doi.org/10.1109/ICACCS54159.2022.9785067
Zaheri, S., Leath, J., & Stroud, D. (2020). Toxic Comment Classification. SMU Data Science Review, 3(1). Retrieved from https://scholar.smu.edu/datasciencereview/vol3/iss1/13
Nassif, A. B., Elnagar, A., Elgendy, O., & Afadar, Y. (2022). Arabic fake news detection based on deep contextualized embedding models. Neural Computing and Applications, 34(18), 16019–16032. https://doi.org/10.1007/s00521-022-07206-4
Devlin, J., Chang, M.-W., Lee, K., Google, & AI Language. (2021). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Retrieved from https://github.com/tensorflow/tensor2tensor
Maslej-Krešňáková, V., Sarnovský, M., & Jacková, J. (2022). Use of Data Augmentation Techniques in Detection of Antisocial Behavior Using Deep Learning Methods. Future Internet, 14(9). https://doi.org/10.3390/fi14090260
Georgakopoulos, S. V., Tasoulis, S. K., Vrahatis, A. G., & Plagianakos, V. P. (2018). Convolutional Neural Networks for Toxic Comment Classification. Proceedings of the 10th Hellenic Conference on Artificial Intelligence, 1–6. https://doi.org/10.1145/3200947.3208069
Chakrabarty, N. (2020). A Machine Learning Approach to Comment Toxicity Classification. Advances in Intelligent Systems and Computing, 999, 183–193. https://doi.org/10.1007/978-981-13-9042-5_16
Mounir, J. H., et al. (2019). Social Media Cyberbullying Detection using Machine Learning. International Journal of Advanced Computer Science and Applications, 10(5). https://doi.org/10.14569/IJACSA.2019.0100587
Dalvi, R. R., Chavan, S. B., & Halbe, A. (2020). Detecting A Twitter Cyberbullying Using Machine Learning. 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS), 297–301. https://doi.org/10.1109/ICICCS48265.2020.9120893
Nalini, K., & Sheela, L. J. (2015). Classification of tweets using text classifier to detect cyber bullying. Advances in Intelligent Systems and Computing, 338, 637–645. https://doi.org/10.1007/978-3-319-13731-5_69
Nurrahmi, H., & Nurjanah, D. (2018). Indonesian Twitter Cyberbullying Detection using Text Classification and User Credibility. 2018 International Conference on Information and Communication Technology (ICOIACT), 543–548. https://doi.org/10.1109/ICOIACT.2018.8350758
Shah, R., Vidyavihar, S., Chopdekar, R., & Somaiya, S. K. J. (2020). Machine Learning based Approach for Detection of Cyberbullying Tweets. International Journal of Computer Applications, 175(37), 975–8887. https://doi.org/10.5120/ijca2020920946
Di Capua, M., Di Nardo, E., & Petrosino, A. (2016). Unsupervised cyber bullying detection in social networks. Proceedings of the International Conference on Pattern Recognition, 0, 432–437. https://doi.org/10.1109/ICPR.2016.7899672
