Abstract
Most studies on text classification are focused on the English language. However, short texts such as SMS are influenced by regional languages. This makes the automatic text classification task challenging due to the multilingual, informal, and noisy nature of language in the text. In this work, we propose a novel multi-cascaded deep learning model called McM for bilingual SMS classification. McM exploits n-gram level information as well as long-term dependencies of text for learning. Our approach aims to learn a model without any code-switching indication, lexical normalization, language translation, or language transliteration. The model relies entirely upon the text as no external knowledge base is utilized for learning. For this purpose, a 12 class bilingual text dataset is developed from SMS feedbacks of citizens on public services containing mixed Roman Urdu and English languages. Our model achieves high accuracy for classification on this dataset and outperforms the previous model for multilingual text classification, highlighting language independence of McM.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
These embeddings are also made available along with dataset.
References
Denecke, K.: Using SentiWordNet for multilingual sentiment analysis. In: International Conference on Data Engineering Workshop, pp. 507–512 (2008)
Fatima, M., et al.: Multilingual SMS-based author profiling: data and methods. Nat. Lang. Eng. (NLE) 24(5), 695–724 (2018)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning (ICML), pp. 448–456 (2015)
Medrouk, L., Pappa, A.: Deep learning model for sentiment analysis in multi-lingual corpus. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017. LNCS, vol. 10634, pp. 205–212. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70087-8_22
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems (NIPS), pp. 3111–3119 (2013)
Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Peters, M., et al.: Deep contextualized word representations. In: Conference of the North American Chapter of the Association for Computational Linguistics (ACACL): Human Language Technologies, Volume 1 (Long Papers), pp. 2227–2237 (2018)
Rafae, A., Qayyum, A., Moeenuddin, M., Karim, A., Sajjad, H., Kamiran, F.: An unsupervised method for discovering lexical variations in Roman Urdu informal text. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 823–828 (2015)
Reimers, N., Gurevych, I.: Reporting score distributions makes a difference: performance study of lstm-networks for sequence tagging. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 338–348 (2017)
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. (IPM) 45(4), 427–437 (2009)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. (JMLR) 15(1), 1929–1958 (2014)
Subramani, S., Michalska, S., Wang, H., Du, J., Zhang, Y., Shakeel, H.: Deep learning for multi-class identification from domestic violence online posts. IEEE Access 7, 46210–46224 (2019)
Wang, X., Jiang, W., Luo, Z.: Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: International Conference on Computational Linguistics (COLING): Technical Papers, pp. 2428–2437 (2016)
Wang, Z., Zhang, Y., Lee, S., Li, S., Zhou, G.: A bilingual attention network for code-switched emotion prediction. In: International Conference on Computational Linguistics (COLING): Technical Papers, pp. 1624–1634 (2016)
Williams, A., Srinivasan, M., Liu, C., Lee, P., Zhou, Q.: Why do bilinguals code-switch when emotional? Insights from immigrant parent-child interactions. Emotion (Washington, DC) (2019)
Zhou, X., Wan, X., Xiao, J.: Attention-based LSTM network for cross-lingual sentiment classification. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 247–256 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Shakeel, M.H., Karim, A., Khan, I. (2019). A Multi-cascaded Deep Model for Bilingual SMS Classification. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11953. Springer, Cham. https://doi.org/10.1007/978-3-030-36708-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-36708-4_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36707-7
Online ISBN: 978-3-030-36708-4
eBook Packages: Computer ScienceComputer Science (R0)