Hybrid Retrieval-Augmented Generation for Telugu Question Answering Using BM25, Dense Embeddings, and Multilingual Transformers

This paper has introduced TQA-RAG which is a retrieval-augmented question answering framework that has been de-signed specifically to overcome the difficulties faced in the development of question answering systems for low-resource languages like Telugu. This paper has identified some major issues faced in the conventional approach of developing such question answering systems which include lack of sufficient semantic understanding and context grounding, and fact in-consistency in knowledge-intensive applications. To deal with these problems, this framework makes use of lexical search by employing BM25 algorithm, semantic search using multilingual representations, reranking with contextual similarity, and generating answers using multilingual trans-formers like a fine-tuned version of Multilingual Text-to-Text Transfer Transformer. The Telugu language dataset and corpus for training and testing have been prepared in the form of a structured dataset consisting of question–answer pairs. The experimental analysis proves that the framework de-veloped has performed better than the transformer-based stan-dalone QA system in terms of retrieval accuracy and answer generation quality. The use of retrieval techniques has enabled capturing lexical relevance along with contextual similarities to provide better passage selection and contextual grounding for retrieval. In addition, generation from grounded facts has resulted in increased factual consistency and minimised hallucination. Overall, the findings of this research demonstrate that retrieval-augmented generation provides an effective and scal-able solution for Telugu question answering. The proposed TQA-RAG framework contributes to the advancement of intelligent language technologies for low-resource languages and establishes a robust foundation for context-aware, knowledge-driven question answering systems in multilingual environments.  

[1]      Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. Attention Is All You Need. Advances in Neural Information Processing Systems. 30 (2017), 5998–6008. [2]    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu. Exploring the Limits of Transfer Learn-ing with a Unified Text-to-Text Transformer. Journal of Machine Learning Research. 21(140) (2020), 1–67. [3]   Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of NAACL-HLT. (2021), 483–498. [4]  Stephen Robertson, Hugo Zaragoza. The Probabilistic Rel-evance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval. 3(4) (2009), 333– 389. [5]   Nils Reimers, Iryna Gurevych. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceed-ings of EMNLP-IJCNLP. (2019), 3982–3992. [6]     Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceed-ings of NAACL-HLT. (2019), 4171–4186. [7]  Zhenyu Ji, Nayeon Lee, Rita Frieske, Tiantian Yu, Dan Su, Yan Xu, Eric Ishii, Younghun Bang, Andrea Madotto, Pascale Fung. Survey of Hallucination in Natural Lan-guage Generation. ACM Computing Surveys. 55(12) (2023), 1–38. [8]     Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems. (2020), 9459–9474. [9]    William A. Woods. Progress in Natural Language Un-derstanding: An Application to Lunar Geology. AFIPS Conference Proceedings. 42 (1973), 441–450. [10]  Karen Sparck Jones, Steve Walker, Stephen E. Robertson. A Probabilistic Model of Information Retrieval: De-velopment and Comparative Experiments. Information Processing and Management. 36(6) (2000), 779–808.