Retrieval-Augmented Generative AI Chatbot

Abstract

This work consists on the development and evaluation of a chatbot that integrates retrieval-augmented generation (RAG) to tackle the issue of hallucination in large language models (LLMs). It begins with an introduction that outlines the evolution of chatbots from simple rule-based systems to advanced models using transformers. Then a detailed history of chatbots, their various categories, and their advantages and disadvantages is provided. It discusses the hallucination problem and introduces the RAG approach, which combines retrieval-based and generative techniques to improve the accuracy and reliability of chatbot responses. The related work section reviews existing literature on methods to mitigate hallucination in LLMs and examines techniques that tackle each stage within the RAG process. Next, a description of the datasets used is given, including the MS MARCO question-answering and passage retrieval datasets, and the ”Guia Tecnico do Alojamento Local.” The preprocessing steps and ´ dataset characteristics are thoroughly explained. The methods chapter outlines the six-phase methodology: data preprocessing, embedding model, vector database, conversational chain, response generation, and interface and deployment. Each phase is elaborated to illustrate the process of constructing the RAG chatbot. The results of the chatbot’s performance are presented using various metrics for retrieval and generation. It presents findings from experiments conducted with the local accommodation dataset and the MS MARCO dataset, demonstrating the chatbot’s enhanced performance due to the RAG approach. Finally, the conclusion summarizes the thesis’ contributions. It also suggests avenues for future research.