Introduction
This project aims to create a chatbot that reduces hallucination, ensuring factual accuracy responses for international students through a multi-agent system integrating Retrieval Augmented Generation (RAG) and Graph-based Retrieval Augmented Generation (GraphRAG).
Method and Technology
The chatbot was developed based on a framework built around natural language processing (NLP). Initially, the multi-agent system, RAG, and GraphRAG components were created separately. The team follows an iterative approach, developing each part individually and integrating them once completed. Large language models (LLMs) like Llama 3 and Qwen2 are deployed locally on the university’s high-performance computing (HPC) system, maximizing the model’s performance.
Results & Limitations
The current accuracy of the model is 58%, signaling that further improvements are needed. This chatbot is still in its testing phase and the client will receive the final product by the predetermined deadline. A limitation includes the inability to use the latest CUDA version due to driver restrictions, hindering optimal GPU performance. This is a year-long project, and the client expects a fully functional chatbot with ongoing development.
Conclusion
As clearly stated, delivering a virtual assistant chatbot capable of providing factually correct answers for international students with minimal hallucinations is our top priority. Team members have equipped themselves with essential AI and Machine Learning knowledge, along with other soft skills such as project management skills to be able to deliver the project successfully. We anticipate further improvements for the chatbot over the coming months.