Gen AI Intensive Course Capstone 2025Q1

 

This is my first time participating in one of the Kaggle's challenges. I attended all Generative AI Q&A sessions but I had so many things to do after that week that I didn't get the chance to work on the Capstone project, but here I am. 

Problem: Sometimes it is hard to keep up with some board games. Even reading the instructions, it is fairly easy to get a bit confused and not know what to do. 

For the development of the project I have chosen to make a help bot for the board game “Perudo”. I chose the game because I didn't know anything about it and I wanted that at the end of the project, and after asking questions, I could understand what it is about and how to play it. This would help me to evaluate the performance of the program. I first searched for pages from which to extract the content and use them as documents to be embedded. I manually added some markdowns and the source of the information.

I thought it would be interesting to evaluate the quality of each of the documents, so I used the Pointwise text quality prompt and modified its content a bit to adapt it to something simpler and obtained a rating of the quality of information provided by the document.

 Each document that exceeds the score of 4 points, obtained after using a structured output, will be appended to a list of approved documents that will then be used for the construction of a database.

 

For the model configuration, I will use a temperature of 0 because the information should come from the documents that were used to reduce possible hallucinations.

Finally, a RAG system of questions and answers can be constructed for the chosen board game.

 I modified the prompt to not include references to the fact that the information was consulted from external documents in order to make it more transparent to the user.  After asking a few questions, I can say that I am satisfied with the result obtained.

The concepts I used in this project are:
- Structured output and controlled generation.
- Embedding.
- Retrieval augmented generation.
- Gen AI evaluation. 

Limitations and opportunities for improvement
Right now the model is limited to only accepting text and defining questions manually. In addition, its knowledge is limited to 4 documents and a single board game.
It would be interesting to expand the knowledge base, maybe integrate it with some API that provides information from more board games. Even an agent that can control in a more human way the interaction or process images to determine the most appropriate way to interact with the game, give some advice for game opportunities or determine based on probabilities who will win!