Retrieval Augmented Generation (RAG) Chatbot

I developed this RAG (Retrieval-Augmented Generation) chatbot to empower Clinical Research Monitors, particularly Principal Investigators (PIs), to interactively query key information extracted from technical and regulatory PDF documents.

The pipeline starts by fetching PDFs via an internal API, then pre-processing them through OCR (Optical Character Recognition) to extract text. The text is chunked and converted into vector embeddings—numerical representations that capture the semantic meaning of each chunk.

Embeddings enable efficient semantic search: when a user submits a question, the system performs a similarity search over the embedding store to retrieve the most relevant document sections. These retrieved chunks are then passed to a Large Language Model (LLM) to generate precise, context-aware answers—this combination of retrieval and generation is known as RAG.