AI Profile & Document RAG API

This project is a localized Retrieval-Augmented Generation (RAG) system that allows users to store personal profiles or documents and query them using a Large Language Model (LLM). It uses a local vector database for storage and a local LLM for generating answers, ensuring data privacy and low latency.

🚀 What This Project Does

The application provides a REST API to:

1. Store Documents: Accept text content (like personal profiles), split it into manageable chunks, and store them in a vector database with associated metadata.
2. Contextual Search: Retrieve the most relevant pieces of information from the database based on a user's question.
3. Local AI Generation: Use the retrieved context to answer questions accurately using a local LLM, filtering by specific users if requested.
4. Static Embedding: Includes a script (embed.py) to quickly ingest specific files (like k8s.txt) into the vector store.

🛠️ Tools Used

• FastAPI: A modern, high-performance web framework for building APIs with Python.
• ChromaDB: An open-source embedding database (vector store) used to store and search document chunks.
• Ollama: A platform for running LLMs locally.
  • Embedding Model: nomic-embed-text for converting text into vector representations.
  • Chat Model: tinyllama for generating answers based on retrieved context.
• Pydantic: Used for data validation and settings management within the API.

📋 Project Structure

• app.py: The main FastAPI application containing the /documents (POST) and /ask (GET) endpoints.
• embed.py: A utility script to manually embed text files (e.g., k8s.txt) into the database.
• db/: Local directory where ChromaDB persists its data.
• k8s.txt: Sample text file used for static embedding.

⚙️ Setup and Usage

Prerequisites

1. Install Ollama: Ensure Ollama is installed and running.
2. Pull Models:

   ollama pull nomic-embed-text
   ollama pull tinyllama

Installation

1. Set up a virtual environment:

   python -m venv venv
   source venv/bin/activate

2. Install dependencies:

   pip install fastapi ollama chromadb uvicorn

Running the App

Start the API server:

uvicorn app:app --reload

API Endpoints

• POST /documents: Send a JSON body with user_name and content to store data.
• GET /ask: Query the system using a question parameter (optional user filter available).