Guide 📄 Talk to your PDFs — build a private document AI (free weekend project)

Everyone has The Pile: 40 PDFs of notes, agreements, reports, question papers — all "I'll read it later", none of it searchable in the way you actually need ("what did that contract say about refund timelines?").

This weekend you fix it. You'll build a private AI that has actually read your documents, answers your questions, and cites the page it got it from. Free tools, no code, works offline if you want. The fancy name is RAG — here's the no-jargon version.

🧠 The one-minute mental model

Imagine a librarian who has read every book in your pile and made index cards: each card holds one paragraph plus where it came from. When you ask a question, the librarian (1) pulls the 5–8 most relevant cards (retrieval), (2) hands them to a fast reader who writes your answer from those cards only (generation).

Retrieve-then-read = RAG. Two reasons it beats pasting files into a chat: it scales past what fits in a chat window, and answers stay grounded in YOUR text instead of the AI's imagination.

🧰 The free stack

• Ollama — runs open models (Llama, Qwen, Phi) on your machine. Free. 8 GB RAM is enough for the small models.
  
• AnythingLLM — the friendly app that holds your documents, does the embedding/retrieval, and gives you a chat window that cites sources. Free, desktop, drag-and-drop.
  
• Optional: an n8n flow instead of AnythingLLM if you want it automated (Dropbox folder → indexed → Telegram Q&A bot). Our 🪄 n8n self-hosted guide gets that base running in 30 minutes.
  

🛠️ Build path A — 30 minutes, zero code

1. Install Ollama (ollama.com → download → done). Open a terminal:
   Code:

   ollama pull llama3.1:8b
   (low RAM? use
   Code:

   ollama pull phi3:mini
   ).
   
2. Install AnythingLLM (desktop app, free).
   
3. Inside it: create a workspace named after the pile — "Semester 5", "Client X", "Flat paperwork".
   
4. Drag your PDFs in. Hit Embed. That's the librarian writing index cards.
   
5. Set the chat model to your Ollama model (settings → LLM → Ollama → pick it).
   
6. Ask: "summarise chapter 2", "what are the refund terms?", "make 10 quiz questions from unit 3". Watch it cite the source file + page.
   

Settings that matter (defaults are fine, tweak if answers feel off): chunk size 512–1000 characters, a little overlap between chunks, and "top-k" (cards pulled) around 5–8.

🔧 Build path B — the n8n tinkerer route

For when you want automation: folder watch → extract text → embed → store in a vector DB (Qdrant or Postgres+pgvector) → chat trigger answers from the store. High-level recipe:

1. n8n running (30 min with our Docker guide).
   
2. Workflow 1: Read files → split text → create embeddings (Ollama has a free embeddings model:
   Code:

   ollama pull nomic-embed-text
   ) → upsert into Qdrant.
   
3. Workflow 2: Chat trigger → embed the question → search Qdrant → stuff top results into the prompt → answer with citations.
   

Same librarian, more plumbing. Start with path A; graduate when you outgrow it.

🩹 When answers are bad (the fix list)

• Scanned PDFs (photos of pages) give the librarian blank cards — run OCR first (free: OCRmyPDF, or upload to Google Drive → open as Docs).
  
• Giant books dilute results — split the 600-page tome into chapter files.
  
• Bad file names hurt — unit3_thermo_notes.pdf beats scan_0042.pdf; the name is context.
  
• The summary-sheet trick: add a one-page "what this workspace is" doc listing every file and topic. Retrieval quality jumps.
  
• Tables are the weak spot — for heavy tables, ask for the page reference and read it yourself.
  

🔒 Privacy & cost
The config above is fully local: documents, embeddings, and the model never leave your machine. Total cost: ₹0. Optional upgrades — API embeddings or a stronger cloud model — start a few rupees per hundred pages, but you don't need them to learn the pattern. Either way: this is your private pile; even local, don't feed it other people's confidential docs without permission.

🎓 When you're ready to level up
The same concept powers serious systems: pgvector inside your own Postgres, hybrid search (keyword + vector), agents that read the same store (hello, 🤖 first AI agent guide), and SQL on the metadata (🗄️ SQL weekend plan if that's new). One weekend project, and suddenly "RAG" in job posts stops looking scary.

🏁 The 7-day challenge
Day 1: path A with 5 documents you know well. Days 2–6: ask it real questions daily — every wrong answer, apply one fix from the list. Day 7: show one friend. The smile they make is why we build.

— Sir-Vigu
Founder, Data-Forums