Expose the underlying sources
Source visibility and retrieval metadata let the person testing the prototype check an answer against the documents.
Document retrieval & local-model RAG
A working document-based RAG application with local model inference, source visibility and retrieval metadata for manual verification.
- My role
- Hands-on architecture & implementation
- Delivery stage
- Proof of concept
- Customer
- Anonymised
THE CUSTOMER PROBLEM
What needed to change.
The proof of concept explored whether a document-based assistant could retrieve useful evidence, generate an answer with local model inference and expose enough source information for manual checking.
MY RESPONSIBILITY
What I owned.
Personally built the document-ingestion, embedding, retrieval and LLM-integration workflows, using Ollama for local model inference and Streamlit for the interface.
CONSTRAINTS & PRIORITIES
What shaped the approach.
- Retrieved material needed source visibility and metadata so answers could be compared with the underlying documents.
- Ollama supported local model inference for the evaluated document and context workflow.
- The work remained a technical proof of concept with manual verification.
ARCHITECTURE & DELIVERY
How the work fitted together.
- 01
Ingest documents
Python workflows handled document ingestion and embedding.
- 02
Retrieve relevant context
LangChain and ChromaDB supported the retrieval workflow, with source information exposed for inspection.
- 03
Generate and inspect an answer
Ollama provided local model inference and Streamlit presented the application for manual verification.
TECHNICAL JUDGEMENT
Decisions and responsibilities.
Evaluate local inference
Ollama allowed documents and retrieved context to be processed through a local model during the proof of concept.
Keep the delivery stage explicit
The result was a working prototype for retrieval and answer checking. Production hardening was outside the confirmed scope.
THE RESULT
What was delivered.
A working proof of concept for testing document retrieval and checking generated answers against their sources.