Simon Stirling.Download CVContact
All projects
01 / INDEPENDENT ENGINEERING DEMONSTRATORPublic without sign-in
AI evaluation & retrieval

Knowledge LabAnswers you can trace back to evidence.

Explore a fictional knowledge base, inspect citations and compare retrieval decisions. Optional local Ollama inference adds a separately validated model draft.

Open live demoView repository
Knowledge Lab showing a support question and source-backed evidence from the fictional company knowledge base.
The public application with synthetic example data.
01 / FUNCTION & ABILITIES

Answers you can trace back to evidence.

01

Inspect exact evidence

Read source passages, document owners, dates and rendered citations rather than accepting an unexplained answer.

02

Compare retrieval behaviour

Compare a keyword baseline with governed BM25 retrieval, source-status filters, role filters and coverage guards.

03

Exercise difficult questions

Try contradictory documents, undocumented topics and a restricted synthetic record. Inspect why the system answers or abstains.

04

Evaluate and extend

Rerun labelled development and verification cases, inspect recorded model reviews, or import a custom JSON knowledge pack.

02 / PLANNED BOUNDARIES, VISIBLE FLOW

From the whiteboard
to a working application.

Play a scenario, step through the decisions or select a component to inspect its responsibility. The diagrams follow the implemented design.

THE ARCHITECTURE BOARD01 / Knowledge Lab
Illustrated flow
Enterprise knowledge assistant & evaluation lab: architecture and data flowThe upper path is the public browser demo. The lower-right branch runs only when the optional local service is configured. Select a component or use the readable step list below.PUBLIC BROWSER / EVIDENCE MODEPUBLIC SYNTHETIC SOURCE PACKOPTIONAL LOCAL SERVICE / LOOPBACK ONLYcanonical passagesquestioneligible passagessource linkspassage IDschecked contextcited JSON draftA visitor selects a question, retrieval configuration and simulated role in the browser.Question + settingsSupport investigationBM25 uses English normalisation and explicit synonyms. Status, role, coverage and conflict checks determine which passages are usable.Governed retrievalBM25 + source filtersThe public mode quotes source text and renders passage links. Missing or conflicting support stays visible.Exact evidenceQuote or abstainVisitors inspect source-backed output and labelled evaluation results. A model draft, when enabled locally, is distinct from exact evidence.Answer inspectionCitations + evaluationValidated ingestion produces a synthetic public source pack with passage identifiers, ownership, status and role metadata.Canonical JSON packDocuments + metadataThe loopback adapter reloads source text and enforces its configured role before allowing a model request.Python admissionCanonical context checksThe optional local model returns one concise JSON answer with admitted passage references. Code validates those references before rendering.Local OllamaLangChain / qwen3:8bfollow the evidence, then the decision
Step 1 / 4 · click any component
FOLLOW THE FLOW

Load canonical sources

The browser loads the synthetic source pack and its passage metadata.

Read all 4 steps without animation

Find an answer without making a model request.

The upper path is the public browser demo. The lower-right branch runs only when the optional local service is configured. Arrows and moving dots are an explanation of the implementation, not live telemetry.
03 / A FIVE-MINUTE WALKTHROUGH

Try it for yourself.

Keep the live demo open
  1. 01

    Open a support question

    Open the public demo and choose Support target. Read the quoted answer, then open its citation to inspect the original passage.

  2. 02

    Change the retrieval rules

    Switch between governed retrieval and the keyword baseline. Compare which draft, archived or authoritative sources appear.

  3. 03

    Test the boundaries

    Choose Conflicting sources, Missing answer and Restricted record. Compare the support and administrator demo roles.

  4. 04

    Inspect the evaluation

    Open Evaluation lab and compare the development and verification sets across all six configurations. Custom packs disable the sample labels.

  5. 05

    Try local inference if wanted

    Follow the repository setup, run npm run dev:local with Ollama and qwen3:8b installed, then expand Optional local inference and check the service before drafting an answer.

04 / ENGINEERING JUDGEMENT

Why it works this way.

Evidence before generation

Public evidence mode quotes canonical passages and runs entirely in the browser. It is useful without a model account or a hosted inference service.

Server-owned context admission

The optional loopback Python adapter reloads the canonical pack, applies its own role and validates admitted context before calling Ollama.

Keep measured claims bounded

The synthetic cases expose regressions and model errors. Passing them does not establish general accuracy or independent human acceptance.

05 / INSPECT THE EVIDENCE

Tested behaviour.
Explicit limits.

  • 68 recorded retrieval/pipeline cases and 46 generated-answer content checks pass in the accepted local Qwen3 8B run.

  • Seven invalid-context probes are rejected. Earlier retrieval and model failures remain in the repository.

  • Source-linked agent review and reproducible setup are published with the project.

Read the recorded validation