Commit Graph

131 Commits

Author SHA1 Message Date
Mathias Bergqvist
9cc6c2d053 feat(extract): implement PDF extraction via pdftotext 2026-04-23 15:53:46 +02:00
Mathias Bergqvist
43a46d07e5 feat(extract): add Text() dispatcher with md/txt passthrough 2026-04-23 15:45:20 +02:00
Mathias Bergqvist
820d1c93a7 docs: add implementation plan for PDF extraction and entity resolution 2026-04-23 15:44:13 +02:00
Mathias Bergqvist
e74320a8e8 feat(ingestion): wire watcher into server startup + fix Procfile env vars
Some checks failed
cd / Build and deploy (push) Successful in 10s
CI / Lint / Test / Vet (push) Failing after 5s
CI / Mirror to GitHub (push) Has been skipped
- Start background watcher on startup when INGEST_WATCH_INTERVAL > 0
- Procfile: add INGEST_WATCH_INTERVAL=30 and INGEST_SVC_URL for supervisor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:09:00 +02:00
Mathias Bergqvist
1b0706f270 chore(brain): rename CLAUDE.md to schema.md for clarity
CLAUDE.md has a specific meaning in the Claude Code ecosystem (agent
instructions). The wiki schema for the ingestion pipeline should live
in schema.md to avoid confusion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:06:32 +02:00
Mathias Bergqvist
2ae6bfe81e fix(brain): enforce mutual exclusivity and clarify brain_ingest schema
- Return error when both path and content are supplied simultaneously
- Improve tool description to clearly state the two valid call forms
- Add per-field descriptions so LLMs understand what each parameter requires

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:03:03 +02:00
Mathias Bergqvist
a6dce972d6 feat(brain): add path field to brain_ingest for /ingest-path routing
Adds an optional path field to brain_ingest so Claude can ingest files
or directories directly by path without embedding content in the call.
Routing: path set → /ingest-path; content+source set → /ingest; neither → error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:01:05 +02:00
Mathias Bergqvist
2f4b577131 fix(ingestion): address code review issues in api and watcher packages
- Strip internal error detail from 500 responses (leak prevention)
- Add path containment assertion in /write handler
- Use Go 1.22 method-prefixed mux routes for automatic 405 responses
- Clarify watch_interval log when watcher not yet wired
- Consolidate validation tests into table-driven TestIngest_Validation
- Watcher: return nil after successful quarantine to avoid double-logging
- Watcher: append timestamp suffix to processed dest if file already exists

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:59:39 +02:00
Mathias Bergqvist
a25bb18c54 feat(ingestion): add /ingest and /ingest-path HTTP handlers
Wires pipeline.Run into the HTTP layer so callers can ingest raw text
or files/directories without touching the filesystem directly. Rewrites
main.go to parse LLM and watcher env vars and build pipeline.Config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:54:28 +02:00
Mathias Bergqvist
78531bb238 feat(ingestion): add background file watcher for brain/raw/
Polls brain/raw/ on a configurable ticker, derives human-readable source
names from filenames, runs the pipeline, and moves files to
processed/YYYY-MM-DD/ on success or failed/ on error with a log.md entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:54:03 +02:00
Mathias Bergqvist
04fefe8e9c fix(ingestion): wrap naked error returns and harden mustJSON helper
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:51:19 +02:00
Mathias Bergqvist
103f4d90bf feat(ingestion): add pipeline orchestrator with prompt builder
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:45:19 +02:00
Mathias Bergqvist
9b11719481 feat(ingestion): add content chunking and LLM JSON output parser 2026-04-22 22:37:14 +02:00
Mathias Bergqvist
d405346f07 feat(ingestion): add wiki index rebuilder and audit log 2026-04-22 22:36:55 +02:00
Mathias Bergqvist
bf8a3fc11c feat(ingestion): add OpenAI-compatible LLM HTTP client with 429 retry 2026-04-22 22:29:24 +02:00
Mathias Bergqvist
ae5a4d04f0 feat(ingestion): add wiki page merge logic 2026-04-22 22:28:55 +02:00
Mathias Bergqvist
3a0424a6b4 feat(ingestion): add wiki inventory loader 2026-04-22 22:28:53 +02:00
Mathias Bergqvist
08dd7b9365 docs(brain): add wiki schema document for ingest prompt 2026-04-22 22:25:52 +02:00
Mathias Bergqvist
91e02b930c feat(ingestion): add wiki package with Page types and slug generation 2026-04-22 22:25:45 +02:00
Mathias Bergqvist
c7341a2607 feat(config): add IngestSvcURL and KBRetrievalURL to supervisor config 2026-04-22 22:24:27 +02:00
Mathias Bergqvist
b5a0085c0a feat(brain): add brain_ingest, brain_search tools and extend search to wiki/ 2026-04-22 22:16:02 +02:00
Mathias Bergqvist
d6daa37c71 docs: add brain ingestion pipeline implementation plan 2026-04-22 22:14:59 +02:00
Mathias Bergqvist
62fc3989f2 docs: add brain ingestion pipeline design spec 2026-04-22 22:05:19 +02:00
Mathias Bergqvist
c9310b1079 fix(ingestion): always append .md extension to written filenames
All checks were successful
cd / Build and deploy (push) Successful in 9s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 4s
brain_write with a custom filename omitted the .md extension, causing
search to skip the file (search.go filters on HasSuffix .md).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 19:23:07 +02:00
Mathias Bergqvist
ca8a691241 fix(exec): strip trailing result-schema JSON from local model output
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
Small models (phi4-mini) produce correct markdown analysis but then
append the old {status/phase/skill} JSON schema out of training habit.
stripResultJSON() detects and removes these trailing fences so Claude
Code receives clean prose regardless of model behaviour.

Non-schema json blocks (config examples etc) are preserved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:55:53 +02:00
Mathias Bergqvist
214f607007 fix(config): make no-JSON instruction unmissable in protocols.md
All checks were successful
cd / Build and deploy (push) Successful in 7s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
Local models (phi4-mini, qwen3-coder-30b) ignore soft instructions
and revert to JSON from their training. Move the prohibition to the
very top with bold caps before any other content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:51:51 +02:00
Mathias Bergqvist
0e08dfffb8 fix(config): rewrite all skill discipline files for simplified model
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
Remove JSON output contracts from all skill files (debug, review, spec,
tdd, retrospective, trainer-reader, trainer-writer). Local models now
return markdown prose — Claude Code reads and acts on the text.

Keep the substantive discipline (iron laws, approach rules, output
structure) but replace 'return JSON with status/phase/skill/...' with
clear markdown format instructions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:46:52 +02:00
Mathias Bergqvist
caef05bea4 fix(config): rewrite protocols.md for simplified model
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 3s
Remove JSON output contract, verification rules, escalation, and scope
limits that applied to the old Claude subprocess workers. Local models
are now consultants returning markdown prose, not JSON executors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:43:04 +02:00
Mathias Bergqvist
ca1a16873c feat(ingestion): add Dockerfile and extend CD to build+push ingestion image
All checks were successful
cd / Build and deploy (push) Successful in 9s
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 3s
Ingestion server is a pure-Go HTTP binary — alpine runtime, no node.js.
CD now builds both supervisor and ingestion images on every push,
updates both deployment.yaml files in the infra repo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:37:11 +02:00
Mathias Bergqvist
63c238c650 fix(config): update model names to match LiteLLM host/name format
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 4s
Replace ollama/ prefix with iguana/ and koala/ prefixes to match
actual model IDs exposed by LiteLLM on this cluster.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:34:53 +02:00
Mathias Bergqvist
ce45592730 refactor: replace orchestrator/verifier chain with direct LiteLLM calls
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
Drop the three-layer Claude subprocess orchestration (local model →
Claude verifier → cloud escalation). Skills now call LiteLLM directly
and return plain text to Claude Code, which decides what to do with it.

- Delete executor, orchestrator, verifier, result, attempts packages
- Simplify LiteLLMExecutor: Run(Request)→Result becomes Complete(model,sys,user)→(string,int64,error)
- Replace ExecutorFn with CompleteFunc in all 6 skill configs
- Rewrite all skill handlers to call Complete and return {"text","model","duration_ms"}
- Simplify config/models: remove Verifier/LlamaSwapURL, add ModelFor
- Bump version to v0.5.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:19:09 +02:00
Mathias Bergqvist
823de23213 feat(exec): log per-attempt chain verdicts for pass rate visibility
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 4s
2026-04-22 15:40:15 +02:00
Mathias Bergqvist
78d3939caa feat(config): wire protocols.md into every worker as shared behavioral contract 2026-04-22 15:39:25 +02:00
Mathias Bergqvist
f2bc39b500 feat(skills): inject brain context into review, debug, spec, tdd before spawning workers 2026-04-22 15:37:56 +02:00
Mathias Bergqvist
3625e1268d feat(ingestion): simplify brain to knowledge/ — write and search use same dir 2026-04-22 15:36:10 +02:00
Mathias Bergqvist
47df642836 feat(brain): add Query client for skill handler context injection 2026-04-22 15:34:09 +02:00
Mathias Bergqvist
235d70ad0b docs: document hyperguild scope reset — drop parametric learning, simplify brain 2026-04-22 15:27:52 +02:00
Mathias Bergqvist
7d5289ac54 chore: bump version to v0.4.0
All checks were successful
cd / Build and deploy (push) Successful in 6s
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-22 13:38:23 +02:00
Mathias Bergqvist
3d8fc9dacd feat(skills): wire session.Append into retrospective and trainer 2026-04-22 13:37:43 +02:00
Mathias Bergqvist
f9f804cd49 feat(skills): wire session.Append and PrependHistory into tdd 2026-04-22 13:37:06 +02:00
Mathias Bergqvist
85f142ade0 feat(skills): wire session.Append and PrependHistory into spec 2026-04-22 13:36:35 +02:00
Mathias Bergqvist
0dfad02513 feat(skills): wire session.Append and PrependHistory into review and debug 2026-04-22 13:36:13 +02:00
Mathias Bergqvist
c44eb680b2 feat(exec): surface AttemptRecord slice on Result for session logging 2026-04-22 13:35:38 +02:00
Mathias Bergqvist
38ada998a2 feat(session): add AttemptsFrom converter for exec.AttemptRecord 2026-04-22 13:35:09 +02:00
Mathias Bergqvist
74547c2bdf feat(session): export PrependHistory for shared use across skills 2026-04-22 13:34:48 +02:00
Mathias Bergqvist
587c0d3b1c chore: bump startup log to v0.3.1 — CD pipeline smoke test
All checks were successful
cd / Build and deploy (push) Successful in 33s
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-22 12:18:27 +02:00
Mathias Bergqvist
bb61f2992b fix(cd): connect to Gitea SSH via localhost:30022 NodePort
All checks were successful
cd / Build and deploy (push) Successful in 5s
CI / Lint / Test / Vet (push) Successful in 1m8s
CI / Mirror to GitHub (push) Successful in 3s
gitea.d-ma.be:30022 is refused externally — the NodePort is only
reachable on koala locally. Use HostName 127.0.0.1 in SSH config
so git@gitea.d-ma.be connects to localhost:30022 instead.
2026-04-21 19:43:06 +02:00
Mathias Bergqvist
3ba72d9b28 fix(cd): replace heredoc with printf to avoid YAML parse error
Some checks failed
cd / Build and deploy (push) Failing after 5s
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 3s
Unindented heredoc content inside a YAML literal block breaks parsing.
Gitea silently drops workflows with YAML errors, causing the CD job
to never trigger.
2026-04-21 19:41:09 +02:00
Mathias Bergqvist
b4f0fbc3ea chore: retrigger CD with SSH port fix
All checks were successful
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-21 19:35:30 +02:00
Mathias Bergqvist
12943ee6f4 fix(cd): use NodePort 30022 for Gitea SSH in infra repo update
All checks were successful
CI / Lint / Test / Vet (push) Successful in 1m9s
CI / Mirror to GitHub (push) Successful in 3s
gitea.d-ma.be port 22 is rejected (NPM only proxies HTTP/HTTPS).
The runner runs on koala where the Gitea SSH NodePort 30022 is
reachable locally. Use SSH config override instead of ssh-keyscan.
2026-04-21 19:28:28 +02:00