Commit Graph

184 Commits

Author SHA1 Message Date
Mathias Bergqvist
ed4966927c feat(hyperguild): brain HTTP REST client
Adds brainClient with Query and Write methods against the brain's
HTTP REST endpoints (/query, /write). Constructor reads BRAIN_URL env
var, defaulting to http://koala:30330 — the Tailscale-exposed
NodePort that serves both MCP and REST.

Tests cover success, transport error, and non-200 cases via
httptest fakes; URL override is verified via t.Setenv.
2026-05-03 21:32:48 +02:00
Mathias Bergqvist
3c4e8e8bb8 feat(hyperguild): tier subcommand
Adds the tier subcommand to the hyperguild CLI. Reuses
internal/tier.Detect verbatim, sources probe URLs from
ANTHROPIC_PROBE_URL (default https://api.anthropic.com) and
LITELLM_BASE_URL (no default — empty triggers airplane).

Human-readable output by default; --json emits the same Info struct
as the supervisor's tier MCP returns. Tests cover all three tier
states via httptest fakes.
2026-05-03 21:27:33 +02:00
Mathias Bergqvist
5c88eff46f feat(hyperguild): subcommand router skeleton
Lays down the cmd/hyperguild/ entry point. Defines the subcommand
contract (ctx, args, stdin, stdout, stderr) error, the dispatch()
function that's testable without os.Exit, and stubs for tier / brain /
mode that return errNotImplemented. Subsequent commits replace each
stub.

Part of Plan 4 (hyperguild CLI) of the hyperguild migration.
2026-05-03 21:21:08 +02:00
Mathias Bergqvist
646a86f2c3 docs(specs): fix brain URL port (3300 → 30330)
Pod-internal port is 3300; Tailscale-exposed NodePort is 30330.
External clients including the planned hyperguild CLI hit 30330.
2026-05-03 21:01:51 +02:00
Mathias Bergqvist
adf0504116 docs(specs): hyperguild CLI design — Plan 4 of hyperguild migration
Implementation-level spec for the hyperguild CLI: a stdlib Go binary
at cmd/hyperguild/ with subcommands tier, brain query/write, and mode
(cloud|client-local|sovereign). Replaces the supervisor's tier MCP and
provides shell-friendly access to the brain HTTP REST API.

Six measurable success criteria, seven out-of-scope items, six risks.
Decisions logged: stdlib flag + inline router (no cobra), reuse
internal/tier verbatim, BRAIN_URL env override, mode subcommand writes
.mcp.json with per-mode template plus placeholder for the Plan 6
routing pod.
2026-05-03 20:59:45 +02:00
Mathias Bergqvist
d44427e71f docs: document brain MCP endpoint at koala:30330
All checks were successful
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 3s
- README architecture diagram now shows two MCP servers (supervisor +
  brain) with the brain hosted by ingestion directly.
- Connect-a-project example includes both servers.
- .context/PROJECT.md replaces the boilerplate "Knowledge base access"
  block with the actual hyperguild MCP endpoints.
- Adapters regenerated via task context:sync.

Captures the transitional state where two MCPs coexist; the supervisor
MCP will shrink as skill workers move to SKILL.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:03:45 +02:00
Mathias Bergqvist
2635cdcaa7 chore: add brain MCP server alongside supervisor
The brain MCP at koala:30330 hosts the brain_* and session_log tools
formerly on supervisor. Supervisor stays connected during the
transition; its skill workers and the brain duplication will be
removed in a later plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:02:28 +02:00
Mathias Bergqvist
e922471229 fix(context-sync): short-circuit when root AGENT.md is unreachable
All checks were successful
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 3s
In CI's clean checkout the tree-walk for ~/dev/.context/AGENT.md
finds nothing, leaving ROOT_CONTEXT empty. The script previously
proceeded to regenerate AGENTS.md, .cursorrules,
.aider.conventions.md, and .context/system-prompt.txt as
project-only — but the committed versions are root+project, so
the drift gate added in cc401d9 fails CI on every push.

When no root context is reachable, only regenerate CLAUDE.md
(which is project-only by design — Claude Code walks up the tree
itself to find the root). The root-bearing adapters are left
untouched, eliminating the false-positive drift.

Local runs (with root context reachable) are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:55:43 +02:00
Mathias Bergqvist
87ff1f907c fix(ingestion): silence errcheck on resp.Body.Close in integration test
Some checks failed
CI / Lint / Test / Vet (push) Failing after 3s
CI / Mirror to GitHub (push) Has been skipped
CI's golangci-lint flagged the un-checked deferred Close. Match the
existing project pattern (defer func() { _ = ... }()).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:55:29 +02:00
Mathias Bergqvist
9cc179dec6 feat: brain MCP migration — extract brain_* + session_log into ingestion pod
Some checks failed
CI / Lint / Test / Vet (push) Failing after 2s
CI / Mirror to GitHub (push) Has been skipped
Slice 1 of the larger SKILL.md + routing-MCP architecture migration.
Adds an MCP HTTP handler to the ingestion service at POST /mcp,
exposing 5 tools (brain_query, brain_write, brain_ingest_raw,
brain_ingest, session_log). Plan and 9 implementation commits
preserved on the feat/brain-mcp-migration branch.

NodePort 30330 wired in infra repo separately (commit 008548e).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:05:08 +02:00
Mathias Bergqvist
370d30e376 feat(ingestion): mount MCP handler at POST /mcp
The ingestion server now exposes both REST and MCP on the same port
(3300). MCP shares brainDir, pipeline config, and LLM client with the
REST handlers — single source of process state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 19:41:05 +02:00
Mathias Bergqvist
bd0c1d75fd feat(ingestion): implement session_log MCP tool
Appends a JSON line to brainDir/sessions/<session_id>.jsonl using the
session package copied in Task 2. Required for upcoming pass-rate
logging.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 13:04:40 +02:00
Mathias Bergqvist
8c87460bff feat(ingestion): implement brain_ingest MCP tool
Wraps pipeline.Run with the existing LLM client. Mirrors the HTTP
/ingest and /ingest-path semantics — accepts either path or
content+source, validates mutual exclusion, surfaces an explicit error
when the LLM client is not configured (test-mode).

ctx is threaded through to pipeline.Run for cancellation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 13:02:02 +02:00
Mathias Bergqvist
809d435480 feat(ingestion): implement brain_ingest_raw MCP tool
Wraps pipeline.RunRaw directly. Same dry-run semantics as the HTTP
/ingest-raw endpoint. Test exercises a single concept page; asserts
returned path and that no file is written under dry_run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 10:28:05 +02:00
Mathias Bergqvist
e4a94df4fc feat(ingestion): extract WriteNote helper and add brain_write MCP tool
api.WriteNote captures the file-write logic that was previously inline
in Handler.Write. The existing HTTP endpoint now delegates to it; the
new MCP brain_write tool reuses the same function. Path-traversal
guard is strengthened to explicitly reject filenames containing path
separators or "..", so the rejection is surfaced before filepath.Base
strips the suspicious component (the previous defense-in-depth prefix
check became unreachable for these inputs after Base normalisation).
HTTP error code for caller-input errors shifts from 500 to 400, which
is semantically correct and not exercised by any existing test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 10:25:38 +02:00
Mathias Bergqvist
7dcb5610fe feat(ingestion): implement brain_query MCP tool
Wraps the existing search.Query function. Same BM25 over
brain/knowledge/ and brain/wiki/ that the HTTP /query endpoint serves.
Plan note: handleCall switch replaces the single-line stub from Task 1
— no unknownToolError type to remove since Task 1 inlined the error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 09:56:40 +02:00
Mathias Bergqvist
63c8d114e8 feat(ingestion): add session package for JSONL log persistence
Copy of internal/session from the supervisor module — the ingestion
service needs it for the upcoming session_log MCP tool. The supervisor
copy will be removed in the supervisor-retirement plan; until then
the two packages are intentionally identical and pinned (no edits).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 09:54:24 +02:00
Mathias Bergqvist
54f7d373bd feat(ingestion): add MCP server skeleton with tools/list
Adds an MCP HTTP handler under ingestion/internal/mcp. Implements
initialize, tools/list, and the JSON-RPC notification skip from prior
work. Tool dispatch is stubbed (returns unknown-tool error) and will be
filled in by subsequent tasks.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 09:43:23 +02:00
Mathias Bergqvist
a412eee427 docs: add brain MCP migration plan
13 TDD-disciplined tasks moving brain_* and session_log out of the
supervisor pod and into the ingestion pod's MCP handler. Slice 1 of
the larger SKILL.md + routing-MCP architecture migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 09:27:28 +02:00
Mathias Bergqvist
3d6f33881b chore: remove sources: from context:sync to disable Task source-cache
Some checks failed
CI / Lint / Test / Vet (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
Task's sources: declaration cached the regeneration on the assumption
that adapter outputs depend only on .context/AGENT.md (or PROJECT.md +
.skills/). The cache occasionally skipped legitimate runs after manual
edits to root or template content not detectable from those source paths.
context-sync.sh is idempotent and cheap; running it on every invocation
is the right default. The freshness gate (git status --porcelain) is
unaffected — it always checked the actual git tree state.
2026-04-30 23:14:20 +02:00
Mathias Bergqvist
07e3f341ef chore: re-sync context adapters after root prose cleanup
Some checks failed
CI / Lint / Test / Vet (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
Root AGENT.md dropped a stale paragraph; adapters that embed root
(AGENTS.md, .cursorrules, .aider.conventions.md, system-prompt.txt)
need to be regenerated to match. CLAUDE.md is project-only by design
and unchanged.
2026-04-30 23:00:31 +02:00
Mathias Bergqvist
5c532e708c chore: drop HANDROLLED sentinel machinery from context-sync.sh
Some checks failed
CI / Lint / Test / Vet (push) Failing after 2s
CI / Mirror to GitHub (push) Has been skipped
Nothing in the workspace uses the HANDROLLED escape hatch anymore — the
infra repo (its only consumer) was migrated to the standard pattern.
Removing the dead code: HANDROLLED_MARKER constant, skip_if_handrolled
function, and call sites in each generator. Sync output for any non-
HANDROLLED file (i.e., every file we have) is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 22:48:28 +02:00
Mathias Bergqvist
a34c66d7cd fix: update scripts/context-sync.sh with HANDROLLED sentinel support
Some checks failed
CI / Lint / Test / Vet (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
Phase 4 sweep updated .gitignore and Taskfile.yml but missed this
script. Copy the canonical from ~/dev/project-template/ so the
HANDROLLED escape hatch works in this project (a CLAUDE.md or
adapter file containing '<!-- HANDROLLED: do not regenerate -->'
near the top is now skipped on regeneration).
2026-04-29 16:41:56 +02:00
Mathias Bergqvist
cc401d92d6 chore: commit adapters; add context freshness gate to task check
Adapters are now tracked so non-Mac hosts get full agent context after
a plain git pull. task check runs context:sync first and fails on drift
via git status --porcelain over the 5 adapter paths.
2026-04-29 15:59:52 +02:00
Mathias Bergqvist
9bdf00f51f refactor(mcp): connect Claude Code via direct HTTP, remove stdio bridge
All checks were successful
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Successful in 3s
Claude Code now supports MCP servers over HTTP natively (type: "http"
in .mcp.json). The stdio↔HTTP bridge binary was a workaround for the
older stdio-only constraint and is no longer needed — the supervisor
NodePort on koala (30320) is reachable over Tailscale from any client
machine.

Removed:
- cmd/bridge/ (Go source, ~60 lines)
- bin/supervisor-bridge artifact
- Taskfile bridge:build target and the build aggregate's reference
- README "Build the bridge binary" instruction

Updated:
- .mcp.json switched to {type:"http", url:"http://koala:30320/mcp"}
- README architecture diagram and "Connect a project" section

Behavioural prerequisite for this change shipped in 7f7524c
(notifications fix). Verified end-to-end: tier tool call returns
{tier:2, label:"lan-only"} via direct HTTP, no shim.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 10:38:58 +02:00
Mathias Bergqvist
7f7524c859 fix(mcp): do not respond to JSON-RPC notifications
All checks were successful
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
The supervisor's MCP HTTP handler was answering every parsed request,
including notifications (messages with no id field). Per JSON-RPC 2.0,
notifications must not receive a response. The Apr-29 incident saw
Claude Code's MCP client receive a -32601 error for the standard
notifications/initialized handshake step and disconnect immediately
after a successful initialize.

Skip writing the response when req.ID == nil. Cover both the
known-method (notifications/initialized) and unknown-method paths
with tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 08:39:20 +02:00
Mathias Bergqvist
0a70d9e972 feat(pipeline): add POST /ingest-raw for direct batch ingestion without LLM
All checks were successful
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Has been skipped
Allows callers to provide pre-structured RawPage data directly, bypassing the
LLM extraction step. The pipeline still handles slug computation, frontmatter,
link canonicalization, source back-references, and dedup — only the extraction
is skipped. Useful when a more capable model or manual curation produces the
structured data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v0.5.0
2026-04-24 11:15:59 +02:00
Mathias Bergqvist
3e9a648115 fix(pipeline): repair invalid JSON escape sequences from LLM output before parsing
All checks were successful
CI / Lint / Test / Vet (push) Successful in 11s
CI / Mirror to GitHub (push) Has been skipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.4.1
2026-04-23 22:04:27 +02:00
Mathias Bergqvist
923a665365 fix(pipeline): skip RawPages with empty title in BuildPages instead of producing broken paths
All checks were successful
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Has been skipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.4.0
2026-04-23 19:55:37 +02:00
Mathias Bergqvist
537aebc302 feat(pipeline): update system prompt for new LLM JSON contract (no slugs)
- Change prompt to reflect new output format: title, type, subtype, domain, content
- Remove slug/path generation responsibility from LLM — pipeline now handles it
- Wikilinks change from [[slug|Display Name]] to [[Display Name]] only
- LLM no longer includes frontmatter or paths in output

docs(schema): update LLM output format and wikilink convention for Level 3

- Specify JSON schema: title, type, subtype, domain, content fields
- Remove frontmatter requirements from schema output (handled by pipeline)
- Simplify wikilink format to [[Display Name]] — no slug or pipe
- Pipeline now responsible for slug generation and frontmatter construction

These changes shift slug/frontmatter generation from LLM to pipeline,
reducing cognitive load on the model and improving control over output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:45:21 +02:00
Mathias Bergqvist
de35d4dbb0 feat(pipeline): wire ParseRawPages+BuildPages+CanonicalizeLinks into Run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:07:33 +02:00
Mathias Bergqvist
26855f69b0 feat(pipeline): add CanonicalizeLinks — convert [[Display Name]] to [[slug|Display Name]] 2026-04-23 18:59:10 +02:00
Mathias Bergqvist
a7b363d589 fix(pipeline): quote YAML scalar fields in buildFrontmatter to prevent injection 2026-04-23 18:56:39 +02:00
Mathias Bergqvist
7b57051af8 feat(pipeline): add BuildPages — compute slugs/paths/frontmatter from RawPage 2026-04-23 18:50:37 +02:00
Mathias Bergqvist
a620f6cb01 fix(pipeline): guard empty-title bridge + skip stale integration tests until task4
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 18:46:07 +02:00
Mathias Bergqvist
26b5636b43 feat(pipeline): replace ParsePages with ParseRawPages + RawPage type
Strips slug authority from the LLM. The new RawPage type carries only
{title, type, subtype, domain, content} — no paths or frontmatter.
Pipeline will derive slugs deterministically (Task 4).

pipeline.go gets a temporary bridge stub (TODO task4) to keep the
package compiling between tasks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 18:41:33 +02:00
Mathias Bergqvist
989f375aec docs: add Level 3 implementation plan 2026-04-23 17:37:45 +02:00
Mathias Bergqvist
6403d5e444 docs: add Level 3 slug authority design spec 2026-04-23 17:23:22 +02:00
Mathias Bergqvist
ab19968ae2 feat: POST /backfill-refs — retroactive source back-reference injection
All checks were successful
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
Walks wiki/sources/, extracts wikilinks from each source page, and injects
## Sources back-refs into all linked concept and entity pages. All refs from
all sources are accumulated in memory before writing, so multiple sources
referencing the same concept are merged in a single write. Running the
endpoint multiple times is safe — wiki.Merge deduplicates bullet items.
2026-04-23 16:50:11 +02:00
Mathias Bergqvist
1605624668 feat(pipeline): add POST /backfill-refs endpoint to retroactively inject source back-references 2026-04-23 16:50:00 +02:00
Mathias Bergqvist
55fa0b503a feat: source back-references on concept and entity pages
All checks were successful
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
After each ingestion, every concept and entity page linked from the
source page gains a ## Sources entry pointing back to that source.
Pages already on disk (from prior ingestions) are loaded and updated,
so re-ingesting a new source accumulates references over time.
Deduplication is handled by wiki.Merge's existing bullet-section logic.
2026-04-23 16:36:40 +02:00
Mathias Bergqvist
3c2bd9268c feat(pipeline): wire source back-reference injection into Run 2026-04-23 16:36:22 +02:00
Mathias Bergqvist
29727ec2a5 feat(pipeline): inject source back-references into concept and entity pages 2026-04-23 16:35:47 +02:00
Mathias Bergqvist
0a075088b2 docs: add source back-references implementation plan 2026-04-23 16:33:41 +02:00
Mathias Bergqvist
1bfe501d09 fix(cd): only deploy when CI passes on main
All checks were successful
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-23 16:24:59 +02:00
Mathias Bergqvist
3607920601 fix(lint): resolve all errcheck violations in ingestion module
All checks were successful
cd / Build and deploy (push) Successful in 10s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-23 16:20:59 +02:00
Mathias Bergqvist
a6c39e8691 feat: PDF extraction and fuzzy entity resolution
Some checks failed
cd / Build and deploy (push) Successful in 11s
CI / Lint / Test / Vet (push) Failing after 5s
CI / Mirror to GitHub (push) Has been skipped
- New extract package: Text() dispatcher for .md/.txt passthrough and
  PDF extraction via pdftotext subprocess
- wiki.Entry gains Aliases []string, loaded from YAML frontmatter
- Fuzzy entity resolution in pipeline: normalizes titles (lowercase,
  strip articles, collapse hyphens) and matches proposed pages against
  existing inventory slugs and aliases to prevent proliferation
- Watcher and API handler now use extract.Text() instead of os.ReadFile
- Dockerfile: apk add poppler-utils in Alpine runtime stage
2026-04-23 16:03:02 +02:00
Mathias Bergqvist
a37d18bf7a chore(docker): add poppler-utils for PDF text extraction 2026-04-23 16:02:12 +02:00
Mathias Bergqvist
2975eadc87 feat(watcher,api): use extract.Text() for file reading — fixes PDF ingestion 2026-04-23 16:01:36 +02:00
Mathias Bergqvist
53e46781b1 feat(pipeline): resolve proposed pages against inventory before writing 2026-04-23 16:00:31 +02:00