Files
hyperguild/docs/superpowers/plans/2026-04-29-brain-mcp-migration.md
Mathias Bergqvist a412eee427 docs: add brain MCP migration plan
13 TDD-disciplined tasks moving brain_* and session_log out of the
supervisor pod and into the ingestion pod's MCP handler. Slice 1 of
the larger SKILL.md + routing-MCP architecture migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 09:27:28 +02:00

1827 lines
55 KiB
Markdown

# Brain MCP Migration Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Move the `brain_*` and `session_log` MCP tools out of the supervisor pod and into the ingestion pod, so Mode 1 (Claude Code, no supervisor) can reach the brain via direct HTTP MCP without the supervisor running.
**Architecture:** A new `ingestion/internal/mcp` package adds a notification-aware MCP HTTP handler to the existing ingestion server, alongside its current REST endpoints. The handler dispatches `brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`, and `session_log` directly to existing pipeline/search/wiki packages within the same process — no HTTP round-trip to itself, no new module. A NodePort service exposes the MCP endpoint over Tailscale at `koala:30330/mcp`. The supervisor pod's brain skill remains in place during this slice; deletion is deferred to a later plan.
**Tech Stack:** Go 1.24, net/http stdlib mux, encoding/json, testify, k3s manifests in the separate `infra` repo, Flux GitOps for deploy.
---
## File Structure
**New files (ingestion module):**
- `ingestion/internal/mcp/server.go` — JSON-RPC handler with notification skip
- `ingestion/internal/mcp/server_test.go` — covers parse, dispatch, notifications
- `ingestion/internal/mcp/handlers.go``brain_*` and `session_log` tool implementations
- `ingestion/internal/mcp/handlers_test.go` — covers each tool against a tmp brain dir
- `ingestion/internal/session/session.go` — copy of `internal/session/session.go` (will dedupe in supervisor-retirement plan)
- `ingestion/internal/session/session_test.go` — round-trip Append/Read
**Modified files (ingestion module):**
- `ingestion/cmd/server/main.go:66-72` — register MCP handler at `POST /mcp`
- `ingestion/internal/api/handler.go` — extract pure `WriteNote` helper for reuse from MCP
**New files (infra repo):**
- `infra/k3s/apps/supervisor/ingestion-nodeport.yaml` — exposes `ingestion:3300` as NodePort 30330
**Modified files (this repo, root):**
- `.mcp.json` — add `brain` server, leave `supervisor` until Mode 1 verified
- `README.md` — document the brain MCP endpoint and updated `.mcp.json` example
- `CLAUDE.md` — note the dual-MCP transitional state
**Out of scope for this plan:**
- Removing the supervisor's brain skill (Plan 7)
- Removing `internal/session` from supervisor module (Plan 7)
- Adding `brain_search` (kb-retrieval not deployed; would be added with that integration)
---
## Task 1: Add MCP server skeleton to ingestion
**Files:**
- Create: `ingestion/internal/mcp/server.go`
- Create: `ingestion/internal/mcp/server_test.go`
The MCP server in supervisor (`internal/mcp/server.go`) already has notification handling correct from prior work. We copy that shape — adapted to ingestion's module path — without pulling in supervisor's `registry` abstraction. Dispatch goes through a single `Server.handleCall` switch in this slice; if a third MCP server emerges later we can extract a registry then.
- [ ] **Step 1: Write the failing tests**
```go
// ingestion/internal/mcp/server_test.go
package mcp_test
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func body(t *testing.T, v any) *bytes.Buffer {
t.Helper()
b, err := json.Marshal(v)
require.NoError(t, err)
return bytes.NewBuffer(b)
}
func TestServerInitialize(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
"jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": map[string]any{},
}))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
var resp map[string]any
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
result := resp["result"].(map[string]any)
assert.Equal(t, "2024-11-05", result["protocolVersion"])
}
func TestServerToolsList(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
"jsonrpc": "2.0", "id": 2, "method": "tools/list",
}))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
var resp map[string]any
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
tools := resp["result"].(map[string]any)["tools"].([]any)
names := make([]string, 0, len(tools))
for _, t := range tools {
names = append(names, t.(map[string]any)["name"].(string))
}
assert.ElementsMatch(t, []string{
"brain_query", "brain_write", "brain_ingest_raw", "brain_ingest", "session_log",
}, names)
}
func TestServerNotificationGetsNoBody(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
"jsonrpc": "2.0", "method": "notifications/initialized",
}))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
assert.Empty(t, strings.TrimSpace(rr.Body.String()))
}
func TestServerUnknownMethodReturnsError(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
"jsonrpc": "2.0", "id": 3, "method": "unknown/method",
}))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
var resp map[string]any
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
assert.NotNil(t, resp["error"])
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | head -30`
Expected: FAIL — package does not exist, `mcp.NewServer` undefined.
- [ ] **Step 3: Write the minimal server implementation**
```go
// ingestion/internal/mcp/server.go
// Package mcp implements an MCP HTTP handler for the ingestion service.
// Exposed tools: brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log.
package mcp
import (
"context"
"encoding/json"
"net/http"
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
)
type request struct {
JSONRPC string `json:"jsonrpc"`
ID any `json:"id"`
Method string `json:"method"`
Params json.RawMessage `json:"params"`
}
type response struct {
JSONRPC string `json:"jsonrpc"`
ID any `json:"id,omitempty"`
Result any `json:"result,omitempty"`
Error *rpcError `json:"error,omitempty"`
}
type rpcError struct {
Code int `json:"code"`
Message string `json:"message"`
}
// Server handles MCP JSON-RPC over HTTP for the ingestion service.
type Server struct {
brainDir string
pipeline pipeline.Config
llm pipeline.CompleteFunc // may be nil in tests
}
// NewServer constructs a Server bound to brainDir. pipelineCfg supplies the
// LLM-backed pipeline; llm may be nil for non-LLM tools only.
func NewServer(brainDir string, pipelineCfg *pipeline.Config, llm pipeline.CompleteFunc) *Server {
cfg := pipeline.Config{}
if pipelineCfg != nil {
cfg = *pipelineCfg
}
return &Server{brainDir: brainDir, pipeline: cfg, llm: llm}
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var req request
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, nil, -32700, "parse error")
return
}
// JSON-RPC 2.0 notifications (no id) must not receive a response.
if req.ID == nil {
return
}
var result any
var rpcErr *rpcError
switch req.Method {
case "initialize":
result = map[string]any{
"protocolVersion": "2024-11-05",
"capabilities": map[string]any{"tools": map[string]any{}},
"serverInfo": map[string]any{"name": "ingestion-brain", "version": "0.1.0"},
}
case "tools/list":
result = map[string]any{"tools": s.tools()}
case "tools/call":
var p struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
}
if err := json.Unmarshal(req.Params, &p); err != nil {
rpcErr = &rpcError{Code: -32602, Message: "invalid params"}
break
}
out, err := s.handleCall(r.Context(), p.Name, p.Arguments)
if err != nil {
rpcErr = &rpcError{Code: -32000, Message: err.Error()}
break
}
result = map[string]any{
"content": []map[string]any{{"type": "text", "text": string(out)}},
}
default:
rpcErr = &rpcError{Code: -32601, Message: "method not found: " + req.Method}
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(response{
JSONRPC: "2.0",
ID: req.ID,
Result: result,
Error: rpcErr,
})
}
func writeError(w http.ResponseWriter, id any, code int, msg string) {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(response{
JSONRPC: "2.0",
ID: id,
Error: &rpcError{Code: code, Message: msg},
})
}
// handleCall dispatches a tools/call. Stub for Task 1; expanded in later tasks.
func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) {
return nil, &unknownToolError{name: name}
}
type unknownToolError struct{ name string }
func (e *unknownToolError) Error() string { return "unknown tool: " + e.name }
```
```go
// ingestion/internal/mcp/handlers.go (placeholder, filled in later tasks)
package mcp
import "encoding/json"
// tools returns the tool descriptors. Schemas are filled per-tool in subsequent tasks.
func (s *Server) tools() []map[string]any {
str := func(desc string) map[string]any {
return map[string]any{"type": "string", "description": desc}
}
int_ := func(desc string) map[string]any {
return map[string]any{"type": "integer", "description": desc}
}
schema := func(required []string, props map[string]any) json.RawMessage {
b, _ := json.Marshal(map[string]any{
"type": "object", "required": required, "properties": props,
})
return b
}
return []map[string]any{
{
"name": "brain_query",
"description": "BM25 full-text search across brain/knowledge/ and brain/wiki/ markdown files.",
"inputSchema": schema([]string{"query"}, map[string]any{
"query": str("search terms"),
"limit": int_("max results, default 5"),
}),
},
{
"name": "brain_write",
"description": "Write a raw knowledge note to brain/knowledge/.",
"inputSchema": schema([]string{"content"}, map[string]any{
"content": str("markdown content"),
"filename": str("optional filename"),
"type": str("optional frontmatter type"),
"domain": str("optional frontmatter domain"),
}),
},
{
"name": "brain_ingest_raw",
"description": "Ingest pre-structured pages into the brain wiki, bypassing the LLM extraction step.",
"inputSchema": schema([]string{"source", "pages"}, map[string]any{
"source": str("source name"),
"pages": map[string]any{"type": "array"},
"dry_run": map[string]any{"type": "boolean"},
}),
},
{
"name": "brain_ingest",
"description": "Ingest content into the brain wiki via the LLM extraction pipeline.",
"inputSchema": schema([]string{}, map[string]any{
"content": str("raw content; required when path is empty"),
"source": str("source name; required when path is empty"),
"path": str("file path; mutually exclusive with content+source"),
"dry_run": map[string]any{"type": "boolean"},
}),
},
{
"name": "session_log",
"description": "Append a structured entry to brain/sessions/<session_id>.jsonl.",
"inputSchema": schema([]string{"session_id"}, map[string]any{
"session_id": str("session identifier"),
"skill": str("skill name"),
"phase": str("phase within the skill"),
"project_root": str("absolute project root"),
"final_status": str("ok | error | skipped"),
"file_path": str("optional file produced"),
"model_used": str("optional model identifier"),
"duration_ms": int_("optional duration in ms"),
"message": str("optional free-text"),
}),
},
}
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestServer' 2>&1 | tail -20`
Expected: All four `TestServer*` tests PASS.
- [ ] **Step 5: Commit**
```bash
cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git add ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): add MCP server skeleton with tools/list
Adds an MCP HTTP handler under ingestion/internal/mcp. Implements
initialize, tools/list, and the JSON-RPC notification skip from prior
work. Tool dispatch is stubbed (returns unknown-tool error) and will be
filled in by subsequent tasks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 2: Add session package to ingestion
**Files:**
- Create: `ingestion/internal/session/session.go`
- Create: `ingestion/internal/session/session_test.go`
We deliberately copy `internal/session/session.go` from supervisor rather than introduce a third Go module. Plan 7 (supervisor retirement) will delete the supervisor copy. Until then both exist; they will not drift because no edits happen during transition.
- [ ] **Step 1: Write the failing test**
```go
// ingestion/internal/session/session_test.go
package session_test
import (
"path/filepath"
"testing"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/session"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestAppendAndRead(t *testing.T) {
dir := t.TempDir()
sid := "test-session"
e1 := session.Entry{
SessionID: sid,
Timestamp: time.Now().UTC().Truncate(time.Second),
Skill: "tdd",
Phase: "red",
FinalStatus: "ok",
}
e2 := session.Entry{
SessionID: sid,
Timestamp: time.Now().UTC().Truncate(time.Second),
Skill: "tdd",
Phase: "green",
FinalStatus: "ok",
}
require.NoError(t, session.Append(dir, sid, e1))
require.NoError(t, session.Append(dir, sid, e2))
got, err := session.Read(dir, sid)
require.NoError(t, err)
require.Len(t, got, 2)
assert.Equal(t, "red", got[0].Phase)
assert.Equal(t, "green", got[1].Phase)
// File path is sessionsDir/<sid>.jsonl
_, statErr := filepath.Abs(filepath.Join(dir, sid+".jsonl"))
require.NoError(t, statErr)
}
func TestReadMissingReturnsEmpty(t *testing.T) {
got, err := session.Read(t.TempDir(), "nope")
require.NoError(t, err)
assert.Empty(t, got)
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd ingestion && go test ./internal/session/... -v 2>&1 | head -20`
Expected: FAIL — package does not exist.
- [ ] **Step 3: Write the implementation**
Copy `/Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/internal/session/session.go` verbatim into `ingestion/internal/session/session.go`, changing only the package comment header to reference the ingestion path.
```go
// ingestion/internal/session/session.go
package session
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"time"
)
// Entry is one skill invocation record, appended to the session JSONL log.
type Entry struct {
SessionID string `json:"session_id"`
Timestamp time.Time `json:"timestamp"`
Skill string `json:"skill"`
Phase string `json:"phase,omitempty"`
ProjectRoot string `json:"project_root,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
Attempts []Attempt `json:"attempts,omitempty"`
FinalStatus string `json:"final_status"`
FilePath string `json:"file_path,omitempty"`
ModelUsed string `json:"model_used,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Message string `json:"message,omitempty"`
}
// Attempt represents one subprocess invocation within a skill call.
type Attempt struct {
Attempt int `json:"attempt"`
Model string `json:"model"`
Tier string `json:"tier"`
DurationMs int64 `json:"duration_ms"`
WarmStart bool `json:"warm_start"`
Verified bool `json:"verified"`
Verdict string `json:"verdict,omitempty"`
Feedback string `json:"feedback,omitempty"`
OutputSummary string `json:"output_summary,omitempty"`
RunnerOutput string `json:"runner_output,omitempty"`
}
// Append writes entry as a single JSON line to sessionsDir/{sessionID}.jsonl.
func Append(sessionsDir, sessionID string, entry Entry) error {
if err := os.MkdirAll(sessionsDir, 0o755); err != nil {
return fmt.Errorf("create sessions dir: %w", err)
}
path := filepath.Join(sessionsDir, sessionID+".jsonl")
f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return fmt.Errorf("open session log: %w", err)
}
line, err := json.Marshal(entry)
if err != nil {
_ = f.Close()
return fmt.Errorf("marshal entry: %w", err)
}
if _, err = fmt.Fprintf(f, "%s\n", line); err != nil {
_ = f.Close()
return fmt.Errorf("write entry: %w", err)
}
if err = f.Close(); err != nil {
return fmt.Errorf("close session log: %w", err)
}
return nil
}
// Read returns all entries for sessionID. Returns empty slice if no log exists.
func Read(sessionsDir, sessionID string) ([]Entry, error) {
path := filepath.Join(sessionsDir, sessionID+".jsonl")
f, err := os.Open(path)
if errors.Is(err, fs.ErrNotExist) {
return []Entry{}, nil
}
if err != nil {
return nil, fmt.Errorf("open session log: %w", err)
}
defer f.Close() //nolint:errcheck
var entries []Entry
scanner := bufio.NewScanner(f)
scanner.Buffer(make([]byte, 0, 256*1024), 1<<20)
for scanner.Scan() {
line := scanner.Bytes()
if len(line) == 0 {
continue
}
var e Entry
if err := json.Unmarshal(line, &e); err != nil {
return nil, fmt.Errorf("parse entry: %w", err)
}
entries = append(entries, e)
}
return entries, scanner.Err()
}
```
- [ ] **Step 4: Run test to verify it passes**
Run: `cd ingestion && go test ./internal/session/... -v 2>&1 | tail -10`
Expected: Both tests PASS.
- [ ] **Step 5: Commit**
```bash
git add ingestion/internal/session
git commit -m "$(cat <<'EOF'
feat(ingestion): add session package for JSONL log persistence
Copy of internal/session from the supervisor module — the ingestion
service needs it for the upcoming session_log MCP tool. The supervisor
copy will be removed in the supervisor-retirement plan; until then
the two packages are intentionally identical and pinned (no edits).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 3: Implement brain_query MCP tool
**Files:**
- Modify: `ingestion/internal/mcp/handlers.go` (add `brainQuery` method + dispatch)
- Modify: `ingestion/internal/mcp/server.go` (wire `handleCall` to call `brainQuery`)
- Create test in: `ingestion/internal/mcp/handlers_test.go`
`brain_query` calls the existing pure function `search.Query(brainDir, query, limit)`. No HTTP self-call needed.
- [ ] **Step 1: Write the failing test**
```go
// ingestion/internal/mcp/handlers_test.go
package mcp_test
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func toolCall(t *testing.T, srv http.Handler, name string, args map[string]any) map[string]any {
t.Helper()
bodyBytes, err := json.Marshal(map[string]any{
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": map[string]any{"name": name, "arguments": args},
})
require.NoError(t, err)
req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(bodyBytes))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
require.Equal(t, http.StatusOK, rr.Code)
var resp map[string]any
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
return resp
}
func TestBrainQueryReturnsResults(t *testing.T) {
brainDir := t.TempDir()
knowledge := filepath.Join(brainDir, "knowledge")
require.NoError(t, os.MkdirAll(knowledge, 0o755))
require.NoError(t, os.WriteFile(
filepath.Join(knowledge, "tdd.md"),
[]byte("# TDD\n\nTest-driven development is iterative.\n"),
0o644,
))
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_query", map[string]any{"query": "tdd"})
require.Nil(t, resp["error"])
result := resp["result"].(map[string]any)
content := result["content"].([]any)
require.NotEmpty(t, content)
text := content[0].(map[string]any)["text"].(string)
assert.Contains(t, text, "tdd.md")
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | head -20`
Expected: FAIL — `unknown tool: brain_query`.
- [ ] **Step 3: Implement brainQuery**
Add to `ingestion/internal/mcp/handlers.go`:
```go
// (append to existing handlers.go)
import (
"context"
"encoding/json"
"fmt"
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
)
type brainQueryArgs struct {
Query string `json:"query"`
Limit int `json:"limit,omitempty"`
}
func (s *Server) brainQuery(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainQueryArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.Query == "" {
return nil, fmt.Errorf("query is required")
}
if a.Limit == 0 {
a.Limit = 5
}
results, err := search.Query(s.brainDir, a.Query, a.Limit)
if err != nil {
return nil, fmt.Errorf("search: %w", err)
}
return json.Marshal(map[string]any{"results": results})
}
```
Update `ingestion/internal/mcp/server.go`'s `handleCall` to dispatch:
```go
// ingestion/internal/mcp/server.go — replace handleCall and unknownToolError block
func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error) {
switch name {
case "brain_query":
return s.brainQuery(ctx, args)
default:
return nil, fmt.Errorf("unknown tool: %s", name)
}
}
```
Add the `"fmt"` import to `server.go` and remove the now-unused `unknownToolError` type and its `Error()` method.
- [ ] **Step 4: Run test to verify it passes**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainQuery' 2>&1 | tail -10`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/handlers_test.go ingestion/internal/mcp/server.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_query MCP tool
Wraps the existing search.Query function. Same BM25 over
brain/knowledge/ and brain/wiki/ that the HTTP /query endpoint serves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 4: Extract WriteNote helper and implement brain_write
**Files:**
- Modify: `ingestion/internal/api/handler.go` (extract `WriteNote` from `Handler.Write`)
- Modify: `ingestion/internal/mcp/handlers.go` (call `api.WriteNote`)
- Modify: `ingestion/internal/mcp/server.go` (dispatch `brain_write`)
- Modify: `ingestion/internal/api/handler_test.go` (regression: existing /write still works)
- Modify: `ingestion/internal/mcp/handlers_test.go` (new: brain_write writes a file)
The current `Handler.Write` mixes JSON decoding, validation, frontmatter construction, path safety, and disk writing. We extract a pure `WriteNote` helper so MCP and HTTP both call it.
- [ ] **Step 1: Write the failing tests**
Add to `ingestion/internal/mcp/handlers_test.go`:
```go
func TestBrainWriteCreatesFile(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_write", map[string]any{
"content": "# Test\n\nbody",
"filename": "test.md",
"type": "note",
"domain": "personal",
})
require.Nil(t, resp["error"])
// File should exist under brain/knowledge/test.md with frontmatter
got, err := os.ReadFile(filepath.Join(brainDir, "knowledge", "test.md"))
require.NoError(t, err)
assert.Contains(t, string(got), "type: note")
assert.Contains(t, string(got), "domain: personal")
assert.Contains(t, string(got), "# Test")
}
func TestBrainWriteRejectsTraversal(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_write", map[string]any{
"content": "x",
"filename": "../escape.md",
})
assert.NotNil(t, resp["error"])
}
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainWrite' 2>&1 | head -20`
Expected: FAIL — `unknown tool: brain_write`.
- [ ] **Step 3: Extract WriteNote in api/handler.go**
Replace the body of `Handler.Write` with a thin wrapper that calls a new exported function `WriteNote`. Read `ingestion/internal/api/handler.go:88-142` first to confirm the exact code.
```go
// ingestion/internal/api/handler.go — add new exported function before func (h *Handler) Write
// and replace the body of Write to call WriteNote.
// WriteNote writes a markdown file to brainDir/knowledge/<filename>, optionally
// prefixed with YAML frontmatter built from typ and domain. Returns the path
// relative to brainDir (forward-slashed). Filename traversal is rejected.
func WriteNote(brainDir, content, filename, typ, domain string) (string, error) {
if content == "" {
return "", fmt.Errorf("content is required")
}
if filename == "" {
filename = fmt.Sprintf("%s-auto.md", time.Now().UTC().Format("2006-01-02-150405"))
}
rawDir := filepath.Join(brainDir, "knowledge")
if err := os.MkdirAll(rawDir, 0o755); err != nil {
return "", fmt.Errorf("create raw dir: %w", err)
}
finalContent := content
if typ != "" || domain != "" {
var fm strings.Builder
fm.WriteString("---\n")
if typ != "" {
fmt.Fprintf(&fm, "type: %s\n", typ)
}
if domain != "" {
fmt.Fprintf(&fm, "domain: %s\n", domain)
}
fm.WriteString("---\n")
finalContent = fm.String() + content
}
base := filepath.Base(filename)
if !strings.HasSuffix(base, ".md") {
base += ".md"
}
dest := filepath.Join(rawDir, base)
if !strings.HasPrefix(filepath.Clean(dest)+string(os.PathSeparator),
filepath.Clean(rawDir)+string(os.PathSeparator)) {
return "", fmt.Errorf("invalid filename")
}
if err := os.WriteFile(dest, []byte(finalContent), 0o644); err != nil {
return "", fmt.Errorf("write: %w", err)
}
rel, _ := filepath.Rel(brainDir, dest)
return filepath.ToSlash(rel), nil
}
```
Replace `Handler.Write` body to call `WriteNote`:
```go
func (h *Handler) Write(w http.ResponseWriter, r *http.Request) {
var req writeRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid JSON")
return
}
relPath, err := WriteNote(h.brainDir, req.Content, req.Filename, req.Type, req.Domain)
if err != nil {
h.logger.Error("write failed", "err", err)
writeError(w, http.StatusBadRequest, err.Error())
return
}
writeJSON(w, map[string]string{"path": relPath})
}
```
- [ ] **Step 4: Add brainWrite handler and dispatch**
Append to `ingestion/internal/mcp/handlers.go`:
```go
import (
// add to existing import block
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
)
type brainWriteArgs struct {
Content string `json:"content"`
Filename string `json:"filename,omitempty"`
Type string `json:"type,omitempty"`
Domain string `json:"domain,omitempty"`
}
func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainWriteArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
relPath, err := api.WriteNote(s.brainDir, a.Content, a.Filename, a.Type, a.Domain)
if err != nil {
return nil, err
}
return json.Marshal(map[string]string{"path": relPath})
}
```
Update `handleCall` switch:
```go
case "brain_write":
return s.brainWrite(ctx, args)
```
- [ ] **Step 5: Run all ingestion tests**
Run: `cd ingestion && go test -count=1 ./... 2>&1 | tail -15`
Expected: All packages PASS, including the existing `api` tests (regression: HTTP /write still works) and the new `mcp` tests.
- [ ] **Step 6: Commit**
```bash
git add ingestion/internal/api/handler.go ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): extract WriteNote helper and add brain_write MCP tool
api.WriteNote captures the file-write logic that was previously inline
in Handler.Write. The existing HTTP endpoint now delegates to it; the
new MCP brain_write tool reuses the same function. Path-traversal
guard is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 5: Implement brain_ingest_raw
**Files:**
- Modify: `ingestion/internal/mcp/handlers.go` (add `brainIngestRaw`)
- Modify: `ingestion/internal/mcp/server.go` (dispatch)
- Modify: `ingestion/internal/mcp/handlers_test.go` (smoke test against tmp brain)
`pipeline.RunRaw(brainDir, source, pages, dryRun)` is already pure — direct call.
- [ ] **Step 1: Write the failing test**
Add to `ingestion/internal/mcp/handlers_test.go`:
```go
import (
// existing + add:
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
)
func TestBrainIngestRawDryRun(t *testing.T) {
brainDir := t.TempDir()
require.NoError(t, os.MkdirAll(filepath.Join(brainDir, "wiki", "concepts"), 0o755))
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_ingest_raw", map[string]any{
"source": "test-source",
"dry_run": true,
"pages": []map[string]any{
{
"title": "Test Concept",
"type": "concept",
"content": "## Definition\nA test concept.",
},
},
})
require.Nil(t, resp["error"])
result := resp["result"].(map[string]any)
content := result["content"].([]any)
text := content[0].(map[string]any)["text"].(string)
var parsed struct {
Pages []string `json:"pages"`
}
require.NoError(t, json.Unmarshal([]byte(text), &parsed))
assert.Contains(t, parsed.Pages[0], "wiki/concepts/test-concept.md")
// dry_run: no file should exist
_, err := os.Stat(filepath.Join(brainDir, "wiki", "concepts", "test-concept.md"))
assert.True(t, os.IsNotExist(err))
}
var _ = pipeline.RawPage{} // keep import linked for future tests
```
- [ ] **Step 2: Run test, verify failure**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngestRaw' 2>&1 | head -20`
Expected: FAIL — `unknown tool: brain_ingest_raw`.
- [ ] **Step 3: Implement**
Append to `ingestion/internal/mcp/handlers.go`:
```go
type brainIngestRawArgs struct {
Source string `json:"source"`
Pages []pipeline.RawPage `json:"pages"`
DryRun bool `json:"dry_run,omitempty"`
}
func (s *Server) brainIngestRaw(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainIngestRawArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.Source == "" {
return nil, fmt.Errorf("source is required")
}
if len(a.Pages) == 0 {
return nil, fmt.Errorf("pages must be non-empty")
}
result, err := pipeline.RunRaw(s.brainDir, a.Source, a.Pages, a.DryRun)
if err != nil {
return nil, fmt.Errorf("ingest: %w", err)
}
pages := result.Pages
if pages == nil {
pages = []string{}
}
warnings := result.Warnings
if warnings == nil {
warnings = []string{}
}
return json.Marshal(map[string]any{"pages": pages, "warnings": warnings})
}
```
Update import block to include `pipeline` package.
Update `handleCall` switch:
```go
case "brain_ingest_raw":
return s.brainIngestRaw(ctx, args)
```
- [ ] **Step 4: Run tests**
Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_ingest_raw MCP tool
Wraps pipeline.RunRaw directly. Same dry-run semantics as the HTTP
/ingest-raw endpoint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 6: Implement brain_ingest (LLM path)
**Files:**
- Modify: `ingestion/internal/mcp/handlers.go` (add `brainIngest`)
- Modify: `ingestion/internal/mcp/server.go` (dispatch + LLM availability check)
- Modify: `ingestion/internal/mcp/handlers_test.go` (path-vs-content+source mutex test)
`brain_ingest` takes either `path` (file) or `content+source`. Calls `pipeline.Run` which uses the LLM. The MCP server's `s.pipeline` config holds the `Complete` function.
- [ ] **Step 1: Write the failing tests**
Add to `ingestion/internal/mcp/handlers_test.go`:
```go
func TestBrainIngestRejectsBoth(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_ingest", map[string]any{
"content": "x",
"source": "y",
"path": "/tmp/foo.md",
})
assert.NotNil(t, resp["error"])
}
func TestBrainIngestRequiresOne(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "brain_ingest", map[string]any{})
assert.NotNil(t, resp["error"])
}
```
- [ ] **Step 2: Run, verify FAIL**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestBrainIngest' 2>&1 | head -20`
Expected: FAIL — `unknown tool: brain_ingest`.
- [ ] **Step 3: Implement**
Append to `ingestion/internal/mcp/handlers.go`:
```go
import (
// add:
"github.com/mathiasbq/hyperguild/ingestion/internal/extract"
"path/filepath"
"strings"
)
type brainIngestArgs struct {
Content string `json:"content,omitempty"`
Source string `json:"source,omitempty"`
Path string `json:"path,omitempty"`
DryRun bool `json:"dry_run,omitempty"`
}
func (s *Server) brainIngest(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainIngestArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.Path != "" && a.Content != "" {
return nil, fmt.Errorf("path and content+source are mutually exclusive")
}
if a.Path == "" && a.Content == "" {
return nil, fmt.Errorf("either path or content+source is required")
}
if s.pipeline.Complete == nil {
return nil, fmt.Errorf("LLM not configured: set INGEST_LLM_URL")
}
if a.Path != "" {
text, err := extract.Text(a.Path)
if err != nil {
return nil, fmt.Errorf("extract: %w", err)
}
source := a.Source
if source == "" {
source = filepath.Base(strings.TrimSuffix(a.Path, filepath.Ext(a.Path)))
}
return s.runIngest(ctx, text, source, a.DryRun)
}
if a.Source == "" {
return nil, fmt.Errorf("source is required when content is provided")
}
return s.runIngest(ctx, a.Content, a.Source, a.DryRun)
}
func (s *Server) runIngest(ctx context.Context, content, source string, dryRun bool) (json.RawMessage, error) {
result, err := pipeline.Run(ctx, s.pipeline, s.brainDir, content, source, dryRun)
if err != nil {
return nil, fmt.Errorf("ingest: %w", err)
}
pages := result.Pages
if pages == nil {
pages = []string{}
}
warnings := result.Warnings
if warnings == nil {
warnings = []string{}
}
return json.Marshal(map[string]any{"pages": pages, "warnings": warnings})
}
```
Update `handleCall` switch:
```go
case "brain_ingest":
return s.brainIngest(ctx, args)
```
- [ ] **Step 4: Run tests**
Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10`
Expected: All PASS.
- [ ] **Step 5: Commit**
```bash
git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement brain_ingest MCP tool
Wraps pipeline.Run with the existing LLM client. Mirrors the HTTP
/ingest and /ingest-path semantics — accepts either path or
content+source, validates mutual exclusion, surfaces an explicit error
when the LLM client is not configured (test-mode).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 7: Implement session_log
**Files:**
- Modify: `ingestion/internal/mcp/handlers.go` (add `sessionLog`)
- Modify: `ingestion/internal/mcp/server.go` (dispatch)
- Modify: `ingestion/internal/mcp/handlers_test.go` (verify JSONL line written)
Writes to `${BRAIN_DIR}/sessions/<session_id>.jsonl` via the new `ingestion/internal/session` package.
- [ ] **Step 1: Write the failing test**
Add to `ingestion/internal/mcp/handlers_test.go`:
```go
func TestSessionLogAppends(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil)
resp := toolCall(t, srv, "session_log", map[string]any{
"session_id": "session-x",
"skill": "tdd",
"phase": "red",
"final_status": "ok",
})
require.Nil(t, resp["error"])
got, err := os.ReadFile(filepath.Join(brainDir, "sessions", "session-x.jsonl"))
require.NoError(t, err)
assert.Contains(t, string(got), `"skill":"tdd"`)
assert.Contains(t, string(got), `"phase":"red"`)
}
func TestSessionLogRequiresSessionID(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
resp := toolCall(t, srv, "session_log", map[string]any{"skill": "tdd"})
assert.NotNil(t, resp["error"])
}
```
- [ ] **Step 2: Run, verify FAIL**
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestSessionLog' 2>&1 | head -20`
Expected: FAIL.
- [ ] **Step 3: Implement**
Append to `ingestion/internal/mcp/handlers.go`:
```go
import (
// add:
"path/filepath"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/session"
)
type sessionLogArgs struct {
SessionID string `json:"session_id"`
Skill string `json:"skill,omitempty"`
Phase string `json:"phase,omitempty"`
ProjectRoot string `json:"project_root,omitempty"`
FinalStatus string `json:"final_status,omitempty"`
FilePath string `json:"file_path,omitempty"`
ModelUsed string `json:"model_used,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Message string `json:"message,omitempty"`
}
func (s *Server) sessionLog(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a sessionLogArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.SessionID == "" {
return nil, fmt.Errorf("session_id is required")
}
entry := session.Entry{
SessionID: a.SessionID,
Timestamp: time.Now().UTC(),
Skill: a.Skill,
Phase: a.Phase,
ProjectRoot: a.ProjectRoot,
FinalStatus: a.FinalStatus,
FilePath: a.FilePath,
ModelUsed: a.ModelUsed,
DurationMs: a.DurationMs,
Message: a.Message,
}
dir := filepath.Join(s.brainDir, "sessions")
if err := session.Append(dir, a.SessionID, entry); err != nil {
return nil, fmt.Errorf("append: %w", err)
}
return json.Marshal(map[string]string{"status": "ok", "session_id": a.SessionID})
}
```
Update `handleCall`:
```go
case "session_log":
return s.sessionLog(ctx, args)
```
- [ ] **Step 4: Run tests**
Run: `cd ingestion && go test -count=1 ./internal/mcp/... 2>&1 | tail -10`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add ingestion/internal/mcp/handlers.go ingestion/internal/mcp/server.go ingestion/internal/mcp/handlers_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): implement session_log MCP tool
Appends a JSON line to brainDir/sessions/<session_id>.jsonl using the
copied session package. Required for upcoming pass-rate logging.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 8: Wire MCP handler into ingestion server
**Files:**
- Modify: `ingestion/cmd/server/main.go:66-72` (mount MCP at POST /mcp)
- [ ] **Step 1: Write a smoke test that exercises the live handler over HTTP**
Add a new file `ingestion/internal/mcp/integration_test.go`:
```go
package mcp_test
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestMCPMountedHandler(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil)
mux := http.NewServeMux()
mux.Handle("POST /mcp", srv)
ts := httptest.NewServer(mux)
defer ts.Close()
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0", "id": 1, "method": "tools/list",
})
resp, err := http.Post(ts.URL+"/mcp", "application/json", bytes.NewReader(body))
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
out, _ := io.ReadAll(resp.Body)
assert.Contains(t, string(out), `"brain_query"`)
}
```
- [ ] **Step 2: Run, expect FAIL until step 3**
This test passes today (it constructs its own mux). Skip step 2 — the test verifies the wiring shape; the actual wiring change in step 3 ensures `cmd/server/main.go` also mounts it.
Run: `cd ingestion && go test ./internal/mcp/... -v -run 'TestMCPMountedHandler' 2>&1 | tail -10`
Expected: PASS (test is module-local and constructs its own mux).
- [ ] **Step 3: Wire into main.go**
Modify `ingestion/cmd/server/main.go`:
Add to imports:
```go
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
```
After `h := api.NewHandler(...)` and before the `mux := http.NewServeMux()` block, construct the MCP server:
```go
mcpSrv := mcp.NewServer(brainDir, &pipelineCfg, llmClient.Complete)
```
Add the route after the existing POST handlers (around line 72):
```go
mux.Handle("POST /mcp", mcpSrv)
```
Update the startup log line to mention MCP:
```go
logger.Info("ingestion server starting",
"addr", addr,
"brain_dir", brainDir,
"llm_url", llmURL,
"llm_model", llmModel,
"chunk_size", chunkSize,
"watch_interval", watchIntervalLog,
"mcp_enabled", true,
)
```
- [ ] **Step 4: Build and run locally as a smoke test**
```bash
cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
mkdir -p /tmp/brain-mcp-test/{wiki,sessions,knowledge}
INGEST_BRAIN_DIR=/tmp/brain-mcp-test \
INGEST_PORT=33301 \
INGEST_WATCH_INTERVAL=0 \
go run ./ingestion/cmd/server/ &
SERVER_PID=$!
sleep 2
# tools/list
curl -sS -X POST http://localhost:33301/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | head -c 500
echo ""
# notifications/initialized → empty body
curl -sS -i -X POST http://localhost:33301/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"notifications/initialized"}' | head -10
echo ""
# brain_query smoke
curl -sS -X POST http://localhost:33301/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"x"}}}' \
| head -c 200
echo ""
kill $SERVER_PID
wait $SERVER_PID 2>/dev/null
rm -rf /tmp/brain-mcp-test
```
Expected:
- tools/list returns JSON with all 5 brain tool names.
- notifications/initialized returns HTTP/1.1 200 with `Content-Length: 0`.
- brain_query returns `{"results":null}` (empty brain).
- [ ] **Step 5: Run full ingestion test suite**
Run: `cd ingestion && go test -race -count=1 ./... 2>&1 | tail -15`
Expected: All packages PASS.
- [ ] **Step 6: Commit**
```bash
git add ingestion/cmd/server/main.go ingestion/internal/mcp/integration_test.go
git commit -m "$(cat <<'EOF'
feat(ingestion): mount MCP handler at POST /mcp
The ingestion server now exposes both REST and MCP on the same port
(3300). MCP shares brainDir, pipeline config, and LLM client with the
REST handlers — single source of process state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 9: Add NodePort service in infra repo
**Files (in `~/Documents/local-dev/AI/infra`):**
- Create: `infra/k3s/apps/supervisor/ingestion-nodeport.yaml`
- Modify: `infra/k3s/apps/supervisor/kustomization.yaml`
This is in a different repo. The branch and PR happen there. **Confirm with the user before pushing**`infra` repo changes touch live cluster routing.
- [ ] **Step 1: Create the NodePort manifest**
Read `infra/k3s/apps/supervisor/supervisor-nodeport.yaml` first to mirror its style.
```yaml
# infra/k3s/apps/supervisor/ingestion-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: ingestion-nodeport
namespace: supervisor
spec:
type: NodePort
selector:
app: ingestion
ports:
- name: http
port: 3300
targetPort: 3300
nodePort: 30330
```
- [ ] **Step 2: Add to kustomization.yaml**
Modify `infra/k3s/apps/supervisor/kustomization.yaml` resources list:
```yaml
resources:
- namespace.yaml
- deployment.yaml
- service.yaml
- secrets.enc.yaml
- supervisor-nodeport.yaml
- ingestion-deployment.yaml
- ingestion-service.yaml
- ingestion-nodeport.yaml # added
```
- [ ] **Step 3: Commit (in infra repo)**
```bash
cd /Users/mathias/Documents/local-dev/AI/infra
git checkout -b feat/ingestion-nodeport
git add k3s/apps/supervisor/ingestion-nodeport.yaml k3s/apps/supervisor/kustomization.yaml
git commit -m "$(cat <<'EOF'
feat(supervisor): expose ingestion as NodePort 30330 for direct MCP
Pairs with hyperguild's brain MCP migration — Claude Code can now
reach brain MCP at http://koala:30330/mcp without going through the
supervisor pod.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
- [ ] **Step 4: Confirm with user before push**
Pause and ask: "Push the infra change to gitea origin? Flux will reconcile within ~30s and add the NodePort service. Confirm?"
If confirmed:
```bash
cd /Users/mathias/Documents/local-dev/AI/infra
git push origin feat/ingestion-nodeport
```
(Or merge to main per your existing GitOps workflow — depends on whether feature branches are reconciled by Flux. Existing pattern is direct-to-main commits.)
- [ ] **Step 5: Verify NodePort live**
```bash
kubectl --request-timeout=10s -n supervisor get svc ingestion-nodeport 2>&1
```
Expected output includes a service with TYPE=NodePort and PORT(S) `3300:30330/TCP`.
---
## Task 10: End-to-end smoke test against deployed brain MCP
**Prerequisite:** Hyperguild commits from Tasks 1-8 are pushed to gitea origin and the new ingestion image has rolled out to the cluster (~2 minutes after push, per CD pattern).
- [ ] **Step 1: Push the hyperguild changes**
```bash
cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git push origin feat/brain-mcp-migration
```
- [ ] **Step 2: Open a PR to main, get it reviewed, merge**
(Manual step. Or fast-forward to main if reviewing solo.)
- [ ] **Step 3: Wait for image rollout**
```bash
TARGET=$(git -C /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration rev-parse HEAD)
for i in $(seq 1 30); do
CURRENT=$(kubectl --request-timeout=5s -n supervisor get deploy ingestion -o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null | sed 's/.*://')
if [ "$CURRENT" = "$TARGET" ]; then
echo "[$i] image rolled to $CURRENT"
break
fi
echo "[$i] still on ${CURRENT:0:12} — waiting 10s"
sleep 10
done
kubectl --request-timeout=8s -n supervisor get pods -l app=ingestion -o wide
```
- [ ] **Step 4: Smoke test the live endpoint**
```bash
echo "=== initialize ==="
curl -sS -X POST http://koala:30330/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}'
echo ""
echo "=== tools/list ==="
curl -sS -X POST http://koala:30330/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | head -c 600
echo ""
echo "=== brain_query (existing brain) ==="
curl -sS -X POST http://koala:30330/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"brain_query","arguments":{"query":"tdd","limit":2}}}'
echo ""
echo "=== session_log ==="
curl -sS -X POST http://koala:30330/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"session_log","arguments":{"session_id":"smoke-test","skill":"plan-execution","phase":"verify","final_status":"ok"}}}'
echo ""
```
Expected:
- initialize returns `protocolVersion: "2024-11-05"` and `serverInfo.name: "ingestion-brain"`.
- tools/list returns 5 tools.
- brain_query returns whatever's in the brain (or `{"results":null}` if empty).
- session_log returns `{"status":"ok","session_id":"smoke-test"}`.
If any of these fail: do not proceed to Task 11. Investigate the failure first.
---
## Task 11: Update .mcp.json to use brain MCP
**Files:**
- Modify: `.mcp.json` (in repo root)
The supervisor MCP entry stays for now (it still works, and the migration of skill workers is a future plan). We add `brain` as a second MCP server.
- [ ] **Step 1: Read current `.mcp.json`**
```bash
cat /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration/.mcp.json
```
Expected:
```json
{
"mcpServers": {
"supervisor": {
"type": "http",
"url": "http://koala:30320/mcp"
}
}
}
```
- [ ] **Step 2: Add brain alongside**
```json
{
"mcpServers": {
"supervisor": {
"type": "http",
"url": "http://koala:30320/mcp"
},
"brain": {
"type": "http",
"url": "http://koala:30330/mcp"
}
}
}
```
- [ ] **Step 3: User restarts Claude Code in this project**
Manual handoff to user: "Restart Claude Code. Then `/mcp` should list two servers: `supervisor` and `brain`. The `brain` server should expose 5 tools (brain_query, brain_write, brain_ingest, brain_ingest_raw, session_log). Try a `brain_query` and confirm it returns results."
- [ ] **Step 4: Commit only after user confirms**
```bash
git add .mcp.json
git commit -m "$(cat <<'EOF'
chore: add brain MCP server alongside supervisor
The brain MCP at koala:30330 hosts the brain_* and session_log tools
formerly on supervisor. Supervisor stays connected during the
transition; its skill workers and the brain duplication will be
removed in a later plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 12: Update README and CLAUDE.md
**Files:**
- Modify: `README.md`
- Modify: `CLAUDE.md` (project-level)
Document the new MCP endpoint and the dual-MCP transitional state. Keep edits surgical — this is documentation, not a marketing rewrite.
- [ ] **Step 1: Update README architecture diagram and connect-a-project section**
Read README.md first. Update the diagram to show two MCP endpoints. Update the "Connect a project" `.mcp.json` example to include both servers.
```markdown
## Connect a project
Create `.mcp.json` in your project root:
```json
{
"mcpServers": {
"supervisor": {
"type": "http",
"url": "http://koala:30320/mcp"
},
"brain": {
"type": "http",
"url": "http://koala:30330/mcp"
}
}
}
```
Two MCP servers are exposed today, both reachable over Tailscale:
- **`supervisor`** at `koala:30320` — skill workers (`tdd_*`, `review`, `debug`, `spec`, `retrospective`, `trainer`, `tier`).
- **`brain`** at `koala:30330` — knowledge access (`brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`) and `session_log`.
The `brain` MCP is hosted by the ingestion service directly. The skill workers will move out of `supervisor` in a later plan.
```
- [ ] **Step 2: Add a brief note in CLAUDE.md about the dual MCP**
Append to the "Knowledge base access" section in `CLAUDE.md`:
```markdown
- **Brain MCP**: `http://koala:30330/mcp` — preferred path for brain_query, brain_write, brain_ingest_raw, brain_ingest, session_log.
- **Supervisor MCP**: `http://koala:30320/mcp` — skill workers (tdd, review, debug, spec, retrospective, trainer) until they migrate to SKILL.md.
```
- [ ] **Step 3: Commit**
```bash
git add README.md CLAUDE.md
git commit -m "$(cat <<'EOF'
docs: document brain MCP endpoint at koala:30330
Updates the connect-a-project example and the CLAUDE.md knowledge
base section. Captures the transitional state where two MCPs coexist;
the supervisor MCP will shrink as skill workers move to SKILL.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 13: Final verification
- [ ] **Step 1: Re-run `task check` on the worktree**
```bash
cd /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
task check 2>&1 | tail -20
```
Expected: lint clean, all tests pass, vet clean, govulncheck clean.
- [ ] **Step 2: Verify both MCPs respond from this Claude Code session**
After user has restarted Claude Code:
- `/mcp` lists supervisor (connected) and brain (connected).
- Calling `brain_query` via the brain MCP returns results.
- Calling `tier` via the supervisor MCP returns tier info.
- [ ] **Step 3: Verify `brain/sessions/` has the smoke-test entry**
```bash
kubectl --request-timeout=8s -n supervisor exec deploy/ingestion -- ls /app/brain/sessions/ 2>&1 | head -5
```
Expected: includes `smoke-test.jsonl` (from Task 10 step 4).
- [ ] **Step 4: Update plan checkbox status and merge**
Mark all checkboxes complete in this plan file. Optionally squash the worktree branch into a single feature merge commit before merging to main:
```bash
cd /Users/mathias/Documents/local-dev/AI/hyperguild
git checkout main
git merge --no-ff feat/brain-mcp-migration -m "feat: brain MCP migration (extract brain + session_log into ingestion pod)"
git push origin main
```
- [ ] **Step 5: Clean up the worktree**
Use `superpowers:finishing-a-development-branch` skill, or manually:
```bash
git worktree remove /Users/mathias/Documents/local-dev/AI/hyperguild/.worktrees/feat-brain-mcp-migration
git branch -d feat/brain-mcp-migration
```
---
## Self-review
**Spec coverage:**
- ✅ MCP server skeleton (Task 1)
- ✅ session package (Task 2)
- ✅ brain_query (Task 3)
- ✅ brain_write with extracted helper (Task 4)
- ✅ brain_ingest_raw (Task 5)
- ✅ brain_ingest (Task 6)
- ✅ session_log (Task 7)
- ✅ Mount into ingestion server (Task 8)
- ✅ NodePort exposure (Task 9, separate repo)
- ✅ Live cluster verification (Task 10)
-`.mcp.json` switch (Task 11)
- ✅ Docs (Task 12)
- ✅ Final verification + cleanup (Task 13)
**Placeholder scan:** No "TBD", "implement later", or hand-wave error handling. Each step has actual code or actual commands.
**Type consistency:**
- `mcp.NewServer(brainDir string, *pipeline.Config, pipeline.CompleteFunc)` — used consistently across Tasks 1, 3-7, and 8.
- `pipeline.RawPage` shape — referenced in Task 5's test args, matches existing definition.
- `session.Entry` shape — referenced in Task 7, matches Task 2's package definition.
- `api.WriteNote(brainDir, content, filename, type, domain)` — defined Task 4, called from Task 4's MCP handler with positional args matching.
**Dependencies on out-of-repo work:**
- Task 9 modifies the `~/dev/AI/infra` repo. Pause point built in (Step 4 of Task 9).
- Task 10 depends on CD completing (image build + Flux reconcile, ~2 min).
**Reversibility:**
- All hyperguild changes are additive until Task 11. The `.mcp.json` change in Task 11 is one line, easily reverted.
- The infra-repo NodePort can be reverted by deleting the new manifest line and re-pushing.
- The supervisor MCP is untouched throughout. If brain MCP fails verification, fall back to supervisor MCP without code changes.