From 86419387bf1825ef70971d2f148b68d1519e2872 Mon Sep 17 00:00:00 2001 From: dimitar Date: Sun, 2 Aug 2026 22:55:53 +0200 Subject: [PATCH] docs: solution architecture & agent-era roadmap (solution.md) --- solution.md | 260 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 260 insertions(+) create mode 100644 solution.md diff --git a/solution.md b/solution.md new file mode 100644 index 0000000..7a01783 --- /dev/null +++ b/solution.md @@ -0,0 +1,260 @@ +# gitflow — Solution Documentation & Agent-Era Roadmap + +## 1. What gitflow is + +gitflow is a single-binary CLI agent written in Go that discovers every Git +repository under a parent directory, snapshots each one's state, and +presents the findings — on demand, on a schedule, or in an interactive +terminal UI. When an AI provider is configured, it suggests the next +concrete action for every repository that needs attention. + +It operates on the user's own machine: no daemon, no central server, no +telemetry. The full scan pipeline runs in-process with bounded concurrency. +Every external call (git porcelain commands, AI provider HTTP requests, +webhook dispatches) has a timeout and never blocks the next scan frame. + +### 1.1 Architecture at a glance + +``` +┌─────────────────────────────────────────────────────────┐ +│ CLI (cobra) — scan · watch · tui · config · completion │ +├─────────────────────────────────────────────────────────┤ +│ App (orchestration) │ +│ discover → scan (errgroup, 8 workers) → ScanResult │ +├───────────────┬───────────────┬─────────────────────────┤ +│ Presenter │ AI (agent) │ Scheduler (interval loop)│ +│ table / json │ OpenAI │ Webhooks (Slack/Discord) │ +│ compact / TUI │ Ollama/Anthropic │ Notify (desktop) │ +├───────────────┴───────────────┴─────────────────────────┤ +│ Domain model (pkg/status): RepoInfo, ScanResult, Summary │ +├─────────────────────────────────────────────────────────┤ +│ Git porcelain wrapper (internal/git) │ +│ Scanner (discovery + concurrent status) │ +├─────────────────────────────────────────────────────────┤ +│ Config (viper: flags > env > ~/.gitflow.yaml > defaults) │ +└─────────────────────────────────────────────────────────┘ +``` + +Every layer is tested in isolation (httptest for providers and webhooks, +canned porcelain output for the git parser, bubbletea model tests for TUI +state transitions). Cross-compilation is verified for linux/amd64, +darwin/arm64, and windows/amd64. + +### 1.2 What it does today + +| Capability | How | +|---|---| +| **Discover repos** | `filepath.WalkDir` with depth limits and glob exclusions; detects working trees (`.git` dir or `.git` gitfile), bare repos (`*.git` dir with HEAD/objects/refs) | +| **Scan status** | `git --porcelain=v2 --branch -z` (NUL-separated, zero quoting issues), `git stash list`, `git config --get-regexp remote.*.url`; 10 s timeout per repo | +| **Classify** | 8 states: `clean`, `modified`, `ahead`, `behind`, `diverged`, `detached`, `bare`, `error` | +| **Render** | Colourised table, indented JSON, compact one-liner; dark/light theme; `NO_COLOR` | +| **Watch** | `tea.Tick`-driven interval loop with change detection (`▲` markers, `status.Changed`), graceful SIGINT/SIGTERM | +| **Notify** | `notify-send -t 30000 -u critical` (persist-until-clicked, 30 s cap) or macOS `osascript display notification` | +| **AI suggest** | System prompt + status table → structured JSON; three providers (OpenAI, Ollama, Anthropic); priority-coloured output | +| **AI execute** | Allowlist-guarded command execution with per-command y/N confirmation | +| **Custom rules** | Threshold evaluator: `field op value` → label (e.g. `behind >= 10 → stale`) | +| **Webhooks** | Slack (Block Kit), Discord (embeds), generic JSON POST; per-webhook rate-limit + retry+backoff | +| **TUI** | bubbletea + lipgloss: repo list with cursor, detail pane, AI panel, countdown status bar | +| **Completions** | bash, zsh, fish, PowerShell via cobra generators | + +### 1.3 Quality posture + +- 13 packages, all tested with `go test -race` +- 77–91% line coverage across domain packages +- gofmt + go vet + golangci-lint in CI (Go 1.24 / 1.26 matrix) +- Functional-options constructors, no `init()`, enum zero values as + `Unknown`, `defer Close()` immediately after open + +--- + +## 2. The agent era — from scanner to collaborator + +The current tool does one thing well: **observe and report**. You run it, it +tells you what's happening, and — when AI is enabled — it suggests what to +do next. This is the "Level 1" of an agent: *a contextual advisor*. + +Level 2 is **an autonomous participant** that can act within guardrails +(like `--ai-execute` but with richer context and a persistent memory of +what it did across scans). + +Level 3 is **a team-level orchestrator** that understands what every +developer is working on, cross-references repositories, integrates with +issue trackers and CI, and proactively drives work forward — essentially a +"repo ops agent" that lives alongside the team. + +The following sections map the concrete features that bridge from today to +that vision. + +--- + +## 3. Extended features — the agent-era roadmap + +### 3.1 Agent memory & statefulness + +Today each scan is stateless. The change-detection map lives only for the +duration of the watch session. An agent needs memory. + +| Feature | Description | +|---|---| +| **Scan journal** | Persist each scan result (SQLite or flat JSONL) with a monotonic scan ID. Enables historic diffs: "what changed between Tuesday and today?" | +| **Repo identity fingerprint** | Hash `remote.origin.url + branch` to track the same logical repo across renames and directory moves. De-duplicate when the same repo appears at multiple paths (due to symlinks, mounts, or multiple clones). | +| **Action log** | Record every AI suggestion that was accepted or executed and its outcome (did the `git pull` succeed? Did the `commit` pass CI?). Feed this back into the prompt so the agent learns what worked. | +| **Dwell-time thresholds** | Flag a repo only after it has been `modified` for N scans (e.g. "dirty for 3 hours"). Suppress noise from transient in-progress work. | +| **Session resumption** | When watch restarts, reload the previous scan journal and continue from where it left off instead of starting from zero. | + +### 3.2 Autonomous actions (Level 2 agent) + +Today `--ai-execute` asks for a y/N confirmation per command. An agent +should be able to act within a declared policy — you tell it *what you're +comfortable with*, and it stays inside that boundary. + +| Feature | Description | +|---|---| +| **Policy files** | `.gitflow-policy.yaml` per repo or per directory tree: `auto_pull: true`, `auto_stash: true`, `max_push_commits: 5`, `require_ci_green: true`. The agent checks the policy before acting. | +| **Pre-flight hooks** | Before executing a suggested command, run a user-defined script (`pre-pull.sh`) that can abort. Example: "don't pull if the VPN is down" or "don't commit if the test suite is red". | +| **Post-action hooks** | After a successful action, trigger a script: rebuild, re-run tests, notify the team. | +| **Dry-run mode** | `--ai-execute --dry-run` prints every command it *would* run without executing anything. Useful for trust-building and policy tuning. | +| **Action batching** | Group suggestions across repos (e.g. "pull main on these 5 repos" as a single user confirmation). | +| **Rollback** | If an action fails, revert to the previous state (e.g. `git stash pop` if a pull created a merge conflict). Record the rollback in the action log. | + +### 3.3 Multi-repository operations + +Today each repo is scanned in isolation. Many teams manage dozens or +hundreds of microservices in a monorepo-adjacent layout. + +| Feature | Description | +|---|---| +| **Cross-repo dependency view** | Parse `go.mod`, `package.json`, `Cargo.toml`, `requirements.txt` to build a dependency graph. Flag downstream repos when an upstream dependency has changed. | +| **Consistent-branch operations** | "Create a `feat/oauth-update` branch on all 12 service repos that import the auth library." The agent creates the branches, updates the dependency, and opens PRs. | +| **Cross-cutting search** | `gitflow grep "deprecated-call" --all-repos` — search across every repo with a single command. | +| **Bulk status dashboards** | A grouped view by team, by project, by technology stack. Surface the "health of the fleet" at a glance. | + +### 3.4 CI & issue-tracker integration + +An agent that bridges the gap between "I saw this problem" and "I opened a +ticket for it" closes the loop. + +| Feature | Description | +|---|---| +| **Auto-PR creation** | When AI suggests `push` and the policy allows it, create the PR automatically with a body that includes the scan context and the AI's reasoning. | +| **CI status overlay** | Pull the latest CI status for each repo's current branch and show it in the scan table as an extra column (✅ CI green, ❌ CI red, ⏳ running). | +| **Issue auto-link** | When a repo is in `diverged` or `behind` state for N days, automatically open a GitHub/GitLab/Jira issue assigned to the last committer. | +| **Release-train awareness** | Know the release schedule: "main is frozen until Friday" → suppress push suggestions, flag repos that merged after the cut. | + +### 3.5 Natural-language interface + +Today you invoke `gitflow scan` and optionally `--ai` for suggestions. The +next step is conversational. + +| Feature | Description | +|---|---| +| **Chat mode** | `gitflow chat` opens a REPL: "what's stale right now?", "create a branch on all go services", "summarise last week's changes across the monorepo". The AI interprets intent, calls the appropriate scan/action functions, and responds. | +| **Contextual reasoning** | The agent remembers the last scan and can answer follow-ups: "tell me more about the web repo" without re-scanning. | +| **Voice / assistant integration** | Expose an MCP (Model Context Protocol) server so tools like Claude Code, Copilot Chat, or a custom Slack bot can invoke gitflow as a tool. The agent becomes a sub-agent of a larger workflow. | +| **Summarisation** | "What happened this week?" → the agent scans the journal and produces a human-readable summary with trends, anomalies, and top risks. | + +### 3.6 Team coordination & multi-user awareness + +Today gitflow is a single-user tool. Multi-user scenarios need coordination +and shared state. + +| Feature | Description | +|---|---| +| **Shared scan journal** | A central (optional) server or a shared SQLite file on a network mount so the team sees the same view. | +| **Conflict prediction** | If two developers are working on the same file in different branches, flag it *before* it becomes a merge conflict. | +| **Review-load balancing** | "Which team member has the fewest open reviews?" — suggest assigning PRs to balance the load. | +| **Standup summaries** | `gitflow standup` — "here's what changed since yesterday, here's what needs attention today, here's the AI-suggested priority order." | + +### 3.7 Operational resilience & security + +| Feature | Description | +|---|---| +| **Secret detection** | Scan staged diffs for accidentally-committed keys, tokens, or credentials. Flag as `critical` and prevent the commit in policy-enforcement mode. | +| **Dependency vulnerability overlay** | Cross-reference `go.mod` / `package.json` versions with known CVEs. Surface repos running vulnerable dependencies. | +| **Disk-space forecasting** | Track `.git` directory growth over time; flag repos approaching a size threshold where `git gc` or shallow cloning would help. | +| **Signed-commit enforcement** | Flag repos and branches where unsigned commits have landed. | +| **Branch-protection audit** | Verify that protected branches (main/master) have required reviews, status checks, and signed-commit policies enabled on the remote. | + +### 3.8 Platform & ecosystem + +| Feature | Description | +|---|---| +| **MCP server mode** | `gitflow serve --mcp` exposes a Model Context Protocol endpoint so any MCP-compatible client (Claude Desktop, Continue.dev, etc.) can use gitflow as a tool. | +| **Prometheus metrics endpoint** | `gitflow serve --metrics :9090` exports scan summaries, repo counts, attention ratios, webhook latencies. Grafana dashboard out of the box. | +| **Kubernetes CronJob deployment** | Run gitflow as a periodic job in a cluster, scanning repos cloned into a shared volume. Push metrics to Prometheus and alerts to Slack. | +| **Systemd timer unit** | `gitflow watch` as a user-level systemd service with a timer, so it survives reboots. | +| **VSCode / JetBrains extension** | Sidebar panel showing the scan status of the current project's dependency repos, with one-click actions (pull, stash, create PR). | + +--- + +## 4. From where we are to where we're going + +``` + gitflow today + ┌────────┐ ┌──────────────────────────────────┐ + │ scan │──────────▶ table / json / compact │ + │ watch │──────────▶ ▲ change markers │ + │ tui │──────────▶ AI suggestions (on demand) │ + └────────┘ └──────────────────────────────────┘ + + gitflow agent-era + ┌────────────┐ ┌──────────────────────────────────┐ + │ scan │ │ journal (SQLite) │ + │ watch │───────│ policy engine │ + │ tui │ │ chat REPL │ + │ chat │ │ MCP server │ + │ serve │ │ Prometheus metrics │ + │ standup │ │ CI / issue-tracker bridges │ + │ grep │ │ dependency graph │ + └────────────┘ │ cross-repo operations │ + │ team coordination │ + └──────────────────────────────────┘ +``` + +The current codebase is structured so that each of these extensions drops +in cleanly: + +- **Journal** → new `internal/journal` package with a `Journal` interface + (in-memory for tests, SQLite for production); `app.ScanOnce` writes to it +- **Policy** → `internal/policy` evaluates a `.gitflow-policy.yaml` per + repo against the scan result and the AI suggestion before execution +- **Chat** → `internal/chat` reuses the AI provider interface and adds a + REPL loop with conversation history +- **MCP** → `internal/mcp` exposes the existing `app.ScanOnce`, `ai.Suggest`, + and `rules.Eval` as MCP tools over stdio or HTTP +- **CI bridge** → `internal/ci` with a provider interface (GitHub Actions, + GitLab CI, Jenkins) that fetches status per branch +- **Dependency graph** → `internal/deps` parser registry that reads + `go.mod`, `package.json`, etc. and builds a `DepGraph` struct + +None of these require restructuring the core domain model — `RepoInfo`, +`ScanResult`, `Summary`, and `Suggestion` are already general enough to +carry the extra metadata these features would attach. + +--- + +## 5. Immediate next steps (near-term, low-risk) + +These can be implemented in the current architecture without any +restructuring and without breaking the existing CLI contract: + +1. **Scan journal (SQLite)** — adds persistence to watch mode; enables + `gitflow history` and `gitflow diff SCAN1 SCAN2` +2. **Dwell-time rules** — extend `rules` with a `scans:` matcher ("flag + only after N consecutive scans"), config-only change +3. **Cross-repo grep** — `gitflow grep ` that walks discovered + repos and runs `git grep` concurrently, presents unified output +4. **Auto-PR** — `--ai-execute` creates a PR via `gh pr create` or the + GitHub API when the suggestion is `create_pr` and a token is present +5. **`gitflow chat`** — a REPL that holds the last scan result in memory + and answers natural-language questions about it + +These five items together would move gitflow from a "scanner with AI +suggestions" to a "repo management agent" without a single breaking change +to the existing surface. Each one is independently shippable. + +--- + +*Document version: 1.0 — reflects the codebase at commit series +`feat/tui-webhooks-m1` through `feat/tui-webhooks-m6`, merged into `main` +with `--no-ff`.*