# Scalegion — Autonomous Agent System & Repository Architecture

This document serves as the authoritative guide for AI coding agents working on the Scalegion platform.

---

## 1. Multi-Backend 3-Instance Architecture

Scalegion decouples social operations, marketing scheduling, and autonomous code development across three dedicated backend instances:

```
┌────────────────────────────────────────────────────────┐
│             FRONTEND UI (Vercel CD)                    │
│  - React 19 + TypeScript + Vite                        │
│  - Real-time Dashboards, Task Kanban, Social Hub       │
└───────────────────────────┬────────────────────────────┘
                            │
       ┌────────────────────┼────────────────────┐
       ▼                    ▼                    ▼
┌──────────────┐   ┌─────────────────┐   ┌───────────────────────────┐
│  INSTANCE 1  │   │   INSTANCE 2    │   │        INSTANCE 3         │
│  SOCIAL HUB  │   │ BACKOFFICE &    │   │    AUTONOMOUS AGENT-      │
│  PUBLISHING  │   │ SCHEDULER API   │   │     DEVELOPER WORKER      │
│              │   │ api.scalegion.com │   │        2.29.4.203         │
│ (Postiz SaaS/│   │ 168.119.53.183  │   │  (8GB ARM64 Hetzner Host) │
│  Self-hosted)│   │  /backoffice/   │   │      /agent-worker/       │
└──────────────┘   └────────┬────────┘   └─────────────┬─────────────┘
                            │                          │
                            │ Task Queue & Issues API  │
                            └──────────────────────────┘
```

### The 3 Dedicated Backend Roles:

1. **Instance 1: Social Hub Instance (Postiz / Publishing)**
   - Dedicated service for social network integrations (X/Twitter, LinkedIn, Instagram, Facebook, TikTok, YouTube).
   - Manages OAuth channel bindings, rate-limited content dispatch, and network-specific asset transformations.

2. **Instance 2: Backoffice Scheduler & Ideation Hub (`api.scalegion.com` / `168.119.53.183`)**
   - Express monolith (`server.js`) + MySQL + `node-cron`.
   - Ideation Pipeline (Harvester -> Classifier -> LLM) & weekly content batch engine.
   - Central Task Queue, APM Error Ingestion (`/api/track-error`), and Agent Dispatch API (`/api/agent/issues`).
   - Executive Memory ("Business Brain" per project).
   - Deploy script: `./deploy_backoffice.sh` (or `./deploy_backoffice_api.sh`).

3. **Instance 3: Autonomous Agent-Developer Worker (`2.29.4.203`)**
   - Dedicated Hetzner 8GB host (`aarch64` Ampere Altra).
   - Base OS: 8GB NVMe Swap + Docker Engine + VM memory tunings.
   - Project-Dedicated Persistent Sandboxes: `/opt/agent-runner/workspaces/<project-slug>`.
   - Single-Flight Project-Affinity Queue Daemon (`worker_daemon.js`).
   - Headless Antigravity CLI Execution Engine.
   - cgroup resource limits (1.8 CPU / 3.5GB RAM).
   - Fast Git Scrub & Automated GitHub / Bitbucket PR Delivery.
   - Deploy script: `./deploy_agent_worker.sh` (or `./agent-worker/deploy_worker.sh`).

---

## 2. Directory Structure & Separation of Concerns

* **`backoffice/`** — The Control Plane and Marketing API.
  * `server.js` — API gateway for projects, social posts, tasks, and executive memory.
  * `ideation/` — Trend scraping, news classification, and post brief assembly.
  * `services/` — Connectors for GA4, GSC, Ahrefs, Stripe, Meta Ads, Postiz, and Video pipelines.
  * `migrations/` — Plain JavaScript sequential database migrations.
  * `apply_schema_sql.py` — Database schema migrator (formerly `run.py`).
  * `test_db_connection.js` — Quick database connectivity test.

* **`agent-worker/`** — The Dedicated Autonomous Code Development Worker.
  * `Dockerfile` — Base container image (`devagent-base:latest`) equipped with Node.js, Python, Git, `gh`, and Antigravity runner.
  * `setup_worker_host.sh` — Bootstraps swap, sysctl memory tunings, and directories on the Hetzner worker.
  * `run_task.sh` — Runs tasks inside isolated Docker containers with cgroup caps, base branch detection, soft git scrub, verification loop, and `gh pr create` (GitHub) or Bitbucket REST API pull requests.
  * `worker_daemon.js` — Single-flight, project-affinity polling coordinator.
  * `deploy_worker.sh` — Deployment script syncing `agent-worker/` to the worker instance.

* **`features/` & `components/`** — Client-side React application.
  * `features/TasksView.tsx` — Operational task board and APM error manager.
  * `features/SocialHubView.tsx` — Automated marketing campaigns, weekly batch calendar.
  * `features/ProjectSettingsView.tsx` — GitHub and Bitbucket repository integration and project settings.
  * `components/AgentChat.tsx` — Business agent chat that learns and persists executive memory.

---

## 3. Worker Execution Conventions & Guardrails

1. **Never Push Directly to `main` (in PR Mode):**
   Automated development tasks execute on an isolated branch (`agent/<task_id>`) and either auto-deploy to the target branch upon passing verification or open a Pull Request (via GitHub CLI `gh pr create` or Bitbucket REST API).
2. **Preserve Warm Dependencies:**
   Between tasks, clean the workspace using:
   ```bash
   git clean -fd -e node_modules -e .venv -e target -e .cache
   ```
   Never delete dependency trees between tasks within the same project.
3. **Dynamic Verification:**
   Verify code changes using the repository's native verification command (`npm test`, `npm run build`, or `pytest`).
4. **Project Affinity:**
   The worker daemon drains all pending tasks for Project A before switching context to Project B to maximize AST and build cache hits.
5. **Inactivity Hibernation:**
   Containers idle for > 30 minutes are automatically stopped to reclaim host RAM while preserving volume state on disk.
6. **Pre-Cost Quota Estimation Guardrail:**
   Before claiming batch dev tasks, the worker daemon queries `/api/projects/:id/check-quota` with the estimated task cost ($0.15–$0.40). If the project is at or over its plan quota, the worker refuses to claim the task and flags it for human review.

---

## 4. Script & Utility Index

| New Script | Previous Name | Description |
|---|---|---|
| `deploy_backoffice.sh` / `deploy_backoffice_api.sh` | `deploy_backoffice.sh` | Deploys Control Plane / Social Hub / Ideation API to Hetzner VPS (`168.119.53.183`). |
| `deploy_agent_worker.sh` | *New* | Deploys the Development Agent Worker to the dedicated Hetzner worker host. |
| `query_projects_db.py` | `run_db_query.py` | Queries `projects` table in production MySQL database. |
| `query_recent_tasks.sh` | `run_db_query.sh` | Queries recent `tasks` in production MySQL database. |
| `ssh_query_vps_tasks.exp` | `testdb.exp` | Queries database tasks remotely on the API VPS. |
| `ssh_run_script_on_vps.exp` | `do_ssh.exp` | Copies and runs a diagnostic Python script on the API VPS. |
| `backoffice/apply_schema_sql.py` | `backoffice/run.py` | Applies `schema.sql` to the MySQL database. |
| `backoffice/test_db_connection.js` | `backoffice/testdb.js` | Verifies MySQL connection from the backoffice. |

---

## 5. Executive Memory ("Business Brain")

Scalegion maintains a persistent memory store (`project_memories` table) for each business tenant:
* **Memory Types:** `ICP`, `Objective`, `Product`, `Vertical`, `Other`.
* **Endpoints:** `GET /api/projects/:id/memories`, `POST /api/projects/:id/memories`, `DELETE /api/projects/:id/memories/:memoryId`.
* **Prompt Injection:** Hydrated into every AI agent context under `EXECUTIVE MEMORY / WHO WE ARE:` to ensure hyper-tailored strategies and code changes.

---

## 6. Deployment & Git Synchronization Rules

1. **Commit and Push is Mandatory**:
   Every code change, feature update, and deployment task MUST be committed (`git commit`) and pushed (`git push origin main`) to GitHub. Vercel CD deployments trigger strictly off GitHub pushes; unpushed local changes will NEVER reach production or preview environments.
2. **Post-Deployment Verification**:
   After triggering a push or running backend deploy scripts (`./deploy_backoffice.sh`), verify that the git working directory is clean (`git status`) and origin is up to date. Never leave working tree changes uncommitted after completing a task.

---

## 7. Economics Governance & 10× Average Profit Floor (Revenue ≥ 10× AI COGS)

Scalegion enforces a strict founder requirement: **valid 10× profits minimum on average vs AI expenses** (`Platform Revenue ≥ 10 × AI COGS`, meaning AI COGS must never exceed 10% of recognized revenue).

### 1. Per-Plan Monthly AI Budgets
Budgets are derived directly from customer subscription prices:
* **Free / Scout Tier**: $1.50 lifetime trial cap.
* **Solo ($39/mo)**: Max AI COGS **$3.90/mo** per seat.
* **Growth ($119/mo)**: Max AI COGS **$11.90/mo**.
* **Scale ($299/mo)**: Max AI COGS **$29.90/mo**.
* **Enterprise**: Custom negotiated SLA (default 10% of monthly contract value).

### 2. Quota Thresholds & Throttling
* **70% Budget Utilization (Soft Warning)**: Visual yellow banner displayed on Project Settings and Admin Costs; non-essential speculative background jobs deprioritized.
* **100% Budget Utilization (Hard Throttle)**: Non-critical background AI calls (ideation harvesting, automated blog generation, autonomous code development) are throttled with HTTP 429 until the next billing cycle or quota top-up. Critical interactive user requests prompt the user to upgrade or add credits.

### 3. Autonomous Agent-Worker Pre-Cost Verification
* Prior to dispatching or executing autonomous dev tasks, the worker daemon estimates the task cost ($0.15–$0.40 depending on expected turns).
* If the project has exceeded its quota (or if the kill-switch is active), the daemon skips task execution, leaves the task in `pending`, flags `requires_approval = 1`, and generates an alert.

### 4. Platform-Level Rolling 30d Metric & Global Kill-Switch
* The Admin Costs dashboard (`/admin/costs`) tracks rolling 30-day recognized revenue vs AI COGS and displays the implied multiple. If the multiple falls below 10×, a red margin warning is flagged.
* **Emergency Kill-Switch**: If the rolling 30-day revenue-to-AI-cost multiple drops below **8×** (i.e. AI COGS > 12.5% of revenue) for longer than 48 consecutive hours, the platform activates the Global Kill-Switch. Nonessential background batch jobs are paused system-wide until founder/admin review or override.

---

## 8. CEO Dogfood Ops & Multi-Tenant Boundary

Founder-locked scope (2026-09-19):
* **Scalegion CEO Scope:** The internal Scalegion CEO autonomous agent and hourly loop (`runHourlyCeoOpsLoop` in `backoffice/services/ceoOps.js`) manage Scalegion's own dogfood project (`proj_1777457741023`) ONLY.
* **Tenant Isolation:** The CEO ops loop never queues, claims, mutates, or runs tasks for any other customer project or external tenant.
* **External Product & Tenant Interaction:** External products, customer tenants, and third-party agents interact exclusively through public unauthenticated endpoints:
  - Project-scoped feedback: `POST /api/projects/:projectId/feedback` (or `POST /api/error-proxy`)
  - Project-scoped error tracking: `POST /api/track-error`
  - Project-scoped dev issues: `GET /api/agent/issues?project=<project_id>&status=todo` (supports `type=code`)
  - Tenant claim-next endpoint: `POST /api/projects/:projectId/tasks/claim-next` (Atomically claims next pending automation=1 task for the project without manual ID specification)
  - Queue metrics: `GET /api/agent/metrics` (Latency, time-in-todo averages, and tenant queue breakdown)
* **Hourly Loop Core Duties:**
  1. List open dogfood tasks for `proj_1777457741023`.
  2. File new feedback/issues when product or growth is blocked.
  3. Prioritize high-intent traffic and organic attention without low-quality directory spray (directories produce ~0 traffic and are deprioritized or flagged).
  4. Never queue tasks or actions for other tenants.
* **Project Filtering:**
  - Admin & Agent API endpoints support strict project filtering:
    - `GET /api/projects/:projectId/tasks` (admin task list for project)
    - `GET /api/tasks?project_id=<id>&status=<status>&type=<type>`
    - `GET /api/agent/issues?project=<id>&status=todo` (supports `type=code`)
    - `POST /api/projects/:projectId/tasks/claim-next` (Single-flight claim next task)
    - `GET /api/agent/outreach/tasks?project=<id>`

* **Email Marketing AI Addon (`email_marketing_ai`, $29/mo Entitlement):**
  - All email endpoints (events, contacts CRUD, campaigns, stats) are gated with HTTP 402 if the project does not have the `email_marketing_ai` addon active in `advisor_addons`.
  - Addons Management: `GET /api/projects/:id/addons`, `POST /api/projects/:id/addons/:advisorId/activate` (or enable), `POST /api/projects/:id/addons/:advisorId/deactivate`, `POST /api/projects/:id/addons/:advisorId/toggle`.
  - Addon status: `GET /api/projects/:id/email/addon-status`
  - Addon enable: `POST /api/projects/:id/email/addon/enable` with `{ "path": "standard" | "dev_task" }`. Path `dev_task` automatically creates an AI Developer code task (`automation: 1`, `type: "code"`) to implement integration into the tenant's codebase.
  - Events ingestion: `POST /api/projects/:id/email/events` with `{ "event": "user.registered", "email": "...", "name": "...", "properties": { ... } }`. Upserts contact, executes matching trigger campaigns, logs messages with delivery status into `email_messages`. Query events via `GET /api/projects/:id/email/events`.
  - Contacts CRUD: `GET`, `POST`, `PATCH`, `PUT`, `DELETE`, `/batch`, `/:contactId/consent`, `/:contactId/resend-confirmation`.
  - Stats & Telemetry: `GET /api/projects/:id/email/stats` (overview alias), `GET /api/projects/:id/email/stats/overview`, `GET /api/projects/:id/email/stats/campaigns/:id`, `GET /api/projects/:id/email/messages`.

* **Question & Executive Memory Tasks:**
  - Task type `question`: Used when autonomous dev or advisory agents need executive founder decision, clarification, or direction.
  - Required fields: `question_text`, `answer_text`, `memory_write`.
  - Task List UI: Shows prominent `❓ Needs answer` badge (vs `👤 Needs work` or `🤖 Auto`).
  - Task Drawer: Features prominent Answer box with Submit button that marks the task `done` and writes the decision to Project Executive Memory (`POST /api/projects/:id/memories` with `type: "Decision" | "Note"`, `source: "task:{taskId}"`).
  - Notifications & Deep-links: All assignment and review emails for questions and tasks must include a prominent project header, project URL, question text, and deep link directly to `/project/{projectId}/tasks?task={taskId}` (secondary link to `/project/{projectId}`).

* **Automated Health Probes & Canaries:**
  - Automated probes from CEO monitor / hourly ops (e.g. `probe — ignore`, `health probe`, `safe to close/cancel`) verify telemetry, queue responsiveness, and gating logic.
  - Probes and canaries testing commit-hash gates and metric gates strictly prove rejection (HTTP 400). Code tasks (`type: "code"` or `automation: 1`) MUST NOT be marked `review` or `done` without a verified non-empty `commit_hash` (or `commit_hashes`), or an explicit human override flag (`human_override: true`). Runner must not move tasks to review on no-op / session-without-commits and must auto-release such sessions to `todo` with an error note.
  - Auto-Unclaim Policy: The runner must push commits or fail-closed and release claims within session limits. Code tasks remaining `in_progress` with null/empty `commit_hash` for > 60 minutes are automatically unclaimed by the control plane watchdog and runner daemon back to `todo` with an explanatory note to prevent multi-hour empty claims.
  - Discard/cancellation of probe spam is supported via `status: "cancelled"` or `status: "deleted"` with `discard_probe: true` or `hard_discard: true`.

---

## 9. Metric-Gated Acceptance for Monetization, KPI, and Metric Tasks

To protect platform and product integrity, commit-alone completion is strictly forbidden for metric-sensitive tasks. Closing code without proving that live metrics moved constitutes a false success.

1. **Gate Activation**:
   - Tasks with metadata flag `acceptance.metric_gate` or type/tags matching `monetization`, `kpi`, or `seo_metric` are metric-gated.
2. **Rejection Rules (HTTP 400)**:
   - **Missing Verification Payload**: Any attempt to mark a gated task `done` via `PATCH /api/agent/issues/:id` or `PUT /api/tasks/:id` without a verification payload is rejected with HTTP 400.
   - **No Metric Change (`after == before`)**: A commit that produces no metric change is rejected with HTTP 400. `after` must demonstrably improve or move compared to `before`.
   - **Missing Commit Hash**: Standard commit verification remains mandatory in addition to the metric delta (`commit_hash` or `commit_hashes` required for code/automation tasks on `review` and `done` unless `human_override` is explicitly set).
3. **Structured Verification Format**:
   ```json
   {
     "metric": "none_direct_stores",
     "before": 3900,
     "after": 3200,
     "source_url": "https://cash-it.com/admin/stores?filter=none_direct",
     "checked_at": "2026-09-21T12:00:00Z"
   }
   ```
4. **Autonomous Developer Obligations**:
   - The AI Developer must attach the structured verification object under `verification` or within resolution notes.
   - Unresolved merchant/external blockers (e.g. Lemon Squeezy store publishing, external KYC) or tasks whose metrics have not yet moved MUST stay open (`todo`/`review`) and must NOT be marked `done`.
   - Cash-it Wave 20/21 P0s (`51c1673c`, `a94c6fd1`, `975d50cd`) and related batches must remain open (`todo`) until live production metrics demonstrably improve. Specifically, metric P0 `975d50cd` must stay open until `unprocessed_views` moves (while AI blocked=true, ~10.73M flat); moving to review/done without verified improvement is strictly rejected.

---

## 10. External-Blocker Gate, Lemon Merchant Publish, and Dogfood Scope Integrity

1. **Metric/External-Blocker Gate (External HTTP 200)**:
   - Code Criticals whose acceptance criteria require external HTTP 200 (such as Lemon buy URLs `/buy/legion-solo`, `/buy/legion-growth`, `/buy/launch-sprint`, or store root not 403) MUST NOT move to `done` or `review` unless verification payload proves those URLs are HTTP 200, OR explicit founder `human_override: true`.
   - Unrelated `commit_hash` alone is strictly insufficient when acceptance metrics still fail (store 403 / buy 404). Attempting to close or move to review without HTTP 200 verification rejects with HTTP 400.
2. **Founder/Merchant-Only Publish Tasks**:
   - The runner daemon must not claim founder/merchant-only Lemon publish tasks for code sessions; leave `todo` until the founder publishes live checkout (HTTP 200). If claimed in error, control plane watchdog and runner daemon auto-unclaim them back to `todo`.
3. **Dogfood Project Scope (`proj_1777457741023`)**:
   - The Scalegion dogfood project (`proj_1777457741023`) manages Scalegion tasks and organic distribution only.
   - Foreign-project SEVs (such as Cash-it) must not be filed or claimed on dogfood `proj_1777457741023`. Filing or claiming rejects with HTTP 400.
4. **Resolution Notes & Commit Integrity**:
   - Any gate change requires a non-empty `commit_hash` (unless human override) and a non-empty `resolution_note` containing before/after probe evidence.