# MI AI Framework — Developer Reference Guide > Complete reference for teams building on the MI AI stack. This JSON endpoint is designed for AI agent and programmatic consumption. **Version:** 3.22.1 **Generated:** 2026-07-05T11:54:27.465Z **Canonical URL:** https://manifest.miglobal.com.mx/stack --- ## Table of Contents - [Overview](#overview) - [Technology Stack](#tech-stack) - [Project Structure](#project-structure) - [User Management & RBAC](#user-management) - [Database Schema (70+ tables)](#database-schema) - [Internationalization (i18n)](#i18n) - [File Storage (S3/MinIO)](#file-storage) - [UI Components & Patterns](#components) - [Document Viewers & Editors](#document-viewers) - [App Shell & Navigation](#app-shell) - [Bug & Improvement System](#bug-system) - [TLI AI Chat System](#tli-system) - [Testing & Observability](#testing) - [Android APK (WebView + Bluetooth Printing)](#android) - [Claude Code Agents, Skills, Slash Commands & MCP](#agents) - [How to Implement Agents, Skills, Commands & Hooks](#self-improvement) - [The Required Workflow: DEV → /approved → Production](#git-workflow) - [Environments & Infrastructure](#environments) - [CI/CD & Production Deployment](#cicd) - [Common Development Commands](#common-commands) - [Starter Template](#starter-template) - [External API v1 & API Key Management](#api-management) --- ## Overview MI AI Framework is a full-stack TypeScript application for logistics and warehouse management (pallets, shipments, containers, billing, fleet). It is a monolithic Express + React app with PostgreSQL, deployed via PM2 behind Nginx with Let's Encrypt SSL. The app supports bilingual (EN / es-MX) UI, role-based access, real-time scanning workflows, and an AI chat system (TLI). ## Technology Stack | Layer | Technologies | | --- | --- | | Frontend | React 18, TypeScript 5, Vite 5, Tailwind CSS 3, shadcn/ui (Radix primitives), wouter (router), TanStack React Query, react-i18next, Zustand (minimal), Framer Motion | | Backend | Express.js 4, TypeScript, Drizzle ORM, Passport.js (session auth), connect-pg-simple, Zod validation, Multer (uploads), node-cron | | Database | PostgreSQL 16 (localhost:5432, user: master, password: master). DEV db: mi_ai_manifest_dev, PROD db: mi_ai_manifest | | Storage | MinIO on QNAP NAS (S3-compatible, bucket: mi-ai-manifest). Region: us-west-1 | | Email | AWS SES (us-east-1). V1 SESClient for internal, V2 SESv2Client for customer-facing. MJML + Handlebars templates. | | AI | Anthropic Claude SDK (chat, analysis, artifact generation) | | Testing | Vitest (unit/integration), Playwright (E2E via MCP server, DISPLAY=:99) | | Observability | Sentry (@sentry/node + @sentry/react), opt-in via SENTRY_DSN env var | | API Docs | Scalar interactive explorer at /api/docs (OpenAPI 3.1, hand-maintained in server/docs/openapi.ts) | | Infrastructure | PM2 process manager, Nginx reverse proxy, Let's Encrypt SSL, GitHub Actions CI/CD | | Mobile | Android WebView wrapper with LPAPI SDK for Bluetooth label printing (DP26s printer) | | PDF | jsPDF, pdfkit, pdf-lib, JsBarcode, QRCode | | Excel | xlsx (SheetJS) | | Scanning | quagga (barcode), html5-qrcode, react-qr-code | ## Project Structure ``` mi-ai-manifest/ ├── client/src/ # React frontend │ ├── components/ # 100+ UI components │ ├── lib/ # queryClient, utils, PDF generators │ ├── hooks/ # React hooks (useColumnPreferences, useDebounce, usePermissions, useIsMobile) │ ├── pages/ # Top-level page components │ └── i18n/config.ts # i18n configuration ├── client/public/locales/ # Translation JSON files (en/, es-MX/) — 23 namespaces per language ├── server/ │ ├── index.ts # Server entry point (Express + Passport + session) │ ├── routes.ts # Main API routes (~14k lines) │ ├── routes/ # Extracted sub-routers (tliRoutes, publicTracking, publicPreferences, publicStack) │ ├── dbStorage.ts # Database access layer (~12.6k lines, IStorage interface) │ ├── db.ts # PostgreSQL connection (pg driver) │ ├── s3Storage.ts # S3/MinIO integration │ └── docs/openapi.ts # OpenAPI 3.1 spec ├── shared/ │ └── schema.ts # Drizzle ORM schema (~5k lines, 70+ tables) ├── tests/ # Vitest test files ├── migrations/ # SQL migration files (applied in order) ├── android-print-middleware/ # Android WebView wrapper └── dist/ # Build output ``` **Path Aliases** (vite.config.ts): `@/` → `client/src/`, `@shared/` → `shared/`, `@assets/` → `attached_assets/` ## User Management & RBAC ### Roles - **admin**: Full system access — user management, settings, all CRUD, RBAC configuration - **supervisor**: Extended permissions — reporting, manage all records, delete pallets - **logistics**: Shipment and container management - **customer**: View-only access filtered by assigned customers (via user_customer_access junction table) - **user**: Basic pallet scanning, can only manage own pallets ### Authentication - Session-based via Passport.js with bcrypt password hashing - Sessions stored in PostgreSQL via connect-pg-simple - Cookie: connect.sid, httpOnly, secure, sameSite: lax, 7-day expiration - All endpoints require auth except /api/login and public routes ### Backend Middleware - `requireAuth` — all authenticated endpoints - `requireAdmin` — admin-only operations - `requireSupervisor` — supervisor+ operations - `requireUserOrAbove` — user, supervisor, admin ### Frontend Permission System - `usePermissions()` hook returns `{ isAdmin, isSupervisor, canEdit, canDelete }` - `` / `` wrapper components for conditional rendering ### API Endpoints - POST /api/auth/login — username + password - POST /api/auth/logout — destroy session - GET /api/auth/me — current session user - GET/POST /api/users — list / create - GET/PUT/DELETE /api/users/:id — read / update / delete - PUT /api/users/:id/language — update language preference ## Database Schema (70+ tables) Schema-first approach: Drizzle ORM generates migrations from shared/schema.ts. Uses createInsertSchema/createSelectSchema for automatic Zod validation. ### Core Tables by Domain | Domain | Key Tables | |--------|-----------| | Auth/Users | users (roles: admin, supervisor, logistics, customer, user) | | Customers | customers, customerContacts, customerAddresses, customerItemFields, customerImportTemplates | | Shipments | shipments, shipmentDocuments, shipmentStops, stopPalletAssignments, shipmentTrackers | | Containers | containers, containerItems, containerDocuments | | Pallets | pallets, scannedItems, palletWarehouseHistory, palletOwnershipHistory | | Status Config | shipmentStatuses, containerStatuses + history tables | | Fleet | drivers, vehicles, vehicleLicensePlates, maintenanceRecords, shipmentFleetAssignments | | Billing | bills, billLineItems, payments, billDisputes, billingCompanies, billingAuditLog | | Freight/Carriers | freightCompanies, carriers, carrierDocuments, customsBrokers | | Rail | railCarriers, railPorts, railShipments, railShipmentItems | | Documentation | documentationCategories, documentationPages, pageDocumentationLinks | | Docks | facilities, dockDoors, dockAppointments | | Photos | photos (entity-specific paths for containers, pallets, shipments) | | AI Chat | tli_threads, tli_messages, tli_artifacts, tli_usage, tli_user_quotas | | Bug Tracking | improvements, improvement_attachments, improvement_comments, improvement_status_history | | System | changelog, emailLogs, emailConfiguration, notificationSettings | ### Database Access ```bash # DEV database (safe for testing) PGPASSWORD=master psql -h localhost -U master -d mi_ai_manifest_dev # PRODUCTION database (caution: real data) PGPASSWORD=master psql -h localhost -U master -d mi_ai_manifest ``` ## Internationalization (i18n) Full bilingual support (English `en` and Spanish Mexico `es-MX`) via react-i18next with HTTP backend. ### Configuration - Supported: ['en', 'es-MX'], Fallback: 'en' - Default namespace: 'common' - Load path: /locales/{{lng}}/{{ns}}.json?v={version} - Detection: cookie → localStorage → navigator - Cookie: i18nextLng (7-day expiry) ### 23 Namespaces common, auth, navigation, pallets, shipments, containers, settings, customers, billing, maintenance, admin, changelog, improvements, documentation, multiStop, tli, import, rail, truckloads, dock-schedule, photos, incoming, recovery ### File Location `client/public/locales/{en|es-MX}/{namespace}.json` — loaded at runtime via HTTP backend ### Usage Patterns ```typescript // Single namespace (default) const { t } = useTranslation('customers'); t('customer_name') // ✓ // Multiple namespaces — first is default const { t } = useTranslation(['orders', 'common']); t('order_status') // ✓ resolves from 'orders' namespace t('common:save') // ✓ explicit namespace prefix for 'common' t('orders:order_status') // ✗ WRONG — redundant prefix, will fail ``` ### Key Rule When adding translation keys, ALWAYS add to BOTH `en` and `es-MX` files for the appropriate namespace. ## File Storage (S3/MinIO) ### Architecture - Primary: MinIO on QNAP NAS (S3-compatible, bucket: mi-ai-manifest) - Region: us-west-1 ### S3 Path Structure ``` {bucket}/ ├── {entity-type}/{entityId}/{photoType}_{timestamp}_{uuid}.{ext} ├── {entity-type}/{entityId}/thumbnails/{file}.jpg ├── documents/{entityId}/{docType}_{timestamp}.pdf ├── companies/{companyId}/logo_{timestamp}.{ext} ├── uploads/improvements/improvement-{ts}-{randomId}.{ext} ├── documentation/images/{uuid} └── documentation/videos/{uuid} ``` ### Upload Endpoints | Endpoint | Max Size | Storage | |----------|----------|---------| | POST /api/photos/upload | 10 MB | S3 (memory) | | POST /api/upload | 10 MB | Local (disk) | | POST /api/objects/upload | N/A | S3 presigned | | POST /api/improvements/:id/attachments | 50 MB | Local (disk) | | POST /api/documentation/admin/images/upload | 10 MB | S3 (memory) | | POST /api/documentation/admin/videos/upload | 50 MB | S3 (memory) | ### Presigned URL TTL Uploads: 15 min, Photos: 24h, Logos: 1h ## UI Components & Patterns ### Component Library Built on shadcn/ui (Radix UI primitives) with Tailwind CSS. All components in `client/src/components/ui/`. ### Key UI Patterns - **Data Grid**: Column definitions with sorting, filtering, persistent preferences via useColumnPreferences hook. No pagination — full dataset loaded. - **Modal Pattern**: ResponsiveModal — Dialog on desktop, Drawer on mobile. Nested modals require z-[60] on DialogContent. - **Form Pattern**: React Hook Form + Zod validation - **Inline Editing**: InlineDateEditor for grid cell date editing - **Filter Pattern**: TextFilterContent, DateFilterContent, NumberFilterContent components - **Toast Pattern**: 1 toast max visible, auto-dismiss - **Loading Pattern**: Skeleton, LoadingSpinner, Loader2 icon - **Role-Based UI**: usePermissions() hook, WithPermission/WithRole components - **Responsive**: useIsMobile() hook for mobile/desktop layouts - **Server State**: React Query with apiRequest helper, 5-minute staleTime - **Debounce**: useDebounce hook, 300ms delay for search inputs - **Status Badges**: StatusBadge with dynamic colors/icons from database configuration ### Critical Dual Component Pattern Two SEPARATE component trees for pallet functionality: 1. **PalletManagement.tsx** — User view (Home > Pallets tab), shows user's own pallets 2. **AdminPalletOverview.tsx** — Admin view (Settings > Pallets), shows all pallets When adding features or fixing bugs, ALWAYS update BOTH components. ## Document Viewers & Editors ### Supported Formats | Extension | Viewer | |-----------|--------| | .pdf | react-pdf v10 (pdfjs-dist v5) — pagination, zoom 50-200%, keyboard shortcuts | | .xlsx, .xls | SheetJS v0.18 — multi-sheet tabs, HTML table rendering | | .docx | mammoth v1.11 — DOCX only (not .doc) | | .jpg, .png, .gif, .bmp, .webp | Image Viewer with zoom/pan | | .mp4, YouTube, Vimeo, Loom | react-player v3.4 | | .md | react-markdown v10 | ### Image Annotation (Fabric.js) Drawing (pencil, highlighter), shapes (rect, circle, arrow, polygon), text tool, eraser, undo/redo, color picker, serializable JSON output. ### Rich Text Editor (TipTap v3.15) StarterKit, Underline, TextAlign, Link, Image, Table, Highlight, Color, Placeholder. Bilingual EN/ES tabs for content editing. ## App Shell & Navigation ### Routing (wouter) App.tsx defines top-level routes: - `/track/:token` → Public tracking page (no auth) - `/preferences/:token` → Public notification preferences (no auth) - `/stack` → Developer reference guide (no auth) - `/docks` → Dock schedule page - `/improvements` → Bug/feature tracking page - `/manual/*` → Documentation viewer - `/*` → ManifestMakerApp (main authenticated shell) ### ManifestMakerApp Tabs All authenticated navigation happens inside ManifestMakerApp.tsx via tabs: - Scanner (pallet scanning workflow) - Pallets, Shipments, Containers, Truckloads, Rail - Billing, Customers, Fleet - Settings (admin) ### Settings Menu Items | Item | Roles | Description | |------|-------|-------------| | Companies | admin | Company management | | Customers | admin | Customer accounts & contacts | | Email | admin | Email templates & notifications | | Status Manager | admin | Custom status workflows | | Users | admin | User CRUD, roles, access | | Deleted Items Recovery | admin | Restore soft-deleted records | | Mobile App | all | Android APK download | | Product Settings | all | App-level configuration | | Update History | all | Changelog viewer | ## Bug & Improvement System ### Overview Full-featured bug and feature request tracking built into the app. Multi-step submission wizard, screenshot annotation (Fabric.js), video uploads, comment threads, email notifications. ### Status Lifecycle created → in_progress → completed|cancelled OR created → rejected (forward-only) ### API Endpoints | Method | Path | Description | |--------|------|-------------| | GET | /api/improvements | List all (filterable) | | POST | /api/improvements | Create improvement | | GET | /api/improvements/:id | Get by ID | | PATCH | /api/improvements/:id | Update | | DELETE | /api/improvements/:id | Soft delete | | PATCH | /api/improvements/:id/status | Update status (admin) | | PATCH | /api/improvements/:id/resolution | Set resolution (admin) | | POST | /api/improvements/:id/attachments | Upload attachment | | GET | /api/improvements/:id/attachments | List attachments | | POST | /api/improvements/:id/comments | Add comment | | GET | /api/improvements/:id/comments | List comments | | GET | /api/improvements/stats | Aggregate counts | | GET | /api/improvements/export/markdown | Export as Markdown | ## TLI AI Chat System ### Architecture Server-Sent Events (SSE) streaming with Anthropic Claude SDK. ### SSE Event Types - `connected` — stream established - `text` — streamed text chunk - `tool_use` — tool invocation - `tool_result` — tool result - `complete` — includes messageId, input/output tokens, latency - `error` — error message ### Database Tables - tli_threads — userId, title, topic, topicContext (JSONB), language, model, visibility - tli_messages — threadId, role, content, toolCalls, toolResults, artifacts, token counts, latency - tli_artifacts — artifactType (excel|pdf|word|csv|chart), fileUrl, contentData - tli_usage — per-request token tracking with estimated cost - tli_user_quotas — daily/monthly token limits, isUnlimited flag ### Features Multi-topic support, language-aware responses, artifact generation to S3, token tracking/billing, quota enforcement, thread sharing, blind spot analysis, email composition tool. ## Testing & Observability ### Vitest (Unit/Integration) ```bash npm run test # Run all tests npm run test:watch # Watch mode npm run test:coverage # Coverage report (v8 provider) ``` Config: `vitest.config.ts`, resolves `@/` and `@shared/` aliases. Tests in `tests/` directory. ### Playwright (E2E) MCP server with DISPLAY=:99 (Chromium). Used by /checkbugs and /checkimprovements for visual verification of fixes. ### Sentry Error Tracking Opt-in via env vars: `SENTRY_DSN` (backend) and `VITE_SENTRY_DSN` (frontend). - Only 5xx errors reported (4xx excluded) - Cookies stripped from events (no PII) - 10% traces sample rate ### Scalar API Docs Interactive explorer at `/api/docs`. Raw spec at `/api/docs/openapi.json`. Hand-maintained OpenAPI 3.1 in `server/docs/openapi.ts`. Tags: Auth, Users, Pallets, Shipments, Containers, Customers, Billing, Improvements, Photos, TLI, Documentation. ## Android APK (WebView + Bluetooth Printing) ### Architecture React Web App → WebView → window.DzLPAPI bridge → LPAPI SDK → Bluetooth SPP → DP26s label printer ### Build ```bash cd android-print-middleware && \ JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 \ ANDROID_HOME=/home/master/android-sdk \ ./gradlew assembleRelease ``` ### Versioning - versionCode: integer, increment by 1 each release - versionName: semantic (e.g., "1.78.0") ### Deployment 1. Build release APK 2. Copy as `app-fresh.apk` (auto-update endpoint) and `app-v{version}.apk` (archive) 3. UpdateManager in app checks every 24h for new version ### JavaScript Bridge (window.DzLPAPI) `openPrinterSync()`, `startJob()`, `drawLabelText()`, `commitJob()`, `printLabel()` ## Claude Code Agents, Skills, Slash Commands & MCP Everything the Claude Code agent uses to operate on this codebase lives in the `.claude/` directory. There are four distinct primitives — **agents** (specialist sub-agents the orchestrator can delegate to), **skills** (reusable workflows the user can invoke directly with `/skill-name`), **commands** (project-specific shortcuts), and **hooks** (deterministic shell commands the runtime fires automatically). Plus MCP servers that extend the toolset. ### Sub-agents (`.claude/agents/*.md`) Specialist agents the main Claude session can spawn via the Task tool. Each `.md` file is a frontmatter+prompt definition. | Agent | Tools | Purpose | |-------|-------|---------| | **changelog-manager** | Read, Write, Edit, Bash, mcp__github, mcp__filesystem | Build changelogs, release notes, bilingual content from improvements DB; bump versions | | **debugging** | Read, Write, Edit, Glob, Grep, Bash, mcp__github, mcp__filesystem, mcp__sequential-thinking, mcp__context7 | Investigate errors, analyze stack traces, identify root causes, propose fixes | | **documentation** | Read, Write, Edit, Glob, Grep, Bash, mcp__filesystem, mcp__github, mcp__context7 | Maintain architecture docs, API references, setup guides, progress tracking | | **frontend-testing** | Read, Write, Edit, Glob, Grep, Bash, Task, mcp__playwright, mcp__filesystem, mcp__github, mcp__sequential-thinking | Create + run Playwright E2E tests on DISPLAY=:99 to catch UI regressions | | **full-stack-developer** | Read, Write, Edit, Glob, Grep, Bash, Task, mcp__filesystem, mcp__sequential-thinking, mcp__context7, mcp__github | Build features end-to-end: frontend UI + backend routes + DB models + integration | | **github-manager** | Read, Write, Edit, Glob, Grep, Bash, mcp__github, mcp__filesystem, mcp__sequential-thinking | Version control, commits, PRs, releases, semver, repo health | | **orchestrating** | Read, Write, Edit, Glob, Grep, Bash, Task, mcp__github, mcp__filesystem, mcp__sequential-thinking | Coordinate other agents, assign tasks, resolve conflicts, track progress | | **ubuntu-system-admin** | Read, Write, Edit, Glob, Grep, Bash, mcp__filesystem, mcp__sequential-thinking, mcp__github | Server config, security, networking, firewall, nginx, SSL, infra | ### Skills (`.claude/skills/*.md`) — user-invocable Run with `/`. Skills are stateless task definitions: title, allowed tools, optional argument hint, prompt body. | Skill | Allowed tools | Argument hint | Purpose | |-------|---------------|---------------|---------| | **/db-migrate** | Read, Write, Edit, Grep, Glob, Bash | `` | Create + apply idempotent DB migration: write SQL → run on DEV → verify | | **/learn-from-fix** | Read, Write, Edit, Grep, Glob, Bash | `` | After a fix, extract root-cause pattern and save it to CLAUDE.md + memory | | **/review-code** | Read, Grep, Glob, Bash | (none) | Review uncommitted changes for quality, reuse, performance, correctness | | **/self-eval** | Read, Grep, Glob, Bash | (none) | 8-category checklist: bugs, missing i18n, untested logic, CLAUDE.md compliance | ### Slash Commands (`.claude/commands/*.md`) — project-specific Heavier scripted workflows. Distinct from skills in that they're often tied to project deployment/CI rather than general patterns. | Command | Argument | Description | |---------|----------|-------------| | **/approved** | `[version-type] [changelog-summary]` | **The ONLY sanctioned path from DEV to production.** Bumps version → builds → commits → tags → pushes to git → waits for CI/CD (GitHub Actions, Coolify, etc.) → runs prod migrations → inserts changelog + docs → rebuilds APK → health-checks. Deploy-target-agnostic; the workflow contract is identical whether prod runs on GitHub Actions, Coolify, Vercel, fly.io, or a custom VPS. **Never bypass with a manual git push or pm2 restart.** Source: `.claude/commands/approved.md` | | **/bugloop** | (none) | Automated bug-fix loop. Use with `/loop 30m /bugloop` for continuous triage | | **/checkbug** | `{ticketNumber}` | Fetch and analyze a specific bug from production DB | | **/checkbugs** | (none) | Batch review all open bugs, fix them, deploy to DEV | | **/checkimprovements** | (none) | Review open feature requests, plan implementation, build them | | **/inbox** | `[recent\|\|search ]` | Check dev email inbox for bug reports + screenshots | | **/getreadytoclear** | (none) | Prepare SESSION_HANDOFF.md before context clearing | ### Hooks (`.claude/settings.json` → `hooks`) Deterministic shell commands the Claude Code runtime executes automatically. Hooks don't run "as Claude" — they run before/after the tool call regardless of what the model wanted to do. | Hook | Matcher | What it does | |------|---------|--------------| | **PreToolUse** | `Bash` | Blocks `rm -rf /`, `DROP DATABASE` before they reach the shell. Exits non-zero with a message; agent must adjust | | **PostToolUse** | `Edit`\|`Write` | When an EN locale file (`locales/en/*.json`) is edited, warns if the matching es-MX counterpart is missing | | **Stop** | (any) | Runs the Vitest suite after every Claude response so regressions surface immediately | ### MCP Servers (`.claude/settings.json` → `mcpServers`) Extend Claude's toolbelt with capabilities the host doesn't provide natively. | Server | Purpose | |--------|---------| | **playwright** | Browser automation for E2E testing (DISPLAY=:99, Chromium) | | **context7** | Up-to-date library documentation — used by debugging/full-stack-developer agents | | **filesystem** | Sandboxed file operations | | **github** | GitHub repository integration (PRs, issues, releases) | | **sequential-thinking** | Multi-step reasoning capabilities | | **postgres** | Direct database access (production-read or DEV-read/write depending on env) | | **git** | Git operations beyond Bash | ### Permissions (`.claude/settings.json` → `permissions`) Allow/deny lists that gate every tool call. Designed so the common case auto-approves but anything risky requires user confirmation. - **allow**: git read ops, `npm run build/test/check`, `pm2 status/logs`, DEV pm2 restarts, DEV database queries, file operations within the project - **deny**: `rm -rf /`, `DROP DATABASE`, `DROP TABLE` on prod, production PM2 restarts (these require explicit user approval each time) ### Discovery from outside Agents and tools that need this list can fetch it from: - `GET /api/public/stack` (canonical, JSON) - `GET /stack.json` (short URL, same JSON) - `GET /stack.md` (Markdown rendering) - `GET /stack` (Markdown if requested by an LLM/curl/python UA, HTML SPA for browsers) - `GET /llms.txt` (top-level discovery + auth model) - `GET /sitemap.xml` (lists every public discovery URL) ## How to Implement Agents, Skills, Commands & Hooks Copy-pasteable templates and the conventions every `.claude/` file in this project follows. Use these when building a new agent, skill, command, or hook. ### The flywheel Session N writes code + hooks enforce invariants → memory persisted in `~/.claude/projects/*/memory/` + CLAUDE.md → Session N+1 reads memory, doesn't repeat past mistakes → monotonic quality improvement. **Hooks are the safety net; memory is the institutional knowledge.** ### File-system layout ``` .claude/ ├── agents/ # Sub-agents (frontmatter + prompt). Spawned via Task tool. │ ├── debugging.md │ ├── full-stack-developer.md │ └── … ├── skills/ # User-invocable workflows (/skill-name) │ ├── db-migrate.md │ ├── review-code.md │ └── … ├── commands/ # Project-specific slash commands │ ├── approved.md │ ├── checkbugs.md │ └── … ├── settings.json # hooks, mcpServers, permissions (checked into git) └── settings.local.json # personal overrides (gitignored) ``` ### Implementing a new sub-agent Create `.claude/agents/.md` with frontmatter: ```markdown --- name: my-agent description: One sentence the orchestrator uses to decide when to spawn this agent. Be specific about inputs and outputs. tools: Read, Write, Edit, Glob, Grep, Bash, mcp__github, mcp__sequential-thinking --- # My Agent You are a specialist that does X. Your job is to: 1. Step one 2. Step two Constraints: - Always … - Never … When done, produce: ``` **Rules:** - `name` must be kebab-case and unique across `.claude/agents/`. - `description` is what the orchestrator sees — write it from the perspective of "should I delegate to this agent?". - `tools` lists only what the agent actually needs. Less is more — narrower tool surface = safer agent. - The agent body is the system prompt. Be terse, specific, and end with the output contract. ### Implementing a new skill Create `.claude/skills/.md`: ```markdown --- name: my-skill description: One-sentence description shown to the user when they type / user-invocable: true allowed-tools: Read, Grep, Glob, Bash argument-hint: "" # Optional, omit if none --- # /my-skill Steps Claude should follow when this skill runs: 1. … 2. … If \${argument} is empty, ask the user for clarification. ``` **Rules:** - `user-invocable: true` makes it appear in `/help` and tab-completion. - `allowed-tools` is enforced — the skill can't use tools outside this list even if it tries. - Skills should be **stateless** — assume nothing about prior conversation state. ### Implementing a new slash command (heavier project workflow) Same shape as a skill, lives under `.claude/commands/`. Use commands when the workflow is **specific to this app/repo** (e.g. `/approved` is the MI AI Manifest release flow). Use skills when the workflow is **generally applicable** (e.g. `/review-code`). ### Adding a hook Hooks run deterministically — they're NOT prompted to Claude. Add them to `.claude/settings.json`: ```json { "hooks": { "PreToolUse": [ { "matcher": "Bash", "hooks": [{ "type": "command", "command": "cmd=$(jq -r '.tool_input.command'); if echo \"$cmd\" | grep -qE 'rm\\s+-rf\\s+/'; then echo 'BLOCKED' >&2; exit 2; fi; exit 0" }] } ], "PostToolUse": [ { "matcher": "Edit|Write", "hooks": [{ "type": "command", "command": "…shell snippet…" }] } ], "Stop": [ { "hooks": [{ "type": "command", "command": "npm run test 2>&1 | tail -3" }] } ] } } ``` **Rules:** - Exit 0 = pass; exit 2 = block the tool call and surface stderr to Claude. - The tool call payload is piped to the hook on stdin as JSON. Parse with `jq -r '.tool_input.'`. - Matchers are regex against the tool name (e.g. `Bash`, `Edit|Write`). - Keep them fast — they run on every matching tool call. ### Adding an MCP server ```json { "mcpServers": { "my-server": { "command": "npx", "args": ["-y", "@me/my-mcp-server"], "env": { "API_KEY": "${MY_API_KEY}" } } } } ``` The server's tools become available to Claude under the prefix `mcp__my-server__*`. They must also appear in the relevant agent's `tools:` frontmatter to be usable by that agent. ### Memory architecture Two parallel persistence layers: 1. **CLAUDE.md** (git-checked, team-shared, always in context). Use for: architecture rules, workflows, gotchas, anything that *must* be loaded every session. 2. **Auto Memory** (`~/.claude/projects/-home-master-projects-mi-ai-manifest/memory/`). Use for: corrections, user feedback, ephemeral discoveries. Files are pointer-indexed in `MEMORY.md` and loaded on demand. Types: `user`, `feedback`, `project`, `reference`. ### Discoverability — make the catalog readable for other agents Every entry in `.claude/{agents,skills,commands}/` flows automatically into: - The Settings → API Keys reference (for permissions UI) - The stack guide (this section, served as JSON/Markdown/HTML) - `/llms.txt` discovery file To add your new agent/skill/command to the **discovery surface**, just commit the `.md` file — the next release picks it up via `server/lib/stackContent.ts`'s agents section. (When making material changes, also bump the version in `package.json` so the JSON's `version` field reflects what's in production.) ## The Required Workflow: DEV → /approved → Production **This is the mandatory development cycle for every team and every project built on this stack.** It is not optional, not "ideal-case" — it is the only path code takes from a developer's machine to a production user. Skipping any step (especially pushing directly to production) is not allowed. The same workflow applies regardless of deploy target — whether your production environment is hosted on GitHub Actions + a self-managed VPS (this app), **Coolify**, Vercel, fly.io, Render, AWS, or anything else. The `/approved` slash command abstracts the deploy mechanism; the **workflow contract** stays the same. ### The cycle every change must follow ``` ┌────────────────────┐ ┌────────────────────┐ ┌─────────────────────┐ ┌────────────────────┐ │ 1. WRITE on LOCAL │ → │ 2. TEST on DEV │ → │ 3. USER APPROVAL │ → │ 4. /approved │ │ (your machine) │ │ (hosted DEV env) │ │ (explicit, human) │ │ (deploys to prod) │ └────────────────────┘ └────────────────────┘ └─────────────────────┘ └────────────────────┘ ``` **Step 1 — Write & build locally.** Run the code on your own machine first. `npm run dev` for hot reload, `npm run check` for TypeScript, `npm run test` for the suite. Iterate until the change works locally. This is where 95% of debugging happens — never skip it by editing the DEV server directly. **Step 2 — Push to your hosted DEV environment and test there.** Local works ≠ DEV works (different DB, different env vars, real S3 instead of mocks, real session middleware, real reverse proxy). Run `npm run build && pm2 restart -dev` (or your equivalent on Coolify/Vercel/etc.) and verify the actual deployed DEV URL behaves as expected. Run E2E tests (Playwright via MCP if available). Reproduce the original bug-report flow end-to-end. **Step 3 — Get explicit user approval.** When testing on DEV looks good, **stop and wait**. Show the user what changed, what was tested, and what the verified URLs are. **Never** assume "looks good to me, I'll push" — that is not a substitute for user approval. The user is the only authorized trigger for Step 4. **Step 4 — Run `/approved` to deploy to production.** Only the `/approved` slash command pushes to production. It bumps the version, builds, commits, tags, pushes to git, waits for CI/CD (GitHub Actions / Coolify / etc.) to deploy, runs DB migrations on prod, inserts the changelog entry and documentation page on prod, rebuilds the Android APK if applicable, and health-checks the result. The 13-step contract is documented under "CI/CD & Production Deployment" below; the source lives in `.claude/commands/approved.md`. ### What `/approved` is — and what it is NOT `/approved` is **the only sanctioned production-deploy entry point**. It is NOT: - a manual `git push` to main - a manual SSH-into-prod + `pm2 restart` - a "hotfix" that bypasses the build step - a "small change so it's fine" justification If you find yourself wanting to deploy without `/approved`, the answer is always: fix `/approved` itself, then run it. Most "I need to skip the workflow" cases are actually "the workflow needs an extra step" — add the step to `approved.md` so every future deploy benefits. ### Deploy targets — the workflow is the same regardless | Production host | What `/approved` does on push | Where the contract lives | |---|---|---| | **GitHub Actions** (this app) | Triggers `.github/workflows/deploy-production.yml` on tag push | Workflow YAML | | **Coolify** | Coolify auto-pulls main on push and rebuilds the container | Coolify config + git webhook | | **Vercel / Netlify / Render** | Push to main triggers automatic build & deploy | Vercel/Netlify/Render dashboard | | **fly.io** | `flyctl deploy` runs inside `/approved` after the git push | `fly.toml` | | **Custom VPS** | `/approved` SSHs in, pulls, builds, restarts the process manager | `.claude/commands/approved.md` | In every case: **dev → test → approve → `/approved` → prod**. The deploy target plugs into Step 4 — it doesn't replace Steps 1-3. ### Git commit format All commits should use this format (the format `/approved` produces automatically): ``` : Co-Authored-By: Claude ``` Types: `feat` (new feature), `fix` (bug fix), `docs`, `style`, `refactor`, `test`, `chore`, `perf`. ### What you must NEVER do - **Push to production without going through `/approved`.** No exceptions. - **Test only on your local machine.** Local works ≠ deployed works. - **Add test/synthetic data to the production database.** Use DEV for that. - **Run destructive git commands** (`reset --hard`, `push --force`) without explicit user approval. - **Skip the build step.** Both PM2 processes run `dist/index.js` — without rebuilding, nothing you changed actually runs. - **Modify production .env files directly.** The deploy pipeline handles `.env` backup/restore; manual edits get clobbered on next deploy. - **Push doc-only changes "just to update".** They still go through `/approved` so the changelog + stack data version stays accurate. ## Environments & Infrastructure ### Two-Server Architecture | | DEV Server | PRODUCTION Server | |---|---|---| | **Hostname** | manifestdev.miglobal.com.mx | manifest.miglobal.com.mx | | **IP** | 45.22.197.163 | 108.61.207.152 | | **SSH** | standard port 22 | port 2215 | | **App Port** | 5000 | 7000 | | **Database** | mi_ai_manifest_dev | mi_ai_manifest | | **PM2 Process** | mi-ai-manifest-dev | mi-ai-manifest-prod | | **Env File** | .env.dev | .env | | **Contains** | Test data, safe to experiment | REAL CUSTOMER DATA | ### PM2 Configuration (ecosystem.config.cjs) Both processes run `dist/index.js` (compiled output, not TypeScript): - Node args: `--max-old-space-size=512` - Auto-restart at 1GB memory - Fork mode, single instance - Logs: `/home/master/.pm2/logs/mi-ai-manifest-{dev|prod}-{out|error}.log` ### Reverse Proxy Nginx with Let's Encrypt SSL on both servers. Certbot auto-renewal runs twice daily via `certbot.timer`. Deploy hook reloads nginx after renewal. ### Database PostgreSQL 16 on localhost:5432 (both servers). User: master, Password: master. Each server has its own database. ### S3 Storage MinIO on QNAP NAS (S3-compatible). Bucket: mi-ai-manifest. Region: us-west-1. Used for photos, documents, logos, attachments. ## CI/CD & Production Deployment ### GitHub Actions Pipeline File: `.github/workflows/deploy-production.yml` **Trigger**: Push to `main` branch (except `**.md` and `docs/**` changes). Also supports manual `workflow_dispatch`. **Pipeline Steps** (runs on ubuntu-latest, SSHs into production server): 1. Backup production `.env` to `/tmp/` (critical — git reset would destroy it) 2. `git stash` any local changes, `git fetch origin main`, `git reset --hard origin/main` 3. Restore `.env` from backup 4. Validate DATABASE_URL doesn't point to dev database (auto-fixes if it does) 5. `npm install --production=false` (need devDependencies for build) 6. `npm run build` (Vite frontend + esbuild backend) 7. Apply all SQL migrations: `for migration in migrations/0*.sql; do psql -f "$migration"; done` 8. `pm2 restart mi-ai-manifest-prod` (falls back to `pm2 start ecosystem.config.cjs`) 9. Wait 5s, then health-check: `curl http://localhost:7000` expects HTTP 200 **Timeout**: 10 minutes total. If any step fails, deployment stops and reports failure. ### The /approved Slash Command (Full Release Automation) When the user says `/approved`, Claude Code executes a 13-step release: 1. **Calculate version** — read current from package.json, bump patch/minor/major 2. **Update package.json** — write new version 3. **Sync stack guide** — when `.claude/` (agents, skills, commands, hooks, MCP, permissions) or the release workflow itself changed since the last release, update all three discovery surfaces so the public stack reference matches reality: - `server/lib/stackContent.ts` — canonical agent-readable source for `/api/public/stack`, `/stack.json`, `/stack.md` - `client/src/components/stack-guide/StackAgentsGuide.tsx` — HTML/React rendering of the agents catalog - `client/src/components/stack-guide/StackApprovedWorkflow.tsx` (`APPROVED_WORKFLOW_CONTENT`) — HTML rendering of this 13-step list itself 4. **Build** — `npm run build` on dev server (stack content is bundled here) 5. **Git commit** — `chore: Release vX.X.X` with changelog summary 6. **Git tag** — annotated tag `vX.X.X` 7. **Push to GitHub** — `git push origin main --tags` → triggers GitHub Actions 8. **Wait for deployment** — ~90s for build + restart on production 9. **Run migrations** — SSH to production, apply SQL migrations 10. **Changelog entry** — insert into `changelog_entries` table on production DB 11. **Documentation page** — insert release notes into `documentation_pages` table 12. **Android APK** — increment versionCode, build with Gradle, deploy `app-fresh.apk` + versioned archive 13. **Health check** — verify PM2 online + HTTP 200 **Why Step 3 matters:** the `/stack` discovery surface is consumed by AI agents (via JSON/Markdown) and humans (via the HTML view). If the listing drifts from `.claude/`, agents will spawn sub-agents that don't exist, miss new skills, or invoke obsolete tools. Treating stack sync as part of every release keeps the catalog honest. ### SSH to Production ```bash ssh -p 2215 master@108.61.207.152 # Default directory: /home/master (NOT the project dir) # Project: /home/master/projects/mi-ai-manifest ``` ### Database Migrations - Migrations are in `migrations/0*.sql` (numbered, applied in order) - All migrations use idempotent SQL (`IF NOT EXISTS`, `ADD COLUMN IF NOT EXISTS`) - Safe to re-run — the CI/CD pipeline runs all of them on every deploy - For new schema changes: create migration file → apply to DEV first → verify → include in release ## Common Development Commands ### Building & Running ```bash npm run dev # Development mode with hot reload (tsx server/index.ts) npm run check # TypeScript type checking npm run build # Production build (vite frontend + esbuild backend → dist/) npm start # Start production server (node dist/index.js) ``` ### Testing ```bash npm run test # Run all tests (Vitest) npm run test:watch # Watch mode npm run test:coverage # Coverage report (v8 provider) ``` ### PM2 Process Management ```bash pm2 status # Check all processes pm2 logs mi-ai-manifest-dev --lines 50 --nostream # View DEV logs pm2 logs mi-ai-manifest-prod --lines 50 --nostream # View PROD logs pm2 restart mi-ai-manifest-dev # Restart DEV (after build) pm2 restart all --update-env # Restart with env changes ``` ### Database ```bash npm run db:push # Apply schema changes via Drizzle ORM npm run seed:test-data # Seed test data (DEV ONLY) # DEV database PGPASSWORD=master psql -h localhost -U master -d mi_ai_manifest_dev # PRODUCTION database (via SSH) ssh -p 2215 master@108.61.207.152 "PGPASSWORD=master psql -h localhost -U master -d mi_ai_manifest" # Backup before dangerous operations PGPASSWORD=master pg_dump -h localhost -U master -d mi_ai_manifest -F c -f backup_$(date +%Y%m%d_%H%M%S).backup ``` ### Git ```bash git status # Check working tree git log --oneline -10 # Recent commits git diff # Unstaged changes git add # Stage specific files (never use git add -A blindly) ``` ## Starter Template The stack guide includes copy-pasteable starter files for bootstrapping a new app on this stack: 1. **package.json** — React/Express dependencies 2. **vite.config.ts** — Vite setup with path aliases (@/, @shared/) 3. **tailwind.config.ts** — Tailwind config with shadcn/ui theme 4. **ecosystem.config.cjs** — PM2 ecosystem configuration 5. **server/index.ts** — Express server with session/passport/CORS 6. **server/db.ts** — PostgreSQL connection via Drizzle + pg 7. **shared/schema.ts** — Drizzle ORM schema with Zod validation 8. **client/src/App.tsx** — React app with wouter routing 9. **client/src/lib/queryClient.ts** — React Query config + apiRequest helper 10. **client/src/i18n/config.ts** — i18next configuration 11. **client/src/hooks/usePermissions.ts** — Role-based permission hook 12. **client/src/hooks/useDebounce.ts** — Debounce hook (300ms) 13. **client/public/locales/en/common.json** — English translations 14. **client/public/locales/es-MX/common.json** — Spanish translations Visit https://manifest.miglobal.com.mx/stack#starter-template for the full code of each file. ## External API v1 & API Key Management ### Surface `/api/v1/external/*` — bearer-token-authenticated REST API for third-party apps (partner integrations, internal scripts, customer portals). Distinct from `/api/public/*` (one-time tracking tokens, not reusable) and `/api/*` (session-authed, used by the web app itself). ### Authentication ``` Authorization: Bearer . ``` Both halves issued at creation time. Secret is bcrypt-hashed at rest and shown only once. ### Three Scopes | Scope | What it sees | Use case | |---|---|---| | customer | Only shipments with matching customer_id | Customer ERP/portal integrations | | partner | All shipments, optionally filterable | 3PL/carrier dispatch and tracking apps | | internal | Full access, no rate limit | Internal scripts, admin tools | ### Rate Limits (3 tiers) | Tier | per minute | per hour | per day | |---|---|---|---| | standard | 60 | 1,000 | 10,000 | | elevated | 300 | 10,000 | 100,000 | | unlimited | ∞ | ∞ | ∞ | PostgreSQL-backed sliding fixed-window counters in `api_rate_limit` table. Per-key override via `rate_limit_override` jsonb. ### Permissions Model ```json { "shipments": ["read", "write_status"], "pallets": ["read"], "items": ["read"], "products": ["read"], "documents": ["read", "write"], "customers": ["read"], "webhooks": ["manage"] } ``` Wildcards: `{"*": ["*"]}` (internal) grants everything. `{"shipments": ["*"]}` grants all actions on that resource. ### Endpoints **Meta**: GET /health, /version, /whoami **Shipments**: GET /shipments (cursor pagination), /shipments/:id, /shipments/:id/pallets, /items, /stops, /trackers, /by-bol/:bolNumber **Items** (fully denormalized — pallet + shipment + stops + customer + scanned_by user + order + product master + trackers + optional documents/photos/history): GET /items/by-serial/:serialNumber, /items/by-upc/:upc, /items/by-lpn/:lpn, /items/by-sku/:sku, /items/by-order/:orderId, /items (flexible search via ?serial=&upc=&lpn=&sku=&make=&model=&order_id=&line_number=) **Products**: GET /products/by-upc/:upc — resolve UPC → make/model/description/harmonized_code from the product master **Documents**: GET /shipments/:id/documents, /documents/:docId/download (streamed proxy from MinIO — endpoint never exposed) **Writes**: PATCH /shipments/:id/status, /stops/:stopId/status; POST /stops/:stopId/pod (multipart) **Webhooks**: GET/POST/DELETE /webhooks; GET /webhooks/:id/deliveries **Customers**: GET /customers (partner/internal only) ### Webhooks HMAC-SHA256 signing (`X-MIT-Signature: t=,v1=` Stripe-style). Exponential backoff retry (1m, 5m, 30m, 2h, 12h, 24h), 6 attempts then dead. Auto-disable after 20 consecutive failures. Events: shipment.created, shipment.status_changed, shipment.delivered, shipment.stop.status_changed, shipment.stop.delivered, shipment.document_uploaded, pallet.shipped, pallet.delivered. Webhook secrets are AES-256-GCM encrypted at rest (not bcrypt) so the dispatcher can sign with the plaintext. ### Logging & Audit | Table | Retention | Purpose | |---|---|---| | api_request_logs | 90 days | Every authed request: status, duration, bytes, IP | | api_denial_logs | 365 days | Every 4xx denial: invalid_key, revoked, rate_limited, permission_denied, customer_scope_violation | | api_usage_daily | indefinite | Precomputed daily aggregates for dashboard | | api_key_events | indefinite | Admin audit trail (created, rotated, revoked, etc.) | | api_webhook_deliveries | 90 days (terminal) | Per-event delivery log with retry state | Cleanup runs daily at 03:30 via `apiRetentionJob`. ### Security Guarantees - Customer scoping enforced at the DB layer (WHERE clause), not just request layer - Customer-scope violations return 404 (not 403) to prevent enumeration - All denials logged with IP and user_agent - 24-hour secret rotation grace period - Optional IP allowlist (CIDR) and CORS origin allowlist per key - HTTPS-only (nginx redirects) ### Admin Dashboard Settings → API Keys & External API: - List/create/edit/revoke/rotate keys - Per-key 30-day usage chart + top endpoints - Recent requests + denials with replay buttons for failed webhook deliveries - Global metrics: total req/24h, top keys, top endpoints, error rate - Webhook subscription manager per key ### Files | File | Role | |---|---| | migrations/0137_external_api_infrastructure.sql | 7 new tables + indexes | | server/middleware/apiKeyAuth.ts | Bearer parse, bcrypt compare, scoping helpers | | server/middleware/apiRateLimit.ts | PG-backed fixed-window rate limiter | | server/routes/externalApi.ts | /api/v1/external/* endpoints | | server/routes/apiKeyAdminRoutes.ts | /api/admin/api-keys/* admin endpoints | | server/services/apiKeyService.ts | Generate/rotate/revoke + audit logging | | server/services/secretCrypto.ts | AES-GCM webhook secret encryption | | server/services/webhookEmitter.ts | Enqueues deliveries from write endpoints | | server/services/webhookDispatcher.ts | Background HTTP sender with retry | | server/services/apiRetentionJob.ts | Daily log pruning + aggregate refresh | | client/src/components/ApiKeyManagement.tsx | Admin dashboard UI | ### Docs - docs/guides/EXTERNAL_API_GUIDE.md — integrator getting started - docs/guides/EXTERNAL_API_REFERENCE.md — every endpoint - docs/guides/EXTERNAL_API_WEBHOOKS.md — event catalog + HMAC verification - docs/stack/api-management.md — full system documentation