feat: endpoints, outbox, PGP, app home and retention (phase e) #8

Merged
KofTwentyTwo merged 16 commits from feature/phase-e-endpoints-home into develop 2026-10-14 14:17:43 +00:00
KofTwentyTwo commented 2026-10-14 03:53:01 +00:00 (Migrated from github.com)

Phase (e) of docs/DESIGN.md §14. Stacked on #7 → #6 → #5, and it needs QRun-IO/qqq#1170.

  • Endpoints (aliases) and chroot.

    • Paths are validated: absolute, [a-z0-9._/-], unique, and no clash with app names.
    • Kinds are BULK_LOAD, INBOX (the consumer is chosen by consumer_name), VIEW, REPORT and OUTBOX.
    • ENDPOINTS_ONLY is the chroot, and it's the default for service accounts. Endpoints grant nothing: the target must be reachable by the user through /qqq/v1.
  • Outbox. FileGatewayOutbox.put(...) is registered in every deployment, since it needs only the DB and object storage. On top of that:

    • outbox files are listed in the OUTBOX endpoint;
    • the first complete download marks a file DOWNLOADED and calls onDownloaded on the consumer;
    • rm marks it REMOVED;
    • retention marks old files EXPIRED.

    This is the contract agreed with qbit-edi.

  • PGP is optional per account. Uploads can be decrypted stream to stream before the consumer runs, and downloads encrypted on the fly to the account's key. Keys come from env var names and stay in memory; public keys are validated on save.

  • App home. Typed widgets only (G5):

    • KPIs: transfers today, failed jobs (24h), active sessions, MB in and out, locked accounts;
    • full-width tables: recent failures, top partners (7d), active sessions, and gateway servers from the new file_gateway_server heartbeat table.
  • Record actions (standard Review/Result): Unlock account, Disable account, Revoke key and End session, alongside Retry upload.

  • Retention. A daily process covers logs (180d), archive and outbox (30d), results (90d), stale RECEIVING uploads and dead servers.

  • FileGatewayIntegrations. ESB tables (accounts, endpoints, upload jobs), Quick Search (accounts, endpoints, upload jobs) and API (all tables).

  • docs/k8s/. The gateway Deployment and Service:

    • same image, FILE_GATEWAY_SFTP_ENABLED=true, and the host key from a Secret;
    • read-only root filesystem with an in-memory /tmp;
    • 2 replicas and a PDB;
    • LoadBalancer port 22 only, with no passive range, externalTrafficPolicy: Local and LB IP 10.100.22.65.
  • UX. Plain labels for fields, enum values ("Endpoints only") and child tabs (SSH keys, Upload jobs, Connections), and readable UTC timestamps on the home.

  • Demo server. FileGatewayDemoServer (test scope) serves the Next UI with realistic data for screenshots.

Not here: the Preferences "File transfer" section waits for QRun-IO/qqq#1162 and will be a small follow-up PR.

Tests: 88. JaCoCo is at 92% instructions; checkstyle is clean. Screenshots of the home and the account, upload-job and endpoint records are attached by the agent in its report (light and dark, 390 and 1440, no horizontal scroll).

🤖 Generated with Claude Code

Phase (e) of `docs/DESIGN.md` §14. **Stacked on #7 → #6 → #5**, and it needs QRun-IO/qqq#1170. - **Endpoints (aliases) and chroot.** - Paths are validated: absolute, `[a-z0-9._/-]`, unique, and no clash with app names. - Kinds are BULK_LOAD, INBOX (the consumer is chosen by `consumer_name`), VIEW, REPORT and OUTBOX. - `ENDPOINTS_ONLY` is the chroot, and it's the default for service accounts. Endpoints grant nothing: the target must be reachable by the user through `/qqq/v1`. - **Outbox.** `FileGatewayOutbox.put(...)` is registered in **every** deployment, since it needs only the DB and object storage. On top of that: - outbox files are listed in the OUTBOX endpoint; - the first complete download marks a file DOWNLOADED and calls `onDownloaded` on the consumer; - `rm` marks it REMOVED; - retention marks old files EXPIRED. This is the contract agreed with qbit-edi. - **PGP** is optional per account. Uploads can be decrypted stream to stream before the consumer runs, and downloads encrypted on the fly to the account's key. Keys come from env var names and stay in memory; public keys are validated on save. - **App home.** Typed widgets only (G5): - KPIs: transfers today, failed jobs (24h), active sessions, MB in and out, locked accounts; - full-width tables: recent failures, top partners (7d), active sessions, and gateway servers from the new `file_gateway_server` heartbeat table. - **Record actions** (standard Review/Result): Unlock account, Disable account, Revoke key and End session, alongside Retry upload. - **Retention.** A daily process covers logs (180d), archive and outbox (30d), results (90d), stale RECEIVING uploads and dead servers. - **`FileGatewayIntegrations`.** ESB tables (accounts, endpoints, upload jobs), Quick Search (accounts, endpoints, upload jobs) and API (all tables). - **`docs/k8s/`.** The gateway Deployment and Service: - same image, `FILE_GATEWAY_SFTP_ENABLED=true`, and the host key from a Secret; - read-only root filesystem with an in-memory `/tmp`; - 2 replicas and a PDB; - LoadBalancer **port 22 only**, with no passive range, `externalTrafficPolicy: Local` and LB IP 10.100.22.65. - **UX.** Plain labels for fields, enum values ("Endpoints only") and child tabs (SSH keys, Upload jobs, Connections), and readable UTC timestamps on the home. - **Demo server.** `FileGatewayDemoServer` (test scope) serves the Next UI with realistic data for screenshots. **Not here:** the Preferences "File transfer" section waits for QRun-IO/qqq#1162 and will be a small follow-up PR. **Tests:** 88. JaCoCo is at 92% instructions; checkstyle is clean. Screenshots of the home and the account, upload-job and endpoint records are attached by the agent in its report (light and dark, 390 and 1440, no horizontal scroll). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.