Merge feature/phase-a-sftp-server into feature/phase-b-upload

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AGENTS.md

@@ -14,18 +14,54 @@ code and QInstance. Design: `docs/DESIGN.md`.
 ## Build commands
 
 ```bash
-./mvnw compile
-./mvnw test
-./mvnw verify  # Checkstyle; JaCoCo: 70% instructions, 90% classes
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify  # Checkstyle; JaCoCo: 70% instructions, 90% classes
 ```
 
-Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`.
+Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell.
+Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement.
 
 ## Architecture
 
 Package: `com.kingsrook.qbits.filegateway`. Entry point: `FileGatewayQBitProducer implements
 QBitMetaDataProducer<FileGatewayQBitConfig>`.
 
+- `metadata/`: seven stable tables, enum/picker sources, joins, typed child widgets, key/account
+  validators and streamed Review/Result admin actions. Producer adds metadata to the QInstance.
+- `model/`: QRecordEntity mappings; DDL in `docs/ddl/file_gateway.sql` uses UTC zone-less timestamps.
+- `permissions/`: view/admin/connect constants and the custom permission checker; system-written
+  logs/jobs/outbox tables remain read-only to people.
+- `FileGatewayRuntime`: host-owned startup after the UI API server, in-memory host-key parsing,
+  injectable environment and clock/test hooks. Production polling interval is 60 seconds.
+- `sftp/`: public-key-only MINA listener, per-account transactional authentication, per-pod IP
+  windows, token refresh/revocation and system-session bookkeeping restricted to owned tables.
+- `vfs/`: custom NIO filesystem; per-session UI API metadata/saved-object caches. Every API call
+  checks its operation spec and sends user credentials plus `sftp/<sessionId>` client detail.
+- `api/`: isolated consumer/outbox contract. Never import gateway implementation packages here.
+
+Phase (b) adds `upload/` (bounded streaming channels, database quota reservations, worker claims,
+scanner/consumer pipeline and UI API bulk load) and `storage/` (the object-storage API adapter
+and ranged read channels). Production uses object storage; the memory implementation is tests-only.
+The runtime starts a bounded worker pool with SFTP and releases owned claims on stop. Sidecars,
+results and archive are readable; sidecars/results can be acknowledged. Retry upload is a
+standard Review/Result record action. The upload-job schema now includes `results_size_bytes`.
+
+Claims compare expected status under an account-row transaction; core's optimistic version check
+alone does not add a SQL WHERE guard. Fresh worker user sessions outlive SSH connections and are
+revoked after delivery. Bulk process multipart field: `theFile`; mapping frontend step: `fileMapping`
+(resumes `receiveFileMapping`). RC.2 object storage owns S3 part size at connection level. The
+private invoker selects multipart only when a file part is present, so the mapping request
+restreams the file with its values. See README adoption prerequisites and compatibility notes.
+
+Saved-view/report downloads, endpoint routing/chroot, outbox delivery, Quick Search/ESB,
+preferences, retention and the app home belong to later phases. The core-valid alert type is `filegateway.lockout`; the design's
+underscore-only name is rejected by core validation.
+
+Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension
+snapshots the working tree (excluding target) and isolated test home, and asserts the test temp
+directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must remain test-scoped.
+
 ## Invariants and naming
 
 - Data access only through `/qqq/v1` (`UiApiInvoker`, channel `sftp`) as the signed-in user. The

CHANGELOG.md

@@ -6,4 +6,35 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
 
 ### Added
 
-- Initial scaffold
+- Phase (b) bounded streaming uploads through the object-storage system API, SHA-256, concurrent
+  byte quotas, temporary-name promotion, dedupe and transfer auditing.
+- Bounded upload workers with guarded claims, fresh user sessions, scanner/quarantine hooks,
+  named consumers, retry scheduling, stale recovery and shutdown requeue.
+- Built-in bulk-load consumer using only spec-defined UI API operations, saved mappings,
+  inserted/error summaries, per-row examples, notifications and a Retry upload record action.
+- Readable sidecars, results CSV and ranged archive downloads with account isolation and
+  acknowledgement. Added `results_size_bytes` to the upload-job schema.
+- Real MINA upload acceptance coverage, including permissions, auditing, limits, reordering,
+  consumer retries, quarantine, worker concurrency and the unchanged no-disk guard.
+
+- Phase (a) SSH-key-only SFTP runtime with in-memory host keys, explicit modern algorithms,
+  account/key expiry, numeric CIDR allowlists, transactional account lockout/session limits,
+  per-IP limits, user-session refresh and live revocation.
+- Virtual listing/stat of permitted apps, tables, saved bulk-load profiles, views and reports
+  through the strict UI API invoker, with per-session caches and paged saved-object queries.
+- Seven gateway tables and record entities, H2/PostgreSQL DDL, enum sources, dependent endpoint
+  pickers, joins, typed child lists, permissions and channel walls.
+- SSH key/account validation and Review/Result actions to unlock/disable accounts, revoke keys
+  and end sessions; connection and operation auditing, including denied mutations.
+- Isolated consumer/outbox API contract and complete configuration for subsequent phases.
+- Real SFTP/strict UI API acceptance tests, no-disk guard, schema round trips and adoption docs.
+
+### Compatibility
+
+- Local invoker integration builds use `4.1.0-FG-SNAPSHOT`; CI retains the POM's
+  `4.1.0-SNAPSHOT` default. RDBMS remains a host-provided runtime dependency.
+- Alert type `filegateway.lockout` uses the dotted syntax required by core validation.
+- Saved-view/report downloads, endpoint routing/chroot, outbox delivery and app home remain later
+  phases. Object-storage RC.2 controls S3 part size through its connection configuration.
+- With the private core build, the file-mapping request resends its file to select multipart
+  encoding; URL-encoded process values are not accepted by the v1 route.

CLAUDE.md

@@ -14,18 +14,54 @@ code and QInstance. Design: `docs/DESIGN.md`.
 ## Build commands
 
 ```bash
-./mvnw compile
-./mvnw test
-./mvnw verify  # Checkstyle; JaCoCo: 70% instructions, 90% classes
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify  # Checkstyle; JaCoCo: 70% instructions, 90% classes
 ```
 
-Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`.
+Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell.
+Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement.
 
 ## Architecture
 
 Package: `com.kingsrook.qbits.filegateway`. Entry point: `FileGatewayQBitProducer implements
 QBitMetaDataProducer<FileGatewayQBitConfig>`.
 
+- `metadata/`: seven stable tables, enum/picker sources, joins, typed child widgets, key/account
+  validators and streamed Review/Result admin actions. Producer adds metadata to the QInstance.
+- `model/`: QRecordEntity mappings; DDL in `docs/ddl/file_gateway.sql` uses UTC zone-less timestamps.
+- `permissions/`: view/admin/connect constants and the custom permission checker; system-written
+  logs/jobs/outbox tables remain read-only to people.
+- `FileGatewayRuntime`: host-owned startup after the UI API server, in-memory host-key parsing,
+  injectable environment and clock/test hooks. Production polling interval is 60 seconds.
+- `sftp/`: public-key-only MINA listener, per-account transactional authentication, per-pod IP
+  windows, token refresh/revocation and system-session bookkeeping restricted to owned tables.
+- `vfs/`: custom NIO filesystem; per-session UI API metadata/saved-object caches. Every API call
+  checks its operation spec and sends user credentials plus `sftp/<sessionId>` client detail.
+- `api/`: isolated consumer/outbox contract. Never import gateway implementation packages here.
+
+Phase (b) adds `upload/` (bounded streaming channels, database quota reservations, worker claims,
+scanner/consumer pipeline and UI API bulk load) and `storage/` (the object-storage API adapter
+and ranged read channels). Production uses object storage; the memory implementation is tests-only.
+The runtime starts a bounded worker pool with SFTP and releases owned claims on stop. Sidecars,
+results and archive are readable; sidecars/results can be acknowledged. Retry upload is a
+standard Review/Result record action. The upload-job schema now includes `results_size_bytes`.
+
+Claims compare expected status under an account-row transaction; core's optimistic version check
+alone does not add a SQL WHERE guard. Fresh worker user sessions outlive SSH connections and are
+revoked after delivery. Bulk process multipart field: `theFile`; mapping frontend step: `fileMapping`
+(resumes `receiveFileMapping`). RC.2 object storage owns S3 part size at connection level. The
+private invoker selects multipart only when a file part is present, so the mapping request
+restreams the file with its values. See README adoption prerequisites and compatibility notes.
+
+Saved-view/report downloads, endpoint routing/chroot, outbox delivery, Quick Search/ESB,
+preferences, retention and the app home belong to later phases. The core-valid alert type is `filegateway.lockout`; the design's
+underscore-only name is rejected by core validation.
+
+Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension
+snapshots the working tree (excluding target) and isolated test home, and asserts the test temp
+directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must remain test-scoped.
+
 ## Invariants and naming
 
 - Data access only through `/qqq/v1` (`UiApiInvoker`, channel `sftp`) as the signed-in user. The

README.md

@@ -1,18 +1,24 @@
 # QBit File Gateway
 
-An SFTP server for QQQ applications. Users and service accounts sign in with their SSH keys and see
-the application as a folder tree of apps and tables. Uploading a file into a table's
-`upload/<profile>/` folder runs a saved bulk-load profile as that user; getting a file from
-`views/` or `reports/` exports a saved view or runs a saved report. The gateway reaches data only
-through the `/qqq/v1` UI API with the signed-in user's own session (client channel `sftp`), so
-permissions, masking and audits apply exactly as in the UI. Design: [docs/DESIGN.md](docs/DESIGN.md).
+An SSH-key-only SFTP server that presents a QQQ application's permitted apps and tables as a
+virtual folder tree. Application metadata and saved objects are read through the spec-defined
+`/qqq/v1` UI API as the signed-in user, using `UiApiInvoker` on channel `sftp`.
 
-This repository currently holds the scaffold; the gateway is built in phases described in the design.
+Phases (a) and (b) implement authentication, listing/stat, account administration, uploads into
+saved bulk-load profiles, background consumer delivery, and readable sidecars/results/archive.
+Saved-view/report downloads, endpoint routing/chroot, outbox delivery, preferences and the app
+home follow in later phases. No native filesystem, host-key file, home directory or spool is
+used by the gateway. See [the approved design](docs/DESIGN.md).
 
 ## Requirements
 
-Java 21 and QQQ 4.1.0-SNAPSHOT (UI API invoker and client channels, until 4.1.0-RC.6). The Maven
-wrapper pins Maven 3.9.11.
+Java 21, Apache MINA SSHD 2.16.0, and the QQQ 4.1 invoker APIs. The host supplies the RDBMS
+backend, qbit-object-storage 0.1.0-RC.2, and core saved bulk-load profile, saved view and saved
+report metadata. The gateway's
+RDBMS module dependency is test-scoped. Maven 3.9.11 is pinned by the wrapper.
+
+Until the invoker core PR merges, local builds use the private `4.1.0-FG-SNAPSHOT` artifacts.
+The POM default remains `4.1.0-SNAPSHOT` for CI, to be replaced by 4.1.0-RC.6 when released.
 
 ## Install
 
@@ -24,32 +30,224 @@ wrapper pins Maven 3.9.11.
 </dependency>
 ```
 
+## Adopting
+
+Apply [docs/ddl/file_gateway.sql](docs/ddl/file_gateway.sql) to the host's database. It supports
+H2 and PostgreSQL and stores UTC timestamps without a zone, matching QQQ's RDBMS binding.
+The gateway does not create or migrate tables at runtime.
+
+Register metadata while building the QInstance:
+
+```java
+FileGatewayQBitConfig config = new FileGatewayQBitConfig()
+   .withBackendName("rdbms")
+   .withSftpEnabled(Boolean.parseBoolean(System.getenv("FILE_GATEWAY_SFTP_ENABLED")))
+   .withHostKeyEnvVarName("FILE_GATEWAY_SSH_HOST_KEY");
+
+new FileGatewayQBitProducer().withQBitConfig(config).produce(qInstance);
+```
+
+After starting the host's `QApplicationJavalinServer`, start SFTP only in the gateway deployment:
+
+```java
+FileGatewayRuntime gateway = null;
+if(config.getSftpEnabled())
+{
+   gateway = FileGatewayRuntime.start(qInstance, server);
+}
+// Call gateway.stop() during application shutdown when gateway is non-null.
+```
+
+Supply an unencrypted OpenSSH private host key as the value of the named environment variable.
+It is parsed in memory; a missing key fails startup. `allowGeneratedHostKey` is for development
+only and generates an ephemeral key with a warning. Each deployment uses the same host application
+and QInstance; expose its SFTP listener separately from the UI HTTP port.
+
+Create an ACTIVE account with a QQQ `userId`, then add an Ed25519, ECDSA, or RSA public key of at
+least 3072 bits. Public-key validation runs on insert and update and computes its unique SHA-256
+fingerprint, type and size. The host authentication module must assign the user's permissions
+when `createAutomatedSessionForUser` is called, including `file_gateway.connect`.
+
 ## Tables and permissions
 
-Planned tables (none are produced yet), all prefixed `file_gateway_`: accounts, SSH keys, aliases,
-upload jobs, the transfer log and the connection log. Permissions use the full `file_gateway`
-prefix.
+All seven tables are registered, with camelCase QQQ fields mapped to snake_case database columns.
+`backendName` selects the host backend; physical table names match the QQQ table names.
+
+| Table | Purpose |
+|---|---|
+| `file_gateway_account` | User/service login, limits, CIDRs, expiry and lockout state |
+| `file_gateway_ssh_key` | Account child keys and computed fingerprints |
+| `file_gateway_alias` | Endpoint definitions; routing is a later phase |
+| `file_gateway_upload_job` | Receiving, queued and completed uploads, retries, results and consumer references |
+| `file_gateway_outbox_file` | Stable schema for later outbound delivery |
+| `file_gateway_transfer_log` | Listings, uploads, object downloads and denied operations |
+| `file_gateway_connection_log` | Accept, auth outcome, heartbeat and disconnect |
+
+One File Gateway app lists Accounts, Endpoints, Upload Jobs, Transfers and Connections. Keys,
+outbox files and related jobs/logs appear in typed child lists. Endpoint saved-object pickers
+are filtered by the selected table. There is no HTML widget or navigation process.
+
+| Permission | Grants |
+|---|---|
+| `file_gateway.connect` | SFTP login, checked again when the user session is refreshed |
+| `file_gateway.view` | App and read access to gateway tables |
+| `file_gateway.admin` | Read access and management of accounts, keys, endpoints and record actions |
+
+System-written job/outbox/audit tables are read-only to people, including admins. Record actions
+use the standard Review/Result flow: Unlock account, Disable account, Revoke key, End session and
+Retry upload. End session is enforced by the owning listener's next revocation poll. Retry upload
+requeues FAILED/REJECTED jobs with a completed stored object and resets their attempts; it
+rechecks eligibility under an account lock when applying the action.
+
+The `sftp` channel cannot access gateway, Authentik, audit, credential, API-key, secret, password,
+permission, role, user-session or webhook administration surfaces. The `assistant` channel also
+cannot access gateway tables.
+
+## Listing and authentication
+
+The root lists visible apps; apps contain child apps and readable tables. Table folders include:
+
+- `upload/<profile-slug>/` when the user has insert permission, a visible profile and a bulk-insert process;
+- `views/<slug>.csv` and `.xlsx` (or account-enabled JSON);
+- `reports/<slug>.<format>` when the saved-report table and render process are available;
+- account-scoped `results/<jobId>-<file>.results.csv` and `archive/<yyyy-mm-dd>/<jobId>-<file>`.
+
+Hidden paths return no-such-file. Slug collisions append record IDs. Saved-object queries are
+paged. Metadata and saved objects are cached per connection for 60 and 30 seconds respectively.
+Saved-view/report files report size zero and remain unreadable until phase (c). Stored artifacts
+report their actual lengths and support ranged reads.
+
+Only public-key auth is advertised. Password, keyboard-interactive, GSSAPI, host-based auth,
+shell, exec and forwarding are disabled. The algorithm lists exclude CBC, SHA-1 MAC/KEX and
+`ssh-rsa` signatures. Idle timeout is 15 minutes; authentication timeout is 30 seconds.
+
+Account failure counters, operation-rate windows and session reservations are serialized with a database row lock across
+listeners. Per-IP failure and connection windows are held in each pod. Unlocking an account also
+clears an IP lock caused solely by that account on its next attempt; mixed-user IP locks expire
+independently. User sessions are re-minted after five minutes on the next operation, expire after
+ten minutes, and are revoked on close. The 60-second poll closes disabled/expired/revoked sessions
+and heartbeats connection rows; three-minute-old heartbeats no longer count as active sessions.
+
+Lockouts raise core `QAlerts` signals. **Core compatibility:** the alert type is
+`filegateway.lockout`, because the core validator rejects the design's underscore-only
+`file_gateway_lockout`. Without an alert sink the core logging fallback is used.
+
+## Uploads and object storage
+
+Configure an object-storage bucket alias matching `config.fileGatewayBucketAlias` (default
+`file-gateway`). Production always uses `ObjectStorage.asSystem("file-gateway", runId)` and only
+the allow-listed `com.kingsrook.qbits.objectstorage.api` package. Tests inject a memory store.
+Objects use `file-gateway/uploads/{accountId}/{yyyy}/{mm}/{dd}/{uuid}/{fileName}`;
+results and quarantine use separate `file-gateway/results/` and `file-gateway/quarantine/` prefixes.
+The original object backs the virtual archive; it is not copied after a successful import.
+
+`uploadPartSizeMb` bounds the gateway pipe and `uploadReorderWindowMb` bounds pending writes
+and gaps. The storage API streams unknown-length content and owns multipart aborts. Its actual
+S3 part size is configured on the **object-storage connection** (`multipartPartSizeMb`); set it
+to 8 for the design's default part size. RC.2 does not expose a per-put part-size override through
+its API. Account file-size and UTC daily-upload quotas are reserved in the database while writing,
+including concurrent uploads on other pods. Failed transfers retain the received-byte charge.
+
+The host's `QJavalinMetaData.uploadedFileArchiveTableName` must name a storage-capable table
+backed by object storage, not a filesystem backend. The bulk-insert UI API uses that table for
+its process input. The test harness uses core memory storage. Configure the host HTTP server's
+request limits for the permitted file sizes; these limits also apply to loopback UI API calls.
+
+A normal close queues the job. Names ending in `.filepart`, `.tmp`, `.part` or `.partial` stay
+RECEIVING after close until renamed within the same profile folder. Each put is a new job.
+Dedupe skips content that already SUCCEEDED for the same account, table, profile and consumer
+within `dedupeWindow`; its `.done` text identifies the earlier job.
+
+The worker mints a fresh account-user session, requires `file_gateway.connect`, sends every
+application request through the `sftp` UI API channel, refreshes credentials after five minutes,
+and revokes them after delivery. It follows the bulk-insert process through upload, mapping,
+review and result. Profiles must contain their value mappings; the worker fails instead of
+interactively guessing a mapping. Results contain core summary lines and row examples where
+core supplies them; a blank CSV field column means no field identifier was supplied by core.
+
+**Process steps:** the mapping step sends its values as form fields only; the invoker encodes
+them as multipart because the step operation declares a multipart body (QRun-IO/qqq#1170). The
+file is uploaded once, at process init.
+
+Workers are bounded by `uploadWorkerThreads`. Claims compare the expected status inside a
+short account-row transaction (core's expected-version check alone is not a SQL conditional
+update). Completion also checks the owning pod and claim timestamp. Retries back off by
+1 minute, 5 minutes, 15 minutes and 1 hour, then fail on the fifth attempt. The configured
+schedule also includes a 6-hour slot; the five-attempt cap takes precedence. Consumer-requested
+delays can lengthen the scheduled wait. Stale RUNNING jobs requeue with a three-start cap;
+shutdown releases this pod's claims. Delivery is at least once, so custom consumers must dedupe
+on job ID and digest.
+
+Optional `FileGatewayScannerInterface` implementations return CLEAN, INFECTED or ERROR.
+INFECTED objects are copied into quarantine and removed from the original key; ERROR retries.
+Consumers receive an `InboundFile` with fresh object streams and run with the user's `QContext`.
+Unknown consumer names are rejected.
+
+`<file>.processing` appears during receiving/queueing/running. Finished jobs expose `.done` or
+`.error` text. Sidecars and results disappear after acknowledgement (`rm` on either artifact)
+or seven days. Archive files remain until later retention processing. Core notifications go to
+the account user (service account: `ownerUserId`) on FAILED/PARTIAL/REJECTED and on SUCCEEDED
+when `notifyOnSuccess` is enabled. Without an alerts sink, sending is a no-op.
+
+Existing phase-(a) databases need this additional column before adopting phase (b):
+
+```sql
+ALTER TABLE file_gateway_upload_job ADD COLUMN results_size_bytes BIGINT;
+```
 
 ## Configuration
 
-`FileGatewayQBitConfig` holds the optional `tableMetaDataCustomizer`, `sftpEnabled` (only the
-gateway deployment runs the server) and `sftpPort` (default 2222).
+Every option has a getter and fluent `withX(...)` setter. Configuration for later phases remains
+present without enabling those features.
 
-## Adopting in another app
+| Options | Defaults |
+|---|---|
+| `backendName`, `sftpEnabled`, `sftpPort` | `rdbms`, `false`, `2222` |
+| `hostKeyEnvVarName`, `allowGeneratedHostKey` | `FILE_GATEWAY_SSH_HOST_KEY`, `false` |
+| `lockoutThreshold`, `lockoutWindow`, `lockoutDuration` | 5, 15 minutes, 30 minutes |
+| `maxConnectionsPerIpPerMinute` | 30 |
+| `defaultMaxConcurrentSessions`, `defaultMaxOperationsPerMinute` | 4, 600 |
+| `defaultMaxFileSizeMb` | 100 |
+| `defaultMaxUploadMbPerDay`, `defaultMaxDownloadMbPerDay` | null (unlimited) |
+| `fileGatewayBucketAlias` | `file-gateway` |
+| `uploadPartSizeMb`, `uploadReorderWindowMb`, `downloadReadAheadMb`, `maxGeneratedFileMb` | 8, 64, 16, 1024 |
+| `uploadWorkerThreads`, `dedupeWindow`, `jobStaleAfter` | 4, 7 days, 30 minutes |
+| `temporaryUploadSuffixes` | `.filepart`, `.tmp`, `.part`, `.partial` |
+| `scanner`, `pgpPrivateKeyEnvVarName`, `pgpPassphraseEnvVarName` | null |
+| `logRetention`, `archiveRetention`, `outboxRetention`, `resultsRetention`, `receivingStaleAfter` | 180 days, 30 days, 30 days, 90 days, 1 hour |
+| `tableMetaDataCustomizer` | null |
 
-```java
-new FileGatewayQBitProducer()
-   .withQBitConfig(new FileGatewayQBitConfig())
-   .produce(qInstance);
-```
+## Consumer and outbox API
 
-## Build
+The isolated `com.kingsrook.qbits.filegateway.api` package contains `FileGatewayConsumer`,
+`InboundFile` (including remote address and a fresh stream supplier), `FileAcceptance`,
+`OutboxFileRef`, and the consumer metadata registry. `FileGatewayOutbox` delegates to a registered
+`FileGatewayOutboxProvider`; without one it reports unavailable and throws
+`FileGatewayUnavailableException`. The consumer pipeline is active in phase (b); endpoint binding
+and the outbox provider remain phase (e).
+
+## Build and acceptance tests
 
 ```bash
-./mvnw verify   # Checkstyle; JaCoCo 70% instructions, 90% classes
+./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify
 ```
 
-On Windows, use `./mvnw.cmd`.
+PowerShell:
+
+```powershell
+./mvnw.cmd -o '-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT' verify
+```
+
+The tests use H2 from the adopter DDL, a strict UI API server, generated in-memory SSH keys and
+real MINA clients. They exercise permissions, auth failures, lockout, competing listeners,
+revocation/refresh, pagination/caching, audit, key validation, contingent pickers, API isolation,
+metadata registration and record serialization. Checkstyle and JaCoCo gates require 70%
+instruction coverage and 90% class coverage.
+
+Surefire isolates `user.home` under `target/fg-home` and `java.io.tmpdir` under `target/fg-tmp`.
+The disk guard creates those directories before tests, compares working-tree and home snapshots,
+and requires an empty temp directory after each protocol test. Build output under `target/`
+is excluded from the working-tree snapshot.
 
 ## License
 

docs/DESIGN.md

@@ -390,7 +390,7 @@ Everything is streamed. Nothing is written to disk.
 - **Lockout.**
   - 5 failed auths within 15 minutes, per username or per IP, lock that username or IP for 30
     minutes. All three numbers are configurable.
-  - A lock sets `locked_until` and raises an alert (`file_gateway_lockout`, `QAlerts.raise`).
+  - A lock sets `locked_until` and raises an alert (`filegateway.lockout`, `QAlerts.raise`; core alert-type names use dots, not the table prefix).
   - The record action "Unlock account" clears it.
   - Lockout state lives in the database, so it holds across pods.
   - A per-IP limit on new connections (30 per minute) applies before auth.

docs/ddl/file_gateway.sql

@@ -0,0 +1,164 @@
+-- File Gateway schema. PostgreSQL and H2. Timestamps are UTC, matching QQQ RDBMS binding.
+
+CREATE TABLE file_gateway_account (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   username VARCHAR(1024) NOT NULL UNIQUE,
+   label VARCHAR(1024),
+   account_type VARCHAR(1024),
+   user_id VARCHAR(1024),
+   owner_user_id VARCHAR(1024),
+   status VARCHAR(1024),
+   root_mode VARCHAR(1024),
+   allowed_cidrs TEXT,
+   expires_at TIMESTAMP,
+   formats VARCHAR(1024),
+   max_concurrent_sessions INTEGER,
+   max_operations_per_minute INTEGER,
+   max_file_size_mb INTEGER,
+   max_upload_mb_per_day INTEGER,
+   max_download_mb_per_day INTEGER,
+   dedupe_uploads BOOLEAN,
+   notify_on_success BOOLEAN,
+   pgp_public_key TEXT,
+   pgp_decrypt_uploads BOOLEAN,
+   pgp_encrypt_downloads BOOLEAN,
+   operation_window_at TIMESTAMP,
+   operation_window_count INTEGER,
+   failed_login_count INTEGER,
+   failed_login_window_at TIMESTAMP,
+   locked_until TIMESTAMP,
+   last_login_at TIMESTAMP,
+   last_login_ip VARCHAR(1024)
+);
+
+CREATE TABLE file_gateway_ssh_key (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   account_id INTEGER,
+   label VARCHAR(1024),
+   public_key TEXT,
+   fingerprint VARCHAR(1024) NOT NULL UNIQUE,
+   key_type VARCHAR(1024),
+   key_bits INTEGER,
+   expires_at TIMESTAMP,
+   revoked_at TIMESTAMP,
+   last_used_at TIMESTAMP,
+   last_used_ip VARCHAR(1024)
+);
+
+CREATE TABLE file_gateway_alias (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   name VARCHAR(1024) NOT NULL UNIQUE,
+   path VARCHAR(1024) NOT NULL UNIQUE,
+   kind VARCHAR(1024),
+   account_id INTEGER,
+   table_name VARCHAR(1024),
+   saved_bulk_load_profile_id INTEGER,
+   saved_view_id INTEGER,
+   saved_report_id INTEGER,
+   format VARCHAR(1024),
+   file_name VARCHAR(1024),
+   consumer_name VARCHAR(1024),
+   is_active BOOLEAN,
+   description TEXT
+);
+
+CREATE TABLE file_gateway_upload_job (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   account_id INTEGER,
+   alias_id INTEGER,
+   connection_log_id INTEGER,
+   remote_address VARCHAR(1024),
+   table_name VARCHAR(1024),
+   saved_bulk_load_profile_id INTEGER,
+   consumer_name VARCHAR(1024),
+   consumer_reference VARCHAR(1024),
+   path VARCHAR(1024),
+   file_name VARCHAR(1024),
+   size_bytes BIGINT,
+   sha256 VARCHAR(1024),
+   storage_key VARCHAR(1024),
+   status VARCHAR(1024),
+   duplicate_of_job_id INTEGER,
+   process_uuid VARCHAR(1024),
+   inserted_count INTEGER,
+   error_count INTEGER,
+   result_summary TEXT,
+   results_storage_key VARCHAR(1024),
+   results_size_bytes BIGINT,
+   error_message TEXT,
+   attempts INTEGER,
+   next_attempt_at TIMESTAMP,
+   claimed_by VARCHAR(1024),
+   claimed_at TIMESTAMP,
+   queued_at TIMESTAMP,
+   started_at TIMESTAMP,
+   finished_at TIMESTAMP,
+   acknowledged_at TIMESTAMP
+);
+
+CREATE TABLE file_gateway_outbox_file (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   alias_id INTEGER,
+   file_name VARCHAR(1024),
+   size_bytes BIGINT,
+   sha256 VARCHAR(1024),
+   storage_key VARCHAR(1024),
+   status VARCHAR(1024),
+   first_downloaded_at TIMESTAMP,
+   download_count INTEGER,
+   removed_at TIMESTAMP,
+   expires_at TIMESTAMP,
+   created_by_consumer VARCHAR(1024)
+);
+
+CREATE TABLE file_gateway_transfer_log (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   account_id INTEGER,
+   connection_log_id INTEGER,
+   operation VARCHAR(1024),
+   path VARCHAR(1024),
+   object_kind VARCHAR(1024),
+   object_id VARCHAR(1024),
+   format VARCHAR(1024),
+   bytes BIGINT,
+   sha256 VARCHAR(1024),
+   duration_ms BIGINT,
+   outcome VARCHAR(1024),
+   message TEXT,
+   client_channel VARCHAR(1024),
+   upload_job_id INTEGER
+);
+
+CREATE TABLE file_gateway_connection_log (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   create_date TIMESTAMP,
+   modify_date TIMESTAMP,
+   account_id INTEGER,
+   username VARCHAR(1024),
+   remote_ip VARCHAR(1024),
+   client_version VARCHAR(1024),
+   key_fingerprint VARCHAR(1024),
+   outcome VARCHAR(1024),
+   session_id VARCHAR(1024),
+   pod_name VARCHAR(1024),
+   connected_at TIMESTAMP,
+   last_seen_at TIMESTAMP,
+   disconnected_at TIMESTAMP,
+   disconnect_reason TEXT,
+   bytes_in BIGINT,
+   bytes_out BIGINT,
+   operation_count BIGINT,
+   client_channel VARCHAR(1024)
+);

pom.xml

@@ -27,6 +27,7 @@
       <maven.compiler.target>21</maven.compiler.target>
       <jacoco.version>0.8.15</jacoco.version>
       <h2.version>2.2.224</h2.version>
+      <sshd.version>2.16.0</sshd.version>
    </properties>
 
    <parent>
@@ -72,6 +73,27 @@
    </scm>
 
    <dependencies>
+      <dependency>
+         <groupId>com.kingsrook.qbits</groupId>
+         <artifactId>qbit-object-storage</artifactId>
+         <version>0.1.0-RC.2</version>
+      </dependency>
+      <!-- MINA's Ed25519 key decoder and signer. -->
+      <dependency>
+         <groupId>net.i2p.crypto</groupId>
+         <artifactId>eddsa</artifactId>
+         <version>0.3.0</version>
+      </dependency>
+      <dependency>
+         <groupId>org.apache.sshd</groupId>
+         <artifactId>sshd-core</artifactId>
+         <version>${sshd.version}</version>
+      </dependency>
+      <dependency>
+         <groupId>org.apache.sshd</groupId>
+         <artifactId>sshd-sftp</artifactId>
+         <version>${sshd.version}</version>
+      </dependency>
       <dependency>
          <groupId>com.kingsrook.qqq</groupId>
          <artifactId>qqq-backend-core</artifactId>
@@ -114,6 +136,16 @@
 
    <build>
       <plugins>
+         <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-surefire-plugin</artifactId>
+            <configuration>
+               <systemPropertyVariables>
+                  <java.io.tmpdir>${project.build.directory}/fg-tmp</java.io.tmpdir>
+                  <user.home>${project.build.directory}/fg-home</user.home>
+               </systemPropertyVariables>
+            </configuration>
+         </plugin>
          <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-checkstyle-plugin</artifactId>

src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitConfig.java

@@ -21,46 +21,77 @@
 package com.kingsrook.qbits.filegateway;
 
 
+import java.time.Duration;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 
 
 /*******************************************************************************
- ** Configuration for the file-gateway qbit.
+ ** Configuration shared by the admin application and gateway deployment.
  *******************************************************************************/
 public class FileGatewayQBitConfig implements QBitConfig
 {
-   private MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer;
+   private String backendName = "rdbms";
    private boolean sftpEnabled = false;
-   private int     sftpPort    = 2222;
+   private int sftpPort = 2222;
+   private String hostKeyEnvVarName = "FILE_GATEWAY_SSH_HOST_KEY";
+   private boolean allowGeneratedHostKey = false;
+   private String fileGatewayBucketAlias = "file-gateway";
+   private int uploadPartSizeMb = 8;
+   private int uploadReorderWindowMb = 64;
+   private int downloadReadAheadMb = 16;
+   private int maxGeneratedFileMb = 1024;
+   private int uploadWorkerThreads = 4;
+   private Duration dedupeWindow = Duration.ofDays(7);
+   private Duration jobStaleAfter = Duration.ofMinutes(30);
+   private List<String> temporaryUploadSuffixes = List.of(".filepart", ".tmp", ".part", ".partial");
+   private int lockoutThreshold = 5;
+   private Duration lockoutWindow = Duration.ofMinutes(15);
+   private Duration lockoutDuration = Duration.ofMinutes(30);
+   private int maxConnectionsPerIpPerMinute = 30;
+   private int defaultMaxConcurrentSessions = 4;
+   private int defaultMaxOperationsPerMinute = 600;
+   private int defaultMaxFileSizeMb = 100;
+   private Integer defaultMaxUploadMbPerDay = null;
+   private Integer defaultMaxDownloadMbPerDay = null;
+   private QCodeReference scanner = null;
+   private String pgpPrivateKeyEnvVarName = null;
+   private String pgpPassphraseEnvVarName = null;
+   private Duration logRetention = Duration.ofDays(180);
+   private Duration archiveRetention = Duration.ofDays(30);
+   private Duration outboxRetention = Duration.ofDays(30);
+   private Duration resultsRetention = Duration.ofDays(90);
+   private Duration receivingStaleAfter = Duration.ofHours(1);
+   private MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer = null;
 
 
 
    /*******************************************************************************
-    ** Getter for the optional table customizer.
+    ** Getter for backendName.
     *******************************************************************************/
-   @Override
-   public MetaDataCustomizerInterface<QTableMetaData> getTableMetaDataCustomizer()
+   public String getBackendName()
    {
-      return (tableMetaDataCustomizer);
+      return (backendName);
    }
 
 
 
    /*******************************************************************************
-    ** Fluent setter for the optional table customizer.
+    ** Fluent setter for backendName.
     *******************************************************************************/
-   public FileGatewayQBitConfig withTableMetaDataCustomizer(MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer)
+   public FileGatewayQBitConfig withBackendName(String backendName)
    {
-      this.tableMetaDataCustomizer = tableMetaDataCustomizer;
+      this.backendName = backendName;
       return (this);
    }
 
 
 
    /*******************************************************************************
-    ** Whether this process runs the SFTP server (only the gateway deployment does).
+    ** Getter for sftpEnabled.
     *******************************************************************************/
    public boolean getSftpEnabled()
    {
@@ -70,7 +101,7 @@ public class FileGatewayQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    ** Fluent setter for whether this process runs the SFTP server.
+    ** Fluent setter for sftpEnabled.
     *******************************************************************************/
    public FileGatewayQBitConfig withSftpEnabled(boolean sftpEnabled)
    {
@@ -81,7 +112,7 @@ public class FileGatewayQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    ** Getter for the SFTP listen port (default 2222).
+    ** Getter for sftpPort.
     *******************************************************************************/
    public int getSftpPort()
    {
@@ -91,11 +122,620 @@ public class FileGatewayQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    ** Fluent setter for the SFTP listen port.
+    ** Fluent setter for sftpPort.
     *******************************************************************************/
    public FileGatewayQBitConfig withSftpPort(int sftpPort)
    {
       this.sftpPort = sftpPort;
       return (this);
    }
+
+
+
+   /*******************************************************************************
+    ** Getter for hostKeyEnvVarName.
+    *******************************************************************************/
+   public String getHostKeyEnvVarName()
+   {
+      return (hostKeyEnvVarName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for hostKeyEnvVarName.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withHostKeyEnvVarName(String hostKeyEnvVarName)
+   {
+      this.hostKeyEnvVarName = hostKeyEnvVarName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for allowGeneratedHostKey.
+    *******************************************************************************/
+   public boolean getAllowGeneratedHostKey()
+   {
+      return (allowGeneratedHostKey);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for allowGeneratedHostKey.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withAllowGeneratedHostKey(boolean allowGeneratedHostKey)
+   {
+      this.allowGeneratedHostKey = allowGeneratedHostKey;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fileGatewayBucketAlias.
+    *******************************************************************************/
+   public String getFileGatewayBucketAlias()
+   {
+      return (fileGatewayBucketAlias);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fileGatewayBucketAlias.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withFileGatewayBucketAlias(String fileGatewayBucketAlias)
+   {
+      this.fileGatewayBucketAlias = fileGatewayBucketAlias;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for uploadPartSizeMb.
+    *******************************************************************************/
+   public int getUploadPartSizeMb()
+   {
+      return (uploadPartSizeMb);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for uploadPartSizeMb.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withUploadPartSizeMb(int uploadPartSizeMb)
+   {
+      this.uploadPartSizeMb = uploadPartSizeMb;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for uploadReorderWindowMb.
+    *******************************************************************************/
+   public int getUploadReorderWindowMb()
+   {
+      return (uploadReorderWindowMb);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for uploadReorderWindowMb.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withUploadReorderWindowMb(int uploadReorderWindowMb)
+   {
+      this.uploadReorderWindowMb = uploadReorderWindowMb;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for downloadReadAheadMb.
+    *******************************************************************************/
+   public int getDownloadReadAheadMb()
+   {
+      return (downloadReadAheadMb);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for downloadReadAheadMb.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDownloadReadAheadMb(int downloadReadAheadMb)
+   {
+      this.downloadReadAheadMb = downloadReadAheadMb;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for maxGeneratedFileMb.
+    *******************************************************************************/
+   public int getMaxGeneratedFileMb()
+   {
+      return (maxGeneratedFileMb);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for maxGeneratedFileMb.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withMaxGeneratedFileMb(int maxGeneratedFileMb)
+   {
+      this.maxGeneratedFileMb = maxGeneratedFileMb;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for uploadWorkerThreads.
+    *******************************************************************************/
+   public int getUploadWorkerThreads()
+   {
+      return (uploadWorkerThreads);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for uploadWorkerThreads.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withUploadWorkerThreads(int uploadWorkerThreads)
+   {
+      this.uploadWorkerThreads = uploadWorkerThreads;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for dedupeWindow.
+    *******************************************************************************/
+   public Duration getDedupeWindow()
+   {
+      return (dedupeWindow);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for dedupeWindow.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDedupeWindow(Duration dedupeWindow)
+   {
+      this.dedupeWindow = dedupeWindow;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for jobStaleAfter.
+    *******************************************************************************/
+   public Duration getJobStaleAfter()
+   {
+      return (jobStaleAfter);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for jobStaleAfter.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withJobStaleAfter(Duration jobStaleAfter)
+   {
+      this.jobStaleAfter = jobStaleAfter;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for temporaryUploadSuffixes.
+    *******************************************************************************/
+   public List<String> getTemporaryUploadSuffixes()
+   {
+      return (temporaryUploadSuffixes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for temporaryUploadSuffixes.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withTemporaryUploadSuffixes(List<String> temporaryUploadSuffixes)
+   {
+      this.temporaryUploadSuffixes = temporaryUploadSuffixes;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lockoutThreshold.
+    *******************************************************************************/
+   public int getLockoutThreshold()
+   {
+      return (lockoutThreshold);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lockoutThreshold.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withLockoutThreshold(int lockoutThreshold)
+   {
+      this.lockoutThreshold = lockoutThreshold;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lockoutWindow.
+    *******************************************************************************/
+   public Duration getLockoutWindow()
+   {
+      return (lockoutWindow);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lockoutWindow.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withLockoutWindow(Duration lockoutWindow)
+   {
+      this.lockoutWindow = lockoutWindow;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lockoutDuration.
+    *******************************************************************************/
+   public Duration getLockoutDuration()
+   {
+      return (lockoutDuration);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lockoutDuration.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withLockoutDuration(Duration lockoutDuration)
+   {
+      this.lockoutDuration = lockoutDuration;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for maxConnectionsPerIpPerMinute.
+    *******************************************************************************/
+   public int getMaxConnectionsPerIpPerMinute()
+   {
+      return (maxConnectionsPerIpPerMinute);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for maxConnectionsPerIpPerMinute.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withMaxConnectionsPerIpPerMinute(int maxConnectionsPerIpPerMinute)
+   {
+      this.maxConnectionsPerIpPerMinute = maxConnectionsPerIpPerMinute;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for defaultMaxConcurrentSessions.
+    *******************************************************************************/
+   public int getDefaultMaxConcurrentSessions()
+   {
+      return (defaultMaxConcurrentSessions);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for defaultMaxConcurrentSessions.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDefaultMaxConcurrentSessions(int defaultMaxConcurrentSessions)
+   {
+      this.defaultMaxConcurrentSessions = defaultMaxConcurrentSessions;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for defaultMaxOperationsPerMinute.
+    *******************************************************************************/
+   public int getDefaultMaxOperationsPerMinute()
+   {
+      return (defaultMaxOperationsPerMinute);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for defaultMaxOperationsPerMinute.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDefaultMaxOperationsPerMinute(int defaultMaxOperationsPerMinute)
+   {
+      this.defaultMaxOperationsPerMinute = defaultMaxOperationsPerMinute;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for defaultMaxFileSizeMb.
+    *******************************************************************************/
+   public int getDefaultMaxFileSizeMb()
+   {
+      return (defaultMaxFileSizeMb);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for defaultMaxFileSizeMb.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDefaultMaxFileSizeMb(int defaultMaxFileSizeMb)
+   {
+      this.defaultMaxFileSizeMb = defaultMaxFileSizeMb;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for defaultMaxUploadMbPerDay.
+    *******************************************************************************/
+   public Integer getDefaultMaxUploadMbPerDay()
+   {
+      return (defaultMaxUploadMbPerDay);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for defaultMaxUploadMbPerDay.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDefaultMaxUploadMbPerDay(Integer defaultMaxUploadMbPerDay)
+   {
+      this.defaultMaxUploadMbPerDay = defaultMaxUploadMbPerDay;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for defaultMaxDownloadMbPerDay.
+    *******************************************************************************/
+   public Integer getDefaultMaxDownloadMbPerDay()
+   {
+      return (defaultMaxDownloadMbPerDay);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for defaultMaxDownloadMbPerDay.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withDefaultMaxDownloadMbPerDay(Integer defaultMaxDownloadMbPerDay)
+   {
+      this.defaultMaxDownloadMbPerDay = defaultMaxDownloadMbPerDay;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for scanner.
+    *******************************************************************************/
+   public QCodeReference getScanner()
+   {
+      return (scanner);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for scanner.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withScanner(QCodeReference scanner)
+   {
+      this.scanner = scanner;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for pgpPrivateKeyEnvVarName.
+    *******************************************************************************/
+   public String getPgpPrivateKeyEnvVarName()
+   {
+      return (pgpPrivateKeyEnvVarName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for pgpPrivateKeyEnvVarName.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withPgpPrivateKeyEnvVarName(String pgpPrivateKeyEnvVarName)
+   {
+      this.pgpPrivateKeyEnvVarName = pgpPrivateKeyEnvVarName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for pgpPassphraseEnvVarName.
+    *******************************************************************************/
+   public String getPgpPassphraseEnvVarName()
+   {
+      return (pgpPassphraseEnvVarName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for pgpPassphraseEnvVarName.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withPgpPassphraseEnvVarName(String pgpPassphraseEnvVarName)
+   {
+      this.pgpPassphraseEnvVarName = pgpPassphraseEnvVarName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for logRetention.
+    *******************************************************************************/
+   public Duration getLogRetention()
+   {
+      return (logRetention);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for logRetention.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withLogRetention(Duration logRetention)
+   {
+      this.logRetention = logRetention;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for archiveRetention.
+    *******************************************************************************/
+   public Duration getArchiveRetention()
+   {
+      return (archiveRetention);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for archiveRetention.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withArchiveRetention(Duration archiveRetention)
+   {
+      this.archiveRetention = archiveRetention;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for outboxRetention.
+    *******************************************************************************/
+   public Duration getOutboxRetention()
+   {
+      return (outboxRetention);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for outboxRetention.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withOutboxRetention(Duration outboxRetention)
+   {
+      this.outboxRetention = outboxRetention;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for resultsRetention.
+    *******************************************************************************/
+   public Duration getResultsRetention()
+   {
+      return (resultsRetention);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for resultsRetention.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withResultsRetention(Duration resultsRetention)
+   {
+      this.resultsRetention = resultsRetention;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for receivingStaleAfter.
+    *******************************************************************************/
+   public Duration getReceivingStaleAfter()
+   {
+      return (receivingStaleAfter);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for receivingStaleAfter.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withReceivingStaleAfter(Duration receivingStaleAfter)
+   {
+      this.receivingStaleAfter = receivingStaleAfter;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tableMetaDataCustomizer.
+    *******************************************************************************/
+   public MetaDataCustomizerInterface<QTableMetaData> getTableMetaDataCustomizer()
+   {
+      return (tableMetaDataCustomizer);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tableMetaDataCustomizer.
+    *******************************************************************************/
+   public FileGatewayQBitConfig withTableMetaDataCustomizer(MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer)
+   {
+      this.tableMetaDataCustomizer = tableMetaDataCustomizer;
+      return (this);
+   }
 }

src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducer.java

@@ -21,17 +21,37 @@
 package com.kingsrook.qbits.filegateway;
 
 
+import java.util.List;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayProcesses;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions;
+import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.ClientChannelsMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 
 
 /*******************************************************************************
- ** Entry point for the file-gateway qbit; no tables are produced yet.
+ ** Entry point for the file-gateway metadata; runtime startup belongs to the host.
  *******************************************************************************/
 public class FileGatewayQBitProducer implements QBitMetaDataProducer<FileGatewayQBitConfig>
 {
+   /*******************************************************************************
+    ** The sftp wall owns its list (core's qbit prefixes are transitional and the
+    ** assistant may now read audits): partners never reach identity, audits,
+    ** webhooks or this qbit's own tables, read or write. Core's generic secret,
+    ** credential, permission and session patterns are added on top.
+    *******************************************************************************/
+   public static final java.util.List<String> SFTP_DENIED_PATTERNS = java.util.List.of(
+      "file_gateway_*", "audit*", "authentik_*", "webhooks_*", "worm_audit_*", "secured_data_*", "assistant_*");
+
    public static final String GROUP_ID    = "com.kingsrook.qbits";
    public static final String ARTIFACT_ID = "qbit-file-gateway";
    public static final String VERSION     = "0.1.0";
@@ -108,13 +128,56 @@ public class FileGatewayQBitProducer implements QBitMetaDataProducer<FileGateway
 
 
    /*******************************************************************************
-    ** Register this qbit and return its metadata, without producing tables.
+    ** Register tables, child lists, permissions, channel walls and the application.
     *******************************************************************************/
    @Override
-   public MetaDataProducerMultiOutput produce(QInstance qInstance)
+   public MetaDataProducerMultiOutput produce(QInstance qInstance) throws QException
    {
+      if(!isEnabled())
+      {
+         return (new MetaDataProducerMultiOutput());
+      }
       QBitMetaData qBit = getQBitMetaData();
-      qInstance.addQBit(qBit);
-      return (new MetaDataProducerMultiOutput().with(qBit));
+      MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput().with(qBit);
+      FileGatewayTables.possibleValueSources().forEach(output::add);
+      QAppMetaData app = new QAppMetaData().withName("file_gateway").withLabel("File Gateway")
+         .withIcon(new QIcon("folder_shared")).withPermissionRules(FileGatewayTables.permissionRules("file_gateway"));
+      for(QTableMetaData table : FileGatewayTables.tables(qBitConfig))
+      {
+         if(qBitConfig.getTableMetaDataCustomizer() != null)
+         {
+            table = qBitConfig.getTableMetaDataCustomizer().customizeMetaData(qInstance, table);
+         }
+         table.setSourceQBitName(qBit.getName());
+         output.add(table);
+         if(FileGatewayTables.MAIN.contains(table.getName()))
+         {
+            app.withChild(table);
+         }
+      }
+      app.withSection(new QAppSection().withName("configuration").withLabel("Configuration")
+         .withTables(List.of(FileGatewayTables.ACCOUNT, FileGatewayTables.ALIAS)));
+      app.withSection(new QAppSection().withName("activity").withLabel("Activity")
+         .withTables(List.of(FileGatewayTables.UPLOAD_JOB, FileGatewayTables.TRANSFER_LOG, FileGatewayTables.CONNECTION_LOG)));
+      FileGatewayTables.joins().forEach(output::add);
+      FileGatewayTables.childWidgets().forEach(output::add);
+      output.add(app);
+      FileGatewayProcesses.processes().forEach(output::add);
+      output.add(new QAlertTypeMetaData().withName("filegateway.lockout").withLabel("File Gateway Lockout")
+         .withSourceQBitName(qBit.getName()).withSubjectTableName(FileGatewayTables.ACCOUNT)
+         .withVisibilityPermission(FileGatewayPermissions.VIEW));
+      output.addSelfToInstance(qInstance);
+      ClientChannelsMetaData channels = ClientChannelsMetaData.ofOrWithNew(qInstance)
+         .withChannel("sftp", "SFTP file gateway").withDeniedPattern("sftp", "file_gateway_*")
+         .withDeniedPattern("assistant", "file_gateway_*");
+      for(String pattern : SFTP_DENIED_PATTERNS)
+      {
+         channels.withDeniedPattern("sftp", pattern);
+      }
+      for(String pattern : ClientChannelsMetaData.ASSISTANT_GENERIC_DENIED_PATTERNS)
+      {
+         channels.withDeniedPattern("sftp", pattern);
+      }
+      return (new MetaDataProducerMultiOutput());
    }
 }

src/main/java/com/kingsrook/qbits/filegateway/FileGatewayRuntime.java

@@ -0,0 +1,201 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.security.KeyPair;
+import java.time.Clock;
+import java.time.Duration;
+import java.util.function.Function;
+import com.kingsrook.qbits.filegateway.sftp.SftpProtocolServer;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+import com.kingsrook.qbits.filegateway.storage.ObjectStorageGatewayStore;
+import com.kingsrook.qbits.filegateway.upload.UploadJobWorker;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import org.apache.sshd.common.NamedResource;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.util.security.SecurityUtils;
+
+
+/*******************************************************************************
+ ** Host-started SFTP runtime. Host private keys are parsed in memory and never persisted.
+ *******************************************************************************/
+public final class FileGatewayRuntime implements AutoCloseable
+{
+   private static final QLogger LOG = QLogger.getLogger(FileGatewayRuntime.class);
+   private final UiApiInvoker invoker;
+   private final SftpProtocolServer protocol;
+   private final UploadJobWorker worker;
+
+
+
+   /*******************************************************************************
+    ** Bind the host UI API server to the SFTP channel.
+    *******************************************************************************/
+   private FileGatewayRuntime(QInstance instance, QApplicationJavalinServer server, FileGatewayQBitConfig config, KeyPair key, Clock clock, Duration interval, String podName, GatewayObjectStore store)
+   {
+      invoker = UiApiInvoker.forServer(server, "sftp");
+      protocol = new SftpProtocolServer(instance, config, invoker, key, clock, interval, podName, store);
+      worker = new UploadJobWorker(instance, invoker, config, store, clock, podName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Start only when the registered qbit configuration enables SFTP.
+    *******************************************************************************/
+   public static FileGatewayRuntime start(QInstance instance, QApplicationJavalinServer server) throws Exception
+   {
+      return (start(instance, server, System::getenv));
+   }
+
+
+
+   /*******************************************************************************
+    ** Start with an injectable environment reader.
+    *******************************************************************************/
+   public static FileGatewayRuntime start(QInstance instance, QApplicationJavalinServer server, Function<String, String> environment) throws Exception
+   {
+      var qbit = instance.getQBits().get(FileGatewayQBitProducer.GROUP_ID + ":" + FileGatewayQBitProducer.ARTIFACT_ID);
+      if(qbit == null || !(qbit.getConfig() instanceof FileGatewayQBitConfig config) || !config.getSftpEnabled())
+      {
+         return (null);
+      }
+      FileGatewayRuntime runtime = new FileGatewayRuntime(instance, server, config, hostKey(config, environment), Clock.systemUTC(), Duration.ofSeconds(60), environment.apply("HOSTNAME"), new ObjectStorageGatewayStore(instance, config.getFileGatewayBucketAlias()));
+      runtime.protocol.start();
+      runtime.worker.start(Duration.ofSeconds(60));
+      return (runtime);
+   }
+
+
+
+   /*******************************************************************************
+    ** Start with an explicit ephemeral port, in-memory host key and controllable clock.
+    *******************************************************************************/
+   public static FileGatewayRuntime startForTest(QInstance instance, QApplicationJavalinServer server, FileGatewayQBitConfig config, KeyPair key, Clock clock, Duration pollInterval) throws IOException
+   {
+      return (startForTest(instance, server, config, key, clock, pollInterval, new ObjectStorageGatewayStore(instance, config.getFileGatewayBucketAlias())));
+   }
+
+
+
+   /*******************************************************************************
+    ** Transfer tests supply an in-memory store with no production fallback.
+    *******************************************************************************/
+   public static FileGatewayRuntime startForTest(QInstance instance, QApplicationJavalinServer server, FileGatewayQBitConfig config, KeyPair key, Clock clock, Duration pollInterval, GatewayObjectStore store) throws IOException
+   {
+      FileGatewayRuntime runtime = new FileGatewayRuntime(instance, server, config, key, clock, pollInterval, "test", store);
+      runtime.protocol.start();
+      return (runtime);
+   }
+
+
+
+   /*******************************************************************************
+    ** Read an OpenSSH private key directly from memory.
+    *******************************************************************************/
+   public static KeyPair hostKey(FileGatewayQBitConfig config, Function<String, String> environment) throws Exception
+   {
+      String pem = environment.apply(config.getHostKeyEnvVarName());
+      if(pem == null || pem.isBlank())
+      {
+         if(!config.getAllowGeneratedHostKey())
+         {
+            throw (new IllegalStateException("SFTP host key environment variable is missing: " + config.getHostKeyEnvVarName()));
+         }
+         LOG.warn("Using a generated in-memory SFTP host key; do not use in production");
+         return (KeyUtils.generateKeyPair("ecdsa-sha2-nistp256", 256));
+      }
+      try(var stream = new ByteArrayInputStream(pem.getBytes(StandardCharsets.UTF_8)))
+      {
+         var keys = SecurityUtils.loadKeyPairIdentities(null, NamedResource.ofName("environment-host-key"), stream, null).iterator();
+         if(!keys.hasNext())
+         {
+            throw (new IllegalArgumentException("No SFTP host key in environment variable"));
+         }
+         return (keys.next());
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Stop the SFTP runtime.
+    *******************************************************************************/
+   public void stop() throws IOException
+   {
+      protocol.stop();
+      try
+      {
+         worker.close();
+      }
+      catch(com.kingsrook.qqq.backend.core.exceptions.QException e)
+      {
+         throw (new IOException("Unable to release upload jobs", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Close this runtime.
+    *******************************************************************************/
+   @Override
+   public void close() throws IOException
+   {
+      stop();
+   }
+
+
+
+   /*******************************************************************************
+    ** Access the protocol listener.
+    *******************************************************************************/
+   public SftpProtocolServer getProtocol()
+   {
+      return (protocol);
+   }
+
+
+
+   /*******************************************************************************
+    ** The fixed sftp-channel invoker.
+    *******************************************************************************/
+   public UiApiInvoker getInvoker()
+   {
+      return (invoker);
+   }
+
+   /*******************************************************************************
+    ** Deterministic worker hook for hosts and integration tests.
+    *******************************************************************************/
+   public UploadJobWorker getWorker()
+   {
+      return (worker);
+   }
+
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileAcceptance.java

@@ -0,0 +1,127 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.time.Duration;
+
+
+/*******************************************************************************
+ ** Consumer decision, including its reference or reason and retry delay.
+ *******************************************************************************/
+public final class FileAcceptance
+{
+   /*******************************************************************************
+    ** Delivery decisions.
+    *******************************************************************************/
+   public enum Status
+   {
+      ACCEPTED, REJECTED, RETRY_LATER
+   }
+
+   private final Status status;
+   private final String consumerReference;
+   private final String reason;
+   private final Duration retryAfter;
+
+
+
+   /*******************************************************************************
+    ** Create an immutable decision.
+    *******************************************************************************/
+   private FileAcceptance(Status status, String consumerReference, String reason, Duration retryAfter)
+   {
+      this.status = status;
+      this.consumerReference = consumerReference;
+      this.reason = reason;
+      this.retryAfter = retryAfter;
+   }
+
+
+
+   /*******************************************************************************
+    ** Accept the delivery.
+    *******************************************************************************/
+   public static FileAcceptance accepted(String consumerReference)
+   {
+      return (new FileAcceptance(Status.ACCEPTED, consumerReference, null, null));
+   }
+
+
+
+   /*******************************************************************************
+    ** Reject the delivery.
+    *******************************************************************************/
+   public static FileAcceptance rejected(String reason)
+   {
+      return (new FileAcceptance(Status.REJECTED, null, reason, null));
+   }
+
+
+
+   /*******************************************************************************
+    ** Request a retry.
+    *******************************************************************************/
+   public static FileAcceptance retryLater(String reason, Duration retryAfter)
+   {
+      return (new FileAcceptance(Status.RETRY_LATER, null, reason, retryAfter));
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for status.
+    *******************************************************************************/
+   public Status getStatus()
+   {
+      return (status);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for consumerReference.
+    *******************************************************************************/
+   public String getConsumerReference()
+   {
+      return (consumerReference);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for reason.
+    *******************************************************************************/
+   public String getReason()
+   {
+      return (reason);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for retryAfter.
+    *******************************************************************************/
+   public Duration getRetryAfter()
+   {
+      return (retryAfter);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayConsumer.java

@@ -0,0 +1,49 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+
+
+/*******************************************************************************
+ ** Consumer SPI for at-least-once inbound delivery and download notification.
+ *******************************************************************************/
+public interface FileGatewayConsumer
+{
+   /*******************************************************************************
+    ** Stable lowercase consumer name.
+    *******************************************************************************/
+   String getName();
+
+   /*******************************************************************************
+    ** Accept, reject or request a later retry.
+    *******************************************************************************/
+   FileAcceptance accept(InboundFile file) throws Exception;
+
+
+
+   /*******************************************************************************
+    ** Called after the first complete download.
+    *******************************************************************************/
+   default void onDownloaded(OutboxFileRef ref) throws Exception
+   {
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayConsumersMetaData.java

@@ -0,0 +1,79 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+
+
+/*******************************************************************************
+ ** Host-declared consumer implementations; contains no gateway runtime dependencies.
+ *******************************************************************************/
+public class FileGatewayConsumersMetaData implements QSupplementalInstanceMetaData
+{
+   public static final String NAME = "fileGatewayConsumers";
+   private final List<QCodeReference> consumers = new ArrayList<>();
+
+
+
+   /*******************************************************************************
+    ** Metadata registry name.
+    *******************************************************************************/
+   public String getName()
+   {
+      return (NAME);
+   }
+
+
+
+   /*******************************************************************************
+    ** Get or install the registry.
+    *******************************************************************************/
+   public static FileGatewayConsumersMetaData ofOrWithNew(QInstance instance)
+   {
+      return (QSupplementalInstanceMetaData.ofOrWithNew(instance, NAME, FileGatewayConsumersMetaData::new));
+   }
+
+
+
+   /*******************************************************************************
+    ** Register a consumer implementation.
+    *******************************************************************************/
+   public FileGatewayConsumersMetaData withConsumer(QCodeReference consumer)
+   {
+      consumers.add(consumer);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Registered consumer implementations.
+    *******************************************************************************/
+   public List<QCodeReference> getConsumers()
+   {
+      return (List.copyOf(consumers));
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayOutbox.java

@@ -0,0 +1,78 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.io.InputStream;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+
+
+/*******************************************************************************
+ ** Optional outbox facade; the host runtime owns provider registration.
+ *******************************************************************************/
+public final class FileGatewayOutbox
+{
+   private static volatile FileGatewayOutboxProvider provider;
+
+
+
+   /*******************************************************************************
+    ** Static facade.
+    *******************************************************************************/
+   private FileGatewayOutbox()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Install or clear the runtime provider.
+    *******************************************************************************/
+   public static void setProvider(FileGatewayOutboxProvider provider)
+   {
+      FileGatewayOutbox.provider = provider;
+   }
+
+
+
+   /*******************************************************************************
+    ** Whether the runtime supports outbox writes.
+    *******************************************************************************/
+   public static boolean isAvailable()
+   {
+      return (provider != null);
+   }
+
+
+
+   /*******************************************************************************
+    ** Delegate to the installed provider.
+    *******************************************************************************/
+   public static OutboxFileRef put(String endpointName, String fileName, InputStream content, long sizeBytes) throws FileGatewayUnavailableException, QException
+   {
+      FileGatewayOutboxProvider current = provider;
+      if(current == null)
+      {
+         throw (new FileGatewayUnavailableException());
+      }
+      return (current.put(endpointName, fileName, content, sizeBytes));
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayOutboxProvider.java

@@ -0,0 +1,37 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.io.InputStream;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+
+
+/*******************************************************************************
+ ** Runtime implementation registered with the outbox facade.
+ *******************************************************************************/
+public interface FileGatewayOutboxProvider
+{
+   /*******************************************************************************
+    ** Stream content into the endpoint outbox.
+    *******************************************************************************/
+   OutboxFileRef put(String endpointName, String fileName, InputStream content, long sizeBytes) throws QException;
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayScannerInterface.java

@@ -0,0 +1,41 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+/*******************************************************************************
+ ** Optional streaming scanner invoked before delivery to a consumer.
+ *******************************************************************************/
+@FunctionalInterface
+public interface FileGatewayScannerInterface
+{
+   /*******************************************************************************
+    ** ERROR is transient; INFECTED quarantines the object without delivery.
+    *******************************************************************************/
+   enum Result
+   {
+      CLEAN, INFECTED, ERROR
+   }
+
+   /*******************************************************************************
+    ** Inspect the file with fresh streams; implementations must close opened streams.
+    *******************************************************************************/
+   Result scan(InboundFile file) throws Exception;
+}

src/main/java/com/kingsrook/qbits/filegateway/api/FileGatewayUnavailableException.java

@@ -0,0 +1,41 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+
+
+/*******************************************************************************
+ ** The outbox runtime has not been installed.
+ *******************************************************************************/
+public class FileGatewayUnavailableException extends QException
+{
+
+
+   /*******************************************************************************
+    ** Create the unavailable error.
+    *******************************************************************************/
+   public FileGatewayUnavailableException()
+   {
+      super("File gateway outbox is unavailable");
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/InboundFile.java

@@ -0,0 +1,45 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.io.InputStream;
+import java.time.Instant;
+import java.util.function.Supplier;
+
+
+/*******************************************************************************
+ ** An inbound object; each openStream call supplies a fresh stream.
+ *******************************************************************************/
+public record InboundFile(Integer uploadJobId, String endpointName, String username, String userId,
+                          String remoteAddress, String path, String fileName, long sizeBytes, String sha256,
+                          Instant receivedAt, String storageBucketAlias, String storageKey, Supplier<InputStream> streamSupplier)
+{
+
+
+   /*******************************************************************************
+    ** Open a fresh object-storage stream.
+    *******************************************************************************/
+   public InputStream openStream()
+   {
+      return (streamSupplier.get());
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/api/OutboxFileRef.java

@@ -0,0 +1,33 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.api;
+
+
+import java.time.Instant;
+
+
+/*******************************************************************************
+ ** Identity and metadata of a file waiting for partner pickup.
+ *******************************************************************************/
+public record OutboxFileRef(Integer outboxFileId, String endpointName, String path, String fileName,
+                            long sizeBytes, String sha256, Instant createdAt)
+{
+}

src/main/java/com/kingsrook/qbits/filegateway/metadata/AccountCustomizer.java

@@ -0,0 +1,57 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.metadata;
+
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+
+/*******************************************************************************
+ ** Reject account names that cannot be used as stable SSH login names.
+ *******************************************************************************/
+public class AccountCustomizer implements TableCustomizerInterface
+{
+   /*******************************************************************************
+    ** Partial updates preserve the current username; supplied names must be valid.
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean preview, Optional<List<QRecord>> oldRecords)
+   {
+      for(QRecord record : records)
+      {
+         if(record.getValues().containsKey("username") && (record.getValueString("username") == null || !record.getValueString("username").matches("[a-z0-9._-]{3,64}")))
+         {
+            record.addError(new BadInputStatusMessage("Username must be 3 to 64 lowercase letters, digits, dots, underscores or hyphens"));
+         }
+         for(String limit : List.of("maxConcurrentSessions", "maxOperationsPerMinute", "maxFileSizeMb", "maxUploadMbPerDay", "maxDownloadMbPerDay"))
+         {
+            if(record.getValueInteger(limit) != null && record.getValueInteger(limit) < 1)
+            {
+               record.addError(new BadInputStatusMessage(limit + " must be positive, or blank for an unlimited daily quota"));
+            }
+         }
+      }
+      return (records);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/metadata/FileGatewayEnums.java

@@ -0,0 +1,508 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.metadata;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
+
+
+/*******************************************************************************
+ ** Stable persisted values for the gateway schema.
+ *******************************************************************************/
+public final class FileGatewayEnums
+{
+
+
+   /*******************************************************************************
+    ** Namespace for schema values.
+    *******************************************************************************/
+   private FileGatewayEnums()
+   {
+   }
+
+
+   /*******************************************************************************
+    ** AccountType values.
+    *******************************************************************************/
+   public enum AccountType implements PossibleValueEnum<String>
+   {
+      USER("USER"), SERVICE("SERVICE");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      AccountType(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** AccountStatus values.
+    *******************************************************************************/
+   public enum AccountStatus implements PossibleValueEnum<String>
+   {
+      ACTIVE("ACTIVE"), DISABLED("DISABLED"), LOCKED("LOCKED");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      AccountStatus(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** RootMode values.
+    *******************************************************************************/
+   public enum RootMode implements PossibleValueEnum<String>
+   {
+      FULL_TREE("FULL_TREE"), ENDPOINTS_ONLY("ENDPOINTS_ONLY");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      RootMode(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** AliasKind values.
+    *******************************************************************************/
+   public enum AliasKind implements PossibleValueEnum<String>
+   {
+      BULK_LOAD("BULK_LOAD"), INBOX("INBOX"), VIEW("VIEW"), REPORT("REPORT"), OUTBOX("OUTBOX");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      AliasKind(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** UploadStatus values.
+    *******************************************************************************/
+   public enum UploadStatus implements PossibleValueEnum<String>
+   {
+      RECEIVING("RECEIVING"), QUEUED("QUEUED"), RUNNING("RUNNING"), SUCCEEDED("SUCCEEDED"), PARTIAL("PARTIAL"), FAILED("FAILED"), REJECTED("REJECTED"), DUPLICATE("DUPLICATE"), ABORTED("ABORTED");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      UploadStatus(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** OutboxStatus values.
+    *******************************************************************************/
+   public enum OutboxStatus implements PossibleValueEnum<String>
+   {
+      READY("READY"), DOWNLOADED("DOWNLOADED"), REMOVED("REMOVED"), EXPIRED("EXPIRED");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      OutboxStatus(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Operation values.
+    *******************************************************************************/
+   public enum Operation implements PossibleValueEnum<String>
+   {
+      LIST("LIST"), UPLOAD("UPLOAD"), DOWNLOAD("DOWNLOAD"), RENAME("RENAME"), REMOVE("REMOVE");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      Operation(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** ObjectKind values.
+    *******************************************************************************/
+   public enum ObjectKind implements PossibleValueEnum<String>
+   {
+      APP("APP"), TABLE("TABLE"), PROFILE("PROFILE"), VIEW("VIEW"), REPORT("REPORT"), RESULT("RESULT"), ARCHIVE("ARCHIVE"), ENDPOINT("ENDPOINT"), OUTBOX("OUTBOX");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      ObjectKind(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** TransferOutcome values.
+    *******************************************************************************/
+   public enum TransferOutcome implements PossibleValueEnum<String>
+   {
+      OK("OK"), DENIED("DENIED"), NOT_FOUND("NOT_FOUND"), LIMIT("LIMIT"), ERROR("ERROR");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      TransferOutcome(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** ConnectionOutcome values.
+    *******************************************************************************/
+   public enum ConnectionOutcome implements PossibleValueEnum<String>
+   {
+      CONNECTED("CONNECTED"), AUTH_FAILED("AUTH_FAILED"), LOCKED_OUT("LOCKED_OUT"), IP_DENIED("IP_DENIED"), EXPIRED("EXPIRED"), DISABLED("DISABLED"), LIMIT("LIMIT");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      ConnectionOutcome(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Format values.
+    *******************************************************************************/
+   public enum Format implements PossibleValueEnum<String>
+   {
+      CSV("csv"), XLSX("xlsx"), JSON("json");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      Format(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** KeyType values.
+    *******************************************************************************/
+   public enum KeyType implements PossibleValueEnum<String>
+   {
+      SSH_ED25519("ssh-ed25519"), ECDSA_SHA2_NISTP256("ecdsa-sha2-nistp256"), ECDSA_SHA2_NISTP384("ecdsa-sha2-nistp384"), ECDSA_SHA2_NISTP521("ecdsa-sha2-nistp521"), SSH_RSA("ssh-rsa");
+
+      private final String value;
+
+      /*******************************************************************************
+       ** Construct a persisted value.
+       *******************************************************************************/
+      KeyType(String value)
+      {
+         this.value = value;
+      }
+
+
+      /*******************************************************************************
+       ** Possible value id.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueId()
+      {
+         return (value);
+      }
+
+
+      /*******************************************************************************
+       ** Possible value label.
+       *******************************************************************************/
+      @Override
+      public String getPossibleValueLabel()
+      {
+         return (value);
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/metadata/FileGatewayProcesses.java

@@ -0,0 +1,241 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.metadata;
+
+import java.io.Serializable;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qbits.filegateway.upload.UploadJobs;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+
+/*******************************************************************************
+ ** Administrative record actions use the standard streamed Review and Result flow.
+ *******************************************************************************/
+public final class FileGatewayProcesses
+{
+   public static final String RETRY_UPLOAD = "file_gateway_retry_upload";
+   public static final String UNLOCK = "file_gateway_unlock_account";
+   public static final String DISABLE = "file_gateway_disable_account";
+   public static final String REVOKE = "file_gateway_revoke_key";
+   public static final String END_SESSION = "file_gateway_end_session";
+
+   /*******************************************************************************
+    ** Metadata factory only.
+    *******************************************************************************/
+   private FileGatewayProcesses()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Administrative actions bound to their owning tables.
+    *******************************************************************************/
+   public static List<QProcessMetaData> processes()
+   {
+      return (List.of(action(UNLOCK, "Unlock account", FileGatewayTables.ACCOUNT),
+         action(DISABLE, "Disable account", FileGatewayTables.ACCOUNT),
+         action(REVOKE, "Revoke key", FileGatewayTables.SSH_KEY),
+         action(END_SESSION, "End session", FileGatewayTables.CONNECTION_LOG),
+         action(RETRY_UPLOAD, "Retry upload", FileGatewayTables.UPLOAD_JOB)));
+   }
+
+
+
+   /*******************************************************************************
+    ** A table-bound action, never a navigation entry.
+    *******************************************************************************/
+   private static QProcessMetaData action(String name, String label, String table)
+   {
+      Map<String, Serializable> values = new HashMap<>();
+      values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, table);
+      values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table);
+      values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
+      values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
+      values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, label + " for the selected records.");
+      return (StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Review.class, Apply.class, values)
+         .withName(name).withLabel(label).withTableName(table).withPermissionRules(FileGatewayTables.permissionRules(name)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Direct step invocation must require the same permission as the process route.
+    *******************************************************************************/
+   private static void requireAdmin() throws QPermissionDeniedException
+   {
+      if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(FileGatewayPermissions.ADMIN))
+      {
+         throw (new QPermissionDeniedException("Missing file_gateway.admin"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Preview selections without changing records.
+    *******************************************************************************/
+   public static class Review extends AbstractTransformStep implements ProcessSummaryProviderInterface
+   {
+      private final ProcessSummaryLine summary = new ProcessSummaryLine(Status.OK, 0, null);
+      private final ProcessSummaryLine skipped = new ProcessSummaryLine(Status.INFO, 0, "uploads are not eligible for retry");
+
+      /*******************************************************************************
+       ** Review uses the framework's future and past tense summary lines.
+       *******************************************************************************/
+      public Review()
+      {
+         summary.setSingularFutureMessage("record will be updated");
+         summary.setPluralFutureMessage("records will be updated");
+         summary.setSingularPastMessage("record was updated");
+         summary.setPluralPastMessage("records were updated");
+      }
+
+
+
+      /*******************************************************************************
+       ** Pass only authorized selections to the load phase.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         requireAdmin();
+         for(QRecord record : input.getRecords())
+         {
+            if(RETRY_UPLOAD.equals(input.getProcessName()) && (!List.of("FAILED", "REJECTED").contains(record.getValueString("status"))
+               || record.getValueString("storageKey") == null || record.getValueString("sha256") == null))
+            {
+               skipped.incrementCountAndAddPrimaryKey(record.getValue("id"));
+               continue;
+            }
+            summary.incrementCountAndAddPrimaryKey(record.getValue("id"));
+            output.getRecords().add(record);
+         }
+      }
+
+
+
+      /*******************************************************************************
+       ** Standard Review and Result summary.
+       *******************************************************************************/
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>(List.of(summary));
+         skipped.addSelfToListIfAnyCount(lines);
+         return (lines);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Apply the approved action to the gateway's own bookkeeping tables.
+    *******************************************************************************/
+   public static class Apply extends AbstractLoadStep
+   {
+      /*******************************************************************************
+       ** Whitelist mutations, using sparse updates so concurrent account changes survive.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         requireAdmin();
+         AuditWriter audit = new AuditWriter(QContext.getQInstance());
+         for(QRecord record : input.getRecords())
+         {
+            if(RETRY_UPLOAD.equals(input.getProcessName()))
+            {
+               audit.withAccountLock("id", record.getValue("accountId"), () ->
+               {
+                  QRecord current = audit.query(FileGatewayTables.UPLOAD_JOB, UploadJobs.equal("id", record.getValue("id"))).getFirst();
+                  if(!List.of("FAILED", "REJECTED").contains(current.getValueString("status"))
+                     || current.getValueString("storageKey") == null || current.getValueString("sha256") == null)
+                  {
+                     throw (new QException("Only failed or rejected stored uploads can be retried"));
+                  }
+                  QRecord retry = new QRecord().withValue("id", record.getValue("id"))
+                     .withValue("status", "QUEUED").withValue("attempts", 0).withValue("nextAttemptAt", null)
+                     .withValue("claimedBy", null).withValue("claimedAt", null).withValue("finishedAt", null)
+                     .withValue("acknowledgedAt", null).withValue("errorMessage", null).withValue("resultSummary", null)
+                     .withValue("resultsStorageKey", null).withValue("resultsSizeBytes", null).withValue("processUuid", null)
+                     .withValue("insertedCount", null).withValue("errorCount", null).withValue("queuedAt", Instant.now());
+                  audit.update(FileGatewayTables.UPLOAD_JOB, retry);
+                  output.getRecords().add(retry);
+                  return (null);
+               });
+               continue;
+            }
+            QRecord update = new QRecord().withValue("id", record.getValue("id"));
+            String table;
+            switch(input.getProcessName())
+            {
+               case UNLOCK ->
+               {
+                  table = FileGatewayTables.ACCOUNT;
+                  update.withValue("lockedUntil", null).withValue("failedLoginCount", 0).withValue("failedLoginWindowAt", null);
+                  if("LOCKED".equals(record.getValueString("status")))
+                  {
+                     update.withValue("status", "ACTIVE");
+                  }
+               }
+               case DISABLE ->
+               {
+                  table = FileGatewayTables.ACCOUNT;
+                  update.withValue("status", "DISABLED");
+               }
+               case REVOKE ->
+               {
+                  table = FileGatewayTables.SSH_KEY;
+                  update.withValue("revokedAt", Instant.now());
+               }
+               case END_SESSION ->
+               {
+                  table = FileGatewayTables.CONNECTION_LOG;
+                  update.withValue("disconnectedAt", Instant.now()).withValue("disconnectReason", "Ended by administrator");
+               }
+               default -> throw (new QException("Unsupported gateway action"));
+            }
+            audit.update(table, update);
+            output.getRecords().add(update);
+         }
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/metadata/FileGatewayTables.java

@@ -0,0 +1,432 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.metadata;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissionChecker;
+import com.kingsrook.qqq.backend.core.actions.ai.AiFieldMetaData;
+import com.kingsrook.qqq.backend.core.actions.ai.DataClassification;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
+import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
+import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinType;
+import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
+
+
+/*******************************************************************************
+ ** The seven gateway tables, their stable columns, pickers and child lists.
+ *******************************************************************************/
+public final class FileGatewayTables
+{
+   public static final String ACCOUNT = "file_gateway_account";
+   public static final String SSH_KEY = "file_gateway_ssh_key";
+   public static final String ALIAS = "file_gateway_alias";
+   public static final String UPLOAD_JOB = "file_gateway_upload_job";
+   public static final String OUTBOX_FILE = "file_gateway_outbox_file";
+   public static final String TRANSFER_LOG = "file_gateway_transfer_log";
+   public static final String CONNECTION_LOG = "file_gateway_connection_log";
+   public static final List<String> ALL = List.of(ACCOUNT, SSH_KEY, ALIAS, UPLOAD_JOB, OUTBOX_FILE, TRANSFER_LOG, CONNECTION_LOG);
+   public static final List<String> MAIN = List.of(ACCOUNT, ALIAS, UPLOAD_JOB, TRANSFER_LOG, CONNECTION_LOG);
+   public static final List<String> SYSTEM_WRITTEN = List.of(UPLOAD_JOB, OUTBOX_FILE, TRANSFER_LOG, CONNECTION_LOG);
+
+
+
+   /*******************************************************************************
+    ** Metadata factories only.
+    *******************************************************************************/
+   private FileGatewayTables()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Permission names use the full table or app name.
+    *******************************************************************************/
+   public static QPermissionRules permissionRules(String name)
+   {
+      return (new QPermissionRules().withLevel(PermissionLevel.READ_WRITE_PERMISSIONS).withPermissionBaseName(name)
+         .withCustomPermissionChecker(new QCodeReference(FileGatewayPermissionChecker.class)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Enums and gateway record pickers.
+    *******************************************************************************/
+   public static List<QPossibleValueSource> possibleValueSources()
+   {
+      List<QPossibleValueSource> sources = new ArrayList<>();
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_account_type", FileGatewayEnums.AccountType.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_account_status", FileGatewayEnums.AccountStatus.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_root_mode", FileGatewayEnums.RootMode.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_alias_kind", FileGatewayEnums.AliasKind.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_upload_status", FileGatewayEnums.UploadStatus.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_outbox_status", FileGatewayEnums.OutboxStatus.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_operation", FileGatewayEnums.Operation.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_object_kind", FileGatewayEnums.ObjectKind.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_transfer_outcome", FileGatewayEnums.TransferOutcome.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_connection_outcome", FileGatewayEnums.ConnectionOutcome.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_format", FileGatewayEnums.Format.values()));
+      sources.add(QPossibleValueSource.newForEnum("file_gateway_key_type", FileGatewayEnums.KeyType.values()));
+      ALL.forEach(name -> sources.add(QPossibleValueSource.newForTable(name)));
+      return (sources);
+   }
+
+
+
+   /*******************************************************************************
+    ** Build all seven tables.
+    *******************************************************************************/
+   public static List<QTableMetaData> tables(FileGatewayQBitConfig config)
+   {
+      return (List.of(account(config), ssh_key(config), alias(config), upload_job(config), outbox_file(config), transfer_log(config), connection_log(config)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Common identity and timestamps.
+    *******************************************************************************/
+   private static QTableMetaData base(FileGatewayQBitConfig config, String name, String label)
+   {
+      return (new QTableMetaData().withName(name).withLabel(label).withBackendName(config.getBackendName())
+         .withPrimaryKeyField("id")
+         .withPermissionRules(permissionRules(name))
+         .withField(field("id", QFieldType.INTEGER).withIsEditable(false))
+         .withField(field("createDate", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("modifyDate", QFieldType.DATE_TIME).withIsEditable(false)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Apply the host backend naming convention explicitly.
+    *******************************************************************************/
+   private static QFieldMetaData field(String name, QFieldType type)
+   {
+      return (new QFieldMetaData(name, type).withBackendName(QInstanceEnricher.inferBackendName(name)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for accounts.
+    *******************************************************************************/
+   private static QTableMetaData account(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, ACCOUNT, "Accounts")
+         .withRecordLabelFormatAndFields("%s", "username")
+         .withField(field("username", QFieldType.STRING).withIsRequired(true))
+         .withField(field("label", QFieldType.STRING))
+         .withField(field("accountType", QFieldType.STRING).withPossibleValueSourceName("file_gateway_account_type").withDefaultValue("USER").withIsRequired(true))
+         .withField(field("userId", QFieldType.STRING).withIsRequired(true))
+         .withField(field("ownerUserId", QFieldType.STRING))
+         .withField(field("status", QFieldType.STRING).withPossibleValueSourceName("file_gateway_account_status").withDefaultValue("ACTIVE").withIsRequired(true))
+         .withField(field("rootMode", QFieldType.STRING).withPossibleValueSourceName("file_gateway_root_mode").withDefaultValue("FULL_TREE").withIsRequired(true))
+         .withField(field("allowedCidrs", QFieldType.TEXT))
+         .withField(field("expiresAt", QFieldType.DATE_TIME))
+         .withField(field("formats", QFieldType.STRING).withDefaultValue("csv,xlsx"))
+         .withField(field("maxConcurrentSessions", QFieldType.INTEGER).withDefaultValue(config.getDefaultMaxConcurrentSessions()))
+         .withField(field("maxOperationsPerMinute", QFieldType.INTEGER).withDefaultValue(config.getDefaultMaxOperationsPerMinute()))
+         .withField(field("maxFileSizeMb", QFieldType.INTEGER).withDefaultValue(config.getDefaultMaxFileSizeMb()))
+         .withField(field("maxUploadMbPerDay", QFieldType.INTEGER).withDefaultValue(config.getDefaultMaxUploadMbPerDay()))
+         .withField(field("maxDownloadMbPerDay", QFieldType.INTEGER).withDefaultValue(config.getDefaultMaxDownloadMbPerDay()))
+         .withField(field("dedupeUploads", QFieldType.BOOLEAN).withDefaultValue(true))
+         .withField(field("notifyOnSuccess", QFieldType.BOOLEAN).withDefaultValue(false))
+         .withField(field("pgpPublicKey", QFieldType.TEXT))
+         .withField(field("pgpDecryptUploads", QFieldType.BOOLEAN).withDefaultValue(false))
+         .withField(field("pgpEncryptDownloads", QFieldType.BOOLEAN).withDefaultValue(false))
+         .withField(field("operationWindowAt", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("operationWindowCount", QFieldType.INTEGER).withDefaultValue(0).withIsEditable(false))
+         .withField(field("failedLoginCount", QFieldType.INTEGER).withDefaultValue(0).withIsEditable(false))
+         .withField(field("failedLoginWindowAt", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("lockedUntil", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("lastLoginAt", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("lastLoginIp", QFieldType.STRING).withIsEditable(false));
+      table.withUniqueKey(new UniqueKey("username"));
+      table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(AccountCustomizer.class));
+      table.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(AccountCustomizer.class));
+      table.withSection(SectionFactory.defaultT1("id", "username", "label", "accountType", "userId", "ownerUserId"));
+      table.withSection(SectionFactory.defaultT2("status", "rootMode", "allowedCidrs", "expiresAt", "formats", "maxConcurrentSessions", "maxOperationsPerMinute", "maxFileSizeMb", "maxUploadMbPerDay", "maxDownloadMbPerDay", "dedupeUploads", "notifyOnSuccess", "pgpPublicKey", "pgpDecryptUploads", "pgpEncryptDownloads", "operationWindowAt", "operationWindowCount", "failedLoginCount", "failedLoginWindowAt", "lockedUntil", "lastLoginAt", "lastLoginIp"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withSection(SectionFactory.customT2(SSH_KEY, new QIcon("list")).withWidgetName(ACCOUNT + "_children_" + SSH_KEY));
+      table.withSection(SectionFactory.customT2(UPLOAD_JOB, new QIcon("list")).withWidgetName(ACCOUNT + "_children_" + UPLOAD_JOB));
+      table.withSection(SectionFactory.customT2(CONNECTION_LOG, new QIcon("list")).withWidgetName(ACCOUNT + "_children_" + CONNECTION_LOG));
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for ssh keys.
+    *******************************************************************************/
+   private static QTableMetaData ssh_key(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, SSH_KEY, "SSH Keys")
+         .withRecordLabelFormatAndFields("%s", "label")
+         .withField(field("accountId", QFieldType.INTEGER).withPossibleValueSourceName(ACCOUNT).withIsRequired(true))
+         .withField(field("label", QFieldType.STRING))
+         .withField(field("publicKey", QFieldType.TEXT).withIsRequired(true))
+         .withField(field("fingerprint", QFieldType.STRING).withIsEditable(false))
+         .withField(field("keyType", QFieldType.STRING).withPossibleValueSourceName("file_gateway_key_type").withIsEditable(false))
+         .withField(field("keyBits", QFieldType.INTEGER).withIsEditable(false))
+         .withField(field("expiresAt", QFieldType.DATE_TIME))
+         .withField(field("revokedAt", QFieldType.DATE_TIME))
+         .withField(field("lastUsedAt", QFieldType.DATE_TIME).withIsEditable(false))
+         .withField(field("lastUsedIp", QFieldType.STRING).withIsEditable(false));
+      table.withUniqueKey(new UniqueKey("fingerprint"));
+      table.withSection(SectionFactory.defaultT1("id", "accountId", "label", "publicKey", "fingerprint", "keyType"));
+      table.withSection(SectionFactory.defaultT2("keyBits", "expiresAt", "revokedAt", "lastUsedAt", "lastUsedIp"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(SshKeyCustomizer.class));
+      table.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(SshKeyCustomizer.class));
+      table.getField("publicKey").withSupplementalMetaData(new AiFieldMetaData().withClassification(DataClassification.CONFIDENTIAL));
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for endpoints.
+    *******************************************************************************/
+   private static QTableMetaData alias(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, ALIAS, "Endpoints")
+         .withRecordLabelFormatAndFields("%s", "name")
+         .withField(field("name", QFieldType.STRING))
+         .withField(field("path", QFieldType.STRING))
+         .withField(field("kind", QFieldType.STRING).withPossibleValueSourceName("file_gateway_alias_kind"))
+         .withField(field("accountId", QFieldType.INTEGER).withPossibleValueSourceName(ACCOUNT))
+         .withField(field("tableName", QFieldType.STRING).withPossibleValueSourceName("tables"))
+         .withField(field("savedBulkLoadProfileId", QFieldType.INTEGER).withPossibleValueSourceName("savedBulkLoadProfile").withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, "${input.tableName}"))))
+         .withField(field("savedViewId", QFieldType.INTEGER).withPossibleValueSourceName("savedView").withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, "${input.tableName}"))))
+         .withField(field("savedReportId", QFieldType.INTEGER).withPossibleValueSourceName("savedReport").withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, "${input.tableName}"))))
+         .withField(field("format", QFieldType.STRING).withPossibleValueSourceName("file_gateway_format"))
+         .withField(field("fileName", QFieldType.STRING))
+         .withField(field("consumerName", QFieldType.STRING))
+         .withField(field("isActive", QFieldType.BOOLEAN).withDefaultValue(true))
+         .withField(field("description", QFieldType.TEXT));
+      table.withUniqueKey(new UniqueKey("name"));
+      table.withUniqueKey(new UniqueKey("path"));
+      table.withSection(SectionFactory.defaultT1("id", "name", "path", "kind", "accountId", "tableName"));
+      table.withSection(SectionFactory.defaultT2("savedBulkLoadProfileId", "savedViewId", "savedReportId", "format", "fileName", "consumerName", "isActive", "description"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withSection(SectionFactory.customT2(UPLOAD_JOB, new QIcon("list")).withWidgetName(ALIAS + "_children_" + UPLOAD_JOB));
+      table.withSection(SectionFactory.customT2(OUTBOX_FILE, new QIcon("list")).withWidgetName(ALIAS + "_children_" + OUTBOX_FILE));
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for upload jobs.
+    *******************************************************************************/
+   private static QTableMetaData upload_job(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, UPLOAD_JOB, "Upload Jobs").withRecordVersionField("status")
+         .withRecordLabelFormatAndFields("%s", "id")
+         .withField(field("accountId", QFieldType.INTEGER).withPossibleValueSourceName(ACCOUNT))
+         .withField(field("aliasId", QFieldType.INTEGER).withPossibleValueSourceName(ALIAS))
+         .withField(field("connectionLogId", QFieldType.INTEGER).withPossibleValueSourceName(CONNECTION_LOG))
+         .withField(field("remoteAddress", QFieldType.STRING))
+         .withField(field("tableName", QFieldType.STRING).withPossibleValueSourceName("tables"))
+         .withField(field("savedBulkLoadProfileId", QFieldType.INTEGER).withPossibleValueSourceName("savedBulkLoadProfile"))
+         .withField(field("consumerName", QFieldType.STRING))
+         .withField(field("consumerReference", QFieldType.STRING))
+         .withField(field("path", QFieldType.STRING))
+         .withField(field("fileName", QFieldType.STRING))
+         .withField(field("sizeBytes", QFieldType.LONG))
+         .withField(field("sha256", QFieldType.STRING))
+         .withField(field("storageKey", QFieldType.STRING))
+         .withField(field("status", QFieldType.STRING).withPossibleValueSourceName("file_gateway_upload_status"))
+         .withField(field("duplicateOfJobId", QFieldType.INTEGER).withPossibleValueSourceName(UPLOAD_JOB))
+         .withField(field("processUuid", QFieldType.STRING))
+         .withField(field("insertedCount", QFieldType.INTEGER))
+         .withField(field("errorCount", QFieldType.INTEGER))
+         .withField(field("resultSummary", QFieldType.TEXT))
+         .withField(field("resultsStorageKey", QFieldType.STRING))
+         .withField(field("resultsSizeBytes", QFieldType.LONG))
+         .withField(field("errorMessage", QFieldType.TEXT))
+         .withField(field("attempts", QFieldType.INTEGER))
+         .withField(field("nextAttemptAt", QFieldType.DATE_TIME))
+         .withField(field("claimedBy", QFieldType.STRING))
+         .withField(field("claimedAt", QFieldType.DATE_TIME))
+         .withField(field("queuedAt", QFieldType.DATE_TIME))
+         .withField(field("startedAt", QFieldType.DATE_TIME))
+         .withField(field("finishedAt", QFieldType.DATE_TIME))
+         .withField(field("acknowledgedAt", QFieldType.DATE_TIME));
+      table.withSection(SectionFactory.defaultT1("id", "accountId", "aliasId", "connectionLogId", "remoteAddress", "tableName"));
+      table.withSection(SectionFactory.defaultT2("savedBulkLoadProfileId", "consumerName", "consumerReference", "path", "fileName", "sizeBytes", "sha256", "storageKey", "status", "duplicateOfJobId", "processUuid", "insertedCount", "errorCount", "resultSummary", "resultsStorageKey", "resultsSizeBytes", "errorMessage", "attempts", "nextAttemptAt", "claimedBy", "claimedAt", "queuedAt", "startedAt", "finishedAt", "acknowledgedAt"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withSection(SectionFactory.customT2(TRANSFER_LOG, new QIcon("list")).withWidgetName(UPLOAD_JOB + "_children_" + TRANSFER_LOG));
+      table.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for outbox files.
+    *******************************************************************************/
+   private static QTableMetaData outbox_file(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, OUTBOX_FILE, "Outbox Files")
+         .withRecordLabelFormatAndFields("%s", "id")
+         .withField(field("aliasId", QFieldType.INTEGER).withPossibleValueSourceName(ALIAS))
+         .withField(field("fileName", QFieldType.STRING))
+         .withField(field("sizeBytes", QFieldType.LONG))
+         .withField(field("sha256", QFieldType.STRING))
+         .withField(field("storageKey", QFieldType.STRING))
+         .withField(field("status", QFieldType.STRING).withPossibleValueSourceName("file_gateway_outbox_status"))
+         .withField(field("firstDownloadedAt", QFieldType.DATE_TIME))
+         .withField(field("downloadCount", QFieldType.INTEGER))
+         .withField(field("removedAt", QFieldType.DATE_TIME))
+         .withField(field("expiresAt", QFieldType.DATE_TIME))
+         .withField(field("createdByConsumer", QFieldType.STRING));
+      table.withSection(SectionFactory.defaultT1("id", "aliasId", "fileName", "sizeBytes", "sha256", "storageKey"));
+      table.withSection(SectionFactory.defaultT2("status", "firstDownloadedAt", "downloadCount", "removedAt", "expiresAt", "createdByConsumer"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for transfers.
+    *******************************************************************************/
+   private static QTableMetaData transfer_log(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, TRANSFER_LOG, "Transfers")
+         .withRecordLabelFormatAndFields("%s", "id")
+         .withField(field("accountId", QFieldType.INTEGER).withPossibleValueSourceName(ACCOUNT))
+         .withField(field("connectionLogId", QFieldType.INTEGER).withPossibleValueSourceName(CONNECTION_LOG))
+         .withField(field("operation", QFieldType.STRING).withPossibleValueSourceName("file_gateway_operation"))
+         .withField(field("path", QFieldType.STRING))
+         .withField(field("objectKind", QFieldType.STRING).withPossibleValueSourceName("file_gateway_object_kind"))
+         .withField(field("objectId", QFieldType.STRING))
+         .withField(field("format", QFieldType.STRING).withPossibleValueSourceName("file_gateway_format"))
+         .withField(field("bytes", QFieldType.LONG))
+         .withField(field("sha256", QFieldType.STRING))
+         .withField(field("durationMs", QFieldType.LONG))
+         .withField(field("outcome", QFieldType.STRING).withPossibleValueSourceName("file_gateway_transfer_outcome"))
+         .withField(field("message", QFieldType.TEXT))
+         .withField(field("clientChannel", QFieldType.STRING))
+         .withField(field("uploadJobId", QFieldType.INTEGER).withPossibleValueSourceName(UPLOAD_JOB));
+      table.withSection(SectionFactory.defaultT1("id", "accountId", "connectionLogId", "operation", "path", "objectKind"));
+      table.withSection(SectionFactory.defaultT2("objectId", "format", "bytes", "sha256", "durationMs", "outcome", "message", "clientChannel", "uploadJobId"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata for connections.
+    *******************************************************************************/
+   private static QTableMetaData connection_log(FileGatewayQBitConfig config)
+   {
+      QTableMetaData table = base(config, CONNECTION_LOG, "Connections")
+         .withRecordLabelFormatAndFields("%s", "id")
+         .withField(field("accountId", QFieldType.INTEGER).withPossibleValueSourceName(ACCOUNT))
+         .withField(field("username", QFieldType.STRING))
+         .withField(field("remoteIp", QFieldType.STRING))
+         .withField(field("clientVersion", QFieldType.STRING))
+         .withField(field("keyFingerprint", QFieldType.STRING))
+         .withField(field("outcome", QFieldType.STRING).withPossibleValueSourceName("file_gateway_connection_outcome"))
+         .withField(field("sessionId", QFieldType.STRING))
+         .withField(field("podName", QFieldType.STRING))
+         .withField(field("connectedAt", QFieldType.DATE_TIME))
+         .withField(field("lastSeenAt", QFieldType.DATE_TIME))
+         .withField(field("disconnectedAt", QFieldType.DATE_TIME))
+         .withField(field("disconnectReason", QFieldType.TEXT))
+         .withField(field("bytesIn", QFieldType.LONG))
+         .withField(field("bytesOut", QFieldType.LONG))
+         .withField(field("operationCount", QFieldType.LONG))
+         .withField(field("clientChannel", QFieldType.STRING));
+      table.withSection(SectionFactory.defaultT1("id", "accountId", "username", "remoteIp", "clientVersion", "keyFingerprint"));
+      table.withSection(SectionFactory.defaultT2("outcome", "sessionId", "podName", "connectedAt", "lastSeenAt", "disconnectedAt", "disconnectReason", "bytesIn", "bytesOut", "operationCount", "clientChannel"));
+      table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
+      table.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
+      return (table);
+   }
+
+
+
+   /*******************************************************************************
+    ** Parent and child relationships.
+    *******************************************************************************/
+   public static List<QJoinMetaData> joins()
+   {
+      return (List.of(
+         new QJoinMetaData().withName(ACCOUNT + "_join_" + SSH_KEY).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(ACCOUNT).withRightTable(SSH_KEY).withJoinOn(new JoinOn("id", "accountId")),
+         new QJoinMetaData().withName(ACCOUNT + "_join_" + UPLOAD_JOB).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(ACCOUNT).withRightTable(UPLOAD_JOB).withJoinOn(new JoinOn("id", "accountId")),
+         new QJoinMetaData().withName(ACCOUNT + "_join_" + CONNECTION_LOG).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(ACCOUNT).withRightTable(CONNECTION_LOG).withJoinOn(new JoinOn("id", "accountId")),
+         new QJoinMetaData().withName(ALIAS + "_join_" + UPLOAD_JOB).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(ALIAS).withRightTable(UPLOAD_JOB).withJoinOn(new JoinOn("id", "aliasId")),
+         new QJoinMetaData().withName(ALIAS + "_join_" + OUTBOX_FILE).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(ALIAS).withRightTable(OUTBOX_FILE).withJoinOn(new JoinOn("id", "aliasId")),
+         new QJoinMetaData().withName(UPLOAD_JOB + "_join_" + TRANSFER_LOG).withType(JoinType.ONE_TO_MANY)
+            .withLeftTable(UPLOAD_JOB).withRightTable(TRANSFER_LOG).withJoinOn(new JoinOn("id", "uploadJobId"))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Typed child-record widgets; no home or HTML widget.
+    *******************************************************************************/
+   public static List<QWidgetMetaData> childWidgets()
+   {
+      List<QWidgetMetaData> widgets = new ArrayList<>();
+      for(QJoinMetaData join : joins())
+      {
+         widgets.add(ChildRecordListRenderer.widgetMetaDataBuilder(join)
+            .withName(join.getLeftTable() + "_children_" + join.getRightTable()).withLabel(join.getRightTable().replace("file_gateway_", "").replace("_", " "))
+            .withCanAddChildRecord(SSH_KEY.equals(join.getRightTable())).withMaxRows(100)
+            .getWidgetMetaData().withGridColumns(12).withPermissionRules(permissionRules(join.getRightTable())));
+      }
+      return (widgets);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/metadata/SshKeyCustomizer.java

@@ -0,0 +1,76 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.metadata;
+
+
+import java.security.PublicKey;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.config.keys.PublicKeyEntry;
+import org.apache.sshd.common.digest.BuiltinDigests;
+
+
+/*******************************************************************************
+ ** Validates stored public keys and computes authoritative key metadata.
+ *******************************************************************************/
+public class SshKeyCustomizer implements TableCustomizerInterface
+{
+
+
+
+   /*******************************************************************************
+    ** Parse and validate inserts and partial updates without logging key material.
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecords)
+   {
+      for(QRecord record : records)
+      {
+         QRecord old = oldRecords.orElse(List.of()).stream().filter(candidate -> Objects.equals(candidate.getValue("id"), record.getValue("id"))).findFirst().orElse(null);
+         String text = record.getValues().containsKey("publicKey") ? record.getValueString("publicKey") : old == null ? null : old.getValueString("publicKey");
+         try
+         {
+            PublicKey key = PublicKeyEntry.parsePublicKeyEntry(text).resolvePublicKey(null, Map.of(), null);
+            String type = KeyUtils.getKeyType(key);
+            int bits = KeyUtils.getKeySize(key);
+            if(!("ssh-ed25519".equals(type) || type.startsWith("ecdsa-sha2-nistp") || ("ssh-rsa".equals(type) && bits >= 3072)))
+            {
+               throw (new IllegalArgumentException("Unsupported key type or size"));
+            }
+            record.setValue("fingerprint", KeyUtils.getFingerPrint(BuiltinDigests.sha256, key));
+            record.setValue("keyType", type);
+            record.setValue("keyBits", bits);
+         }
+         catch(Exception e)
+         {
+            record.addError(new BadInputStatusMessage("Use an OpenSSH Ed25519, ECDSA, or RSA key of at least 3072 bits"));
+         }
+      }
+      return (records);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/model/FileGatewayAlias.java

@@ -0,0 +1,600 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.model;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Record entity for endpoints.
+ *******************************************************************************/
+public class FileGatewayAlias extends QRecordEntity
+{
+   public static final String TABLE_NAME = "file_gateway_alias";
+
+   @QField
+   private Integer id;
+
+   @QField
+   private Instant createDate;
+
+   @QField
+   private Instant modifyDate;
+
+   @QField
+   private String name;
+
+   @QField
+   private String path;
+
+   @QField
+   private String kind;
+
+   @QField
+   private Integer accountId;
+
+   @QField
+   private String tableName;
+
+   @QField
+   private Integer savedBulkLoadProfileId;
+
+   @QField
+   private Integer savedViewId;
+
+   @QField
+   private Integer savedReportId;
+
+   @QField
+   private String format;
+
+   @QField
+   private String fileName;
+
+   @QField
+   private String consumerName;
+
+   @QField
+   private Boolean isActive;
+
+   @QField
+   private String description;
+
+
+
+   /*******************************************************************************
+    ** Create an empty entity.
+    *******************************************************************************/
+   public FileGatewayAlias()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Populate from a database record.
+    *******************************************************************************/
+   public FileGatewayAlias(QRecord record) throws QException
+   {
+      populateFromQRecord(record);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id.
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id.
+    *******************************************************************************/
+   public FileGatewayAlias withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate.
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return (createDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate.
+    *******************************************************************************/
+   public FileGatewayAlias withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate.
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return (modifyDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate.
+    *******************************************************************************/
+   public FileGatewayAlias withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for name.
+    *******************************************************************************/
+   public String getName()
+   {
+      return (name);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for name.
+    *******************************************************************************/
+   public FileGatewayAlias withName(String name)
+   {
+      this.name = name;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for path.
+    *******************************************************************************/
+   public String getPath()
+   {
+      return (path);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for path.
+    *******************************************************************************/
+   public FileGatewayAlias withPath(String path)
+   {
+      this.path = path;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for kind.
+    *******************************************************************************/
+   public String getKind()
+   {
+      return (kind);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for kind.
+    *******************************************************************************/
+   public FileGatewayAlias withKind(String kind)
+   {
+      this.kind = kind;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for accountId.
+    *******************************************************************************/
+   public Integer getAccountId()
+   {
+      return (accountId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for accountId.
+    *******************************************************************************/
+   public FileGatewayAlias withAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tableName.
+    *******************************************************************************/
+   public String getTableName()
+   {
+      return (tableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tableName.
+    *******************************************************************************/
+   public FileGatewayAlias withTableName(String tableName)
+   {
+      this.tableName = tableName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for savedBulkLoadProfileId.
+    *******************************************************************************/
+   public Integer getSavedBulkLoadProfileId()
+   {
+      return (savedBulkLoadProfileId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for savedBulkLoadProfileId.
+    *******************************************************************************/
+   public FileGatewayAlias withSavedBulkLoadProfileId(Integer savedBulkLoadProfileId)
+   {
+      this.savedBulkLoadProfileId = savedBulkLoadProfileId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for savedViewId.
+    *******************************************************************************/
+   public Integer getSavedViewId()
+   {
+      return (savedViewId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for savedViewId.
+    *******************************************************************************/
+   public FileGatewayAlias withSavedViewId(Integer savedViewId)
+   {
+      this.savedViewId = savedViewId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for savedReportId.
+    *******************************************************************************/
+   public Integer getSavedReportId()
+   {
+      return (savedReportId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for savedReportId.
+    *******************************************************************************/
+   public FileGatewayAlias withSavedReportId(Integer savedReportId)
+   {
+      this.savedReportId = savedReportId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for format.
+    *******************************************************************************/
+   public String getFormat()
+   {
+      return (format);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for format.
+    *******************************************************************************/
+   public FileGatewayAlias withFormat(String format)
+   {
+      this.format = format;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fileName.
+    *******************************************************************************/
+   public String getFileName()
+   {
+      return (fileName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fileName.
+    *******************************************************************************/
+   public FileGatewayAlias withFileName(String fileName)
+   {
+      this.fileName = fileName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for consumerName.
+    *******************************************************************************/
+   public String getConsumerName()
+   {
+      return (consumerName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for consumerName.
+    *******************************************************************************/
+   public FileGatewayAlias withConsumerName(String consumerName)
+   {
+      this.consumerName = consumerName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for isActive.
+    *******************************************************************************/
+   public Boolean getIsActive()
+   {
+      return (isActive);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isActive.
+    *******************************************************************************/
+   public FileGatewayAlias withIsActive(Boolean isActive)
+   {
+      this.isActive = isActive;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for description.
+    *******************************************************************************/
+   public String getDescription()
+   {
+      return (description);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for description.
+    *******************************************************************************/
+   public FileGatewayAlias withDescription(String description)
+   {
+      this.description = description;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setName(String name)
+   {
+      this.name = name;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setPath(String path)
+   {
+      this.path = path;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setKind(String kind)
+   {
+      this.kind = kind;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setTableName(String tableName)
+   {
+      this.tableName = tableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSavedBulkLoadProfileId(Integer savedBulkLoadProfileId)
+   {
+      this.savedBulkLoadProfileId = savedBulkLoadProfileId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSavedViewId(Integer savedViewId)
+   {
+      this.savedViewId = savedViewId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSavedReportId(Integer savedReportId)
+   {
+      this.savedReportId = savedReportId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFormat(String format)
+   {
+      this.format = format;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFileName(String fileName)
+   {
+      this.fileName = fileName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setConsumerName(String consumerName)
+   {
+      this.consumerName = consumerName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setIsActive(Boolean isActive)
+   {
+      this.isActive = isActive;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setDescription(String description)
+   {
+      this.description = description;
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/model/FileGatewayConnectionLog.java

@@ -0,0 +1,702 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.model;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Record entity for connections.
+ *******************************************************************************/
+public class FileGatewayConnectionLog extends QRecordEntity
+{
+   public static final String TABLE_NAME = "file_gateway_connection_log";
+
+   @QField
+   private Integer id;
+
+   @QField
+   private Instant createDate;
+
+   @QField
+   private Instant modifyDate;
+
+   @QField
+   private Integer accountId;
+
+   @QField
+   private String username;
+
+   @QField
+   private String remoteIp;
+
+   @QField
+   private String clientVersion;
+
+   @QField
+   private String keyFingerprint;
+
+   @QField
+   private String outcome;
+
+   @QField
+   private String sessionId;
+
+   @QField
+   private String podName;
+
+   @QField
+   private Instant connectedAt;
+
+   @QField
+   private Instant lastSeenAt;
+
+   @QField
+   private Instant disconnectedAt;
+
+   @QField
+   private String disconnectReason;
+
+   @QField
+   private Long bytesIn;
+
+   @QField
+   private Long bytesOut;
+
+   @QField
+   private Long operationCount;
+
+   @QField
+   private String clientChannel;
+
+
+
+   /*******************************************************************************
+    ** Create an empty entity.
+    *******************************************************************************/
+   public FileGatewayConnectionLog()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Populate from a database record.
+    *******************************************************************************/
+   public FileGatewayConnectionLog(QRecord record) throws QException
+   {
+      populateFromQRecord(record);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id.
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate.
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return (createDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate.
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return (modifyDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for accountId.
+    *******************************************************************************/
+   public Integer getAccountId()
+   {
+      return (accountId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for accountId.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for username.
+    *******************************************************************************/
+   public String getUsername()
+   {
+      return (username);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for username.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withUsername(String username)
+   {
+      this.username = username;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for remoteIp.
+    *******************************************************************************/
+   public String getRemoteIp()
+   {
+      return (remoteIp);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for remoteIp.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withRemoteIp(String remoteIp)
+   {
+      this.remoteIp = remoteIp;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientVersion.
+    *******************************************************************************/
+   public String getClientVersion()
+   {
+      return (clientVersion);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for clientVersion.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withClientVersion(String clientVersion)
+   {
+      this.clientVersion = clientVersion;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for keyFingerprint.
+    *******************************************************************************/
+   public String getKeyFingerprint()
+   {
+      return (keyFingerprint);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for keyFingerprint.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withKeyFingerprint(String keyFingerprint)
+   {
+      this.keyFingerprint = keyFingerprint;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for outcome.
+    *******************************************************************************/
+   public String getOutcome()
+   {
+      return (outcome);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for outcome.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withOutcome(String outcome)
+   {
+      this.outcome = outcome;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for sessionId.
+    *******************************************************************************/
+   public String getSessionId()
+   {
+      return (sessionId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for sessionId.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withSessionId(String sessionId)
+   {
+      this.sessionId = sessionId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for podName.
+    *******************************************************************************/
+   public String getPodName()
+   {
+      return (podName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for podName.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withPodName(String podName)
+   {
+      this.podName = podName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for connectedAt.
+    *******************************************************************************/
+   public Instant getConnectedAt()
+   {
+      return (connectedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for connectedAt.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withConnectedAt(Instant connectedAt)
+   {
+      this.connectedAt = connectedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lastSeenAt.
+    *******************************************************************************/
+   public Instant getLastSeenAt()
+   {
+      return (lastSeenAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lastSeenAt.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withLastSeenAt(Instant lastSeenAt)
+   {
+      this.lastSeenAt = lastSeenAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for disconnectedAt.
+    *******************************************************************************/
+   public Instant getDisconnectedAt()
+   {
+      return (disconnectedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for disconnectedAt.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withDisconnectedAt(Instant disconnectedAt)
+   {
+      this.disconnectedAt = disconnectedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for disconnectReason.
+    *******************************************************************************/
+   public String getDisconnectReason()
+   {
+      return (disconnectReason);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for disconnectReason.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withDisconnectReason(String disconnectReason)
+   {
+      this.disconnectReason = disconnectReason;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for bytesIn.
+    *******************************************************************************/
+   public Long getBytesIn()
+   {
+      return (bytesIn);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for bytesIn.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withBytesIn(Long bytesIn)
+   {
+      this.bytesIn = bytesIn;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for bytesOut.
+    *******************************************************************************/
+   public Long getBytesOut()
+   {
+      return (bytesOut);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for bytesOut.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withBytesOut(Long bytesOut)
+   {
+      this.bytesOut = bytesOut;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for operationCount.
+    *******************************************************************************/
+   public Long getOperationCount()
+   {
+      return (operationCount);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for operationCount.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withOperationCount(Long operationCount)
+   {
+      this.operationCount = operationCount;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientChannel.
+    *******************************************************************************/
+   public String getClientChannel()
+   {
+      return (clientChannel);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for clientChannel.
+    *******************************************************************************/
+   public FileGatewayConnectionLog withClientChannel(String clientChannel)
+   {
+      this.clientChannel = clientChannel;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setUsername(String username)
+   {
+      this.username = username;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setRemoteIp(String remoteIp)
+   {
+      this.remoteIp = remoteIp;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setClientVersion(String clientVersion)
+   {
+      this.clientVersion = clientVersion;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setKeyFingerprint(String keyFingerprint)
+   {
+      this.keyFingerprint = keyFingerprint;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setOutcome(String outcome)
+   {
+      this.outcome = outcome;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSessionId(String sessionId)
+   {
+      this.sessionId = sessionId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setPodName(String podName)
+   {
+      this.podName = podName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setConnectedAt(Instant connectedAt)
+   {
+      this.connectedAt = connectedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setLastSeenAt(Instant lastSeenAt)
+   {
+      this.lastSeenAt = lastSeenAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setDisconnectedAt(Instant disconnectedAt)
+   {
+      this.disconnectedAt = disconnectedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setDisconnectReason(String disconnectReason)
+   {
+      this.disconnectReason = disconnectReason;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setBytesIn(Long bytesIn)
+   {
+      this.bytesIn = bytesIn;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setBytesOut(Long bytesOut)
+   {
+      this.bytesOut = bytesOut;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setOperationCount(Long operationCount)
+   {
+      this.operationCount = operationCount;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setClientChannel(String clientChannel)
+   {
+      this.clientChannel = clientChannel;
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/model/FileGatewayOutboxFile.java

@@ -0,0 +1,532 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.model;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Record entity for outbox files.
+ *******************************************************************************/
+public class FileGatewayOutboxFile extends QRecordEntity
+{
+   public static final String TABLE_NAME = "file_gateway_outbox_file";
+
+   @QField
+   private Integer id;
+
+   @QField
+   private Instant createDate;
+
+   @QField
+   private Instant modifyDate;
+
+   @QField
+   private Integer aliasId;
+
+   @QField
+   private String fileName;
+
+   @QField
+   private Long sizeBytes;
+
+   @QField
+   private String sha256;
+
+   @QField
+   private String storageKey;
+
+   @QField
+   private String status;
+
+   @QField
+   private Instant firstDownloadedAt;
+
+   @QField
+   private Integer downloadCount;
+
+   @QField
+   private Instant removedAt;
+
+   @QField
+   private Instant expiresAt;
+
+   @QField
+   private String createdByConsumer;
+
+
+
+   /*******************************************************************************
+    ** Create an empty entity.
+    *******************************************************************************/
+   public FileGatewayOutboxFile()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Populate from a database record.
+    *******************************************************************************/
+   public FileGatewayOutboxFile(QRecord record) throws QException
+   {
+      populateFromQRecord(record);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id.
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate.
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return (createDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate.
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return (modifyDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for aliasId.
+    *******************************************************************************/
+   public Integer getAliasId()
+   {
+      return (aliasId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for aliasId.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withAliasId(Integer aliasId)
+   {
+      this.aliasId = aliasId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fileName.
+    *******************************************************************************/
+   public String getFileName()
+   {
+      return (fileName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fileName.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withFileName(String fileName)
+   {
+      this.fileName = fileName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for sizeBytes.
+    *******************************************************************************/
+   public Long getSizeBytes()
+   {
+      return (sizeBytes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for sizeBytes.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withSizeBytes(Long sizeBytes)
+   {
+      this.sizeBytes = sizeBytes;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for sha256.
+    *******************************************************************************/
+   public String getSha256()
+   {
+      return (sha256);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for sha256.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withSha256(String sha256)
+   {
+      this.sha256 = sha256;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for storageKey.
+    *******************************************************************************/
+   public String getStorageKey()
+   {
+      return (storageKey);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for storageKey.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withStorageKey(String storageKey)
+   {
+      this.storageKey = storageKey;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for status.
+    *******************************************************************************/
+   public String getStatus()
+   {
+      return (status);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for status.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withStatus(String status)
+   {
+      this.status = status;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for firstDownloadedAt.
+    *******************************************************************************/
+   public Instant getFirstDownloadedAt()
+   {
+      return (firstDownloadedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for firstDownloadedAt.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withFirstDownloadedAt(Instant firstDownloadedAt)
+   {
+      this.firstDownloadedAt = firstDownloadedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for downloadCount.
+    *******************************************************************************/
+   public Integer getDownloadCount()
+   {
+      return (downloadCount);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for downloadCount.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withDownloadCount(Integer downloadCount)
+   {
+      this.downloadCount = downloadCount;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for removedAt.
+    *******************************************************************************/
+   public Instant getRemovedAt()
+   {
+      return (removedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for removedAt.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withRemovedAt(Instant removedAt)
+   {
+      this.removedAt = removedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for expiresAt.
+    *******************************************************************************/
+   public Instant getExpiresAt()
+   {
+      return (expiresAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for expiresAt.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withExpiresAt(Instant expiresAt)
+   {
+      this.expiresAt = expiresAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createdByConsumer.
+    *******************************************************************************/
+   public String getCreatedByConsumer()
+   {
+      return (createdByConsumer);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createdByConsumer.
+    *******************************************************************************/
+   public FileGatewayOutboxFile withCreatedByConsumer(String createdByConsumer)
+   {
+      this.createdByConsumer = createdByConsumer;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setAliasId(Integer aliasId)
+   {
+      this.aliasId = aliasId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFileName(String fileName)
+   {
+      this.fileName = fileName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSizeBytes(Long sizeBytes)
+   {
+      this.sizeBytes = sizeBytes;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSha256(String sha256)
+   {
+      this.sha256 = sha256;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setStorageKey(String storageKey)
+   {
+      this.storageKey = storageKey;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setStatus(String status)
+   {
+      this.status = status;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFirstDownloadedAt(Instant firstDownloadedAt)
+   {
+      this.firstDownloadedAt = firstDownloadedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setDownloadCount(Integer downloadCount)
+   {
+      this.downloadCount = downloadCount;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setRemovedAt(Instant removedAt)
+   {
+      this.removedAt = removedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setExpiresAt(Instant expiresAt)
+   {
+      this.expiresAt = expiresAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreatedByConsumer(String createdByConsumer)
+   {
+      this.createdByConsumer = createdByConsumer;
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/model/FileGatewaySshKey.java

@@ -0,0 +1,498 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.model;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Record entity for ssh keys.
+ *******************************************************************************/
+public class FileGatewaySshKey extends QRecordEntity
+{
+   public static final String TABLE_NAME = "file_gateway_ssh_key";
+
+   @QField
+   private Integer id;
+
+   @QField
+   private Instant createDate;
+
+   @QField
+   private Instant modifyDate;
+
+   @QField
+   private Integer accountId;
+
+   @QField
+   private String label;
+
+   @QField
+   private String publicKey;
+
+   @QField
+   private String fingerprint;
+
+   @QField
+   private String keyType;
+
+   @QField
+   private Integer keyBits;
+
+   @QField
+   private Instant expiresAt;
+
+   @QField
+   private Instant revokedAt;
+
+   @QField
+   private Instant lastUsedAt;
+
+   @QField
+   private String lastUsedIp;
+
+
+
+   /*******************************************************************************
+    ** Create an empty entity.
+    *******************************************************************************/
+   public FileGatewaySshKey()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Populate from a database record.
+    *******************************************************************************/
+   public FileGatewaySshKey(QRecord record) throws QException
+   {
+      populateFromQRecord(record);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id.
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id.
+    *******************************************************************************/
+   public FileGatewaySshKey withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate.
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return (createDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate.
+    *******************************************************************************/
+   public FileGatewaySshKey withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate.
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return (modifyDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate.
+    *******************************************************************************/
+   public FileGatewaySshKey withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for accountId.
+    *******************************************************************************/
+   public Integer getAccountId()
+   {
+      return (accountId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for accountId.
+    *******************************************************************************/
+   public FileGatewaySshKey withAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for label.
+    *******************************************************************************/
+   public String getLabel()
+   {
+      return (label);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for label.
+    *******************************************************************************/
+   public FileGatewaySshKey withLabel(String label)
+   {
+      this.label = label;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for publicKey.
+    *******************************************************************************/
+   public String getPublicKey()
+   {
+      return (publicKey);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for publicKey.
+    *******************************************************************************/
+   public FileGatewaySshKey withPublicKey(String publicKey)
+   {
+      this.publicKey = publicKey;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fingerprint.
+    *******************************************************************************/
+   public String getFingerprint()
+   {
+      return (fingerprint);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fingerprint.
+    *******************************************************************************/
+   public FileGatewaySshKey withFingerprint(String fingerprint)
+   {
+      this.fingerprint = fingerprint;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for keyType.
+    *******************************************************************************/
+   public String getKeyType()
+   {
+      return (keyType);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for keyType.
+    *******************************************************************************/
+   public FileGatewaySshKey withKeyType(String keyType)
+   {
+      this.keyType = keyType;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for keyBits.
+    *******************************************************************************/
+   public Integer getKeyBits()
+   {
+      return (keyBits);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for keyBits.
+    *******************************************************************************/
+   public FileGatewaySshKey withKeyBits(Integer keyBits)
+   {
+      this.keyBits = keyBits;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for expiresAt.
+    *******************************************************************************/
+   public Instant getExpiresAt()
+   {
+      return (expiresAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for expiresAt.
+    *******************************************************************************/
+   public FileGatewaySshKey withExpiresAt(Instant expiresAt)
+   {
+      this.expiresAt = expiresAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for revokedAt.
+    *******************************************************************************/
+   public Instant getRevokedAt()
+   {
+      return (revokedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for revokedAt.
+    *******************************************************************************/
+   public FileGatewaySshKey withRevokedAt(Instant revokedAt)
+   {
+      this.revokedAt = revokedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lastUsedAt.
+    *******************************************************************************/
+   public Instant getLastUsedAt()
+   {
+      return (lastUsedAt);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lastUsedAt.
+    *******************************************************************************/
+   public FileGatewaySshKey withLastUsedAt(Instant lastUsedAt)
+   {
+      this.lastUsedAt = lastUsedAt;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for lastUsedIp.
+    *******************************************************************************/
+   public String getLastUsedIp()
+   {
+      return (lastUsedIp);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for lastUsedIp.
+    *******************************************************************************/
+   public FileGatewaySshKey withLastUsedIp(String lastUsedIp)
+   {
+      this.lastUsedIp = lastUsedIp;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setLabel(String label)
+   {
+      this.label = label;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setPublicKey(String publicKey)
+   {
+      this.publicKey = publicKey;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFingerprint(String fingerprint)
+   {
+      this.fingerprint = fingerprint;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setKeyType(String keyType)
+   {
+      this.keyType = keyType;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setKeyBits(Integer keyBits)
+   {
+      this.keyBits = keyBits;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setExpiresAt(Instant expiresAt)
+   {
+      this.expiresAt = expiresAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setRevokedAt(Instant revokedAt)
+   {
+      this.revokedAt = revokedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setLastUsedAt(Instant lastUsedAt)
+   {
+      this.lastUsedAt = lastUsedAt;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setLastUsedIp(String lastUsedIp)
+   {
+      this.lastUsedIp = lastUsedIp;
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/model/FileGatewayTransferLog.java

@@ -0,0 +1,634 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.model;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Record entity for transfers.
+ *******************************************************************************/
+public class FileGatewayTransferLog extends QRecordEntity
+{
+   public static final String TABLE_NAME = "file_gateway_transfer_log";
+
+   @QField
+   private Integer id;
+
+   @QField
+   private Instant createDate;
+
+   @QField
+   private Instant modifyDate;
+
+   @QField
+   private Integer accountId;
+
+   @QField
+   private Integer connectionLogId;
+
+   @QField
+   private String operation;
+
+   @QField
+   private String path;
+
+   @QField
+   private String objectKind;
+
+   @QField
+   private String objectId;
+
+   @QField
+   private String format;
+
+   @QField
+   private Long bytes;
+
+   @QField
+   private String sha256;
+
+   @QField
+   private Long durationMs;
+
+   @QField
+   private String outcome;
+
+   @QField
+   private String message;
+
+   @QField
+   private String clientChannel;
+
+   @QField
+   private Integer uploadJobId;
+
+
+
+   /*******************************************************************************
+    ** Create an empty entity.
+    *******************************************************************************/
+   public FileGatewayTransferLog()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Populate from a database record.
+    *******************************************************************************/
+   public FileGatewayTransferLog(QRecord record) throws QException
+   {
+      populateFromQRecord(record);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id.
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id.
+    *******************************************************************************/
+   public FileGatewayTransferLog withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate.
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return (createDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate.
+    *******************************************************************************/
+   public FileGatewayTransferLog withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate.
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return (modifyDate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate.
+    *******************************************************************************/
+   public FileGatewayTransferLog withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for accountId.
+    *******************************************************************************/
+   public Integer getAccountId()
+   {
+      return (accountId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for accountId.
+    *******************************************************************************/
+   public FileGatewayTransferLog withAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for connectionLogId.
+    *******************************************************************************/
+   public Integer getConnectionLogId()
+   {
+      return (connectionLogId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for connectionLogId.
+    *******************************************************************************/
+   public FileGatewayTransferLog withConnectionLogId(Integer connectionLogId)
+   {
+      this.connectionLogId = connectionLogId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for operation.
+    *******************************************************************************/
+   public String getOperation()
+   {
+      return (operation);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for operation.
+    *******************************************************************************/
+   public FileGatewayTransferLog withOperation(String operation)
+   {
+      this.operation = operation;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for path.
+    *******************************************************************************/
+   public String getPath()
+   {
+      return (path);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for path.
+    *******************************************************************************/
+   public FileGatewayTransferLog withPath(String path)
+   {
+      this.path = path;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for objectKind.
+    *******************************************************************************/
+   public String getObjectKind()
+   {
+      return (objectKind);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for objectKind.
+    *******************************************************************************/
+   public FileGatewayTransferLog withObjectKind(String objectKind)
+   {
+      this.objectKind = objectKind;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for objectId.
+    *******************************************************************************/
+   public String getObjectId()
+   {
+      return (objectId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for objectId.
+    *******************************************************************************/
+   public FileGatewayTransferLog withObjectId(String objectId)
+   {
+      this.objectId = objectId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for format.
+    *******************************************************************************/
+   public String getFormat()
+   {
+      return (format);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for format.
+    *******************************************************************************/
+   public FileGatewayTransferLog withFormat(String format)
+   {
+      this.format = format;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for bytes.
+    *******************************************************************************/
+   public Long getBytes()
+   {
+      return (bytes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for bytes.
+    *******************************************************************************/
+   public FileGatewayTransferLog withBytes(Long bytes)
+   {
+      this.bytes = bytes;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for sha256.
+    *******************************************************************************/
+   public String getSha256()
+   {
+      return (sha256);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for sha256.
+    *******************************************************************************/
+   public FileGatewayTransferLog withSha256(String sha256)
+   {
+      this.sha256 = sha256;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for durationMs.
+    *******************************************************************************/
+   public Long getDurationMs()
+   {
+      return (durationMs);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for durationMs.
+    *******************************************************************************/
+   public FileGatewayTransferLog withDurationMs(Long durationMs)
+   {
+      this.durationMs = durationMs;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for outcome.
+    *******************************************************************************/
+   public String getOutcome()
+   {
+      return (outcome);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for outcome.
+    *******************************************************************************/
+   public FileGatewayTransferLog withOutcome(String outcome)
+   {
+      this.outcome = outcome;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for message.
+    *******************************************************************************/
+   public String getMessage()
+   {
+      return (message);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for message.
+    *******************************************************************************/
+   public FileGatewayTransferLog withMessage(String message)
+   {
+      this.message = message;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientChannel.
+    *******************************************************************************/
+   public String getClientChannel()
+   {
+      return (clientChannel);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for clientChannel.
+    *******************************************************************************/
+   public FileGatewayTransferLog withClientChannel(String clientChannel)
+   {
+      this.clientChannel = clientChannel;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for uploadJobId.
+    *******************************************************************************/
+   public Integer getUploadJobId()
+   {
+      return (uploadJobId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for uploadJobId.
+    *******************************************************************************/
+   public FileGatewayTransferLog withUploadJobId(Integer uploadJobId)
+   {
+      this.uploadJobId = uploadJobId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setAccountId(Integer accountId)
+   {
+      this.accountId = accountId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setConnectionLogId(Integer connectionLogId)
+   {
+      this.connectionLogId = connectionLogId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setOperation(String operation)
+   {
+      this.operation = operation;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setPath(String path)
+   {
+      this.path = path;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setObjectKind(String objectKind)
+   {
+      this.objectKind = objectKind;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setObjectId(String objectId)
+   {
+      this.objectId = objectId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setFormat(String format)
+   {
+      this.format = format;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setBytes(Long bytes)
+   {
+      this.bytes = bytes;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setSha256(String sha256)
+   {
+      this.sha256 = sha256;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setDurationMs(Long durationMs)
+   {
+      this.durationMs = durationMs;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setOutcome(String outcome)
+   {
+      this.outcome = outcome;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setMessage(String message)
+   {
+      this.message = message;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setClientChannel(String clientChannel)
+   {
+      this.clientChannel = clientChannel;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter used by QRecordEntity population.
+    *******************************************************************************/
+   public void setUploadJobId(Integer uploadJobId)
+   {
+      this.uploadJobId = uploadJobId;
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/permissions/FileGatewayPermissionChecker.java

@@ -0,0 +1,63 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.permissions;
+
+
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+
+
+/*******************************************************************************
+ ** Read with view or admin; only administrators write configuration tables.
+ *******************************************************************************/
+public class FileGatewayPermissionChecker implements CustomPermissionChecker
+{
+
+
+
+   /*******************************************************************************
+    ** Check the current user against the gateway permission model.
+    *******************************************************************************/
+   @Override
+   public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
+   {
+      boolean write = metadata instanceof QProcessMetaData || input instanceof InsertInput || input instanceof UpdateInput || input instanceof DeleteInput;
+      if(write && metadata instanceof QTableMetaData table && FileGatewayTables.SYSTEM_WRITTEN.contains(table.getName()))
+      {
+         throw (new QPermissionDeniedException("This table is written only by the file gateway"));
+      }
+      QSession session = QContext.getQSession();
+      if(session == null || !(session.hasPermission(FileGatewayPermissions.ADMIN) || (!write && session.hasPermission(FileGatewayPermissions.VIEW))))
+      {
+         throw (new QPermissionDeniedException("Missing file gateway permission"));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/permissions/FileGatewayPermissions.java

@@ -0,0 +1,45 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.permissions;
+
+
+import java.util.List;
+
+
+/*******************************************************************************
+ ** Permissions granted by the host application.
+ *******************************************************************************/
+public final class FileGatewayPermissions
+{
+   public static final String VIEW = "file_gateway.view";
+   public static final String ADMIN = "file_gateway.admin";
+   public static final String CONNECT = "file_gateway.connect";
+   public static final List<String> ALL = List.of(VIEW, ADMIN, CONNECT);
+
+
+
+   /*******************************************************************************
+    ** Constants only.
+    *******************************************************************************/
+   private FileGatewayPermissions()
+   {
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/AuditWriter.java

@@ -0,0 +1,251 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.io.Serializable;
+import java.time.Instant;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
+import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
+
+
+/*******************************************************************************
+ ** System-session access limited to the gateway own tables; application data uses the UI API.
+ *******************************************************************************/
+public class AuditWriter
+{
+   private final QInstance instance;
+   private final ThreadLocal<QBackendTransaction> transaction = new ThreadLocal<>();
+
+
+
+   /*******************************************************************************
+    ** Bind the gateway database instance.
+    *******************************************************************************/
+   public AuditWriter(QInstance instance)
+   {
+      this.instance = instance;
+   }
+
+
+   /*******************************************************************************
+    ** Work within a short system-session scope.
+    *******************************************************************************/
+   @FunctionalInterface
+   public interface Work<T>
+   {
+      /*******************************************************************************
+       ** Execute the database operation.
+       *******************************************************************************/
+      T run() throws QException;
+   }
+
+
+
+   /*******************************************************************************
+    ** Preserve caller context while writing gateway-owned state.
+    *******************************************************************************/
+   public <T> T system(Work<T> work) throws QException
+   {
+      var previous = QContext.capture();
+      try
+      {
+         QContext.init(instance, new QSystemUserSession(instance));
+         try(var ignored = CorePermissionEnforcer.openAuthorizedScope())
+         {
+            return (work.run());
+         }
+      }
+      finally
+      {
+         QContext.init(previous);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Serialize account authentication across pods with a transactional row update.
+    ** Re-reading after acquiring the write lock sees committed counters and sessions.
+    *******************************************************************************/
+   public <T> T withAccountLock(String username, Work<T> work) throws QException
+   {
+      return (withAccountLock("username", username, work));
+   }
+
+
+
+   /*******************************************************************************
+    ** Lock one account row without changing any user-managed fields.
+    *******************************************************************************/
+   public <T> T withAccountLock(String field, Serializable value, Work<T> work) throws QException
+   {
+      return (system(() ->
+      {
+         try(var tx = QBackendTransaction.openFor(new QueryInput(FileGatewayTables.ACCOUNT)))
+         {
+            transaction.set(tx);
+            try
+            {
+               var rows = query(FileGatewayTables.ACCOUNT, new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
+               if(!rows.isEmpty())
+               {
+                  update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", rows.getFirst().getValue("id")).withValue("modifyDate", Instant.now()));
+               }
+               T result = work.run();
+               tx.commit();
+               return (result);
+            }
+            catch(Exception e)
+            {
+               tx.rollback();
+               throw (new QException("Gateway account transaction failed", e));
+            }
+            finally
+            {
+               transaction.remove();
+            }
+         }
+      }));
+   }
+
+
+
+   /*******************************************************************************
+    ** Charge an account-wide minute window atomically, including other gateway pods.
+    *******************************************************************************/
+   public boolean operationAllowed(Integer accountId, Instant now, int maximum) throws QException
+   {
+      return (withAccountLock("id", accountId, () ->
+      {
+         var rows = query(FileGatewayTables.ACCOUNT, new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.EQUALS, accountId)));
+         if(rows.isEmpty())
+         {
+            throw (new QException("Gateway account no longer exists"));
+         }
+         QRecord account = rows.getFirst();
+         Instant window = account.getValueInstant("operationWindowAt");
+         Integer previous = account.getValueInteger("operationWindowCount");
+         boolean reset = window == null || !window.plusSeconds(60).isAfter(now);
+         int count = reset ? 1 : (previous == null ? 0 : previous) + 1;
+         update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", accountId)
+            .withValue("operationWindowAt", reset ? now : window).withValue("operationWindowCount", count));
+         return (count <= maximum);
+      }));
+   }
+
+
+
+   /*******************************************************************************
+    ** Reject accidental use on application data.
+    *******************************************************************************/
+   private void checkTable(String table)
+   {
+      if(!FileGatewayTables.ALL.contains(table))
+      {
+         throw (new IllegalArgumentException("System access is limited to gateway tables"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Read gateway state.
+    *******************************************************************************/
+   public List<QRecord> query(String table, QQueryFilter filter) throws QException
+   {
+      checkTable(table);
+      return (system(() -> new QueryAction().execute(new QueryInput(table).withFilter(filter).withTransaction(transaction.get())).getRecords()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert and require successful persistence.
+    *******************************************************************************/
+   public QRecord insert(String table, QRecord record) throws QException
+   {
+      checkTable(table);
+      QRecord result = system(() -> new InsertAction().execute(new InsertInput(table).withRecord(record).withTransaction(transaction.get())).getRecords().get(0));
+      checkErrors(result);
+      return (result);
+   }
+
+
+
+   /*******************************************************************************
+    ** Update gateway state.
+    *******************************************************************************/
+   public void update(String table, QRecord record) throws QException
+   {
+      checkTable(table);
+      QRecord result = system(() -> new UpdateAction().execute(new UpdateInput(table).withRecords(List.of(record)).withTransaction(transaction.get())).getRecords().get(0));
+      checkErrors(result);
+   }
+
+
+
+   /*******************************************************************************
+    ** Optimistic status transition, called under withAccountLock because core checks
+    ** expected versions before SQL execution rather than in the SQL WHERE clause.
+    *******************************************************************************/
+   public boolean transitionUpload(QRecord changes, String expectedStatus) throws QException
+   {
+      QRecord result = system(() -> new UpdateAction().execute(new UpdateInput(FileGatewayTables.UPLOAD_JOB)
+         .withRecords(List.of(changes)).withTransaction(transaction.get())
+         .withExpectedVersions(Map.of(changes.getValue("id"), expectedStatus))).getRecords().getFirst());
+      if(result.getErrors() != null && !result.getErrors().isEmpty())
+      {
+         return (false);
+      }
+      return (true);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fail closed on bookkeeping errors without printing records.
+    *******************************************************************************/
+   private void checkErrors(QRecord record) throws QException
+   {
+      if(record.getErrors() != null && !record.getErrors().isEmpty())
+      {
+         throw (new QException("Unable to persist gateway state"));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/CidrMatcher.java

@@ -0,0 +1,105 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.net.InetAddress;
+
+
+/*******************************************************************************
+ ** Numeric IPv4 and IPv6 CIDR matching; hostnames never trigger DNS resolution.
+ *******************************************************************************/
+public final class CidrMatcher
+{
+
+
+
+   /*******************************************************************************
+    ** Utility only.
+    *******************************************************************************/
+   private CidrMatcher()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Match an address against comma or whitespace separated networks.
+    *******************************************************************************/
+   public static boolean matches(String address, String cidrs)
+   {
+      if(cidrs == null || cidrs.isBlank())
+      {
+         return (true);
+      }
+      try
+      {
+         byte[] candidate = numeric(address);
+         for(String cidr : cidrs.split("[,\\s]+"))
+         {
+            String[] parts = cidr.split("/", -1);
+            byte[] network = numeric(parts[0]);
+            int bits = parts.length == 1 ? network.length * 8 : Integer.parseInt(parts[1]);
+            if(parts.length > 2 || bits < 0 || bits > network.length * 8)
+            {
+               return (false);
+            }
+            if(candidate.length != network.length)
+            {
+               continue;
+            }
+            boolean match = true;
+            for(int i = 0; i < bits; i++)
+            {
+               int mask = 1 << (7 - i % 8);
+               if((candidate[i / 8] & mask) != (network[i / 8] & mask))
+               {
+                  match = false;
+                  break;
+               }
+            }
+            if(match)
+            {
+               return (true);
+            }
+         }
+      }
+      catch(Exception e)
+      {
+         return (false);
+      }
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Parse numeric addresses only.
+    *******************************************************************************/
+   private static byte[] numeric(String address) throws Exception
+   {
+      if(address == null || !address.matches("[0-9a-fA-F:.]+") || (!address.contains(":") && !address.matches("[0-9]{1,3}(\\.[0-9]{1,3}){3}")))
+      {
+         throw (new IllegalArgumentException("Expected a numeric IP address"));
+      }
+      return (InetAddress.getByName(address).getAddress());
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/FileGatewayAuthenticatorInterface.java

@@ -0,0 +1,37 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.security.PublicKey;
+import org.apache.sshd.server.session.ServerSession;
+
+
+/*******************************************************************************
+ ** Protocol authentication boundary for public-key credentials.
+ *******************************************************************************/
+public interface FileGatewayAuthenticatorInterface
+{
+   /*******************************************************************************
+    ** Resolve and authenticate a protocol credential.
+    *******************************************************************************/
+   boolean authenticate(String username, PublicKey key, ServerSession session);
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/FileGatewayProtocolServer.java

@@ -0,0 +1,46 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.io.IOException;
+
+
+/*******************************************************************************
+ ** Lifecycle of a gateway protocol listener.
+ *******************************************************************************/
+public interface FileGatewayProtocolServer
+{
+   /*******************************************************************************
+    ** Start listening.
+    *******************************************************************************/
+   void start() throws IOException;
+
+   /*******************************************************************************
+    ** Close the listener and live sessions.
+    *******************************************************************************/
+   void stop() throws IOException;
+
+   /*******************************************************************************
+    ** Number of open sessions.
+    *******************************************************************************/
+   int activeSessions();
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/GatewaySession.java

@@ -0,0 +1,129 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.UUID;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
+import org.apache.sshd.common.AttributeRepository.AttributeKey;
+import org.apache.sshd.server.session.ServerSession;
+
+
+/*******************************************************************************
+ ** One SSH connection, its audit identity and refreshable in-process credentials.
+ *******************************************************************************/
+public class GatewaySession
+{
+   public static final AttributeKey<GatewaySession> ATTRIBUTE = new AttributeKey<>();
+   public static final AttributeKey<String> TOKEN = new AttributeKey<>();
+   public final String id = UUID.randomUUID().toString();
+   public final ServerSession ssh;
+   public final String remoteIp;
+   public Integer connectionId;
+   public QRecord account;
+   public QRecord key;
+   public long operations;
+   public String disconnectReason = "Client disconnected";
+   private Instant mintedAt;
+   private final QInstance instance;
+   private final Clock clock;
+
+
+
+   /*******************************************************************************
+    ** Bind an accepted SSH connection.
+    *******************************************************************************/
+   public GatewaySession(QInstance instance, ServerSession ssh, Clock clock)
+   {
+      this.instance = instance;
+      this.ssh = ssh;
+      this.clock = clock;
+      this.remoteIp = ((InetSocketAddress) ssh.getClientAddress()).getAddress().getHostAddress();
+   }
+
+
+
+   /*******************************************************************************
+    ** Mint via the host authentication module and refresh permissions every five minutes.
+    *******************************************************************************/
+   public synchronized String token() throws QException
+   {
+      if(account == null || ssh.isClosing() || !ssh.isOpen())
+      {
+         throw (new QException("Unauthenticated connection"));
+      }
+      if(mintedAt == null || !mintedAt.plus(Duration.ofMinutes(5)).isAfter(clock.instant()))
+      {
+         var module = new QAuthenticationModuleDispatcher().getQModule(instance.getAuthentication());
+         var user = new AuditWriter(instance).system(() -> module.createAutomatedSessionForUser(instance, account.getValueString("userId")));
+         if(user == null || !user.hasPermission(FileGatewayPermissions.CONNECT))
+         {
+            revoke();
+            throw (new QException("no file_gateway.connect"));
+         }
+         String next = UiApiInProcessSessions.register(user, Duration.ofMinutes(10));
+         UiApiInProcessSessions.revoke(ssh.getAttribute(TOKEN));
+         ssh.setAttribute(TOKEN, next);
+         mintedAt = clock.instant();
+      }
+      return (ssh.getAttribute(TOKEN));
+   }
+
+
+
+   /*******************************************************************************
+    ** Release credentials immediately on disconnect or revocation.
+    *******************************************************************************/
+   public synchronized void revoke()
+   {
+      UiApiInProcessSessions.revoke(ssh.removeAttribute(TOKEN));
+      mintedAt = null;
+   }
+
+
+
+   /*******************************************************************************
+    ** Count operations locally and charge the shared account minute window.
+    *******************************************************************************/
+   public synchronized boolean operationAllowed() throws IOException
+   {
+      operations++;
+      Integer maximum = account.getValueInteger("maxOperationsPerMinute");
+      try
+      {
+         return (new AuditWriter(instance).operationAllowed(account.getValueInteger("id"), clock.instant(), maximum == null ? 600 : maximum));
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to verify account operation limit", e));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/SftpProtocolServer.java

@@ -0,0 +1,297 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.io.IOException;
+import java.nio.file.FileSystem;
+import java.nio.file.Path;
+import java.security.KeyPair;
+import java.time.Clock;
+import java.time.Duration;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+import com.kingsrook.qbits.filegateway.storage.ObjectStorageGatewayStore;
+import com.kingsrook.qbits.filegateway.upload.UploadJobs;
+import com.kingsrook.qbits.filegateway.vfs.GatewayFileSystem;
+import com.kingsrook.qbits.filegateway.vfs.GatewayUiApiClient;
+import com.kingsrook.qbits.filegateway.vfs.VirtualTree;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import org.apache.sshd.common.cipher.BuiltinCiphers;
+import org.apache.sshd.common.file.FileSystemFactory;
+import org.apache.sshd.common.kex.BuiltinDHFactories;
+import org.apache.sshd.common.keyprovider.KeyPairProvider;
+import org.apache.sshd.common.mac.BuiltinMacs;
+import org.apache.sshd.common.session.Session;
+import org.apache.sshd.common.session.SessionContext;
+import org.apache.sshd.common.session.SessionListener;
+import org.apache.sshd.common.signature.BuiltinSignatures;
+import org.apache.sshd.core.CoreModuleProperties;
+import org.apache.sshd.server.ServerBuilder;
+import org.apache.sshd.server.SshServer;
+import org.apache.sshd.server.auth.pubkey.UserAuthPublicKeyFactory;
+import org.apache.sshd.server.forward.RejectAllForwardingFilter;
+import org.apache.sshd.server.session.ServerSession;
+import org.apache.sshd.sftp.server.SftpSubsystemFactory;
+
+
+/*******************************************************************************
+ ** Public-key-only SFTP with explicit modern algorithms and an entirely virtual filesystem.
+ *******************************************************************************/
+public class SftpProtocolServer implements FileGatewayProtocolServer
+{
+   private static final QLogger LOG = QLogger.getLogger(SftpProtocolServer.class);
+   private final SshServer server;
+   private final SshKeyAuthenticator authenticator;
+   private final Duration pollInterval;
+   private ScheduledExecutorService poller;
+
+
+
+   /*******************************************************************************
+    ** Configure the listener, security policy and injected runtime dependencies.
+    *******************************************************************************/
+   public SftpProtocolServer(QInstance instance, FileGatewayQBitConfig config, UiApiInvoker invoker, KeyPair hostKey, Clock clock, Duration pollInterval, String podName)
+   {
+      this(instance, config, invoker, hostKey, clock, pollInterval, podName, new ObjectStorageGatewayStore(instance, config.getFileGatewayBucketAlias()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Inject storage for transfer tests; production uses the object-storage implementation.
+    *******************************************************************************/
+   public SftpProtocolServer(QInstance instance, FileGatewayQBitConfig config, UiApiInvoker invoker, KeyPair hostKey, Clock clock, Duration pollInterval, String podName, GatewayObjectStore store)
+   {
+      this.pollInterval = pollInterval;
+      AuditWriter audit = new AuditWriter(instance);
+      authenticator = new SshKeyAuthenticator(instance, config, audit, clock, podName);
+      server = SshServer.setUpDefaultServer();
+      server.setPort(config.getSftpPort());
+      server.setKeyPairProvider(KeyPairProvider.wrap(hostKey));
+      server.setCipherFactories(List.of(BuiltinCiphers.cc20p1305_openssh, BuiltinCiphers.aes128gcm, BuiltinCiphers.aes256gcm, BuiltinCiphers.aes128ctr, BuiltinCiphers.aes192ctr, BuiltinCiphers.aes256ctr));
+      server.setMacFactories(List.of(BuiltinMacs.hmacsha256etm, BuiltinMacs.hmacsha512etm));
+      server.setKeyExchangeFactories(List.of(BuiltinDHFactories.curve25519, BuiltinDHFactories.curve25519_libssh, BuiltinDHFactories.ecdhp256, BuiltinDHFactories.ecdhp384, BuiltinDHFactories.ecdhp521, BuiltinDHFactories.dhg16_512, BuiltinDHFactories.dhg18_512)
+         .stream().filter(BuiltinDHFactories::isSupported).map(ServerBuilder.DH2KEX).toList());
+      server.setSignatureFactories(List.of(BuiltinSignatures.ed25519, BuiltinSignatures.nistp256, BuiltinSignatures.nistp384, BuiltinSignatures.nistp521, BuiltinSignatures.rsaSHA256, BuiltinSignatures.rsaSHA512));
+      server.setPasswordAuthenticator(null);
+      server.setKeyboardInteractiveAuthenticator(null);
+      server.setGSSAuthenticator(null);
+      server.setHostBasedAuthenticator(null);
+      server.setUserAuthFactories(List.of(UserAuthPublicKeyFactory.INSTANCE));
+      server.setPublickeyAuthenticator(authenticator);
+      server.setShellFactory(null);
+      server.setCommandFactory(null);
+      server.setForwardingFilter(RejectAllForwardingFilter.INSTANCE);
+      server.setAgentFactory(null);
+      server.setSubsystemFactories(List.of(new SftpSubsystemFactory.Builder().withFileSystemAccessor(new org.apache.sshd.sftp.server.SftpFileSystemAccessor()
+      {
+         /*******************************************************************************
+          ** MINA must use our seekable channels, not the native FileChannel default.
+          *******************************************************************************/
+         @Override
+         public java.nio.channels.SeekableByteChannel openFile(org.apache.sshd.sftp.server.SftpSubsystemProxy subsystem, org.apache.sshd.sftp.server.FileHandle handle, Path path, String handleId,
+            java.util.Set<? extends java.nio.file.OpenOption> options, java.nio.file.attribute.FileAttribute<?>... attributes) throws IOException
+         {
+            return (java.nio.file.Files.newByteChannel(path, options, attributes));
+         }
+      }).build()));
+      CoreModuleProperties.IDLE_TIMEOUT.set(server, Duration.ofMinutes(15));
+      CoreModuleProperties.AUTH_TIMEOUT.set(server, Duration.ofSeconds(30));
+      server.setFileSystemFactory(new FileSystemFactory()
+      {
+         /*******************************************************************************
+          ** Create the session virtual root.
+          *******************************************************************************/
+         @Override
+         public Path getUserHomeDir(SessionContext session) throws IOException
+         {
+            return (createFileSystem(session).getPath("/"));
+         }
+      
+         /*******************************************************************************
+          ** No native filesystem is consulted.
+          *******************************************************************************/
+         @Override
+         public FileSystem createFileSystem(SessionContext session)
+         {
+            GatewaySession state = session.getAttribute(GatewaySession.ATTRIBUTE);
+            return (new GatewayFileSystem(new VirtualTree(new GatewayUiApiClient(invoker, state), state, audit, clock).withUploads(new UploadJobs(audit, store, config, clock), store)));
+         }
+      });
+      server.addSessionListener(new SessionListener()
+      {
+         /*******************************************************************************
+          ** Persist accept before authentication.
+          *******************************************************************************/
+         @Override
+         public void sessionCreated(Session session)
+         {
+            try
+            {
+               authenticator.accepted((ServerSession) session);
+            }
+            catch(Exception e)
+            {
+               LOG.warn("Unable to audit SFTP accept");
+               session.close(true);
+            }
+         }
+      
+         /*******************************************************************************
+          ** Login bookkeeping starts only after proof of private-key possession.
+          *******************************************************************************/
+         @Override
+         public void sessionEvent(Session session, Event event)
+         {
+            if(event == Event.Authenticated)
+            {
+               try
+               {
+                  authenticator.authenticated((ServerSession) session);
+               }
+               catch(Exception e)
+               {
+                  LOG.warn("Unable to finalize SFTP authentication");
+                  session.close(true);
+               }
+            }
+         }
+
+
+
+         /*******************************************************************************
+          ** Always revoke the in-process token.
+          *******************************************************************************/
+         @Override
+         public void sessionClosed(Session session)
+         {
+            try
+            {
+               authenticator.closed((ServerSession) session);
+            }
+            catch(Exception e)
+            {
+               LOG.warn("Unable to finalize SFTP connection audit");
+            }
+         }
+      });
+   }
+
+
+
+   /*******************************************************************************
+    ** Start the listener and revocation heartbeat.
+    *******************************************************************************/
+   @Override
+   public void start() throws IOException
+   {
+      server.start();
+      poller = Executors.newSingleThreadScheduledExecutor(runnable ->
+      {
+         Thread thread = new Thread(runnable, "file-gateway-revocation");
+         thread.setDaemon(true);
+         return (thread);
+      });
+      poller.scheduleWithFixedDelay(() ->
+      {
+         try
+         {
+            authenticator.poll();
+         }
+         catch(Exception e)
+         {
+            LOG.warn("Unable to verify live SFTP sessions; closing them");
+            server.getActiveSessions().forEach(session -> session.close(true));
+         }
+      }, pollInterval.toMillis(), pollInterval.toMillis(), TimeUnit.MILLISECONDS);
+   }
+
+
+
+   /*******************************************************************************
+    ** Stop polling, revoke sessions and stop the listener.
+    *******************************************************************************/
+   @Override
+   public void stop() throws IOException
+   {
+      if(poller != null)
+      {
+         poller.shutdownNow();
+      }
+      server.stop(true);
+      try
+      {
+         authenticator.shutdown();
+      }
+      catch(com.kingsrook.qqq.backend.core.exceptions.QException e)
+      {
+         throw (new IOException("Unable to finalize gateway shutdown", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Count active SSH connections.
+    *******************************************************************************/
+   @Override
+   public int activeSessions()
+   {
+      return (server.getActiveSessions().size());
+   }
+
+
+
+   /*******************************************************************************
+    ** Bound port, including ephemeral test ports.
+    *******************************************************************************/
+   public int getPort()
+   {
+      return (server.getPort());
+   }
+
+
+
+   /*******************************************************************************
+    ** Server security configuration for inspection.
+    *******************************************************************************/
+   public SshServer getServer()
+   {
+      return (server);
+   }
+
+
+
+   /*******************************************************************************
+    ** Live revocation hook for deterministic tests.
+    *******************************************************************************/
+   public SshKeyAuthenticator getAuthenticator()
+   {
+      return (authenticator);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/sftp/SshKeyAuthenticator.java

@@ -0,0 +1,435 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.sftp;
+
+
+import java.security.PublicKey;
+import java.time.Clock;
+import java.time.Instant;
+import java.util.ArrayDeque;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.digest.BuiltinDigests;
+import org.apache.sshd.server.auth.pubkey.PublickeyAuthenticator;
+import org.apache.sshd.server.session.ServerSession;
+
+
+/*******************************************************************************
+ ** Database-backed SSH key authentication, account limits, IP windows and live revocation.
+ *******************************************************************************/
+public class SshKeyAuthenticator implements PublickeyAuthenticator, FileGatewayAuthenticatorInterface
+{
+   private static final QLogger LOG = QLogger.getLogger(SshKeyAuthenticator.class);
+   private final QInstance instance;
+   private final FileGatewayQBitConfig config;
+   private final AuditWriter audit;
+   private final Clock clock;
+   private final String podName;
+   private final Map<String, GatewaySession> sessions = new ConcurrentHashMap<>();
+   private final Map<String, Deque<Instant>> failures = new HashMap<>();
+   private final Map<String, Deque<Instant>> connections = new HashMap<>();
+   private final Map<String, Instant> ipLocks = new HashMap<>();
+   private final Map<String, String> ipLockAccounts = new HashMap<>();
+   private final Map<String, String> failureAccounts = new HashMap<>();
+
+
+
+   /*******************************************************************************
+    ** Bind authentication and audit dependencies.
+    *******************************************************************************/
+   public SshKeyAuthenticator(QInstance instance, FileGatewayQBitConfig config, AuditWriter audit, Clock clock, String podName)
+   {
+      this.instance = instance;
+      this.config = config;
+      this.audit = audit;
+      this.clock = clock;
+      this.podName = podName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Audit TCP accept and enforce the pre-authentication connection rate.
+    *******************************************************************************/
+   public synchronized void accepted(ServerSession ssh) throws QException
+   {
+      GatewaySession state = new GatewaySession(instance, ssh, clock);
+      ssh.setAttribute(GatewaySession.ATTRIBUTE, state);
+      state.connectionId = audit.insert(FileGatewayTables.CONNECTION_LOG, new QRecord()
+         .withValue("sessionId", state.id).withValue("remoteIp", state.remoteIp).withValue("podName", podName)
+         .withValue("connectedAt", clock.instant()).withValue("lastSeenAt", clock.instant())
+         .withValue("clientChannel", "sftp").withValue("outcome", "AUTH_FAILED")).getValueInteger("id");
+      sessions.put(state.id, state);
+      prune(connections, clock.instant().minusSeconds(60));
+      Deque<Instant> window = connections.computeIfAbsent(state.remoteIp, ignored -> new ArrayDeque<>());
+      window.addLast(clock.instant());
+      if(window.size() > config.getMaxConnectionsPerIpPerMinute())
+      {
+         outcome(state, "LIMIT", "Connection rate limit exceeded");
+         ssh.close(true);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Validate the account and offered public key, then mint the user session.
+    *******************************************************************************/
+   @Override
+   public synchronized boolean authenticate(String username, PublicKey publicKey, ServerSession ssh)
+   {
+      GatewaySession state = ssh.getAttribute(GatewaySession.ATTRIBUTE);
+      if(state == null || ssh.isClosing())
+      {
+         return (false);
+      }
+      try
+      {
+         return (audit.withAccountLock(username, () -> authenticateLocked(username, publicKey, ssh, state)));
+      }
+      catch(Exception e)
+      {
+         LOG.warn("SFTP authentication service failed", e);
+         state.revoke();
+         state.disconnectReason = "Authentication service unavailable";
+         ssh.close(true);
+         return (false);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Account checks and reservations share one database transaction across pods.
+    *******************************************************************************/
+   private boolean authenticateLocked(String username, PublicKey publicKey, ServerSession ssh, GatewaySession state) throws QException
+   {
+      if(state.account != null)
+      {
+         return (username.equals(state.account.getValueString("username")) && state.key != null
+            && KeyUtils.getFingerPrint(BuiltinDigests.sha256, publicKey).equals(state.key.getValueString("fingerprint")));
+      }
+      QRecord account = first(FileGatewayTables.ACCOUNT, "username", username);
+      String fingerprint = KeyUtils.getFingerPrint(BuiltinDigests.sha256, publicKey);
+      audit.update(FileGatewayTables.CONNECTION_LOG, new QRecord().withValue("id", state.connectionId)
+         .withValue("username", username).withValue("keyFingerprint", fingerprint).withValue("clientVersion", ssh.getClientVersion())
+         .withValue("accountId", account == null ? null : account.getValue("id")));
+      Instant now = clock.instant();
+      if(account != null && account.getValue("lockedUntil") == null && username.equals(ipLockAccounts.get(state.remoteIp)))
+      {
+         ipLocks.remove(state.remoteIp);
+         ipLockAccounts.remove(state.remoteIp);
+         failures.remove(state.remoteIp);
+         failureAccounts.remove(state.remoteIp);
+      }
+      ipLocks.entrySet().removeIf(entry -> !entry.getValue().isAfter(now));
+      ipLockAccounts.keySet().retainAll(ipLocks.keySet());
+      if(ipLocks.containsKey(state.remoteIp) || (account != null && (future(account, "lockedUntil") || "LOCKED".equals(account.getValueString("status")))))
+      {
+         return (outcome(state, "LOCKED_OUT", "Authentication locked"));
+      }
+      if(account == null)
+      {
+         return (failed(state, null, "AUTH_FAILED", "Unknown account or key"));
+      }
+      if(!"ACTIVE".equals(account.getValueString("status")))
+      {
+         return (failed(state, account, "DISABLED", "Account disabled"));
+      }
+      if(expired(account, "expiresAt"))
+      {
+         return (failed(state, account, "EXPIRED", "Account expired"));
+      }
+      if(!CidrMatcher.matches(state.remoteIp, account.getValueString("allowedCidrs")))
+      {
+         return (failed(state, account, "IP_DENIED", "Address outside allowlist"));
+      }
+      QRecord key = first(FileGatewayTables.SSH_KEY, "fingerprint", fingerprint);
+      if(key == null || !account.getValue("id").equals(key.getValue("accountId")) || key.getValue("revokedAt") != null)
+      {
+         return (failed(state, account, "AUTH_FAILED", "Unknown or revoked key"));
+      }
+      if(expired(key, "expiresAt"))
+      {
+         return (failed(state, account, "EXPIRED", "Key expired"));
+      }
+      int limit = account.getValueInteger("maxConcurrentSessions") == null ? config.getDefaultMaxConcurrentSessions() : account.getValueInteger("maxConcurrentSessions");
+      ////////////////////////////////////////////////////////////////////////
+      // only live rows: connected, not disconnected, seen in the last 3 min //
+      ////////////////////////////////////////////////////////////////////////
+      QQueryFilter liveFilter = filter("accountId", account.getValue("id"))
+         .withCriteria(new QFilterCriteria("outcome", QCriteriaOperator.EQUALS, "CONNECTED"))
+         .withCriteria(new QFilterCriteria("disconnectedAt", QCriteriaOperator.IS_BLANK))
+         .withCriteria(new QFilterCriteria("lastSeenAt", QCriteriaOperator.GREATER_THAN, now.minusSeconds(180)));
+      long active = audit.query(FileGatewayTables.CONNECTION_LOG, liveFilter).size();
+      if(active >= limit)
+      {
+         return (outcome(state, "LIMIT", "Concurrent session limit exceeded"));
+      }
+      state.account = account;
+      state.key = key;
+      try
+      {
+         state.token();
+      }
+      catch(QException e)
+      {
+         state.account = null;
+         state.key = null;
+         return (failed(state, account, "AUTH_FAILED", "no file_gateway.connect"));
+      }
+      outcome(state, "CONNECTED", null);
+      return (true);
+   }
+
+
+
+   /*******************************************************************************
+    ** MINA has verified the signature, not just accepted an unsigned key probe.
+    *******************************************************************************/
+   public synchronized void authenticated(ServerSession ssh) throws QException
+   {
+      GatewaySession state = ssh.getAttribute(GatewaySession.ATTRIBUTE);
+      if(state == null || state.account == null)
+      {
+         throw (new QException("Missing authenticated gateway identity"));
+      }
+      QRecord account = state.account;
+      QRecord key = state.key;
+      Instant now = clock.instant();
+      audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id"))
+         .withValue("failedLoginCount", 0).withValue("failedLoginWindowAt", null)
+         .withValue("lastLoginAt", now).withValue("lastLoginIp", state.remoteIp));
+      audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id"))
+         .withValue("lastUsedAt", now).withValue("lastUsedIp", state.remoteIp));
+   }
+
+
+
+   /*******************************************************************************
+    ** Persist account failure windows and apply the per-IP lockout.
+    *******************************************************************************/
+   private boolean failed(GatewaySession state, QRecord account, String result, String reason) throws QException
+   {
+      Instant now = clock.instant();
+      prune(failures, now.minus(config.getLockoutWindow()));
+      failureAccounts.keySet().retainAll(failures.keySet());
+      Deque<Instant> window = failures.computeIfAbsent(state.remoteIp, ignored -> new ArrayDeque<>());
+      String username = account == null ? "" : account.getValueString("username");
+      if(window.isEmpty())
+      {
+         failureAccounts.put(state.remoteIp, username);
+      }
+      else if(!username.equals(failureAccounts.get(state.remoteIp)))
+      {
+         failureAccounts.put(state.remoteIp, "");
+      }
+      window.addLast(now);
+      boolean lock = window.size() >= config.getLockoutThreshold();
+      if(lock)
+      {
+         ipLocks.put(state.remoteIp, now.plus(config.getLockoutDuration()));
+         ipLockAccounts.put(state.remoteIp, failureAccounts.get(state.remoteIp));
+      }
+      if(account != null)
+      {
+         Instant start = account.getValueInstant("failedLoginWindowAt");
+         Integer oldCount = account.getValueInteger("failedLoginCount");
+         int count = start == null || !start.plus(config.getLockoutWindow()).isAfter(now) ? 1 : (oldCount == null ? 0 : oldCount) + 1;
+         QRecord update = new QRecord().withValue("id", account.getValue("id")).withValue("failedLoginCount", count)
+            .withValue("failedLoginWindowAt", count == 1 ? now : start);
+         if(count >= config.getLockoutThreshold())
+         {
+            update.setValue("lockedUntil", now.plus(config.getLockoutDuration()));
+            lock = true;
+         }
+         audit.update(FileGatewayTables.ACCOUNT, update);
+      }
+      if(lock)
+      {
+         audit.system(() ->
+         {
+            QAlerts.raise(new AlertSignal("filegateway.lockout").withDedupKey(account == null ? state.remoteIp : account.getValueString("id"))
+               .withSummary("File gateway authentication locked"));
+            return (null);
+         });
+      }
+      return (outcome(state, result, reason));
+   }
+
+
+
+   /*******************************************************************************
+    ** Record the authentication outcome without credentials.
+    *******************************************************************************/
+   private boolean outcome(GatewaySession state, String outcome, String reason) throws QException
+   {
+      audit.update(FileGatewayTables.CONNECTION_LOG, new QRecord().withValue("id", state.connectionId)
+         .withValue("outcome", outcome).withValue("disconnectReason", reason).withValue("lastSeenAt", clock.instant()));
+      if(reason != null)
+      {
+         state.disconnectReason = reason;
+      }
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Revoke credentials and finalize connection accounting.
+    *******************************************************************************/
+   public synchronized void closed(ServerSession ssh) throws QException
+   {
+      GatewaySession state = ssh.removeAttribute(GatewaySession.ATTRIBUTE);
+      if(state == null)
+      {
+         return;
+      }
+      state.revoke();
+      sessions.remove(state.id);
+      if(state.account != null && !ssh.isAuthenticated())
+      {
+         outcome(state, "AUTH_FAILED", "Public key signature was not verified");
+      }
+      if(state.connectionId != null)
+      {
+         audit.update(FileGatewayTables.CONNECTION_LOG, new QRecord().withValue("id", state.connectionId)
+            .withValue("disconnectedAt", clock.instant()).withValue("disconnectReason", state.disconnectReason)
+            .withValue("bytesIn", 0L).withValue("bytesOut", 0L)
+            .withValue("operationCount", state.operations));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Finish any callbacks still queued when MINA has stopped accepting work.
+    *******************************************************************************/
+   public synchronized void shutdown() throws QException
+   {
+      for(GatewaySession state : List.copyOf(sessions.values()))
+      {
+         closed(state.ssh);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Re-read live accounts and keys and heartbeat open connection rows.
+    *******************************************************************************/
+   public synchronized void poll() throws QException
+   {
+      for(GatewaySession state : List.copyOf(sessions.values()))
+      {
+         if(state.account == null)
+         {
+            continue;
+         }
+         QRecord account = first(FileGatewayTables.ACCOUNT, "id", state.account.getValue("id"));
+         QRecord key = first(FileGatewayTables.SSH_KEY, "id", state.key.getValue("id"));
+         QRecord connection = first(FileGatewayTables.CONNECTION_LOG, "id", state.connectionId);
+         if(account == null || key == null || !"ACTIVE".equals(account.getValueString("status"))
+            || expired(account, "expiresAt") || expired(key, "expiresAt") || key.getValue("revokedAt") != null
+            || connection == null || connection.getValue("disconnectedAt") != null)
+         {
+            state.disconnectReason = connection != null && connection.getValueString("disconnectReason") != null
+               ? connection.getValueString("disconnectReason") : "Account or key revoked";
+            state.revoke();
+            state.ssh.close(true);
+         }
+         else
+         {
+            state.account = account;
+            audit.update(FileGatewayTables.CONNECTION_LOG, new QRecord().withValue("id", state.connectionId)
+               .withValue("lastSeenAt", clock.instant()).withValue("operationCount", state.operations));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Find gateway-owned state.
+    *******************************************************************************/
+   private QRecord first(String table, String field, java.io.Serializable value) throws QException
+   {
+      return (audit.query(table, filter(field, value)).stream().findFirst().orElse(null));
+   }
+
+
+
+   /*******************************************************************************
+    ** Equality query for owned tables.
+    *******************************************************************************/
+   private QQueryFilter filter(String field, java.io.Serializable value)
+   {
+      return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Whether a timestamp lies in the future.
+    *******************************************************************************/
+   private boolean future(QRecord row, String field)
+   {
+      Instant value = row.getValueInstant(field);
+      return (value != null && value.isAfter(clock.instant()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Whether an optional expiry has passed.
+    *******************************************************************************/
+   private boolean expired(QRecord row, String field)
+   {
+      Instant value = row.getValueInstant(field);
+      return (value != null && !value.isAfter(clock.instant()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Bound in-memory IP windows to their active interval.
+    *******************************************************************************/
+   private void prune(Map<String, Deque<Instant>> windows, Instant cutoff)
+   {
+      windows.values().forEach(window -> window.removeIf(instant -> !instant.isAfter(cutoff)));
+      windows.entrySet().removeIf(entry -> entry.getValue().isEmpty());
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/storage/GatewayObjectStore.java

@@ -0,0 +1,62 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.storage;
+
+import java.io.IOException;
+import java.io.InputStream;
+
+/*******************************************************************************
+ ** Streaming storage boundary. Production uses the object-storage system API.
+ *******************************************************************************/
+public interface GatewayObjectStore
+{
+   /*******************************************************************************
+    ** Completed immutable object identity, measured while streaming.
+    *******************************************************************************/
+   record Receipt(long size, String sha256)
+   {
+   }
+
+   /*******************************************************************************
+    ** Consume an unknown-length stream; a read failure must abort multipart upload.
+    *******************************************************************************/
+   Receipt put(String key, InputStream content) throws IOException;
+
+   /*******************************************************************************
+    ** Open a fresh stream owned by the caller.
+    *******************************************************************************/
+   InputStream open(String key) throws IOException;
+
+   /*******************************************************************************
+    ** Open an inclusive byte range, owned by the caller.
+    *******************************************************************************/
+   InputStream openRange(String key, long start, long end) throws IOException;
+
+   /*******************************************************************************
+    ** Copy within the configured bucket.
+    *******************************************************************************/
+   void copy(String source, String destination) throws IOException;
+
+   /*******************************************************************************
+    ** Permanently remove an aborted or quarantined source object.
+    *******************************************************************************/
+   void delete(String key) throws IOException;
+}

src/main/java/com/kingsrook/qbits/filegateway/storage/ObjectReadChannel.java

@@ -0,0 +1,207 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.storage;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.channels.ClosedChannelException;
+import java.nio.channels.NonWritableChannelException;
+import java.nio.channels.SeekableByteChannel;
+
+/*******************************************************************************
+ ** Seekable range reads with bounded per-request memory and prompt response cleanup.
+ *******************************************************************************/
+public final class ObjectReadChannel implements SeekableByteChannel
+{
+   /*******************************************************************************
+    ** Open an inclusive range of an immutable object or generated sidecar.
+    *******************************************************************************/
+   @FunctionalInterface
+   public interface Ranges
+   {
+      /*******************************************************************************
+       ** Caller closes the response stream.
+       *******************************************************************************/
+      InputStream open(long start, long end) throws IOException;
+   }
+
+   /*******************************************************************************
+    ** Persist one transfer outcome when the handle closes.
+    *******************************************************************************/
+   @FunctionalInterface
+   public interface Completion
+   {
+      /*******************************************************************************
+       ** Report actual bytes sent and any read failure.
+       *******************************************************************************/
+      void done(long bytes, boolean failed) throws IOException;
+   }
+
+   private final long size;
+   private final Ranges ranges;
+   private final Completion completion;
+   private long bytesRead;
+   private boolean failed;
+   private long position;
+   private boolean open = true;
+
+   /*******************************************************************************
+    ** Bind a stored size and ranged stream supplier.
+    *******************************************************************************/
+   public ObjectReadChannel(long size, Ranges ranges)
+   {
+      this(size, ranges, (bytes, failure) ->
+      {
+         // Direct channel users may omit a transfer audit callback.
+      });
+   }
+
+   /*******************************************************************************
+    ** Bind an audited object read.
+    *******************************************************************************/
+   public ObjectReadChannel(long size, Ranges ranges, Completion completion)
+   {
+      this.size = size;
+      this.ranges = ranges;
+      this.completion = completion;
+   }
+
+   /*******************************************************************************
+    ** At most 64 KiB is buffered, regardless of object size or seek distance.
+    *******************************************************************************/
+   @Override
+   public int read(ByteBuffer destination) throws IOException
+   {
+      checkOpen();
+      if(!destination.hasRemaining())
+      {
+         return (0);
+      }
+      if(position >= size)
+      {
+         return (-1);
+      }
+      int length = (int) Math.min(Math.min(destination.remaining(), 65536), size - position);
+      try(InputStream input = ranges.open(position, position + length - 1))
+      {
+         byte[] bytes = input.readNBytes(length);
+         if(bytes.length == 0)
+         {
+            throw (new IOException("Stored object ended before its recorded size"));
+         }
+         destination.put(bytes);
+         position += bytes.length;
+         bytesRead += bytes.length;
+         return (bytes.length);
+      }
+      catch(IOException e)
+      {
+         failed = true;
+         throw (e);
+      }
+   }
+
+   /*******************************************************************************
+    ** Stored objects are immutable.
+    *******************************************************************************/
+   @Override
+   public int write(ByteBuffer source)
+   {
+      throw (new NonWritableChannelException());
+   }
+
+   /*******************************************************************************
+    ** Current read offset.
+    *******************************************************************************/
+   @Override
+   public long position() throws IOException
+   {
+      checkOpen();
+      return (position);
+   }
+
+   /*******************************************************************************
+    ** Seek without reading preceding bytes.
+    *******************************************************************************/
+   @Override
+   public SeekableByteChannel position(long value) throws IOException
+   {
+      checkOpen();
+      if(value < 0)
+      {
+         throw (new IllegalArgumentException("Negative read offset"));
+      }
+      position = value;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Persisted object length.
+    *******************************************************************************/
+   @Override
+   public long size()
+   {
+      return (size);
+   }
+
+   /*******************************************************************************
+    ** Stored objects cannot be truncated.
+    *******************************************************************************/
+   @Override
+   public SeekableByteChannel truncate(long value)
+   {
+      throw (new NonWritableChannelException());
+   }
+
+   /*******************************************************************************
+    ** Handle state.
+    *******************************************************************************/
+   @Override
+   public boolean isOpen()
+   {
+      return (open);
+   }
+
+   /*******************************************************************************
+    ** No stream is retained between reads.
+    *******************************************************************************/
+   @Override
+   public void close() throws IOException
+   {
+      if(open)
+      {
+         open = false;
+         completion.done(bytesRead, failed);
+      }
+   }
+
+   /*******************************************************************************
+    ** Reject use after close.
+    *******************************************************************************/
+   private void checkOpen() throws ClosedChannelException
+   {
+      if(!open)
+      {
+         throw (new ClosedChannelException());
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/storage/ObjectStorageGatewayStore.java

@@ -0,0 +1,149 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.storage;
+
+import java.io.FilterInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.DigestInputStream;
+import java.security.MessageDigest;
+import java.util.HexFormat;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qbits.objectstorage.api.CopyRequest;
+import com.kingsrook.qbits.objectstorage.api.DeleteRequest;
+import com.kingsrook.qbits.objectstorage.api.ObjectStorage;
+import com.kingsrook.qbits.objectstorage.api.PutRequest;
+import com.kingsrook.qbits.objectstorage.api.Range;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+
+/*******************************************************************************
+ ** Only the allow-listed object-storage API is used, under a named system caller.
+ *******************************************************************************/
+public final class ObjectStorageGatewayStore implements GatewayObjectStore
+{
+   private final AuditWriter context;
+   private final String alias;
+
+   /*******************************************************************************
+    ** Bind the host instance and bucket alias without touching storage at startup.
+    *******************************************************************************/
+   public ObjectStorageGatewayStore(QInstance instance, String alias)
+   {
+      context = new AuditWriter(instance);
+      this.alias = alias;
+   }
+
+   /*******************************************************************************
+    ** Unknown-length puts use object-storage's bounded multipart implementation.
+    *******************************************************************************/
+   @Override
+   public Receipt put(String key, InputStream content) throws IOException
+   {
+      try
+      {
+         MessageDigest digest = MessageDigest.getInstance("SHA-256");
+         var result = context.system(() -> ObjectStorage.asSystem("file-gateway", key).bucket(alias)
+            .put(PutRequest.of(key, new DigestInputStream(content, digest))));
+         return (new Receipt(result.getBytes(), HexFormat.of().formatHex(digest.digest())));
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Object upload failed", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Stream the complete object.
+    *******************************************************************************/
+   @Override
+   public InputStream open(String key) throws IOException
+   {
+      return (read(key, Range.all()));
+   }
+
+   /*******************************************************************************
+    ** Stream only the requested range.
+    *******************************************************************************/
+   @Override
+   public InputStream openRange(String key, long start, long end) throws IOException
+   {
+      return (read(key, Range.bytes(start, end)));
+   }
+
+   /*******************************************************************************
+    ** Closing the returned stream also releases its object response.
+    *******************************************************************************/
+   private InputStream read(String key, Range range) throws IOException
+   {
+      try
+      {
+         var object = context.system(() -> ObjectStorage.asSystem("file-gateway", key).bucket(alias).get(key, range));
+         return (new FilterInputStream(object.getInputStream())
+         {
+            @Override
+            public void close() throws IOException
+            {
+               object.close();
+            }
+         });
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Object read failed", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Server-side quarantine copy.
+    *******************************************************************************/
+   @Override
+   public void copy(String source, String destination) throws IOException
+   {
+      try
+      {
+         context.system(() ->
+         {
+            ObjectStorage.asSystem("file-gateway", source).bucket(alias).copy(CopyRequest.of(source, destination));
+            return (null);
+         });
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Object copy failed", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Remove all versions of a gateway-owned object.
+    *******************************************************************************/
+   @Override
+   public void delete(String key) throws IOException
+   {
+      try
+      {
+         context.system(() -> ObjectStorage.asSystem("file-gateway", key).bucket(alias).delete(DeleteRequest.of(key).permanent()));
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Object deletion failed", e));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/BulkLoadConsumer.java

@@ -0,0 +1,209 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.IOException;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.kingsrook.qbits.filegateway.api.FileAcceptance;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumer;
+import com.kingsrook.qbits.filegateway.api.InboundFile;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+
+/*******************************************************************************
+ ** Drives the real bulk-insert UI process as the user; never inserts application rows directly.
+ *******************************************************************************/
+public final class BulkLoadConsumer implements FileGatewayConsumer
+{
+   private final WorkerUserSession session;
+   private final QRecord job;
+   private QRecord outcome;
+
+   /*******************************************************************************
+    ** One consumer instance per delivery, holding only that process's outcome.
+    *******************************************************************************/
+   public BulkLoadConsumer(WorkerUserSession session, QRecord job)
+   {
+      this.session = session;
+      this.job = job;
+   }
+
+   /*******************************************************************************
+    ** Stable built-in registry name.
+    *******************************************************************************/
+   @Override
+   public String getName()
+   {
+      return ("bulk-load");
+   }
+
+   /*******************************************************************************
+    ** Follow returned frontend steps; the API resumes backend steps after each one.
+    *******************************************************************************/
+   @Override
+   public FileAcceptance accept(InboundFile file) throws Exception
+   {
+      String process = job.getValueString("tableName") + ".bulkInsert";
+      JsonNode profiles = session.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/table/{tableName}/query")
+         .withPathParam("tableName", "savedBulkLoadProfile").withJsonBody(Map.of("filter", UploadJobs.equal("id", job.getValue("savedBulkLoadProfileId"))))).path("records");
+      if(profiles.size() != 1 || !job.getValueString("tableName").equals(profiles.get(0).path("values").path("tableName").asText()))
+      {
+         return (failed("Saved bulk load profile is unavailable"));
+      }
+      JsonNode mapping = new ObjectMapper().readTree(profiles.get(0).path("values").path("mappingJson").asText());
+      Map<String, Object> values = new LinkedHashMap<>();
+      values.put("savedBulkLoadProfileId", job.getValue("savedBulkLoadProfileId"));
+      values.put("version", mapping.path("version").asText());
+      values.put("layout", mapping.path("layout").asText());
+      values.put("hasHeaderRow", mapping.path("hasHeaderRow").asBoolean());
+      values.put("fieldListJSON", mapping.path("fieldList").toString());
+      for(JsonNode field : mapping.path("fieldList"))
+      {
+         if(field.path("doValueMapping").asBoolean() && field.path("valueMappings").isEmpty())
+         {
+            return (failed("this profile needs value mappings saved in it"));
+         }
+      }
+      JsonNode response = session.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/processes/{processName}/init")
+         .withPathParam("processName", process).withFormParam("values", new ObjectMapper().writeValueAsString(Map.of("tableName", job.getValueString("tableName"))))
+         .withMultipartFile("theFile", file.fileName(), "application/octet-stream", file.streamSupplier()));
+      String uuid = response.path("processUUID").asText();
+      job.withValue("processUuid", uuid);
+      for(int steps = 0; steps < 20; steps++)
+      {
+         response = await(process, uuid, response);
+         if("ERROR".equals(response.path("type").asText()))
+         {
+            return (failed(response.path("userFacingError").asText("Bulk load process failed")));
+         }
+         String next = response.path("nextStep").asText();
+         if(next.equals("valueMapping"))
+         {
+            return (failed("this profile needs value mappings saved in it"));
+         }
+         if(response.path("values").has("processResults"))
+         {
+            summarize(response.path("values").path("processResults"));
+            return (FileAcceptance.accepted(uuid));
+         }
+         if(!java.util.List.of("upload", "fileMapping", "review").contains(next))
+         {
+            return (failed("Unexpected bulk load step: " + next));
+         }
+         Map<String, Object> stepValues = next.equals("fileMapping") ? values : Map.of();
+         UiApiRequest step = new UiApiRequest().withMethod(HttpMethod.POST).withPath("/processes/{processName}/{processUUID}/step/{stepName}")
+            .withPathParam("processName", process).withPathParam("processUUID", uuid).withPathParam("stepName", next);
+         if(!stepValues.isEmpty())
+         {
+            // the invoker sends form fields as multipart for this multipart-only operation (qqq#1170)
+            step.withFormParam("values", new ObjectMapper().writeValueAsString(stepValues));
+         }
+         response = session.invoke(step);
+      }
+      return (failed("Bulk load exceeded its step limit"));
+   }
+
+   /*******************************************************************************
+    ** Asynchronous UI API steps retain their job UUID across RUNNING responses.
+    *******************************************************************************/
+   private JsonNode await(String process, String uuid, JsonNode response) throws Exception
+   {
+      String async = response.path("jobUUID").asText();
+      for(int poll = 0; poll < 3600 && ("JOB_STARTED".equals(response.path("type").asText()) || "RUNNING".equals(response.path("type").asText())); poll++)
+      {
+         Thread.sleep(100);
+         response = session.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath("/processes/{processName}/{processUUID}/status/{jobUUID}")
+            .withPathParam("processName", process).withPathParam("processUUID", uuid).withPathParam("jobUUID", async));
+      }
+      if("RUNNING".equals(response.path("type").asText()))
+      {
+         throw (new IOException("Bulk load status timed out"));
+      }
+      return (response);
+   }
+
+   /*******************************************************************************
+    ** Preserve the API's detailed summary (including row examples) for the result artifact.
+    *******************************************************************************/
+   private void summarize(JsonNode lines)
+   {
+      int inserted = 0;
+      int errors = 0;
+      StringBuilder summary = new StringBuilder();
+      StringBuilder csv = new StringBuilder("row,field,message\r\n");
+      for(JsonNode line : lines)
+      {
+         String status = line.path("status").asText();
+         int count = line.path("count").asInt();
+         String message = line.path("message").asText();
+         if("OK".equals(status))
+         {
+            inserted += count;
+         }
+         if("ERROR".equals(status))
+         {
+            errors += count;
+         }
+         summary.append(count).append(" ").append(message).append("\n");
+         csv.append(",,").append(csvCell(count + " " + message)).append("\r\n");
+         for(JsonNode bullet : line.path("bulletsOfText"))
+         {
+            String detail = bullet.asText();
+            summary.append(detail).append("\n");
+            var row = java.util.regex.Pattern.compile("(?i)(?:row[^0-9]*)?([0-9]+)").matcher(detail);
+            csv.append(row.find() ? row.group(1) : "").append(",,").append(csvCell(message + " " + detail)).append("\r\n");
+         }
+      }
+      outcome = new QRecord().withValue("status", errors == 0 ? "SUCCEEDED" : inserted == 0 ? "FAILED" : "PARTIAL")
+         .withValue("insertedCount", inserted).withValue("errorCount", errors).withValue("resultSummary", summary.toString()).withValue("_resultsCsv", csv.toString());
+   }
+
+   /*******************************************************************************
+    ** Quote CSV cells, including embedded commas, newlines and quotes.
+    *******************************************************************************/
+   private String csvCell(String value)
+   {
+      return ("\"" + value.replace("\"", "\"\"") + "\"");
+   }
+
+
+
+   /*******************************************************************************
+    ** Process errors are terminal rather than silently replaying an executed import.
+    *******************************************************************************/
+   private FileAcceptance failed(String message)
+   {
+      outcome = new QRecord().withValue("status", "FAILED").withValue("errorMessage", message).withValue("resultSummary", message);
+      return (FileAcceptance.accepted(job.getValueString("processUuid")));
+   }
+
+   /*******************************************************************************
+    ** Worker persists counts and status along with the consumer reference.
+    *******************************************************************************/
+   public QRecord outcome()
+   {
+      return (outcome);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/UploadChannel.java

@@ -0,0 +1,304 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.FilterInputStream;
+import java.io.IOException;
+import java.io.PipedInputStream;
+import java.io.PipedOutputStream;
+import java.nio.ByteBuffer;
+import java.nio.channels.ClosedChannelException;
+import java.nio.channels.NonReadableChannelException;
+import java.nio.channels.SeekableByteChannel;
+import java.util.TreeMap;
+import java.util.concurrent.CompletableFuture;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+
+/*******************************************************************************
+ ** Sequential storage stream with a bounded SFTP reorder window and backpressure.
+ *******************************************************************************/
+public final class UploadChannel implements SeekableByteChannel
+{
+   /*******************************************************************************
+    ** Persist reservations and terminal state before acknowledging SFTP operations.
+    *******************************************************************************/
+   public interface Lifecycle
+   {
+      /*******************************************************************************
+       ** Atomically reserve the current high-water size against account limits.
+       *******************************************************************************/
+      void reserve(long size) throws IOException;
+
+      /*******************************************************************************
+       ** Commit a completed object.
+       *******************************************************************************/
+      void complete(GatewayObjectStore.Receipt receipt) throws IOException;
+
+      /*******************************************************************************
+       ** Persist an aborted transfer and its reason.
+       *******************************************************************************/
+      void abort(String message, boolean limit) throws IOException;
+   }
+
+   private final GatewayObjectStore store;
+   private final String key;
+   private final Lifecycle lifecycle;
+   private final long window;
+   private final PipedInputStream input;
+   private final PipedOutputStream output;
+   private final CompletableFuture<GatewayObjectStore.Receipt> result = new CompletableFuture<>();
+   private final TreeMap<Long, byte[]> pending = new TreeMap<>();
+   private volatile boolean aborted;
+   private boolean open = true;
+   private long position;
+   private long contiguous;
+   private long size;
+
+   /*******************************************************************************
+    ** A virtual thread consumes the pipe; only bounded buffers hold upload content.
+    *******************************************************************************/
+   public UploadChannel(GatewayObjectStore store, String key, int bufferSize, long window, Lifecycle lifecycle) throws IOException
+   {
+      this.store = store;
+      this.key = key;
+      this.lifecycle = lifecycle;
+      this.window = window;
+      input = new PipedInputStream(bufferSize);
+      output = new PipedOutputStream(input);
+      Thread.startVirtualThread(() ->
+      {
+         try(var source = new FilterInputStream(input)
+         {
+            @Override
+            public int read(byte[] bytes, int offset, int length) throws IOException
+            {
+               if(aborted)
+               {
+                  throw (new IOException("Upload aborted"));
+               }
+               int count = super.read(bytes, offset, length);
+               if(aborted)
+               {
+                  throw (new IOException("Upload aborted"));
+               }
+               return (count);
+            }
+         })
+         {
+            var receipt = store.put(key, source);
+            result.complete(receipt);
+         }
+         catch(Exception e)
+         {
+            result.completeExceptionally(e);
+         }
+         finally
+         {
+            if(aborted)
+            {
+               try
+               {
+                  store.delete(key);
+               }
+               catch(IOException ignored)
+               {
+                  // The caller also deletes after waiting for completion.
+               }
+            }
+         }
+      });
+   }
+
+   /*******************************************************************************
+    ** Fail closed and release the streaming producer before deleting the object.
+    *******************************************************************************/
+   private void fail(String message, boolean limit) throws IOException
+   {
+      aborted = true;
+      open = false;
+      pending.clear();
+      output.close();
+      input.close();
+      try
+      {
+         result.join();
+      }
+      catch(RuntimeException ignored)
+      {
+         // A failing input is how the storage client aborts multipart uploads.
+      }
+      try
+      {
+         store.delete(key);
+      }
+      finally
+      {
+         lifecycle.abort(message, limit);
+      }
+   }
+
+   /*******************************************************************************
+    ** Writes behind the committed prefix and overlapping blocks are unsupported.
+    *******************************************************************************/
+   @Override
+   public synchronized int write(ByteBuffer source) throws IOException
+   {
+      checkOpen();
+      int count = source.remaining();
+      if(count == 0)
+      {
+         return (0);
+      }
+      try
+      {
+         long end = Math.addExact(position, count);
+         var before = pending.floorEntry(position);
+         var after = pending.ceilingEntry(position);
+         if(position < contiguous || (before != null && before.getKey() + before.getValue().length > position)
+            || (after != null && after.getKey() < end) || end - contiguous > window)
+         {
+            throw (new IOException("Upload exceeds reorder window or overlaps an earlier write"));
+         }
+         lifecycle.reserve(Math.max(size, end));
+         byte[] bytes = new byte[count];
+         source.get(bytes);
+         pending.put(position, bytes);
+         size = Math.max(size, end);
+         position = end;
+         while(pending.containsKey(contiguous))
+         {
+            byte[] next = pending.remove(contiguous);
+            output.write(next);
+            contiguous += next.length;
+         }
+         return (count);
+      }
+      catch(Exception e)
+      {
+         boolean limit = e instanceof UploadLimitException;
+         fail(limit ? e.getMessage() : "Upload stream failed", limit);
+         throw (new IOException(limit ? e.getMessage() : "Upload stream failed", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Complete only a contiguous upload; close of a temporary file does not queue it.
+    *******************************************************************************/
+   @Override
+   public synchronized void close() throws IOException
+   {
+      if(!open)
+      {
+         return;
+      }
+      try
+      {
+         if(!pending.isEmpty())
+         {
+            throw (new IOException("Upload has missing blocks"));
+         }
+         output.close();
+         lifecycle.complete(result.join());
+         open = false;
+      }
+      catch(Exception e)
+      {
+         fail("Upload did not complete", false);
+         throw (new IOException("Upload did not complete", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify this handle is usable.
+    *******************************************************************************/
+   private void checkOpen() throws ClosedChannelException
+   {
+      if(!open)
+      {
+         throw (new ClosedChannelException());
+      }
+   }
+
+   /*******************************************************************************
+    ** Upload handles cannot read.
+    *******************************************************************************/
+   @Override
+   public int read(ByteBuffer destination)
+   {
+      throw (new NonReadableChannelException());
+   }
+
+   /*******************************************************************************
+    ** Current SFTP write offset.
+    *******************************************************************************/
+   @Override
+   public synchronized long position() throws IOException
+   {
+      checkOpen();
+      return (position);
+   }
+
+   /*******************************************************************************
+    ** SFTP clients set the offset before each write.
+    *******************************************************************************/
+   @Override
+   public synchronized SeekableByteChannel position(long value) throws IOException
+   {
+      checkOpen();
+      if(value < 0)
+      {
+         throw (new IllegalArgumentException("Negative upload offset"));
+      }
+      position = value;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Highest received offset.
+    *******************************************************************************/
+   @Override
+   public synchronized long size()
+   {
+      return (size);
+   }
+
+   /*******************************************************************************
+    ** Arbitrary truncation would invalidate a streamed object.
+    *******************************************************************************/
+   @Override
+   public SeekableByteChannel truncate(long value) throws IOException
+   {
+      if(value != size)
+      {
+         throw (new IOException("Upload truncation is unsupported"));
+      }
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Handle lifecycle.
+    *******************************************************************************/
+   @Override
+   public boolean isOpen()
+   {
+      return (open);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/UploadJobWorker.java

@@ -0,0 +1,410 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import java.time.Clock;
+import java.time.Duration;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
+import com.kingsrook.qbits.filegateway.api.FileAcceptance;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumer;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumersMetaData;
+import com.kingsrook.qbits.filegateway.api.FileGatewayScannerInterface;
+import com.kingsrook.qbits.filegateway.api.InboundFile;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+
+/*******************************************************************************
+ ** Bounded, at-least-once upload delivery with fenced claims and deterministic polling.
+ *******************************************************************************/
+public class UploadJobWorker implements AutoCloseable
+{
+   private static final QLogger LOG = QLogger.getLogger(UploadJobWorker.class);
+   private static final long[] RETRY_SECONDS = {60, 300, 900, 3600, 21600};
+   private final QInstance instance;
+   private final UiApiInvoker invoker;
+   private final FileGatewayQBitConfig config;
+   private final GatewayObjectStore store;
+   private final AuditWriter audit;
+   private final Clock clock;
+   private final String pod;
+   private volatile boolean stopped;
+   private ScheduledExecutorService pool;
+
+   /*******************************************************************************
+    ** No threads are started until the enabled runtime explicitly starts this worker.
+    *******************************************************************************/
+   public UploadJobWorker(QInstance instance, UiApiInvoker invoker, FileGatewayQBitConfig config, GatewayObjectStore store, Clock clock, String pod)
+   {
+      this.instance = instance;
+      this.invoker = invoker;
+      this.config = config;
+      this.store = store;
+      this.clock = clock;
+      this.pod = pod == null ? java.util.UUID.randomUUID().toString() : pod;
+      audit = new AuditWriter(instance);
+   }
+
+   /*******************************************************************************
+    ** Each scheduled lane handles one job at a time, without an unbounded work queue.
+    *******************************************************************************/
+   public void start(Duration interval)
+   {
+      if(!config.getSftpEnabled())
+      {
+         return;
+      }
+      pool = Executors.newScheduledThreadPool(config.getUploadWorkerThreads(), runnable ->
+      {
+         Thread thread = new Thread(runnable, "file-gateway-upload");
+         thread.setDaemon(true);
+         return (thread);
+      });
+      for(int lane = 0; lane < config.getUploadWorkerThreads(); lane++)
+      {
+         pool.scheduleWithFixedDelay(() ->
+         {
+            try
+            {
+               runOnce();
+            }
+            catch(Exception e)
+            {
+               LOG.warn("Upload worker could not persist job state");
+            }
+         }, interval.toMillis(), interval.toMillis(), TimeUnit.MILLISECONDS);
+      }
+   }
+
+   /*******************************************************************************
+    ** Reconcile stale leases and synchronously deliver at most one eligible job.
+    *******************************************************************************/
+   public boolean runOnce() throws QException
+   {
+      if(stopped)
+      {
+         return (false);
+      }
+      requeueStale();
+      for(QRecord candidate : audit.query(FileGatewayTables.UPLOAD_JOB, UploadJobs.equal("status", "QUEUED")))
+      {
+         if(candidate.getValueInstant("nextAttemptAt") != null && candidate.getValueInstant("nextAttemptAt").isAfter(clock.instant()))
+         {
+            continue;
+         }
+         QRecord claimed = claim(candidate);
+         if(claimed != null)
+         {
+            deliver(claimed);
+            return (true);
+         }
+      }
+      return (false);
+   }
+
+   /*******************************************************************************
+    ** The expected status is checked inside the account transaction; competing pods cannot both win.
+    *******************************************************************************/
+   public QRecord claim(QRecord candidate) throws QException
+   {
+      return (audit.withAccountLock("id", candidate.getValue("accountId"), () ->
+      {
+         QRecord current = find(candidate);
+         if(stopped || !"QUEUED".equals(current.getValueString("status"))
+            || (current.getValueInstant("nextAttemptAt") != null && current.getValueInstant("nextAttemptAt").isAfter(clock.instant())))
+         {
+            return (null);
+         }
+         QRecord changes = new QRecord().withValue("id", current.getValue("id")).withValue("status", "RUNNING")
+            .withValue("claimedBy", pod).withValue("claimedAt", clock.instant()).withValue("startedAt", clock.instant())
+            .withValue("attempts", attempts(current) + 1);
+         return (audit.transitionUpload(changes, "QUEUED") ? find(current) : null);
+      }));
+   }
+
+   /*******************************************************************************
+    ** Requeue abandoned leases up to three starts, fencing late completions by claim time.
+    *******************************************************************************/
+   public void requeueStale() throws QException
+   {
+      for(QRecord candidate : audit.query(FileGatewayTables.UPLOAD_JOB, UploadJobs.equal("status", "RUNNING")))
+      {
+         audit.withAccountLock("id", candidate.getValue("accountId"), () ->
+         {
+            QRecord job = find(candidate);
+            if("RUNNING".equals(job.getValueString("status")) && job.getValueInstant("claimedAt") != null
+               && !job.getValueInstant("claimedAt").plus(config.getJobStaleAfter()).isAfter(clock.instant()))
+            {
+               QRecord changes = new QRecord().withValue("id", job.getValue("id"))
+                  .withValue("status", attempts(job) >= 3 ? "FAILED" : "QUEUED").withValue("claimedBy", null).withValue("claimedAt", null)
+                  .withValue("errorMessage", "Upload worker lease expired").withValue("nextAttemptAt", clock.instant());
+               if(attempts(job) >= 3)
+               {
+                  changes.withValue("finishedAt", clock.instant());
+               }
+               audit.update(FileGatewayTables.UPLOAD_JOB, changes);
+               QRecord account = audit.query(FileGatewayTables.ACCOUNT, UploadJobs.equal("id", job.getValue("accountId"))).getFirst();
+               UploadNotifications.send(audit, job, changes, account);
+            }
+            return (null);
+         });
+      }
+   }
+
+   /*******************************************************************************
+    ** Fresh user context is scoped to each delivery and always restored and revoked.
+    *******************************************************************************/
+   private void deliver(QRecord job) throws QException
+   {
+      var previous = QContext.capture();
+      try
+      {
+         QRecord account = audit.query(FileGatewayTables.ACCOUNT, UploadJobs.equal("id", job.getValue("accountId"))).getFirst();
+         if(!"ACTIVE".equals(account.getValueString("status"))
+            || (account.getValueInstant("expiresAt") != null && !account.getValueInstant("expiresAt").isAfter(clock.instant())))
+         {
+            finish(job, new QRecord().withValue("status", "REJECTED").withValue("errorMessage", "Gateway account is unavailable"), account);
+            return;
+         }
+         try(var user = new WorkerUserSession(instance, invoker, account.getValueString("userId"), "upload-" + job.getValue("id"), clock))
+         {
+            InboundFile file = inbound(job, account);
+            if(config.getScanner() != null)
+            {
+               var scan = QCodeLoader.getAdHoc(FileGatewayScannerInterface.class, config.getScanner()).scan(file);
+               if(scan == FileGatewayScannerInterface.Result.INFECTED)
+               {
+                  String quarantine = "file-gateway/quarantine/" + job.getValue("id") + "/" + job.getValueString("fileName");
+                  store.copy(file.storageKey(), quarantine);
+                  store.delete(file.storageKey());
+                  finish(job, new QRecord().withValue("status", "REJECTED").withValue("storageKey", quarantine)
+                     .withValue("errorMessage", "Scanner rejected infected file"), account);
+                  return;
+               }
+               if(scan != FileGatewayScannerInterface.Result.CLEAN)
+               {
+                  retry(job, "Scanner unavailable", null);
+                  return;
+               }
+            }
+            FileGatewayConsumer consumer = consumer(job, user);
+            if(consumer == null)
+            {
+               finish(job, new QRecord().withValue("status", "REJECTED").withValue("errorMessage", "Unknown consumer: " + job.getValueString("consumerName")), account);
+               return;
+            }
+            FileAcceptance acceptance = consumer.accept(file);
+            if(acceptance.getStatus() == FileAcceptance.Status.RETRY_LATER)
+            {
+               retry(job, acceptance.getReason(), acceptance.getRetryAfter());
+               return;
+            }
+            QRecord outcome = consumer instanceof BulkLoadConsumer bulk ? bulk.outcome() : new QRecord()
+               .withValue("status", acceptance.getStatus() == FileAcceptance.Status.ACCEPTED ? "SUCCEEDED" : "REJECTED")
+               .withValue("errorMessage", acceptance.getReason());
+            outcome.withValue("consumerReference", acceptance.getConsumerReference()).withValue("processUuid", job.getValue("processUuid"));
+            finish(job, outcome, account);
+         }
+      }
+      catch(Exception e)
+      {
+         retry(job, "Consumer delivery failed", null);
+      }
+      finally
+      {
+         QContext.init(previous);
+      }
+   }
+
+   /*******************************************************************************
+    ** Registry lookup is by declared name, with the built-in bulk-load consumer reserved.
+    *******************************************************************************/
+   private FileGatewayConsumer consumer(QRecord job, WorkerUserSession user)
+   {
+      String name = job.getValueString("consumerName");
+      if("bulk-load".equals(name))
+      {
+         return (new BulkLoadConsumer(user, job));
+      }
+      for(var reference : FileGatewayConsumersMetaData.ofOrWithNew(instance).getConsumers())
+      {
+         FileGatewayConsumer consumer = QCodeLoader.getAdHoc(FileGatewayConsumer.class, reference);
+         if(name.equals(consumer.getName()))
+         {
+            return (consumer);
+         }
+      }
+      return (null);
+   }
+
+   /*******************************************************************************
+    ** Immutable delivery envelope; each openStream gets an independent object stream.
+    *******************************************************************************/
+   private InboundFile inbound(QRecord job, QRecord account)
+   {
+      return (new InboundFile(job.getValueInteger("id"), null, account.getValueString("username"), account.getValueString("userId"),
+         job.getValueString("remoteAddress"), job.getValueString("path"), job.getValueString("fileName"), job.getValueLong("sizeBytes"),
+         job.getValueString("sha256"), job.getValueInstant("createDate"), config.getFileGatewayBucketAlias(), job.getValueString("storageKey"), () ->
+         {
+            try
+            {
+               return (store.open(job.getValueString("storageKey")));
+            }
+            catch(IOException e)
+            {
+               throw (new UncheckedIOException(e));
+            }
+         }));
+   }
+
+   /*******************************************************************************
+    ** Retry delays have a five-attempt cap; a consumer can request a longer delay.
+    *******************************************************************************/
+   private void retry(QRecord job, String message, Duration requested) throws QException
+   {
+      if(stopped)
+      {
+         return;
+      }
+      if(attempts(job) >= 5)
+      {
+         QRecord account = audit.query(FileGatewayTables.ACCOUNT, UploadJobs.equal("id", job.getValue("accountId"))).getFirst();
+         try
+         {
+            finish(job, new QRecord().withValue("status", "FAILED").withValue("errorMessage", message), account);
+         }
+         catch(IOException e)
+         {
+            throw (new QException("Unable to finish failed upload", e));
+         }
+         return;
+      }
+      long delay = RETRY_SECONDS[Math.min(Math.max(0, attempts(job) - 1), RETRY_SECONDS.length - 1)];
+      if(requested != null)
+      {
+         delay = Math.max(delay, requested.toSeconds());
+      }
+      persistOwned(job, new QRecord().withValue("status", "QUEUED").withValue("errorMessage", message)
+         .withValue("nextAttemptAt", clock.instant().plusSeconds(delay)).withValue("claimedBy", null).withValue("claimedAt", null));
+   }
+
+   /*******************************************************************************
+    ** Results are bounded by the process summary, never by the original upload size.
+    *******************************************************************************/
+   private void finish(QRecord job, QRecord outcome, QRecord account) throws QException, IOException
+   {
+      String summary = outcome.getValueString("resultSummary");
+      if(summary == null)
+      {
+         summary = outcome.getValueString("errorMessage");
+      }
+      if(summary == null)
+      {
+         summary = outcome.getValueString("status");
+      }
+      String key = "file-gateway/results/" + job.getValue("accountId") + "/" + job.getValue("id") + "/" + java.util.UUID.randomUUID() + ".csv";
+      byte[] csv = ("row,field,message\r\n,,\"" + summary.replace("\"", "\"\"") + "\"\r\n").getBytes(StandardCharsets.UTF_8);
+      String detailed = outcome.getValueString("_resultsCsv");
+      outcome.getValues().remove("_resultsCsv");
+      if(detailed != null)
+      {
+         csv = detailed.getBytes(StandardCharsets.UTF_8);
+      }
+      store.put(key, new ByteArrayInputStream(csv));
+      outcome.withValue("resultSummary", summary).withValue("resultsStorageKey", key).withValue("resultsSizeBytes", (long) csv.length)
+         .withValue("finishedAt", clock.instant()).withValue("nextAttemptAt", null).withValue("claimedBy", null).withValue("claimedAt", null);
+      if(persistOwned(job, outcome))
+      {
+         UploadNotifications.send(audit, job, outcome, account);
+      }
+      else
+      {
+         store.delete(key);
+      }
+   }
+
+   /*******************************************************************************
+    ** Late workers cannot overwrite a requeued or newly claimed job.
+    *******************************************************************************/
+   private boolean persistOwned(QRecord job, QRecord changes) throws QException
+   {
+      return (audit.withAccountLock("id", job.getValue("accountId"), () ->
+      {
+         QRecord current = find(job);
+         if(!"RUNNING".equals(current.getValueString("status")) || !Objects.equals(pod, current.getValueString("claimedBy"))
+            || !Objects.equals(job.getValueInstant("claimedAt"), current.getValueInstant("claimedAt")))
+         {
+            return (false);
+         }
+         return (audit.transitionUpload(changes.withValue("id", job.getValue("id")), "RUNNING"));
+      }));
+   }
+
+   /*******************************************************************************
+    ** Reload a gateway job inside a short bookkeeping transaction.
+    *******************************************************************************/
+   private QRecord find(QRecord job) throws QException
+   {
+      return (audit.query(FileGatewayTables.UPLOAD_JOB, UploadJobs.equal("id", job.getValue("id"))).getFirst());
+   }
+
+   /*******************************************************************************
+    ** Null attempts on older rows count as zero.
+    *******************************************************************************/
+   private int attempts(QRecord job)
+   {
+      return (job.getValueInteger("attempts") == null ? 0 : job.getValueInteger("attempts"));
+   }
+
+   /*******************************************************************************
+    ** Interrupt in-flight delivery and release only this pod's leases.
+    *******************************************************************************/
+   @Override
+   public void close() throws QException
+   {
+      stopped = true;
+      if(pool != null)
+      {
+         pool.shutdownNow();
+      }
+      for(QRecord job : audit.query(FileGatewayTables.UPLOAD_JOB, UploadJobs.equal("claimedBy", pod)))
+      {
+         persistOwned(job, new QRecord().withValue("status", "QUEUED").withValue("claimedBy", null).withValue("claimedAt", null));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/UploadJobs.java

@@ -0,0 +1,325 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.IOException;
+import java.nio.channels.SeekableByteChannel;
+import java.time.Clock;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.Objects;
+import java.util.UUID;
+import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qbits.filegateway.sftp.GatewaySession;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+/*******************************************************************************
+ ** Durable upload lifecycle. Account row transactions serialize quotas and renames across pods.
+ *******************************************************************************/
+public class UploadJobs
+{
+   public static final long MIB = 1024L * 1024L;
+   private final AuditWriter audit;
+   private final GatewayObjectStore store;
+   private final FileGatewayQBitConfig config;
+   private final Clock clock;
+
+   /*******************************************************************************
+    ** Share storage and persistence between the virtual tree and workers.
+    *******************************************************************************/
+   public UploadJobs(AuditWriter audit, GatewayObjectStore store, FileGatewayQBitConfig config, Clock clock)
+   {
+      this.audit = audit;
+      this.store = store;
+      this.config = config;
+      this.clock = clock;
+   }
+
+   /*******************************************************************************
+    ** The target was resolved through the user's metadata; endpoint routing can reuse this seam.
+    *******************************************************************************/
+   public SeekableByteChannel receive(GatewaySession session, String path, String table, Integer profile) throws IOException
+   {
+      String name = path.substring(path.lastIndexOf('/') + 1);
+      if(name.isBlank() || name.contains(":") || name.contains("\\") || name.chars().anyMatch(Character::isISOControl))
+      {
+         throw (new IOException("Invalid upload filename"));
+      }
+      if(!session.operationAllowed())
+      {
+         throw (new IOException("Operation limit exceeded"));
+      }
+      try
+      {
+         String date = DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneOffset.UTC).format(clock.instant());
+         String key = "file-gateway/uploads/" + session.account.getValue("id") + "/" + date + "/" + UUID.randomUUID() + "/" + name;
+         QRecord job = audit.insert(FileGatewayTables.UPLOAD_JOB, new QRecord()
+            .withValue("accountId", session.account.getValue("id")).withValue("connectionLogId", session.connectionId)
+            .withValue("remoteAddress", session.remoteIp).withValue("tableName", table).withValue("savedBulkLoadProfileId", profile)
+            .withValue("consumerName", "bulk-load").withValue("path", path).withValue("fileName", name).withValue("storageKey", key)
+            .withValue("status", "RECEIVING").withValue("sizeBytes", 0L).withValue("attempts", 0).withValue("createDate", clock.instant()));
+         long started = System.nanoTime();
+         return (new UploadChannel(store, key, Math.toIntExact(config.getUploadPartSizeMb() * MIB), config.getUploadReorderWindowMb() * MIB, new UploadChannel.Lifecycle()
+         {
+            @Override
+            public void reserve(long size) throws IOException
+            {
+               reserveBytes(job, size);
+            }
+
+            @Override
+            public void complete(GatewayObjectStore.Receipt receipt) throws IOException
+            {
+               try
+               {
+                  if(session.ssh.isClosing() || !session.ssh.isOpen())
+                  {
+                     throw (new IOException("Connection ended before upload close"));
+                  }
+                  job.withValue("sizeBytes", receipt.size()).withValue("sha256", receipt.sha256());
+                  audit.update(FileGatewayTables.UPLOAD_JOB, new QRecord().withValue("id", job.getValue("id"))
+                     .withValue("sizeBytes", receipt.size()).withValue("sha256", receipt.sha256()));
+                  if(!temporary(name))
+                  {
+                     queue(job, path, name);
+                  }
+                  log(session, job, "OK", null, started);
+               }
+               catch(QException e)
+               {
+                  throw (new IOException("Unable to finalize upload", e));
+               }
+            }
+
+            @Override
+            public void abort(String message, boolean limit) throws IOException
+            {
+               try
+               {
+                  QRecord outcome = new QRecord().withValue("id", job.getValue("id"))
+                     .withValue("status", limit ? "REJECTED" : "ABORTED").withValue("errorMessage", message).withValue("finishedAt", clock.instant());
+                  audit.update(FileGatewayTables.UPLOAD_JOB, outcome);
+                  UploadNotifications.send(audit, job, outcome, session.account);
+                  log(session, job, limit ? "LIMIT" : "ERROR", message, started);
+               }
+               catch(QException e)
+               {
+                  throw (new IOException("Unable to audit aborted upload", e));
+               }
+            }
+         }));
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to create upload job", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Charge the high-water byte count under an account lock, including concurrent uploads.
+    *******************************************************************************/
+   private void reserveBytes(QRecord job, long size) throws IOException
+   {
+      try
+      {
+         String rejection = audit.withAccountLock("id", job.getValue("accountId"), () ->
+         {
+            QRecord account = audit.query(FileGatewayTables.ACCOUNT, equal("id", job.getValue("accountId"))).getFirst();
+            Integer maximum = account.getValueInteger("maxFileSizeMb");
+            if(size > (maximum == null ? config.getDefaultMaxFileSizeMb() : maximum) * MIB)
+            {
+               return ("File size limit exceeded");
+            }
+            Integer daily = account.getValueInteger("maxUploadMbPerDay");
+            if(daily != null)
+            {
+               var midnight = clock.instant().atZone(ZoneOffset.UTC).toLocalDate().atStartOfDay(ZoneOffset.UTC).toInstant();
+               long used = 0;
+               for(QRecord other : audit.query(FileGatewayTables.UPLOAD_JOB, equal("accountId", job.getValue("accountId"))
+                  .withCriteria(new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, midnight))))
+               {
+                  if(!Objects.equals(other.getValue("id"), job.getValue("id")))
+                  {
+                     used += other.getValueLong("sizeBytes") == null ? 0 : other.getValueLong("sizeBytes");
+                  }
+               }
+               if(size > daily * MIB - used)
+               {
+                  return ("Daily upload quota exceeded");
+               }
+            }
+            audit.update(FileGatewayTables.UPLOAD_JOB, new QRecord().withValue("id", job.getValue("id")).withValue("sizeBytes", size));
+            return (null);
+         });
+         if(rejection != null)
+         {
+            throw (new UploadLimitException(rejection));
+         }
+         job.withValue("sizeBytes", size);
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to reserve upload quota", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Close and rename share one idempotent transition, with target-aware deduplication.
+    *******************************************************************************/
+   public void queue(QRecord job, String path, String name) throws QException
+   {
+      audit.withAccountLock("id", job.getValue("accountId"), () ->
+      {
+         QRecord current = audit.query(FileGatewayTables.UPLOAD_JOB, equal("id", job.getValue("id"))).getFirst();
+         if(!"RECEIVING".equals(current.getValueString("status")) || current.getValueString("sha256") == null)
+         {
+            return (null);
+         }
+         QRecord update = new QRecord().withValue("id", job.getValue("id")).withValue("path", path).withValue("fileName", name);
+         if(temporary(name))
+         {
+            audit.update(FileGatewayTables.UPLOAD_JOB, update);
+            return (null);
+         }
+         update.withValue("status", "QUEUED").withValue("queuedAt", clock.instant());
+         QRecord account = audit.query(FileGatewayTables.ACCOUNT, equal("id", job.getValue("accountId"))).getFirst();
+         if(Boolean.TRUE.equals(account.getValueBoolean("dedupeUploads")))
+         {
+            for(QRecord candidate : audit.query(FileGatewayTables.UPLOAD_JOB, equal("accountId", job.getValue("accountId"))
+               .withCriteria(new QFilterCriteria("sha256", QCriteriaOperator.EQUALS, current.getValue("sha256")))
+               .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "SUCCEEDED"))
+               .withCriteria(new QFilterCriteria("finishedAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, clock.instant().minus(config.getDedupeWindow())))))
+            {
+               if(Objects.equals(current.getValue("tableName"), candidate.getValue("tableName"))
+                  && Objects.equals(current.getValue("savedBulkLoadProfileId"), candidate.getValue("savedBulkLoadProfileId"))
+                  && Objects.equals(current.getValue("aliasId"), candidate.getValue("aliasId"))
+                  && Objects.equals(current.getValue("consumerName"), candidate.getValue("consumerName")))
+               {
+                  update.withValue("status", "DUPLICATE").withValue("duplicateOfJobId", candidate.getValue("id"))
+                     .withValue("finishedAt", clock.instant()).withValue("resultSummary", "Skipped: duplicate of upload " + candidate.getValue("id"));
+                  break;
+               }
+            }
+         }
+         audit.update(FileGatewayTables.UPLOAD_JOB, update);
+         return (null);
+      });
+   }
+
+   /*******************************************************************************
+    ** Closed temporary files can be renamed only within their original upload folder.
+    *******************************************************************************/
+   public void rename(Integer accountId, String source, String target) throws IOException
+   {
+      if(!source.substring(0, source.lastIndexOf('/')).equals(target.substring(0, target.lastIndexOf('/'))))
+      {
+         throw (new IOException("Rename must stay inside one upload folder"));
+      }
+      String name = target.substring(target.lastIndexOf('/') + 1);
+      if(name.isBlank() || name.contains(":") || name.contains("\\") || name.chars().anyMatch(Character::isISOControl))
+      {
+         throw (new IOException("Invalid upload filename"));
+      }
+      try
+      {
+         var rows = audit.query(FileGatewayTables.UPLOAD_JOB, equal("accountId", accountId)
+            .withCriteria(new QFilterCriteria("path", QCriteriaOperator.EQUALS, source))
+            .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "RECEIVING")));
+         if(rows.size() != 1 || rows.getFirst().getValueString("sha256") == null || !temporary(rows.getFirst().getValueString("fileName")))
+         {
+            throw (new IOException("Only a completed temporary upload can be renamed"));
+         }
+         queue(rows.getFirst(), target, target.substring(target.lastIndexOf('/') + 1));
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to rename upload", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Account-scoped records for a visible table.
+    *******************************************************************************/
+   public List<QRecord> forTable(Integer accountId, String table) throws IOException
+   {
+      try
+      {
+         return (audit.query(FileGatewayTables.UPLOAD_JOB, equal("accountId", accountId)
+            .withCriteria(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, table))));
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to list upload jobs", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Acknowledging either artifact hides both results and the sidecar.
+    *******************************************************************************/
+   public void acknowledge(QRecord job) throws IOException
+   {
+      try
+      {
+         audit.update(FileGatewayTables.UPLOAD_JOB, new QRecord().withValue("id", job.getValue("id")).withValue("acknowledgedAt", clock.instant()));
+      }
+      catch(QException e)
+      {
+         throw (new IOException("Unable to acknowledge upload", e));
+      }
+   }
+
+   /*******************************************************************************
+    ** Configured temporary filename suffixes.
+    *******************************************************************************/
+   public boolean temporary(String name)
+   {
+      return (config.getTemporaryUploadSuffixes().stream().anyMatch(name::endsWith));
+   }
+
+   /*******************************************************************************
+    ** Common equality filter for gateway bookkeeping only.
+    *******************************************************************************/
+   public static QQueryFilter equal(String field, java.io.Serializable value)
+   {
+      return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
+   }
+
+   /*******************************************************************************
+    ** Upload audit includes measured bytes, digest and the original connection identity.
+    *******************************************************************************/
+   private void log(GatewaySession session, QRecord job, String outcome, String message, long started) throws QException
+   {
+      audit.insert(FileGatewayTables.TRANSFER_LOG, new QRecord().withValue("accountId", session.account.getValue("id"))
+         .withValue("connectionLogId", session.connectionId).withValue("operation", "UPLOAD").withValue("path", job.getValue("path"))
+         .withValue("objectKind", "PROFILE").withValue("objectId", job.getValueString("savedBulkLoadProfileId"))
+         .withValue("uploadJobId", job.getValue("id")).withValue("bytes", job.getValue("sizeBytes")).withValue("sha256", job.getValue("sha256"))
+         .withValue("durationMs", (System.nanoTime() - started) / 1_000_000).withValue("outcome", outcome).withValue("message", message)
+         .withValue("clientChannel", "sftp/" + session.id));
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/UploadLimitException.java

@@ -0,0 +1,37 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.IOException;
+
+/*******************************************************************************
+ ** An upload exceeded the account's byte budget.
+ *******************************************************************************/
+public class UploadLimitException extends IOException
+{
+   /*******************************************************************************
+    ** Safe reason to expose through SFTP and the transfer audit.
+    *******************************************************************************/
+   public UploadLimitException(String message)
+   {
+      super(message);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/UploadNotifications.java

@@ -0,0 +1,80 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.util.List;
+import java.util.Locale;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qqq.backend.core.alerts.AlertLink;
+import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
+import com.kingsrook.qqq.backend.core.alerts.NotificationAudience;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+/*******************************************************************************
+ ** Core notification signals for terminal upload outcomes, including transfer limits.
+ *******************************************************************************/
+public final class UploadNotifications
+{
+   /*******************************************************************************
+    ** Static signal publisher.
+    *******************************************************************************/
+   private UploadNotifications()
+   {
+   }
+
+   /*******************************************************************************
+    ** No alerts producer is a no-op; service account messages go to their owner.
+    *******************************************************************************/
+   public static void send(AuditWriter audit, QRecord job, QRecord outcome, QRecord account) throws QException
+   {
+      String status = outcome.getValueString("status");
+      if(!List.of("FAILED", "PARTIAL", "REJECTED").contains(status)
+         && !("SUCCEEDED".equals(status) && Boolean.TRUE.equals(account.getValueBoolean("notifyOnSuccess"))))
+      {
+         return;
+      }
+      audit.system(() ->
+      {
+         var alerts = AlertsInstanceMetaData.of(QContext.getQInstance());
+         if(alerts == null || alerts.getSink() == null)
+         {
+            return (null);
+         }
+         String recipient = "SERVICE".equals(account.getValueString("accountType")) ? account.getValueString("ownerUserId") : account.getValueString("userId");
+         if(recipient == null || recipient.isBlank())
+         {
+            return (null);
+         }
+         QAlerts.notify(new NotificationSignal().withTitle("Upload " + status.toLowerCase(Locale.ROOT))
+            .withBody(outcome.getValueString("resultSummary") == null ? outcome.getValueString("errorMessage") : outcome.getValueString("resultSummary"))
+            .withSourceName("file-gateway").withSourceKey(job.getValueString("id"))
+            .withSourceTableName(FileGatewayTables.UPLOAD_JOB).withSourceRecordId(job.getValueString("id"))
+            .withAudience(NotificationAudience.user(recipient))
+            .withLink(AlertLink.toRecord(FileGatewayTables.UPLOAD_JOB, job.getValue("id"), "Upload job")));
+         return (null);
+      });
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/upload/WorkerUserSession.java

@@ -0,0 +1,114 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.upload;
+
+import java.io.IOException;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiCredentials;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
+
+/*******************************************************************************
+ ** Independent, short-lived account-user credentials for an upload worker.
+ *******************************************************************************/
+public final class WorkerUserSession implements AutoCloseable
+{
+   private final QInstance instance;
+   private final UiApiInvoker invoker;
+   private final String userId;
+   private final String detail;
+   private final Clock clock;
+   private String token;
+   private Instant minted;
+
+   /*******************************************************************************
+    ** The caller owns restoration of its QContext after the job.
+    *******************************************************************************/
+   public WorkerUserSession(QInstance instance, UiApiInvoker invoker, String userId, String detail, Clock clock) throws QException
+   {
+      this.instance = instance;
+      this.invoker = invoker;
+      this.userId = userId;
+      this.detail = detail;
+      this.clock = clock;
+      refresh();
+   }
+
+   /*******************************************************************************
+    ** Re-mint permissions even after the originating SSH session has disconnected.
+    *******************************************************************************/
+   private void refresh() throws QException
+   {
+      if(minted == null || !minted.plusSeconds(300).isAfter(clock.instant()))
+      {
+         var module = new QAuthenticationModuleDispatcher().getQModule(instance.getAuthentication());
+         var user = new AuditWriter(instance).system(() -> module.createAutomatedSessionForUser(instance, userId));
+         if(user == null || !user.hasPermission(FileGatewayPermissions.CONNECT))
+         {
+            throw (new QException("Missing file_gateway.connect"));
+         }
+         user.setClientChannel("sftp/" + detail);
+         String replacement = UiApiInProcessSessions.register(user, Duration.ofMinutes(10));
+         UiApiInProcessSessions.revoke(token);
+         token = replacement;
+         minted = clock.instant();
+         QContext.init(instance, user);
+      }
+   }
+
+   /*******************************************************************************
+    ** Only published operations, with fixed channel identity and user credentials.
+    *******************************************************************************/
+   public JsonNode invoke(UiApiRequest request) throws IOException, QException
+   {
+      refresh();
+      if(invoker.findOperation(request.getMethod(), request.getPath()).isEmpty())
+      {
+         throw (new IOException("Gateway requires a spec-defined UI API operation"));
+      }
+      var response = invoker.invoke(request.withCredentials(UiApiCredentials.ofInProcessSession(token))
+         .withInvokedBy("file-gateway").withClientDetail(detail));
+      if(!response.isSuccess())
+      {
+         throw (new IOException("UI API request failed with status " + response.getStatus()));
+      }
+      return (response.getJson());
+   }
+
+   /*******************************************************************************
+    ** Never retain worker credentials after a job.
+    *******************************************************************************/
+   @Override
+   public void close()
+   {
+      UiApiInProcessSessions.revoke(token);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/GatewayFileAttributes.java

@@ -0,0 +1,173 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.nio.file.attribute.FileTime;
+import java.nio.file.attribute.GroupPrincipal;
+import java.nio.file.attribute.PosixFileAttributes;
+import java.nio.file.attribute.PosixFilePermission;
+import java.nio.file.attribute.UserPrincipal;
+import java.util.Set;
+
+
+/*******************************************************************************
+ ** Virtual attributes: generated exports have size zero; stored objects report persisted size.
+ *******************************************************************************/
+public record GatewayFileAttributes(VirtualTree.Node node, FileTime time) implements PosixFileAttributes
+{
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute lastModifiedTime.
+    *******************************************************************************/
+   @Override
+   public FileTime lastModifiedTime()
+   {
+      return (time);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute lastAccessTime.
+    *******************************************************************************/
+   @Override
+   public FileTime lastAccessTime()
+   {
+      return (time);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute creationTime.
+    *******************************************************************************/
+   @Override
+   public FileTime creationTime()
+   {
+      return (time);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute isRegularFile.
+    *******************************************************************************/
+   @Override
+   public boolean isRegularFile()
+   {
+      return (!node.directory());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute isDirectory.
+    *******************************************************************************/
+   @Override
+   public boolean isDirectory()
+   {
+      return (node.directory());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute isSymbolicLink.
+    *******************************************************************************/
+   @Override
+   public boolean isSymbolicLink()
+   {
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute isOther.
+    *******************************************************************************/
+   @Override
+   public boolean isOther()
+   {
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute size.
+    *******************************************************************************/
+   @Override
+   public long size()
+   {
+      return (node.size());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute fileKey.
+    *******************************************************************************/
+   @Override
+   public Object fileKey()
+   {
+      return (node.path());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute owner.
+    *******************************************************************************/
+   @Override
+   public UserPrincipal owner()
+   {
+      return (() -> "gateway");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute group.
+    *******************************************************************************/
+   @Override
+   public GroupPrincipal group()
+   {
+      return (() -> "gateway");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual attribute permissions.
+    *******************************************************************************/
+   @Override
+   public Set<PosixFilePermission> permissions()
+   {
+      if(node.directory() && "PROFILE".equals(node.kind()) && node.objectId() != null)
+      {
+         return (Set.of(PosixFilePermission.OWNER_READ, PosixFilePermission.OWNER_WRITE, PosixFilePermission.OWNER_EXECUTE));
+      }
+      return (node.directory() ? Set.of(PosixFilePermission.OWNER_READ, PosixFilePermission.OWNER_EXECUTE) : Set.of(PosixFilePermission.OWNER_READ));
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/GatewayFileSystem.java

@@ -0,0 +1,201 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.nio.file.ClosedFileSystemException;
+import java.nio.file.FileStore;
+import java.nio.file.FileSystem;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+import java.nio.file.WatchService;
+import java.nio.file.attribute.UserPrincipalLookupService;
+import java.nio.file.spi.FileSystemProvider;
+import java.util.List;
+import java.util.Set;
+
+
+/*******************************************************************************
+ ** Session-private NIO filesystem over a UI API virtual tree, with no native mounts.
+ *******************************************************************************/
+public class GatewayFileSystem extends FileSystem
+{
+   private final VirtualTree tree;
+   private final GatewayFileSystemProvider provider;
+   private boolean open = true;
+
+
+
+   /*******************************************************************************
+    ** Create a purely virtual filesystem.
+    *******************************************************************************/
+   public GatewayFileSystem(VirtualTree tree)
+   {
+      this.tree = tree;
+      this.provider = new GatewayFileSystemProvider(this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Access the live virtual tree.
+    *******************************************************************************/
+   public VirtualTree tree()
+   {
+      if(!open)
+      {
+         throw (new ClosedFileSystemException());
+      }
+      return (tree);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem provider.
+    *******************************************************************************/
+   @Override
+   public FileSystemProvider provider()
+   {
+      return (provider);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem close.
+    *******************************************************************************/
+   @Override
+   public void close()
+   {
+      open = false;
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem isOpen.
+    *******************************************************************************/
+   @Override
+   public boolean isOpen()
+   {
+      return (open);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem isReadOnly.
+    *******************************************************************************/
+   @Override
+   public boolean isReadOnly()
+   {
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getSeparator.
+    *******************************************************************************/
+   @Override
+   public String getSeparator()
+   {
+      return ("/");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getRootDirectories.
+    *******************************************************************************/
+   @Override
+   public Iterable<Path> getRootDirectories()
+   {
+      return (List.of(getPath("/")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getFileStores.
+    *******************************************************************************/
+   @Override
+   public Iterable<FileStore> getFileStores()
+   {
+      return (List.of());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem supportedFileAttributeViews.
+    *******************************************************************************/
+   @Override
+   public Set<String> supportedFileAttributeViews()
+   {
+      return (Set.of("basic", "posix"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getPath.
+    *******************************************************************************/
+   @Override
+   public Path getPath(String first, String... more)
+   {
+      return (new GatewayPath(this, first + (more.length == 0 ? "" : "/" + String.join("/", more))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getPathMatcher.
+    *******************************************************************************/
+   @Override
+   public PathMatcher getPathMatcher(String syntaxAndPattern)
+   {
+      throw (new UnsupportedOperationException("Path matching is unavailable"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem getUserPrincipalLookupService.
+    *******************************************************************************/
+   @Override
+   public UserPrincipalLookupService getUserPrincipalLookupService()
+   {
+      throw (new UnsupportedOperationException("Native users are unavailable"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual filesystem newWatchService.
+    *******************************************************************************/
+   @Override
+   public WatchService newWatchService()
+   {
+      throw (new UnsupportedOperationException("Watching is unavailable"));
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/GatewayFileSystemProvider.java

@@ -0,0 +1,388 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.io.IOException;
+import java.net.URI;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.AccessMode;
+import java.nio.file.CopyOption;
+import java.nio.file.DirectoryStream;
+import java.nio.file.FileStore;
+import java.nio.file.FileSystem;
+import java.nio.file.FileSystemAlreadyExistsException;
+import java.nio.file.LinkOption;
+import java.nio.file.OpenOption;
+import java.nio.file.Path;
+import java.nio.file.ProviderMismatchException;
+import java.nio.file.StandardOpenOption;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.nio.file.attribute.FileAttribute;
+import java.nio.file.attribute.FileAttributeView;
+import java.nio.file.attribute.FileTime;
+import java.nio.file.spi.FileSystemProvider;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+
+/*******************************************************************************
+ ** NIO operations resolve against the virtual tree and its explicitly supported upload artifacts.
+ *******************************************************************************/
+public class GatewayFileSystemProvider extends FileSystemProvider
+{
+   private final GatewayFileSystem fileSystem;
+
+
+
+   /*******************************************************************************
+    ** Bind one virtual filesystem.
+    *******************************************************************************/
+   public GatewayFileSystemProvider(GatewayFileSystem fileSystem)
+   {
+      this.fileSystem = fileSystem;
+   }
+
+
+
+   /*******************************************************************************
+    ** Canonical virtual path in this session.
+    *******************************************************************************/
+   private String name(Path path)
+   {
+      if(path.getFileSystem() != fileSystem)
+      {
+         throw (new ProviderMismatchException());
+      }
+      return (path.toAbsolutePath().normalize().toString());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider getScheme.
+    *******************************************************************************/
+   @Override
+   public String getScheme()
+   {
+      return ("gateway");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider newFileSystem.
+    *******************************************************************************/
+   @Override
+   public FileSystem newFileSystem(URI uri, Map<String, ?> env)
+   {
+      throw (new FileSystemAlreadyExistsException());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider getFileSystem.
+    *******************************************************************************/
+   @Override
+   public FileSystem getFileSystem(URI uri)
+   {
+      return (fileSystem);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider getPath.
+    *******************************************************************************/
+   @Override
+   public Path getPath(URI uri)
+   {
+      return (fileSystem.getPath(uri.getPath()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider newByteChannel.
+    *******************************************************************************/
+   @Override
+   public SeekableByteChannel newByteChannel(Path path, Set<? extends OpenOption> options, FileAttribute<?>... attrs) throws IOException
+   {
+      boolean write = options.contains(StandardOpenOption.WRITE) || options.contains(StandardOpenOption.APPEND) || options.contains(StandardOpenOption.CREATE) || options.contains(StandardOpenOption.CREATE_NEW);
+      if(options.contains(StandardOpenOption.APPEND))
+      {
+         fileSystem.tree().deny(name(path), "UPLOAD", "Appending to stored uploads is unsupported");
+      }
+      return (write ? fileSystem.tree().write(name(path)) : fileSystem.tree().read(name(path)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider newDirectoryStream.
+    *******************************************************************************/
+   @Override
+   public DirectoryStream<Path> newDirectoryStream(Path dir, DirectoryStream.Filter<? super Path> filter) throws IOException
+   {
+      List<Path> entries = new ArrayList<>();
+      for(VirtualTree.Node node : fileSystem.tree().list(name(dir)))
+      {
+         Path path = fileSystem.getPath(node.path());
+         if(filter.accept(path))
+         {
+            entries.add(path);
+         }
+      }
+      return (new GatewayDirectoryStream(entries));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider createDirectory.
+    *******************************************************************************/
+   @Override
+   public void createDirectory(Path dir, FileAttribute<?>... attrs) throws IOException
+   {
+      fileSystem.tree().deny(name(dir), "UPLOAD", "mkdir is not supported");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider delete.
+    *******************************************************************************/
+   @Override
+   public void delete(Path path) throws IOException
+   {
+      fileSystem.tree().remove(name(path));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider copy.
+    *******************************************************************************/
+   @Override
+   public void copy(Path source, Path target, CopyOption... options) throws IOException
+   {
+      fileSystem.tree().deny(name(target), "UPLOAD", "Copy is not supported");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider move.
+    *******************************************************************************/
+   @Override
+   public void move(Path source, Path target, CopyOption... options) throws IOException
+   {
+      fileSystem.tree().rename(name(source), name(target));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider isSameFile.
+    *******************************************************************************/
+   @Override
+   public boolean isSameFile(Path path, Path path2)
+   {
+      return (name(path).equals(name(path2)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider isHidden.
+    *******************************************************************************/
+   @Override
+   public boolean isHidden(Path path)
+   {
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider getFileStore.
+    *******************************************************************************/
+   @Override
+   public FileStore getFileStore(Path path)
+   {
+      throw (new UnsupportedOperationException("No native store"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider checkAccess.
+    *******************************************************************************/
+   @Override
+   public void checkAccess(Path path, AccessMode... modes) throws IOException
+   {
+      fileSystem.tree().stat(name(path));
+      for(AccessMode mode : modes)
+      {
+         if(mode == AccessMode.WRITE)
+         {
+            VirtualTree.Node node = fileSystem.tree().stat(name(path));
+            if(!node.directory() || !"PROFILE".equals(node.kind()) || node.objectId() == null)
+            {
+               fileSystem.tree().deny(name(path), "UPLOAD", "Writes require a profile folder");
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider getFileAttributeView.
+    *******************************************************************************/
+   @Override
+   public <V extends FileAttributeView> V getFileAttributeView(Path path, Class<V> type, LinkOption... options)
+   {
+      return (null);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider readAttributes.
+    *******************************************************************************/
+   @Override
+   public <A extends BasicFileAttributes> A readAttributes(Path path, Class<A> type, LinkOption... options) throws IOException
+   {
+      return (type.cast(new GatewayFileAttributes(fileSystem.tree().stat(name(path)), FileTime.from(Instant.now()))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider readAttributes.
+    *******************************************************************************/
+   @Override
+   public Map<String, Object> readAttributes(Path path, String attributes, LinkOption... options) throws IOException
+   {
+      var attrs = readAttributes(path, GatewayFileAttributes.class, options);
+      Map<String, Object> result = new HashMap<>();
+      result.put("isDirectory", attrs.isDirectory());
+      result.put("isRegularFile", attrs.isRegularFile());
+      result.put("isSymbolicLink", false);
+      result.put("isOther", false);
+      result.put("size", attrs.size());
+      result.put("fileKey", attrs.fileKey());
+      result.put("lastModifiedTime", attrs.lastModifiedTime());
+      result.put("lastAccessTime", attrs.lastAccessTime());
+      result.put("creationTime", attrs.creationTime());
+      result.put("permissions", attrs.permissions());
+      result.put("owner", attrs.owner());
+      result.put("group", attrs.group());
+      return (result);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider setAttribute.
+    *******************************************************************************/
+   @Override
+   public void setAttribute(Path path, String attribute, Object value, LinkOption... options) throws IOException
+   {
+      fileSystem.tree().deny(name(path), "UPLOAD", "setattr is not supported");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider createSymbolicLink.
+    *******************************************************************************/
+   @Override
+   public void createSymbolicLink(Path link, Path target, FileAttribute<?>... attrs) throws IOException
+   {
+      fileSystem.tree().deny(name(link), "UPLOAD", "Links are not supported");
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual provider createLink.
+    *******************************************************************************/
+   @Override
+   public void createLink(Path link, Path existing) throws IOException
+   {
+      fileSystem.tree().deny(name(link), "UPLOAD", "Links are not supported");
+   }
+
+
+   /*******************************************************************************
+    ** Single-use directory snapshot held entirely in memory.
+    *******************************************************************************/
+   private static class GatewayDirectoryStream implements DirectoryStream<Path>
+   {
+      private final List<Path> entries;
+      private boolean used;
+
+
+
+      /*******************************************************************************
+       ** Directory snapshot lifecycle.
+       *******************************************************************************/
+      GatewayDirectoryStream(List<Path> entries)
+      {
+         this.entries = entries;
+      }
+
+
+
+      /*******************************************************************************
+       ** Directory snapshot lifecycle.
+       *******************************************************************************/
+      @Override
+      public Iterator<Path> iterator()
+      {
+         if(used)
+         {
+            throw (new IllegalStateException("Directory stream is closed or already iterated"));
+         }
+         used = true;
+         return (entries.iterator());
+      }
+
+
+
+      /*******************************************************************************
+       ** Directory snapshot lifecycle.
+       *******************************************************************************/
+      @Override
+      public void close()
+      {
+         used = true;
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/GatewayPath.java

@@ -0,0 +1,371 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.io.File;
+import java.io.IOException;
+import java.net.URI;
+import java.nio.file.FileSystem;
+import java.nio.file.LinkOption;
+import java.nio.file.Path;
+import java.nio.file.ProviderMismatchException;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.List;
+
+
+/*******************************************************************************
+ ** A path made solely from virtual name segments; never delegates to a native filesystem.
+ *******************************************************************************/
+public final class GatewayPath implements Path
+{
+   private final GatewayFileSystem fileSystem;
+   private final String value;
+
+
+
+   /*******************************************************************************
+    ** Construct a virtual path.
+    *******************************************************************************/
+   public GatewayPath(GatewayFileSystem fileSystem, String value)
+   {
+      this.fileSystem = fileSystem;
+      this.value = value.replaceAll("/+", "/").replaceAll("(?<=.)/$", "");
+   }
+
+
+
+   /*******************************************************************************
+    ** Individual name segments.
+    *******************************************************************************/
+   private List<String> parts()
+   {
+      return (value.isEmpty() || "/".equals(value) ? List.of() : Arrays.asList(value.replaceFirst("^/", "").split("/")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getFileSystem.
+    *******************************************************************************/
+   @Override
+   public FileSystem getFileSystem()
+   {
+      return (fileSystem);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path isAbsolute.
+    *******************************************************************************/
+   @Override
+   public boolean isAbsolute()
+   {
+      return (value.startsWith("/"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getRoot.
+    *******************************************************************************/
+   @Override
+   public Path getRoot()
+   {
+      return (isAbsolute() ? new GatewayPath(fileSystem, "/") : null);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getFileName.
+    *******************************************************************************/
+   @Override
+   public Path getFileName()
+   {
+      List<String> parts = parts();
+      return (parts.isEmpty() ? null : new GatewayPath(fileSystem, parts.getLast()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getParent.
+    *******************************************************************************/
+   @Override
+   public Path getParent()
+   {
+      int slash = value.lastIndexOf('/');
+      return (slash < 0 || "/".equals(value) ? null : new GatewayPath(fileSystem, slash == 0 ? "/" : value.substring(0, slash)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getNameCount.
+    *******************************************************************************/
+   @Override
+   public int getNameCount()
+   {
+      return (parts().size());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path getName.
+    *******************************************************************************/
+   @Override
+   public Path getName(int index)
+   {
+      return (new GatewayPath(fileSystem, parts().get(index)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path subpath.
+    *******************************************************************************/
+   @Override
+   public Path subpath(int beginIndex, int endIndex)
+   {
+      return (new GatewayPath(fileSystem, String.join("/", parts().subList(beginIndex, endIndex))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path startsWith.
+    *******************************************************************************/
+   @Override
+   public boolean startsWith(Path other)
+   {
+      return (other instanceof GatewayPath p && p.fileSystem == fileSystem && (value.equals(p.value) || value.startsWith(p.value.endsWith("/") ? p.value : p.value + "/")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path endsWith.
+    *******************************************************************************/
+   @Override
+   public boolean endsWith(Path other)
+   {
+      return (other instanceof GatewayPath p && p.fileSystem == fileSystem && (value.equals(p.value) || (!p.isAbsolute() && value.endsWith("/" + p.value))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path normalize.
+    *******************************************************************************/
+   @Override
+   public Path normalize()
+   {
+      Deque<String> names = new ArrayDeque<>();
+      for(String part : parts())
+      {
+         if(part.equals(".") || part.isEmpty())
+         {
+            continue;
+         }
+         if(part.equals("..") && !names.isEmpty() && !names.getLast().equals(".."))
+         {
+            names.removeLast();
+         }
+         else if(!part.equals("..") || !isAbsolute())
+         {
+            names.addLast(part);
+         }
+      }
+      return (new GatewayPath(fileSystem, (isAbsolute() ? "/" : "") + String.join("/", names)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path resolve.
+    *******************************************************************************/
+   @Override
+   public Path resolve(Path other)
+   {
+      if(!(other instanceof GatewayPath p) || p.fileSystem != fileSystem)
+      {
+         throw (new ProviderMismatchException());
+      }
+      return (other.isAbsolute() ? other : new GatewayPath(fileSystem, value + (value.endsWith("/") || value.isEmpty() ? "" : "/") + other));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path relativize.
+    *******************************************************************************/
+   @Override
+   public Path relativize(Path other)
+   {
+      if(!(other instanceof GatewayPath p) || p.fileSystem != fileSystem || isAbsolute() != p.isAbsolute())
+      {
+         throw (new IllegalArgumentException("Incompatible virtual paths"));
+      }
+      List<String> left = ((GatewayPath) normalize()).parts();
+      List<String> right = ((GatewayPath) p.normalize()).parts();
+      int common = 0;
+      while(common < left.size() && common < right.size() && left.get(common).equals(right.get(common)))
+      {
+         common++;
+      }
+      List<String> result = new ArrayList<>();
+      for(int i = common; i < left.size(); i++)
+      {
+         result.add("..");
+      }
+      result.addAll(right.subList(common, right.size()));
+      return (new GatewayPath(fileSystem, String.join("/", result)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path toUri.
+    *******************************************************************************/
+   @Override
+   public URI toUri()
+   {
+      try
+      {
+         return (new URI("gateway", null, toAbsolutePath().toString(), null));
+      }
+      catch(java.net.URISyntaxException e)
+      {
+         throw (new IllegalArgumentException(e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path toAbsolutePath.
+    *******************************************************************************/
+   @Override
+   public Path toAbsolutePath()
+   {
+      return (isAbsolute() ? this : new GatewayPath(fileSystem, "/" + value));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path toRealPath.
+    *******************************************************************************/
+   @Override
+   public Path toRealPath(LinkOption... options) throws IOException
+   {
+      Path path = toAbsolutePath().normalize();
+      fileSystem.tree().stat(path.toString());
+      return (path);
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path toFile.
+    *******************************************************************************/
+   @Override
+   public File toFile()
+   {
+      throw (new UnsupportedOperationException("Virtual paths have no native file"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path register.
+    *******************************************************************************/
+   @Override
+   public WatchKey register(WatchService watcher, WatchEvent.Kind<?>[] events, WatchEvent.Modifier... modifiers)
+   {
+      throw (new UnsupportedOperationException("Virtual paths cannot be watched"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path iterator.
+    *******************************************************************************/
+   @Override
+   public Iterator<Path> iterator()
+   {
+      return (parts().stream().map(name -> (Path) new GatewayPath(fileSystem, name)).iterator());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path compareTo.
+    *******************************************************************************/
+   @Override
+   public int compareTo(Path other)
+   {
+      return (value.compareTo(other.toString()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path equals.
+    *******************************************************************************/
+   @Override
+   public boolean equals(Object other)
+   {
+      return (other instanceof GatewayPath path && path.fileSystem == fileSystem && path.value.equals(value));
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path hashCode.
+    *******************************************************************************/
+   @Override
+   public int hashCode()
+   {
+      return (31 * System.identityHashCode(fileSystem) + value.hashCode());
+   }
+
+
+
+   /*******************************************************************************
+    ** Virtual path toString.
+    *******************************************************************************/
+   @Override
+   public String toString()
+   {
+      return (value);
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/GatewayUiApiClient.java

@@ -0,0 +1,126 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.io.IOException;
+import java.nio.file.AccessDeniedException;
+import java.nio.file.NoSuchFileException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import com.kingsrook.qbits.filegateway.sftp.GatewaySession;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiCredentials;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+
+
+/*******************************************************************************
+ ** Every visible object comes from a spec-defined UI API operation as the signed-in user.
+ *******************************************************************************/
+public class GatewayUiApiClient
+{
+   private final UiApiInvoker invoker;
+   private final GatewaySession session;
+
+
+
+   /*******************************************************************************
+    ** Bind this client to one SFTP identity.
+    *******************************************************************************/
+   public GatewayUiApiClient(UiApiInvoker invoker, GatewaySession session)
+   {
+      this.invoker = invoker;
+      this.session = session;
+   }
+
+
+
+   /*******************************************************************************
+    ** Get the permission-filtered app tree.
+    *******************************************************************************/
+   public JsonNode metadata() throws IOException
+   {
+      return (invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath("/metaData")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Query visible saved objects of a table.
+    *******************************************************************************/
+   public JsonNode savedObjects(String source, String table) throws IOException
+   {
+      var records = JsonNodeFactory.instance.arrayNode();
+      int pageSize = 250;
+      for(int skip = 0; ; skip += pageSize)
+      {
+         QQueryFilter filter = new QQueryFilter(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, table))
+            .withOrderBy(new QFilterOrderBy("id", true)).withLimit(pageSize).withSkip(skip);
+         JsonNode page = invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/table/{tableName}/query")
+            .withPathParam("tableName", source).withJsonBody(java.util.Map.of("filter", filter))).path("records");
+         page.forEach(records::add);
+         if(page.size() < pageSize)
+         {
+            return (records);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Apply the fixed channel identity, credentials and contract check on every call.
+    *******************************************************************************/
+   protected JsonNode invoke(UiApiRequest request) throws IOException
+   {
+      if(invoker.findOperation(request.getMethod(), request.getPath()).isEmpty())
+      {
+         throw (new IOException("Gateway requires a spec-defined UI API operation"));
+      }
+      try
+      {
+         var response = invoker.invoke(request.withCredentials(UiApiCredentials.ofInProcessSession(session.token()))
+            .withInvokedBy("file-gateway").withClientDetail(session.id));
+         if(response.getStatus() == 404)
+         {
+            throw (new NoSuchFileException(request.getPath()));
+         }
+         if(response.getStatus() == 401 || response.getStatus() == 403)
+         {
+            throw (new AccessDeniedException(request.getPath()));
+         }
+         if(!response.isSuccess())
+         {
+            throw (new IOException("UI API request failed with status " + response.getStatus()));
+         }
+         return (response.getJson());
+      }
+      catch(com.kingsrook.qqq.backend.core.exceptions.QException e)
+      {
+         throw (new IOException("User session unavailable", e));
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/filegateway/vfs/VirtualTree.java

@@ -0,0 +1,600 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway.vfs;
+
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.AccessDeniedException;
+import java.nio.file.NoSuchFileException;
+import java.time.Clock;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qbits.filegateway.sftp.GatewaySession;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+import com.kingsrook.qbits.filegateway.storage.ObjectReadChannel;
+import com.kingsrook.qbits.filegateway.upload.UploadJobs;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Permission-filtered virtual app and table tree with per-session in-memory caches.
+ *******************************************************************************/
+public class VirtualTree
+{
+   /*******************************************************************************
+    ** A virtual directory or generated file.
+    *******************************************************************************/
+   public record Node(String path, boolean directory, String kind, String objectId, long size, QRecord job)
+   {
+      /*******************************************************************************
+       ** Metadata-backed nodes have no stored size or job.
+       *******************************************************************************/
+      public Node(String path, boolean directory, String kind, String objectId)
+      {
+         this(path, directory, kind, objectId, 0, null);
+      }
+   }
+
+   private final GatewayUiApiClient client;
+   private final GatewaySession session;
+   private final AuditWriter audit;
+   private final Clock clock;
+   private UploadJobs uploads;
+   private GatewayObjectStore store;
+   private JsonNode metadata;
+   private Instant metadataAt;
+   private final Map<String, JsonNode> saved = new HashMap<>();
+   private final Map<String, Instant> savedAt = new HashMap<>();
+
+
+
+   /*******************************************************************************
+    ** Create a session-private tree and caches.
+    *******************************************************************************/
+   public VirtualTree(GatewayUiApiClient client, GatewaySession session, AuditWriter audit, Clock clock)
+   {
+      this.client = client;
+      this.session = session;
+      this.audit = audit;
+      this.clock = clock;
+   }
+
+
+
+   /*******************************************************************************
+    ** Attach the runtime's upload pipeline; endpoint targets will reuse this service.
+    *******************************************************************************/
+   public VirtualTree withUploads(UploadJobs uploads, GatewayObjectStore store)
+   {
+      this.uploads = uploads;
+      this.store = store;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Cache user metadata for sixty seconds.
+    *******************************************************************************/
+   private JsonNode metadata() throws IOException
+   {
+      if(metadataAt == null || !metadataAt.plusSeconds(60).isAfter(clock.instant()))
+      {
+         metadata = client.metadata();
+         metadataAt = clock.instant();
+      }
+      return (metadata);
+   }
+
+
+
+   /*******************************************************************************
+    ** Cache saved objects for thirty seconds.
+    *******************************************************************************/
+   private JsonNode saved(String source, String table) throws IOException
+   {
+      String key = source + "/" + table;
+      if(!savedAt.containsKey(key) || !savedAt.get(key).plusSeconds(30).isAfter(clock.instant()))
+      {
+         saved.put(key, client.savedObjects(source, table));
+         savedAt.put(key, clock.instant());
+      }
+      return (saved.get(key));
+   }
+
+
+
+   /*******************************************************************************
+    ** Resolve only through visible parents; hidden paths are indistinguishable from missing ones.
+    *******************************************************************************/
+   public synchronized Node stat(String path) throws IOException
+   {
+      sessionToken();
+      if("/".equals(path))
+      {
+         return (new Node("/", true, "APP", null));
+      }
+      int split = path.lastIndexOf('/');
+      String parent = split == 0 ? "/" : path.substring(0, split);
+      return (children(parent).stream().filter(node -> node.path().equals(path)).findFirst()
+         .orElseThrow(() -> new NoSuchFileException(path)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Refresh permissions even when a listing uses cached metadata.
+    *******************************************************************************/
+   private void sessionToken() throws IOException
+   {
+      try
+      {
+         session.token();
+      }
+      catch(Exception e)
+      {
+         throw (new AccessDeniedException("User session unavailable"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Audit each directory open once.
+    *******************************************************************************/
+   public synchronized List<Node> list(String path) throws IOException
+   {
+      long started = System.nanoTime();
+      try
+      {
+         sessionToken();
+         if(!session.operationAllowed())
+         {
+            log(path, "LIST", "LIMIT", null, started);
+            throw (new IOException("Operation limit exceeded"));
+         }
+         List<Node> nodes = children(path);
+         log(path, "LIST", "OK", null, started);
+         return (nodes);
+      }
+      catch(NoSuchFileException e)
+      {
+         log(path, "LIST", "NOT_FOUND", null, started);
+         throw (e);
+      }
+      catch(AccessDeniedException e)
+      {
+         log(path, "LIST", "DENIED", null, started);
+         throw (e);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Resolve permitted app and table folders from UI API metadata.
+    *******************************************************************************/
+   private List<Node> children(String path) throws IOException
+   {
+      JsonNode meta = metadata();
+      List<Node> result = new ArrayList<>();
+      if("/".equals(path))
+      {
+         addAppChildren(result, "", meta.path("appTree"), meta);
+         return (result);
+      }
+      String[] segments = path.substring(1).split("/");
+      JsonNode children = meta.path("appTree");
+      String prefix = "";
+      String table = null;
+      int index = 0;
+      for(; index < segments.length; index++)
+      {
+         JsonNode found = null;
+         for(JsonNode child : children)
+         {
+            if(segments[index].equals(child.path("name").asText()))
+            {
+               found = child;
+               break;
+            }
+         }
+         if(found == null)
+         {
+            throw (new NoSuchFileException(path));
+         }
+         prefix += "/" + segments[index];
+         if("TABLE".equalsIgnoreCase(found.path("type").asText()) && meta.path("tables").has(segments[index]))
+         {
+            table = segments[index++];
+            break;
+         }
+         if(!"APP".equalsIgnoreCase(found.path("type").asText()))
+         {
+            throw (new NoSuchFileException(path));
+         }
+         children = found.path("children");
+      }
+      if(table == null)
+      {
+         addAppChildren(result, prefix, children, meta);
+         return (result);
+      }
+      List<String> folders = new ArrayList<>(List.of("results", "archive"));
+      boolean profiles = meta.path("tables").path(table).path("insertPermission").asBoolean(false) && meta.path("tables").has("savedBulkLoadProfile") && meta.path("processes").has(table + ".bulkInsert");
+      if(profiles && !saved("savedBulkLoadProfile", table).isEmpty())
+      {
+         folders.add("upload");
+      }
+      if(meta.path("tables").has("savedView"))
+      {
+         folders.add("views");
+      }
+      if(meta.path("tables").has("savedReport") && meta.path("processes").has("renderSavedReport"))
+      {
+         folders.add("reports");
+      }
+      if(index == segments.length)
+      {
+         for(String folder : folders)
+         {
+            result.add(new Node(prefix + "/" + folder, true, folderKind(folder), null));
+         }
+         return (result);
+      }
+      String folder = segments[index++];
+      if(!folders.contains(folder))
+      {
+         throw (new NoSuchFileException(path));
+      }
+      if("results".equals(folder) || "archive".equals(folder))
+      {
+         return (artifacts(prefix, table, path, folder));
+      }
+      String source = "upload".equals(folder) ? "savedBulkLoadProfile" : "views".equals(folder) ? "savedView" : "savedReport";
+      JsonNode records = saved(source, table);
+      Map<String, Integer> counts = new HashMap<>();
+      for(JsonNode row : records)
+      {
+         counts.merge(slug(row.path("values").path("label").asText()), 1, Integer::sum);
+      }
+      for(JsonNode row : records)
+      {
+         String id = row.path("values").path("id").asText();
+         String name = slug(row.path("values").path("label").asText());
+         if(counts.get(name) > 1)
+         {
+            name += "-" + id;
+         }
+         String item = prefix + "/" + folder + "/" + name;
+         if("upload".equals(folder))
+         {
+            result.add(new Node(item, true, "PROFILE", id));
+         }
+         else
+         {
+            String formats = session.account.getValueString("formats");
+            for(String format : (formats == null ? "csv,xlsx" : formats).split("[,\\s]+"))
+            {
+               if(List.of("csv", "xlsx", "json").contains(format))
+               {
+                  result.add(new Node(item + "." + format, false, folderKind(folder), id));
+               }
+            }
+         }
+      }
+      if(index == segments.length)
+      {
+         return (result);
+      }
+      if(index + 1 == segments.length && "upload".equals(folder) && result.stream().anyMatch(node -> node.path().equals(path)))
+      {
+         return (artifacts(prefix, table, path, "upload"));
+      }
+      throw (new NoSuchFileException(path));
+   }
+
+
+
+   /*******************************************************************************
+    ** Sidecars and results expire after seven days; archive objects remain until retention.
+    *******************************************************************************/
+   private List<Node> artifacts(String prefix, String table, String path, String folder) throws IOException
+   {
+      List<Node> nodes = new ArrayList<>();
+      if("results".equals(folder) && !path.equals(prefix + "/results"))
+      {
+         throw (new NoSuchFileException(path));
+      }
+      if(uploads == null)
+      {
+         return (nodes);
+      }
+      for(QRecord job : uploads.forTable(session.account.getValueInteger("id"), table))
+      {
+         String status = job.getValueString("status");
+         boolean active = List.of("RECEIVING", "QUEUED", "RUNNING").contains(status);
+         boolean visible = job.getValueInstant("acknowledgedAt") == null
+            && job.getValueInstant("createDate").plusSeconds(7 * 86400).isAfter(clock.instant());
+         String name = job.getValueString("fileName");
+         String id = job.getValueString("id");
+         if("upload".equals(folder) && visible && job.getValueString("path").startsWith(path + "/"))
+         {
+            String suffix = active ? ".processing" : List.of("SUCCEEDED", "DUPLICATE").contains(status) ? ".done" : ".error";
+            String item = path + "/" + name + suffix;
+            nodes.removeIf(node -> node.path().equals(item));
+            nodes.add(new Node(item, false, "SIDECAR", id, summary(job).length, job));
+            if("RECEIVING".equals(status) && uploads.temporary(name))
+            {
+               nodes.add(new Node(path + "/" + name, false, "TEMPORARY", id, job.getValueLong("sizeBytes"), job));
+            }
+         }
+         if("results".equals(folder) && visible && job.getValueString("resultsStorageKey") != null)
+         {
+            nodes.add(new Node(prefix + "/results/" + id + "-" + name + ".results.csv", false, "RESULT", id,
+               job.getValueLong("resultsSizeBytes") == null ? 0 : job.getValueLong("resultsSizeBytes"), job));
+         }
+         if("archive".equals(folder) && !active && job.getValueString("sha256") != null
+            && !job.getValueString("storageKey").startsWith("file-gateway/quarantine/"))
+         {
+            String date = job.getValueInstant("createDate").atZone(ZoneOffset.UTC).toLocalDate().toString();
+            String directory = prefix + "/archive/" + date;
+            if(path.equals(prefix + "/archive"))
+            {
+               if(nodes.stream().noneMatch(node -> node.path().equals(directory)))
+               {
+                  nodes.add(new Node(directory, true, "ARCHIVE", null));
+               }
+            }
+            else if(path.equals(directory))
+            {
+               nodes.add(new Node(directory + "/" + id + "-" + name, false, "ARCHIVE", id, job.getValueLong("sizeBytes"), job));
+            }
+         }
+      }
+      return (nodes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Sidecar text is generated solely from durable job state.
+    *******************************************************************************/
+   private byte[] summary(QRecord job)
+   {
+      String text = job.getValueString("status") + "\n";
+      if(job.getValueString("resultSummary") != null)
+      {
+         text += job.getValueString("resultSummary") + "\n";
+      }
+      else if(job.getValueString("errorMessage") != null)
+      {
+         text += job.getValueString("errorMessage") + "\n";
+      }
+      return (text.getBytes(StandardCharsets.UTF_8));
+   }
+
+
+
+   /*******************************************************************************
+    ** Writes require a visible profile, never a guessed table or profile identifier.
+    *******************************************************************************/
+   public SeekableByteChannel write(String path) throws IOException
+   {
+      String parent = path.substring(0, path.lastIndexOf('/'));
+      Node target = stat(parent);
+      if(uploads == null || !target.directory() || !"PROFILE".equals(target.kind()) || target.objectId() == null)
+      {
+         deny(path, "UPLOAD", "Uploads require a bulk load profile folder");
+      }
+      String table = parent.substring(0, parent.lastIndexOf("/upload/"));
+      table = table.substring(table.lastIndexOf('/') + 1);
+      return (uploads.receive(session, path, table, Integer.valueOf(target.objectId())));
+   }
+
+
+
+   /*******************************************************************************
+    ** Read only stored artifacts and generated sidecars; view/report export is phase c.
+    *******************************************************************************/
+   public SeekableByteChannel read(String path) throws IOException
+   {
+      Node node = stat(path);
+      if(node.job() == null || node.directory() || "TEMPORARY".equals(node.kind()))
+      {
+         deny(path, "DOWNLOAD", "This file is not downloadable");
+      }
+      if(!session.operationAllowed())
+      {
+         throw (new IOException("Operation limit exceeded"));
+      }
+      long began = System.nanoTime();
+      ObjectReadChannel.Completion completion = (bytes, failed) -> downloadLog(node, bytes, failed, began);
+      if("SIDECAR".equals(node.kind()))
+      {
+         byte[] content = summary(node.job());
+         return (new ObjectReadChannel(content.length, (start, end) -> new ByteArrayInputStream(content, (int) start, (int) (end - start + 1)), completion));
+      }
+      String key = node.job().getValueString("RESULT".equals(node.kind()) ? "resultsStorageKey" : "storageKey");
+      return (new ObjectReadChannel(node.size(), (start, end) -> store.openRange(key, start, end), completion));
+   }
+
+
+
+   /*******************************************************************************
+    ** Record actual object-download bytes, including ranged reads, once per handle.
+    *******************************************************************************/
+   private void downloadLog(Node node, long bytes, boolean failed, long began) throws IOException
+   {
+      try
+      {
+         audit.insert(FileGatewayTables.TRANSFER_LOG, new QRecord().withValue("accountId", session.account.getValue("id"))
+            .withValue("connectionLogId", session.connectionId).withValue("operation", "DOWNLOAD").withValue("path", node.path())
+            .withValue("objectKind", "SIDECAR".equals(node.kind()) ? "RESULT" : node.kind()).withValue("objectId", node.objectId())
+            .withValue("uploadJobId", node.job().getValue("id")).withValue("bytes", bytes)
+            .withValue("sha256", "ARCHIVE".equals(node.kind()) ? node.job().getValue("sha256") : null)
+            .withValue("durationMs", (System.nanoTime() - began) / 1_000_000).withValue("outcome", failed ? "ERROR" : "OK")
+            .withValue("clientChannel", "sftp/" + session.id));
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Unable to persist download audit", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Acknowledge visible job artifacts; archive removal remains denied.
+    *******************************************************************************/
+   public void remove(String path) throws IOException
+   {
+      Node node = stat(path);
+      if(node.job() == null || !List.of("SIDECAR", "RESULT").contains(node.kind()))
+      {
+         deny(path, "REMOVE", "Only sidecars and results can be acknowledged");
+      }
+      uploads.acknowledge(node.job());
+   }
+
+
+
+   /*******************************************************************************
+    ** Only completed temporary uploads may be renamed within the same visible folder.
+    *******************************************************************************/
+   public void rename(String source, String target) throws IOException
+   {
+      Node node = stat(source);
+      if(!"TEMPORARY".equals(node.kind()))
+      {
+         deny(source, "RENAME", "Only temporary uploads can be renamed");
+      }
+      stat(target.substring(0, target.lastIndexOf('/')));
+      uploads.rename(session.account.getValueInteger("id"), source, target);
+   }
+
+
+
+   /*******************************************************************************
+    ** Expose only app and readable table nodes.
+    *******************************************************************************/
+   private void addAppChildren(List<Node> result, String prefix, JsonNode children, JsonNode meta)
+   {
+      for(JsonNode child : children)
+      {
+         String type = child.path("type").asText().toUpperCase(Locale.ROOT);
+         String name = child.path("name").asText();
+         if(("APP".equals(type) || (!prefix.isEmpty() && "TABLE".equals(type) && meta.path("tables").has(name)))
+            && !name.isEmpty() && !name.contains("/") && !name.equals(".") && !name.equals(".."))
+         {
+            result.add(new Node(prefix + "/" + name, true, type, name));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Map virtual folders to persisted audit kinds.
+    *******************************************************************************/
+   private String folderKind(String folder)
+   {
+      return (switch(folder)
+      {
+         case "upload" -> "PROFILE";
+         case "views" -> "VIEW";
+         case "reports" -> "REPORT";
+         case "results" -> "RESULT";
+         default -> "ARCHIVE";
+      });
+   }
+
+
+
+   /*******************************************************************************
+    ** Stable lowercase label slug, at most eighty characters before a collision suffix.
+    *******************************************************************************/
+   public static String slug(String label)
+   {
+      String value = label.toLowerCase(Locale.ROOT).replaceAll("[^a-z0-9]+", "-").replaceAll("^-|-$", "");
+      if(value.length() > 80)
+      {
+         value = value.substring(0, 80);
+      }
+      return (value.isEmpty() ? "item" : value);
+   }
+
+
+
+   /*******************************************************************************
+    ** Audit and deny unsupported operations.
+    *******************************************************************************/
+   public void deny(String path, String operation, String message) throws IOException
+   {
+      session.operationAllowed();
+      log(path, operation, "DENIED", message, System.nanoTime());
+      throw (new AccessDeniedException(path, null, message));
+   }
+
+
+
+   /*******************************************************************************
+    ** Persist the operation through the system audit writer.
+    *******************************************************************************/
+   private void log(String path, String operation, String outcome, String message, long started) throws IOException
+   {
+      String kind = "TABLE";
+      String id = null;
+      try
+      {
+         Node node = stat(path);
+         kind = List.of("SIDECAR", "TEMPORARY").contains(node.kind()) ? "RESULT" : node.kind();
+         id = node.objectId();
+      }
+      catch(IOException ignored)
+      {
+         // Missing targets have no object id.
+      }
+      try
+      {
+         audit.insert(FileGatewayTables.TRANSFER_LOG, new QRecord().withValue("accountId", session.account.getValue("id"))
+            .withValue("connectionLogId", session.connectionId).withValue("operation", operation).withValue("path", path)
+            .withValue("objectKind", kind).withValue("objectId", id).withValue("bytes", 0L)
+            .withValue("durationMs", (System.nanoTime() - started) / 1_000_000).withValue("outcome", outcome)
+            .withValue("message", message).withValue("clientChannel", "sftp/" + session.id));
+      }
+      catch(Exception e)
+      {
+         throw (new IOException("Unable to persist operation audit", e));
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/ApiContractTest.java

@@ -0,0 +1,136 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.io.ByteArrayInputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.time.Duration;
+import java.time.Instant;
+import com.kingsrook.qbits.filegateway.api.FileAcceptance;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumer;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumersMetaData;
+import com.kingsrook.qbits.filegateway.api.FileGatewayOutbox;
+import com.kingsrook.qbits.filegateway.api.FileGatewayUnavailableException;
+import com.kingsrook.qbits.filegateway.api.InboundFile;
+import com.kingsrook.qbits.filegateway.api.OutboxFileRef;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Independently usable consumer and outbox contracts.
+ *******************************************************************************/
+class ApiContractTest
+{
+   /*******************************************************************************
+    ** The public API may depend on core, but never gateway implementation types.
+    *******************************************************************************/
+   @Test
+   void apiImportsAreIsolated() throws Exception
+   {
+      try(var sources = Files.list(Path.of("src/main/java/com/kingsrook/qbits/filegateway/api")))
+      {
+         for(Path source : sources.filter(p -> p.toString().endsWith(".java")).toList())
+         {
+            String text = Files.readString(source);
+            assertThat(text).as(source.toString()).doesNotContainPattern("com\\.kingsrook\\.qbits\\.filegateway\\.(?!api[.;])");
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Fresh streams, immutable decisions, consumer registration and provider dispatch.
+    *******************************************************************************/
+   @Test
+   void consumerAndOutboxContract() throws Exception
+   {
+      var file = new InboundFile(1, "orders", "alice", "A", "127.0.0.1", "/orders/a.csv", "a.csv", 3, "sha", Instant.EPOCH,
+         "bucket", "key", () -> new ByteArrayInputStream(new byte[] {1, 2, 3}));
+      assertThat(file.openStream()).isNotSameAs(file.openStream());
+      assertThat(file.openStream().readAllBytes()).containsExactly(1, 2, 3);
+      assertThat(file.remoteAddress()).isEqualTo("127.0.0.1");
+      FileGatewayConsumer consumer = new FileGatewayConsumer()
+      {
+         /*******************************************************************************
+          ** Get name.
+          *******************************************************************************/
+         public String getName()
+         {
+            return ("test");
+         }
+
+
+
+         /*******************************************************************************
+          ** Accept.
+          *******************************************************************************/
+         public FileAcceptance accept(InboundFile inbound)
+         {
+            return (FileAcceptance.accepted(inbound.sha256()));
+         }
+      };
+      assertThat(consumer.getName()).isEqualTo("test");
+      var accepted = consumer.accept(file);
+      assertThat(accepted.getStatus()).isEqualTo(FileAcceptance.Status.ACCEPTED);
+      assertThat(accepted.getConsumerReference()).isEqualTo("sha");
+      assertThat(accepted.getReason()).isNull();
+      var rejected = FileAcceptance.rejected("invalid");
+      assertThat(rejected.getStatus()).isEqualTo(FileAcceptance.Status.REJECTED);
+      assertThat(rejected.getReason()).isEqualTo("invalid");
+      var retry = FileAcceptance.retryLater("busy", Duration.ofMinutes(1));
+      assertThat(retry.getStatus()).isEqualTo(FileAcceptance.Status.RETRY_LATER);
+      assertThat(retry.getRetryAfter()).isEqualTo(Duration.ofMinutes(1));
+      var ref = new OutboxFileRef(2, "orders", "/orders/b.csv", "b.csv", 3, "sha", Instant.EPOCH);
+      consumer.onDownloaded(ref);
+      FileGatewayOutbox.setProvider(null);
+      assertThat(FileGatewayOutbox.isAvailable()).isFalse();
+      assertThatThrownBy(() -> FileGatewayOutbox.put("orders", "b.csv", file.openStream(), 3)).isInstanceOf(FileGatewayUnavailableException.class);
+      try
+      {
+         FileGatewayOutbox.setProvider((endpoint, name, stream, size) ->
+         {
+            assertThat(endpoint).isEqualTo("orders");
+            assertThat(name).isEqualTo("b.csv");
+            assertThat(size).isEqualTo(3);
+            return (ref);
+         });
+         assertThat(FileGatewayOutbox.isAvailable()).isTrue();
+         assertThat(FileGatewayOutbox.put("orders", "b.csv", file.openStream(), 3)).isSameAs(ref);
+      }
+      finally
+      {
+         FileGatewayOutbox.setProvider(null);
+      }
+      var instance = new QInstance();
+      var registry = FileGatewayConsumersMetaData.ofOrWithNew(instance);
+      var code = new QCodeReference(consumer.getClass());
+      assertThat(registry.withConsumer(code)).isSameAs(registry);
+      assertThat(registry.getName()).isEqualTo("fileGatewayConsumers");
+      assertThat(FileGatewayConsumersMetaData.ofOrWithNew(instance)).isSameAs(registry);
+      assertThat(registry.getConsumers()).containsExactly(code);
+      assertThatThrownBy(() -> registry.getConsumers().clear()).isInstanceOf(UnsupportedOperationException.class);
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/CidrMatcherTest.java

@@ -0,0 +1,59 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import com.kingsrook.qbits.filegateway.sftp.CidrMatcher;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** IPv4 and IPv6 boundaries, multiple networks and numeric-only fail-closed parsing.
+ *******************************************************************************/
+class CidrMatcherTest
+{
+   /*******************************************************************************
+    ** Networks never require hostname resolution or accept invalid prefix lengths.
+    *******************************************************************************/
+   @ParameterizedTest
+   @CsvSource({
+      "192.0.2.255,192.0.2.0/24,true",
+      "192.0.3.0,192.0.2.0/24,false",
+      "127.0.0.1,0.0.0.0/0,true",
+      "127.0.0.1,127.0.0.1/32,true",
+      "127.0.0.1,127.0.0.1/33,false",
+      "127.0.0.1,127.0.0.1/-1,false",
+      "127.0.0.1,deadbeef/32,false",
+      "127.0.0.1,127.0.0/24,false",
+      "127.0.0.1,'192.0.2.0/24,127.0.0.0/8',true",
+      "2001:db8::ffff,2001:db8::/32,true",
+      "2001:db9::1,2001:db8::/32,false",
+      "::1,::1/128,true",
+      "::1,::1/129,false"
+   })
+   /*******************************************************************************
+    ** Matches numeric networks.
+    *******************************************************************************/
+   void matchesNumericNetworks(String address, String cidrs, boolean allowed)
+   {
+      assertThat(CidrMatcher.matches(address, cidrs)).isEqualTo(allowed);
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/EntityContractTest.java

@@ -0,0 +1,99 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.lang.reflect.Modifier;
+import java.time.Instant;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** The stable schema and record entities must round-trip every persisted field.
+ *******************************************************************************/
+class EntityContractTest
+{
+   /*******************************************************************************
+    ** Verify annotations, metadata completeness and fluent accessors against QRecord conversion.
+    *******************************************************************************/
+   @ParameterizedTest
+   @ValueSource(strings = {"Account", "SshKey", "Alias", "UploadJob", "OutboxFile", "TransferLog", "ConnectionLog"})
+   void entityRoundTrip(String suffix) throws Exception
+   {
+      Class<?> type = Class.forName("com.kingsrook.qbits.filegateway.model.FileGateway" + suffix);
+      var entity = (QRecordEntity) type.getConstructor().newInstance();
+      String tableName = (String) type.getField("TABLE_NAME").get(null);
+      var table = FileGatewayTables.tables(new FileGatewayQBitConfig()).stream().filter(t -> t.getName().equals(tableName)).findFirst().orElseThrow();
+      for(var field : type.getDeclaredFields())
+      {
+         if(Modifier.isStatic(field.getModifiers()))
+         {
+            continue;
+         }
+         assertThat(field.getAnnotation(QField.class)).isNotNull();
+         assertThat(table.getField(field.getName())).as(field.getName()).isNotNull();
+         Object value = field.getType() == String.class ? "value-" + field.getName() : field.getType() == Instant.class ? Instant.parse("2026-10-13T12:34:56Z")
+            : field.getType() == Boolean.class ? true : field.getType() == Long.class ? (Object) 12345678901L : 17;
+         String capitalized = Character.toUpperCase(field.getName().charAt(0)) + field.getName().substring(1);
+         assertThat(type.getMethod("with" + capitalized, field.getType()).invoke(entity, value)).isSameAs(entity);
+         assertThat(type.getMethod("get" + capitalized).invoke(entity)).isEqualTo(value);
+      }
+      QRecord record = entity.toQRecord();
+      var restored = (QRecordEntity) type.getConstructor(QRecord.class).newInstance(record);
+      assertThat(restored.toQRecord().getValues()).containsAllEntriesOf(record.getValues());
+      assertThat(record.getValues().keySet()).containsExactlyInAnyOrderElementsOf(table.getFields().keySet());
+   }
+
+
+
+   /*******************************************************************************
+    ** All documented defaults are available before any runtime has started.
+    *******************************************************************************/
+   @Test
+   void configurationDefaultsAndFluentOptions() throws Exception
+   {
+      var config = new FileGatewayQBitConfig();
+      assertThat(config.getSftpEnabled()).isFalse();
+      assertThat(config.getSftpPort()).isEqualTo(2222);
+      assertThat(config.getAllowGeneratedHostKey()).isFalse();
+      assertThat(config.getHostKeyEnvVarName()).isEqualTo("FILE_GATEWAY_SSH_HOST_KEY");
+      assertThat(config.getDefaultMaxConcurrentSessions()).isEqualTo(4);
+      assertThat(config.getDefaultMaxOperationsPerMinute()).isEqualTo(600);
+      assertThat(config.getLockoutThreshold()).isEqualTo(5);
+      assertThat(config.getTemporaryUploadSuffixes()).containsExactly(".filepart", ".tmp", ".part", ".partial");
+      for(var method : FileGatewayQBitConfig.class.getDeclaredMethods())
+      {
+         if(!method.getName().startsWith("with"))
+         {
+            continue;
+         }
+         var getter = FileGatewayQBitConfig.class.getMethod("get" + method.getName().substring(4));
+         Object value = getter.invoke(config);
+         assertThat(method.invoke(config, value)).isSameAs(config);
+         assertThat(getter.invoke(config)).isEqualTo(value);
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducerTest.java

@@ -20,74 +20,96 @@
 
 package com.kingsrook.qbits.filegateway;
 
-
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+import java.util.List;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.ClientChannelsMetaData;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
 import static org.assertj.core.api.Assertions.assertThat;
 
-
 /*******************************************************************************
- ** Verify the scaffold's registration and configuration contract.
+ ** Registration, channel walls and contingent endpoint pickers.
  *******************************************************************************/
 class FileGatewayQBitProducerTest
 {
    /*******************************************************************************
-    ** Produce the identity and exact config into an otherwise empty instance.
+    ** All stable metadata belongs to one configured qbit.
     *******************************************************************************/
    @Test
-   void producesQBitWithoutTables()
+   void registersTablesAppPickersAndWalls() throws Exception
    {
-      QInstance instance = new QInstance();
-      FileGatewayQBitConfig config = new FileGatewayQBitConfig();
-      FileGatewayQBitProducer producer = new FileGatewayQBitProducer().withQBitConfig(config);
-      MetaDataProducerMultiOutput output = producer.produce(instance);
-      QBitMetaData qBit = instance.getQBits().get("com.kingsrook.qbits:qbit-file-gateway");
-
-      assertThat(qBit).isNotNull();
-      assertThat(qBit.getGroupId()).isEqualTo("com.kingsrook.qbits");
-      assertThat(qBit.getArtifactId()).isEqualTo("qbit-file-gateway");
-      assertThat(qBit.getVersion()).isEqualTo("0.1.0");
-      assertThat(qBit.getConfig()).isSameAs(config);
+      var instance = new QInstance();
+      var config = new FileGatewayQBitConfig().withBackendName("host-db").withTableMetaDataCustomizer((i, t) -> t.withLabel("Custom " + t.getLabel()));
+      var producer = new FileGatewayQBitProducer().withQBitConfig(config).withNamespace("test");
+      producer.produce(instance).addSelfToInstance(instance);
+      assertThat(instance.getTables().keySet()).containsExactlyInAnyOrderElementsOf(FileGatewayTables.ALL);
+      assertThat(instance.getQBits().get("com.kingsrook.qbits:qbit-file-gateway:test").getConfig()).isSameAs(config);
       assertThat(producer.getQBitConfig()).isSameAs(config);
-      assertThat(producer.isEnabled()).isTrue();
-      assertThat(producer.getNamespace()).isNull();
-      assertThat(output.getEach(QBitMetaData.class)).containsExactly(qBit);
-      assertThat(output.getEach(QTableMetaData.class)).isEmpty();
-      assertThat(instance.getTables()).isEmpty();
-      assertThat(config.getSftpEnabled()).isFalse();
-      assertThat(config.getSftpPort()).isEqualTo(2222);
-      assertThat(config.getTableMetaDataCustomizer()).isNull();
+      assertThat(producer.getNamespace()).isEqualTo("test");
+      assertThat(instance.getApps()).containsOnlyKeys("file_gateway");
+      assertThat(instance.getApp("file_gateway").getSections().stream().flatMap(s -> s.getTables().stream()).toList())
+         .containsExactlyElementsOf(FileGatewayTables.MAIN);
+      assertThat(instance.getJoins()).hasSize(6);
+      assertThat(instance.getWidgets()).hasSize(6);
+      assertThat(instance.getPossibleValueSources()).hasSize(19);
+      instance.getTables().values().forEach(t ->
+      {
+         assertThat(t.getBackendName()).isEqualTo("host-db");
+         assertThat(t.getLabel()).startsWith("Custom ");
+         assertThat(t.getField("createDate").getBackendName()).isEqualTo("create_date");
+      });
+      var channels = ClientChannelsMetaData.ofOrWithNew(instance);
+      assertThat(channels.getChannels()).containsKey("sftp");
+      for(String name : List.of("file_gateway_account", "authentik_user", "audit", "api_key", "credential", "secret", "password", "permission", "role", "user_session", "webhooks_endpoint"))
+      {
+         assertThat(channels.isDenied("sftp", name, null)).as(name).isTrue();
+      }
+      assertThat(channels.isDenied("assistant", "file_gateway_ssh_key", null)).isTrue();
+      assertThat(channels.isDenied("sftp", "orders", null)).isFalse();
+      assertThat(instance.getTable(FileGatewayTables.ALIAS).getField("tableName").getPossibleValueSourceName()).isEqualTo("tables");
    }
 
 
 
    /*******************************************************************************
-    ** Configured instances may be namespaced; unconfigured scanning is disabled.
+    ** Package scanning never enables an unconfigured producer.
     *******************************************************************************/
    @Test
-   void supportsNamespacedConfiguration()
+   void disabledWithoutConfig() throws Exception
    {
-      assertThat(new FileGatewayQBitProducer().isEnabled()).isFalse();
-      MetaDataCustomizerInterface<QTableMetaData> customizer = (instance, table) -> table;
-      FileGatewayQBitConfig config = new FileGatewayQBitConfig()
-         .withTableMetaDataCustomizer(customizer)
-         .withSftpEnabled(true)
-         .withSftpPort(2022);
-      FileGatewayQBitProducer producer = new FileGatewayQBitProducer()
-         .withQBitConfig(config)
-         .withNamespace("test");
-      QInstance instance = new QInstance();
-      producer.produce(instance);
-
-      assertThat(producer.getNamespace()).isEqualTo("test");
-      assertThat(instance.getQBits().get("com.kingsrook.qbits:qbit-file-gateway:test").getConfig()).isSameAs(config);
-      assertThat(config.getTableMetaDataCustomizer()).isSameAs(customizer);
-      assertThat(config.getSftpEnabled()).isTrue();
-      assertThat(config.getSftpPort()).isEqualTo(2022);
+      var instance = new QInstance();
+      var producer = new FileGatewayQBitProducer();
+      assertThat(producer.isEnabled()).isFalse();
+      producer.produce(instance).addSelfToInstance(instance);
       assertThat(instance.getTables()).isEmpty();
+      assertThat(instance.getQBits()).isEmpty();
+      assertThat(FileGatewayRuntime.start(instance, null)).isNull();
+      new FileGatewayQBitProducer().withQBitConfig(new FileGatewayQBitConfig()).produce(instance);
+      assertThat(FileGatewayRuntime.start(instance, null, name -> null)).isNull();
+   }
+
+
+
+   /*******************************************************************************
+    ** Each dependent endpoint field has exactly the specified input criterion.
+    *******************************************************************************/
+   @ParameterizedTest
+   @ValueSource(strings = {"savedBulkLoadProfile", "savedView", "savedReport"})
+   void contingentPickerMetadata(String source) throws Exception
+   {
+      var instance = new QInstance();
+      new FileGatewayQBitProducer().withQBitConfig(new FileGatewayQBitConfig()).produce(instance);
+      var field = instance.getTable(FileGatewayTables.ALIAS).getField(source + "Id");
+      assertThat(field.getPossibleValueSourceName()).isEqualTo(source);
+      var filter = field.getPossibleValueSourceFilter();
+      assertThat(filter.getCriteria()).singleElement().satisfies(c ->
+      {
+         assertThat(c.getFieldName()).isEqualTo("tableName");
+         assertThat(c.getOperator().name()).isEqualTo("EQUALS");
+         assertThat(c.getValues()).containsExactly("${input.tableName}");
+      });
+      assertThat(filter.getSubFilters()).isNullOrEmpty();
    }
 }

src/test/java/com/kingsrook/qbits/filegateway/FileGatewayTestServer.java

@@ -0,0 +1,319 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+
+import java.io.Serializable;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.security.KeyPair;
+import java.sql.Connection;
+import java.time.Clock;
+import java.time.Duration;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import com.kingsrook.qbits.filegateway.sftp.AuditWriter;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
+import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.TablesPossibleValueSourceMetaDataProvider;
+import com.kingsrook.qqq.backend.core.model.savedbulkloadprofiles.SavedBulkLoadProfileMetaDataProvider;
+import com.kingsrook.qqq.backend.core.model.savedreports.SavedReportsMetaDataProvider;
+import com.kingsrook.qqq.backend.core.model.savedviews.SavedViewsMetaDataProvider;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
+import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
+import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
+import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1;
+import org.apache.sshd.client.SshClient;
+import org.apache.sshd.client.keyverifier.AcceptAllServerKeyVerifier;
+import org.apache.sshd.client.session.ClientSession;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.config.keys.PublicKeyEntry;
+
+
+/*******************************************************************************
+ ** Real strict UI API server, in-memory H2 and real MINA clients for gateway acceptance tests.
+ *******************************************************************************/
+public class FileGatewayTestServer implements AutoCloseable
+{
+   public final MemoryGatewayObjectStore store = new MemoryGatewayObjectStore();
+   public final QInstance instance = new QInstance();
+   public final FileGatewayQBitConfig config = new FileGatewayQBitConfig().withSftpEnabled(true).withSftpPort(0);
+   public final AuditWriter audit = new AuditWriter(instance);
+   public final QApplicationJavalinServer http;
+   public final FileGatewayRuntime runtime;
+   public final SshClient client;
+   public final java.util.List<ApiCall> calls = new java.util.concurrent.CopyOnWriteArrayList<>();
+
+   /*******************************************************************************
+    ** Non-secret request evidence for the invoker contract assertions.
+    *******************************************************************************/
+   public record ApiCall(String method, String path, String channel, String invokedBy)
+   {
+   }
+   private final Connection keeper;
+   public static final Map<String, Set<String>> GRANTS = new HashMap<>();
+
+
+
+   /*******************************************************************************
+    ** Start the complete in-memory test stack.
+    *******************************************************************************/
+   public FileGatewayTestServer() throws Exception
+   {
+      this(Clock.systemUTC());
+   }
+
+
+
+   /*******************************************************************************
+    ** Controllable time for expiry, session refresh and metadata caching.
+    *******************************************************************************/
+   public FileGatewayTestServer(Clock clock) throws Exception
+   {
+      ConnectionManager.resetConnectionProviders();
+      GRANTS.clear();
+      GRANTS.put("A", Set.of("file_gateway.connect", "alpha.read", "alpha.insert", "beta.read", "beta.insert"));
+      GRANTS.put("B", Set.of("file_gateway.connect", "beta.read", "beta.insert"));
+      GRANTS.put("NONE", Set.of("beta.read"));
+      instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("test").withType(QAuthenticationType.MOCK)
+         .withCustomizer(new QCodeReference(TestAuthentication.class)));
+      RDBMSBackendMetaData backend = new RDBMSBackendMetaData().withName("rdbms").withVendor("h2").withHostName("mem")
+         .withDatabaseName("fg" + UUID.randomUUID().toString().replace("-", "")).withUsername("sa");
+      backend.setJdbcUrl(ConnectionManager.getJdbcUrl(backend) + ";DATABASE_TO_UPPER=FALSE");
+      instance.addBackend(backend);
+      keeper = new ConnectionManager().getConnection(backend);
+      for(String sql : Files.readString(Path.of("docs/ddl/file_gateway.sql")).split(";"))
+      {
+         if(!sql.isBlank())
+         {
+            keeper.createStatement().execute(sql);
+         }
+      }
+      for(String name : List.of("alpha", "beta"))
+      {
+         instance.addTable(new QTableMetaData().withName(name).withLabel(name).withBackendName("rdbms")
+            .withPrimaryKeyField("id").withField(new QFieldMetaData("id", QFieldType.INTEGER))
+            .withField(new QFieldMetaData("name", QFieldType.STRING).withIsRequired(true))
+            .withField(new QFieldMetaData("quantity", QFieldType.INTEGER))
+            .withSection(SectionFactory.defaultT1("id", "name", "quantity"))
+            .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.READ_INSERT_EDIT_DELETE_PERMISSIONS).withPermissionBaseName(name)));
+      }
+      instance.addApp(new QAppMetaData().withName("testapp").withLabel("Test App")
+         .withChild(instance.getTable("alpha")).withChild(instance.getTable("beta"))
+         .withSection(new QAppSection().withName("data").withLabel("Data").withTables(List.of("alpha", "beta"))));
+      instance.addPossibleValueSource(TablesPossibleValueSourceMetaDataProvider.defineTablesPossibleValueSource(instance));
+      new SavedBulkLoadProfileMetaDataProvider().defineAll(instance, "rdbms", null);
+      new SavedViewsMetaDataProvider().defineAll(instance, "rdbms", null);
+      new SavedReportsMetaDataProvider().defineAll(instance, "rdbms", "rdbms", null);
+      new FileGatewayQBitProducer().withQBitConfig(config).produce(instance);
+      new com.kingsrook.qqq.backend.core.model.audits.AuditsMetaDataProvider().withClientChannelField(true).defineAll(instance, "rdbms", null);
+      instance.getTable("alpha").withAuditRules(new com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules()
+         .withAuditLevel(com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel.RECORD));
+      TestAlertSink.notifications.clear();
+      com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData.ofOrWithNew(instance).withSink(new QCodeReference(TestAlertSink.class));
+      for(QTableMetaData table : instance.getTables().values())
+      {
+         if(!table.getName().startsWith("file_gateway_"))
+         {
+            table.withBackendDetails(new RDBMSTableBackendDetails().withTableName(table.getName()));
+            for(QFieldMetaData field : table.getFields().values())
+            {
+               field.setBackendName(QInstanceEnricher.inferBackendName(field.getName()));
+            }
+            createTable(table);
+         }
+      }
+      instance.addBackend(new com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData()
+         .withName("memory").withBackendType(com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule.class));
+      instance.addTable(new QTableMetaData().withName("testUploadArchive").withBackendName("memory").withPrimaryKeyField("reference")
+         .withField(new QFieldMetaData("reference", QFieldType.STRING)).withField(new QFieldMetaData("contents", QFieldType.BLOB)));
+      instance.withSupplementalMetaData(new com.kingsrook.qqq.middleware.javalin.QJavalinMetaData().withUploadedFileArchiveTableName("testUploadArchive"));
+      new QInstanceEnricher(instance).enrich();
+      new QInstanceValidator().validate(instance);
+      QContext.init(instance, new QSystemUserSession(instance));
+      http = new QApplicationJavalinServer(new AbstractQQQApplication()
+      {
+         /*******************************************************************************
+          ** Define q instance.
+          *******************************************************************************/
+         @Override
+         public QInstance defineQInstance()
+         {
+            return (instance);
+         }
+      }).withPort(0).withServeFrontendMaterialDashboard(false).withServeFrontendNext(false)
+         .withJavalinConfigCustomizer(j -> j.routes.before(ctx -> calls.add(new ApiCall(ctx.method().name(), ctx.path(), ctx.header("X-QQQ-Client"), ctx.header("X-QQQ-Invoked-By")))))
+         .withStrictUiApiRoutes(true).withMiddlewareVersionList(List.of(new MiddlewareVersionV1()));
+      http.start();
+      runtime = FileGatewayRuntime.startForTest(instance, http, config, KeyUtils.generateKeyPair("ecdsa-sha2-nistp256", 256), clock, Duration.ofHours(1), store);
+      client = SshClient.setUpDefaultClient();
+      client.setServerKeyVerifier(AcceptAllServerKeyVerifier.INSTANCE);
+      client.start();
+   }
+
+
+
+   /*******************************************************************************
+    ** Create core test tables using their actual metadata.
+    *******************************************************************************/
+   private void createTable(QTableMetaData table) throws Exception
+   {
+      StringBuilder sql = new StringBuilder("CREATE TABLE ").append(table.getName()).append(" (");
+      boolean first = true;
+      for(QFieldMetaData field : table.getFields().values())
+      {
+         if(!first)
+         {
+            sql.append(", ");
+         }
+         first = false;
+         String type = switch(field.getType())
+         {
+            case INTEGER -> "INTEGER";
+            case LONG -> "BIGINT";
+            case DECIMAL -> "DECIMAL";
+            case BOOLEAN -> "BOOLEAN";
+            case DATE_TIME -> "TIMESTAMP";
+            case DATE -> "DATE";
+            case TIME -> "TIME";
+            case BLOB -> "BINARY LARGE OBJECT";
+            default -> "VARCHAR(65535)";
+         };
+         sql.append(field.getBackendName()).append(" ").append(type);
+         if(field.getName().equals(table.getPrimaryKeyField()) && (field.getType() == QFieldType.INTEGER || field.getType() == QFieldType.LONG))
+         {
+            sql.append(" GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY");
+         }
+      }
+      keeper.createStatement().execute(sql.append(")").toString());
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert an active account with schema defaults.
+    *******************************************************************************/
+   public QRecord account(String username, String userId) throws Exception
+   {
+      return (audit.insert("file_gateway_account", new QRecord().withValue("username", username).withValue("label", username).withValue("userId", userId)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert a generated key through the real validation customizer.
+    *******************************************************************************/
+   public QRecord key(QRecord account, KeyPair pair) throws Exception
+   {
+      return (audit.insert("file_gateway_ssh_key", new QRecord().withValue("accountId", account.getValue("id"))
+         .withValue("label", "Test key").withValue("publicKey", PublicKeyEntry.toString(pair.getPublic()))));
+   }
+
+
+
+   /*******************************************************************************
+    ** Authenticate a real MINA SSH client.
+    *******************************************************************************/
+   public ClientSession login(String username, KeyPair key) throws Exception
+   {
+      ClientSession session = client.connect(username, "127.0.0.1", runtime.getProtocol().getPort()).verify(Duration.ofSeconds(10)).getSession();
+      session.addPublicKeyIdentity(key);
+      try
+      {
+         session.auth().verify(Duration.ofSeconds(10));
+         return (session);
+      }
+      catch(Exception e)
+      {
+         session.close(true).await(Duration.ofSeconds(5));
+         throw (e);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Seed visible core saved-object rows under their owner.
+    *******************************************************************************/
+   public QRecord saved(String source, String table, String label) throws Exception
+   {
+      return (audit.system(() -> new InsertAction().execute(new InsertInput(source).withRecord(new QRecord()
+         .withValue("tableName", table).withValue("label", label).withValue("userId", "A").withValue("viewJson", "{}")
+         .withValue("mappingJson", "{}").withValue("columnsJson", "{\"columns\":[{\"name\":\"id\"}]}"))).getRecords().get(0)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Close sessions and all services.
+    *******************************************************************************/
+   @Override
+   public void close() throws Exception
+   {
+      client.stop();
+      runtime.stop();
+      http.stop();
+      keeper.close();
+      QContext.clear();
+   }
+
+
+
+   /*******************************************************************************
+    ** MOCK automation permissions differ by user id.
+    *******************************************************************************/
+   public static class TestAuthentication implements QAuthenticationModuleCustomizerInterface
+   {
+      /*******************************************************************************
+       ** Customize automated session for user.
+       *******************************************************************************/
+      @Override
+      public void customizeAutomatedSessionForUser(QInstance instance, QSession session, Serializable userId)
+      {
+         session.getUser().setFullName("Test user " + userId);
+         session.setPermissions(GRANTS.getOrDefault(userId.toString(), Set.of()));
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/GatewayLifecycleTest.java

@@ -0,0 +1,464 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.AccessDeniedException;
+import java.nio.file.Files;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Executors;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.sftp.GatewaySession;
+import com.kingsrook.qbits.filegateway.vfs.GatewayFileSystem;
+import com.kingsrook.qbits.filegateway.vfs.VirtualTree;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
+import org.apache.sshd.client.session.ClientSession;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.config.keys.writer.openssh.OpenSSHKeyPairResourceWriter;
+import org.apache.sshd.sftp.client.SftpClient;
+import org.apache.sshd.sftp.client.SftpClientFactory;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Time, multiple gateway listeners, runtime configuration and NIO boundaries.
+ *******************************************************************************/
+@ExtendWith(NoDiskExtension.class)
+class GatewayLifecycleTest
+{
+   /*******************************************************************************
+    ** Advance time deterministically without changing the machine clock.
+    *******************************************************************************/
+   static class MutableClock extends Clock
+   {
+      private Instant now = Instant.now();
+      /*******************************************************************************
+       ** Get zone.
+       *******************************************************************************/
+      public ZoneId getZone()
+      {
+         return (ZoneOffset.UTC);
+      }
+
+
+
+      /*******************************************************************************
+       ** With zone.
+       *******************************************************************************/
+      public Clock withZone(ZoneId zone)
+      {
+         return (this);
+      }
+
+
+
+      /*******************************************************************************
+       ** Instant.
+       *******************************************************************************/
+      public Instant instant()
+      {
+         return (now);
+      }
+
+
+
+      /*******************************************************************************
+       ** Advance.
+       *******************************************************************************/
+      void advance(Duration duration)
+      {
+         now = now.plus(duration);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Server-side session for assertions without exposing credentials in logs.
+    *******************************************************************************/
+   private GatewaySession state(FileGatewayTestServer h)
+   {
+      return (h.runtime.getProtocol().getServer().getActiveSessions().iterator().next().getAttribute(GatewaySession.ATTRIBUTE));
+   }
+
+
+
+   /*******************************************************************************
+    ** Read names while retaining the actual opendir protocol path.
+    *******************************************************************************/
+   private List<String> names(SftpClient client, String path) throws Exception
+   {
+      var names = new ArrayList<String>();
+      for(var entry : client.readDir(path))
+      {
+         names.add(entry.getFilename());
+      }
+      return (names);
+   }
+
+
+
+   /*******************************************************************************
+    ** Re-mint at five minutes, revoke the prior token, and enforce changed grants.
+    *******************************************************************************/
+   @Test
+   void refreshPermissionsAndCaches() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var session = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(session))
+         {
+            assertThat(names(sftp, "/testapp")).contains("alpha", "beta");
+            int count = h.calls.size();
+            names(sftp, "/testapp");
+            assertThat(h.calls).hasSize(count);
+            String token = state(h).token();
+            assertThat(UiApiInProcessSessions.resolve(token)).isPresent();
+            FileGatewayTestServer.GRANTS.put("A", Set.of("file_gateway.connect", "beta.read"));
+            clock.advance(Duration.ofMinutes(5));
+            assertThat(names(sftp, "/testapp")).doesNotContain("alpha").contains("beta");
+            assertThat(state(h).token().equals(token)).isFalse();
+            assertThat(UiApiInProcessSessions.resolve(token)).isEmpty();
+            FileGatewayTestServer.GRANTS.put("A", Set.of());
+            clock.advance(Duration.ofMinutes(5));
+            assertThatThrownBy(() -> names(sftp, "/")).isInstanceOf(Exception.class);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Each saved-object list is cached for thirty seconds and paged past 250 rows.
+    *******************************************************************************/
+   @Test
+   void savedObjectsPaginationAndSlugCollisions() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         h.saved("savedView", "alpha", "Duplicate!");
+         h.saved("savedView", "alpha", "Duplicate?");
+         try(var session = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(session))
+         {
+            assertThat(names(sftp, "/testapp/alpha/views")).contains("duplicate-1.csv", "duplicate-2.xlsx");
+            h.saved("savedView", "alpha", "Fresh");
+            assertThat(names(sftp, "/testapp/alpha/views")).doesNotContain("fresh.csv");
+            clock.advance(Duration.ofSeconds(31));
+            assertThat(names(sftp, "/testapp/alpha/views")).contains("fresh.csv");
+            var rows = new ArrayList<QRecord>();
+            for(int i = 0; i < 251; i++)
+            {
+               rows.add(new QRecord().withValue("tableName", "alpha").withValue("label", "Paged " + i).withValue("userId", "A").withValue("viewJson", "{}"));
+            }
+            h.audit.system(() -> new com.kingsrook.qqq.backend.core.actions.tables.InsertAction().execute(
+               new com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput("savedView").withRecords(rows)));
+            clock.advance(Duration.ofSeconds(31));
+            assertThat(names(sftp, "/testapp/alpha/views")).contains("paged-250.csv").hasSize(510);
+         }
+      }
+      assertThat(VirtualTree.slug(" !!! ")).isEqualTo("item");
+      assertThat(VirtualTree.slug("A".repeat(100))).hasSize(80);
+   }
+
+
+
+   /*******************************************************************************
+    ** Revocation covers accounts, keys and explicit session termination, with heartbeats.
+    *******************************************************************************/
+   @ParameterizedTest
+   @ValueSource(strings = {"disabled", "expired", "keyExpired", "end"})
+   void pollEndsRevokedSessions(String reason) throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         var pair = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         var key = h.key(account, pair);
+         try(var session = h.login("alice", pair))
+         {
+            var state = state(h);
+            String token = state.token();
+            clock.advance(Duration.ofSeconds(60));
+            h.runtime.getProtocol().getAuthenticator().poll();
+            var connection = h.audit.query(FileGatewayTables.CONNECTION_LOG, new QQueryFilter()).getFirst();
+            assertThat(connection.getValueInstant("lastSeenAt")).isAfter(connection.getValueInstant("connectedAt"));
+            switch(reason)
+            {
+               case "disabled" -> h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("status", "DISABLED"));
+               case "expired" -> h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("expiresAt", clock.instant().minusSeconds(1)));
+               case "keyExpired" -> h.audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id")).withValue("expiresAt", clock.instant().minusSeconds(1)));
+               case "end" ->
+               {
+                  com.kingsrook.qqq.backend.core.context.QContext.init(h.instance,
+                     new com.kingsrook.qqq.backend.core.model.session.QSession().withPermissions(Set.of("file_gateway.admin")));
+                  new com.kingsrook.qbits.filegateway.metadata.FileGatewayProcesses.Apply().runOnePage(
+                     new com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput()
+                        .withProcessName(com.kingsrook.qbits.filegateway.metadata.FileGatewayProcesses.END_SESSION).withRecords(List.of(connection)),
+                     new com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput());
+               }
+            }
+            h.runtime.getProtocol().getAuthenticator().poll();
+            session.waitFor(Set.of(ClientSession.ClientSessionEvent.CLOSED), Duration.ofSeconds(5));
+            assertThat(session.isClosed()).isTrue();
+            assertThat(UiApiInProcessSessions.resolve(token)).isEmpty();
+            assertThatThrownBy(state::token).isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QException.class);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Competing listeners share the account row lock, never exceeding the database limit.
+    *******************************************************************************/
+   @Test
+   void concurrentListenersAndRestartPreserveLockout() throws Exception
+   {
+      try(var h = new FileGatewayTestServer(); var second = FileGatewayRuntime.startForTest(h.instance, h.http,
+         h.config, KeyUtils.generateKeyPair("ecdsa-sha2-nistp256", 256), Clock.systemUTC(), Duration.ofHours(1));
+         var executor = Executors.newFixedThreadPool(2))
+      {
+         var pair = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         h.key(account, pair);
+         h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("maxConcurrentSessions", 1));
+         var start = new CountDownLatch(1);
+         var tasks = new ArrayList<java.util.concurrent.Future<ClientSession>>();
+         for(var runtime : List.of(h.runtime, second))
+         {
+            tasks.add(executor.submit(() ->
+            {
+               start.await();
+               var session = h.client.connect("alice", "127.0.0.1", runtime.getProtocol().getPort()).verify(Duration.ofSeconds(5)).getSession();
+               session.addPublicKeyIdentity(pair);
+               try
+               {
+                  session.auth().verify(Duration.ofSeconds(5));
+                  return (session);
+               }
+               catch(Exception e)
+               {
+                  session.close(true).await();
+                  return (null);
+               }
+            }));
+         }
+         start.countDown();
+         var sessions = new ArrayList<ClientSession>();
+         for(var task : tasks)
+         {
+            var session = task.get();
+            if(session != null)
+            {
+               sessions.add(session);
+            }
+         }
+         try
+         {
+            assertThat(sessions).hasSize(1);
+         }
+         finally
+         {
+            for(var session : sessions)
+            {
+               session.close(true).await();
+            }
+         }
+         assertThat(h.audit.query(FileGatewayTables.CONNECTION_LOG, new QQueryFilter())).anySatisfy(r -> assertThat(r.getValueString("outcome")).isEqualTo("LIMIT"));
+         h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("lockedUntil", Instant.now().plusSeconds(900)));
+         h.runtime.stop();
+         try(var session = h.client.connect("alice", "127.0.0.1", second.getProtocol().getPort()).verify(Duration.ofSeconds(5)).getSession())
+         {
+            session.addPublicKeyIdentity(pair);
+            assertThatThrownBy(() -> session.auth().verify(Duration.ofSeconds(5))).isInstanceOf(Exception.class);
+         }
+         assertThat(h.audit.query(FileGatewayTables.CONNECTION_LOG, new QQueryFilter())).anySatisfy(r -> assertThat(r.getValueString("outcome")).isEqualTo("LOCKED_OUT"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Connection limits apply before auth; operation limits reset on the next minute.
+    *******************************************************************************/
+   @Test
+   void rateAndOperationLimits() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         var pair = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         h.key(account, pair);
+         h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("maxOperationsPerMinute", 1));
+         try(var session = h.login("alice", pair); var sftp = SftpClientFactory.instance().createSftpClient(session))
+         {
+            names(sftp, "/");
+            try(var other = h.login("alice", pair); var otherSftp = SftpClientFactory.instance().createSftpClient(other))
+            {
+               assertThatThrownBy(() -> names(otherSftp, "/")).isInstanceOf(Exception.class);
+            }
+            assertThatThrownBy(() -> names(sftp, "/")).isInstanceOf(Exception.class);
+            clock.advance(Duration.ofSeconds(61));
+            assertThat(names(sftp, "/")).contains("testapp");
+         }
+         h.config.withMaxConnectionsPerIpPerMinute(1);
+         try(var first = h.login("alice", pair))
+         {
+            assertThatThrownBy(() -> h.login("alice", pair)).isInstanceOf(Exception.class);
+         }
+         assertThat(h.audit.query(FileGatewayTables.TRANSFER_LOG, new QQueryFilter())).anySatisfy(r -> assertThat(r.getValueString("outcome")).isEqualTo("LIMIT"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Rotating usernames cannot evade the per-IP lock, which expires independently.
+    *******************************************************************************/
+   @Test
+   void ipLockoutAcrossUsernames() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         h.config.withLockoutThreshold(3);
+         var good = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), good);
+         var bad = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         for(String username : List.of("unknown-one", "unknown-two", "unknown-three"))
+         {
+            assertThatThrownBy(() -> h.login(username, bad)).isInstanceOf(Exception.class);
+         }
+         assertThatThrownBy(() -> h.login("alice", good)).isInstanceOf(Exception.class);
+         assertThat(h.audit.query(FileGatewayTables.CONNECTION_LOG, new QQueryFilter())).anySatisfy(r -> assertThat(r.getValueString("outcome")).isEqualTo("LOCKED_OUT"));
+         clock.advance(Duration.ofMinutes(31));
+         try(var session = h.login("alice", good))
+         {
+            assertThat(session.isAuthenticated()).isTrue();
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Host secrets are parsed from memory and missing secrets fail startup.
+    *******************************************************************************/
+   @Test
+   void inMemoryHostKeys() throws Exception
+   {
+      var config = new FileGatewayQBitConfig().withHostKeyEnvVarName("TEST_HOST_KEY");
+      assertThatThrownBy(() -> FileGatewayRuntime.hostKey(config, name -> null)).isInstanceOf(IllegalStateException.class);
+      assertThat(FileGatewayRuntime.hostKey(config.withAllowGeneratedHostKey(true), name -> null)).isNotNull();
+      var pair = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+      var output = new ByteArrayOutputStream();
+      OpenSSHKeyPairResourceWriter.INSTANCE.writePrivateKey(pair, "test", null, output);
+      String pem = output.toString(StandardCharsets.UTF_8);
+      var parsed = FileGatewayRuntime.hostKey(config, name -> name.equals("TEST_HOST_KEY") ? pem : null);
+      assertThat(KeyUtils.compareKeyPairs(pair, parsed)).isTrue();
+      assertThatThrownBy(() -> FileGatewayRuntime.hostKey(config, name -> "invalid")).isInstanceOf(Exception.class);
+      try(var h = new FileGatewayTestServer())
+      {
+         h.runtime.stop();
+         h.config.withHostKeyEnvVarName("TEST_HOST_KEY");
+         try(var restarted = FileGatewayRuntime.start(h.instance, h.http, name -> name.equals("TEST_HOST_KEY") ? pem : "test-pod"))
+         {
+            assertThat(restarted.getProtocol().getPort()).isPositive();
+            assertThat(restarted.getProtocol().activeSessions()).isZero();
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** NIO paths never escape to native paths; phase-a mutations are always denied.
+    *******************************************************************************/
+   @Test
+   void virtualNioContract() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var pair = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), pair);
+         h.saved("savedView", "alpha", "Orders");
+         try(var session = h.login("alice", pair))
+         {
+            var state = state(h);
+            try(var fs = (GatewayFileSystem) h.runtime.getProtocol().getServer().getFileSystemFactory().createFileSystem(state.ssh))
+            {
+               var path = fs.getPath("/testapp/alpha/views/orders.csv");
+               assertThat(path.getFileName().toString()).isEqualTo("orders.csv");
+               assertThat(path.getNameCount()).isEqualTo(4);
+               assertThat(path.getName(1).toString()).isEqualTo("alpha");
+               assertThat(path.subpath(1, 3).toString()).isEqualTo("alpha/views");
+               assertThat(path.startsWith("/testapp")).isTrue();
+               assertThat(path.endsWith("views/orders.csv")).isTrue();
+               assertThat(fs.getPath("/testapp").relativize(path).toString()).isEqualTo("alpha/views/orders.csv");
+               assertThat(fs.getPath("../../testapp").toAbsolutePath().normalize().toString()).isEqualTo("/testapp");
+               assertThat(fs.getPath("/testapp", "alpha").resolve("views/orders.csv")).isEqualTo(path);
+               assertThat(path.toRealPath()).isEqualTo(path);
+               assertThat(path.hashCode()).isEqualTo(fs.getPath(path.toString()).hashCode());
+               assertThat(path.toUri().getScheme()).isEqualTo("gateway");
+               assertThat(path.iterator()).toIterable().hasSize(4);
+               assertThatThrownBy(path::toFile).isInstanceOf(UnsupportedOperationException.class);
+               assertThat(Files.readAttributes(path, BasicFileAttributes.class).isRegularFile()).isTrue();
+               assertThat(Files.readAttributes(fs.getPath("/"), BasicFileAttributes.class).isDirectory()).isTrue();
+               assertThatThrownBy(() -> Files.setAttribute(path, "basic:lastModifiedTime", java.nio.file.attribute.FileTime.from(Instant.EPOCH))).isInstanceOf(AccessDeniedException.class);
+               assertThatThrownBy(() -> Files.createSymbolicLink(fs.getPath("/link"), path)).isInstanceOf(AccessDeniedException.class);
+               assertThatThrownBy(() -> Files.createLink(fs.getPath("/link"), path)).isInstanceOf(AccessDeniedException.class);
+               assertThatThrownBy(() -> Files.copy(path, fs.getPath("/copy"))).isInstanceOf(AccessDeniedException.class);
+               assertThat(fs.getFileStores()).isEmpty();
+               assertThat(fs.getRootDirectories()).containsExactly(fs.getPath("/"));
+               assertThat(fs.isReadOnly()).isFalse();
+               assertThat(fs.getSeparator()).isEqualTo("/");
+               assertThatThrownBy(fs::newWatchService).isInstanceOf(UnsupportedOperationException.class);
+               assertThatThrownBy(fs::getUserPrincipalLookupService).isInstanceOf(UnsupportedOperationException.class);
+               assertThatThrownBy(() -> fs.getPathMatcher("glob:*")).isInstanceOf(UnsupportedOperationException.class);
+            }
+         }
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/MemoryGatewayObjectStore.java

@@ -0,0 +1,95 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.MessageDigest;
+import java.util.Arrays;
+import java.util.HexFormat;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import com.kingsrook.qbits.filegateway.storage.GatewayObjectStore;
+
+/*******************************************************************************
+ ** Test-only object storage.
+ *******************************************************************************/
+public class MemoryGatewayObjectStore implements GatewayObjectStore
+{
+   public final Map<String, byte[]> objects = new ConcurrentHashMap<>();
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public Receipt put(String key, InputStream content) throws IOException
+   {
+      try
+      {
+         byte[] bytes = content.readAllBytes();
+         objects.put(key, bytes);
+         return (new Receipt(bytes.length, HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes))));
+      }
+      catch(java.security.NoSuchAlgorithmException e)
+      {
+         throw new IOException(e);
+      }
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public InputStream open(String key) throws IOException
+   {
+      byte[] bytes = objects.get(key);
+      if(bytes == null)
+      {
+         throw (new IOException("Object missing"));
+      }
+      return (new ByteArrayInputStream(bytes));
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public InputStream openRange(String key, long start, long end) throws IOException
+   {
+      try(var in = open(key))
+      {
+         in.skipNBytes(start);
+         return (new ByteArrayInputStream(in.readNBytes((int) (end - start + 1))));
+      }
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void copy(String source, String destination) throws IOException
+   {
+      try(var in = open(source))
+      {
+         objects.put(destination, in.readAllBytes());
+      }
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void delete(String key)
+   {
+      objects.remove(key);
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/MetadataBehaviorTest.java

@@ -0,0 +1,173 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.time.Instant;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayProcesses;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
+import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissionChecker;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiCredentials;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.common.config.keys.PublicKeyEntry;
+import org.apache.sshd.common.digest.BuiltinDigests;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Real database customizers and strict UI API dependent pickers.
+ *******************************************************************************/
+@ExtendWith(NoDiskExtension.class)
+class MetadataBehaviorTest
+{
+   /*******************************************************************************
+    ** Every contingent picker excludes records for the other table via the UI API.
+    *******************************************************************************/
+   @ParameterizedTest
+   @ValueSource(strings = {"savedBulkLoadProfile", "savedView", "savedReport"})
+   void contingentPickerBehavior(String source) throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var own = h.saved(source, "alpha", "Alpha choice");
+         var other = h.saved(source, "beta", "Beta choice");
+         assertThat(own.getErrors()).isEmpty();
+         assertThat(other.getErrors()).isEmpty();
+         var session = new QSession().withPermissions(Set.of("file_gateway.admin"));
+         String token = UiApiInProcessSessions.register(session, java.time.Duration.ofMinutes(1));
+         try
+         {
+            var response = UiApiInvoker.forServer(h.http, "next-ui").invoke(new UiApiRequest().withMethod(HttpMethod.POST)
+               .withPath("/table/{tableName}/possibleValues/{fieldName}").withPathParam("tableName", FileGatewayTables.ALIAS)
+               .withPathParam("fieldName", source + "Id").withCredentials(UiApiCredentials.ofInProcessSession(token))
+               .withJsonBody(Map.of("values", Map.of("tableName", "alpha"))));
+            assertThat(response.getStatus()).as(response.getJson().toString()).isEqualTo(200);
+            assertThat(response.getJson().toString()).contains("Alpha choice").doesNotContain("Beta choice");
+         }
+         finally
+         {
+            UiApiInProcessSessions.revoke(token);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Validation runs on insert and partial update, computes metadata, and enforces uniqueness.
+    *******************************************************************************/
+   @Test
+   void publicKeyValidation() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var account = h.account("alice", "A");
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var stored = h.key(account, key);
+         assertThat(stored.getValueString("fingerprint")).isEqualTo(KeyUtils.getFingerPrint(BuiltinDigests.sha256, key.getPublic()));
+         assertThat(stored.getValueString("keyType")).isEqualTo("ssh-ed25519");
+         assertThat(stored.getValueInteger("keyBits")).isEqualTo(256);
+         assertThatThrownBy(() -> h.key(account, key)).isInstanceOf(QException.class);
+         for(String type : List.of("ssh-rsa", "ssh-dss"))
+         {
+            var weak = KeyUtils.generateKeyPair(type, type.equals("ssh-rsa") ? 2048 : 1024);
+            assertThatThrownBy(() -> h.key(account, weak)).isInstanceOf(QException.class);
+         }
+         assertThat(h.key(account, KeyUtils.generateKeyPair("ssh-rsa", 3072)).getValueInteger("keyBits")).isEqualTo(3072);
+         assertThat(h.key(account, KeyUtils.generateKeyPair("ecdsa-sha2-nistp384", 384)).getValueInteger("keyBits")).isEqualTo(384);
+         for(String text : List.of("not a key", "ssh-unknown invalid", ""))
+         {
+            assertThatThrownBy(() -> h.audit.insert(FileGatewayTables.SSH_KEY, new QRecord().withValue("accountId", account.getValue("id")).withValue("publicKey", text))).isInstanceOf(QException.class);
+         }
+         h.audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", stored.getValue("id")).withValue("label", "Renamed").withValue("fingerprint", "forged"));
+         var read = h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter()).getFirst();
+         assertThat(read.getValueString("fingerprint")).isEqualTo(stored.getValueString("fingerprint"));
+         var weak = KeyUtils.generateKeyPair("ssh-rsa", 2048);
+         assertThatThrownBy(() -> h.audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", stored.getValue("id")).withValue("publicKey", PublicKeyEntry.toString(weak.getPublic())))).isInstanceOf(QException.class);
+         assertThatThrownBy(() -> h.account("Bad name", "A")).isInstanceOf(QException.class);
+         assertThatThrownBy(() -> h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("maxConcurrentSessions", 0))).isInstanceOf(QException.class);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** People can read logs but cannot write them, including administrators.
+    *******************************************************************************/
+   @Test
+   void permissionsAndAdministrativeActions() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var account = h.account("alice", "A");
+         var key = h.key(account, KeyUtils.generateKeyPair("ssh-ed25519", 256));
+         var checker = new FileGatewayPermissionChecker();
+         QContext.init(h.instance, new QSession().withPermissions(Set.of("file_gateway.view")));
+         checker.checkPermissionsThrowing(new QueryInput(FileGatewayTables.ACCOUNT), h.instance.getTable(FileGatewayTables.ACCOUNT));
+         assertThatThrownBy(() -> checker.checkPermissionsThrowing(new UpdateInput(FileGatewayTables.ACCOUNT), h.instance.getTable(FileGatewayTables.ACCOUNT))).isInstanceOf(QException.class);
+         QContext.init(h.instance, new QSession().withPermissions(Set.of("file_gateway.admin")));
+         checker.checkPermissionsThrowing(new InsertInput(FileGatewayTables.ACCOUNT), h.instance.getTable(FileGatewayTables.ACCOUNT));
+         for(String table : FileGatewayTables.SYSTEM_WRITTEN)
+         {
+            assertThatThrownBy(() -> checker.checkPermissionsThrowing(new UpdateInput(table), h.instance.getTable(table))).isInstanceOf(QException.class);
+         }
+         h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("lockedUntil", Instant.now().plusSeconds(900)).withValue("failedLoginCount", 5));
+         var input = new RunBackendStepInput().withProcessName(FileGatewayProcesses.UNLOCK).withRecords(List.of(account));
+         var review = new FileGatewayProcesses.Review();
+         var preview = new RunBackendStepOutput();
+         review.runOnePage(input, preview);
+         assertThat(review.getProcessSummary(preview, false)).hasSize(1);
+         assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter()).getFirst().getValue("lockedUntil")).isNotNull();
+         new FileGatewayProcesses.Apply().runOnePage(input, new RunBackendStepOutput());
+         var unlocked = h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter()).getFirst();
+         assertThat(unlocked.getValue("lockedUntil")).isNull();
+         assertThat(unlocked.getValueInteger("failedLoginCount")).isZero();
+         new FileGatewayProcesses.Apply().runOnePage(input.withProcessName(FileGatewayProcesses.DISABLE), new RunBackendStepOutput());
+         assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter()).getFirst().getValueString("status")).isEqualTo("DISABLED");
+         new FileGatewayProcesses.Apply().runOnePage(input.withProcessName(FileGatewayProcesses.REVOKE).withRecords(List.of(key)), new RunBackendStepOutput());
+         assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter()).getFirst().getValue("revokedAt")).isNotNull();
+         QContext.init(h.instance, new QSession());
+         assertThatThrownBy(() -> new FileGatewayProcesses.Apply().runOnePage(input, new RunBackendStepOutput())).isInstanceOf(QException.class);
+         assertThatThrownBy(() -> checker.checkPermissionsThrowing(new QueryInput(FileGatewayTables.ACCOUNT), h.instance.getTable(FileGatewayTables.ACCOUNT))).isInstanceOf(QException.class);
+         assertThatThrownBy(() -> h.audit.query("alpha", new QQueryFilter())).isInstanceOf(IllegalArgumentException.class);
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/NoDiskExtension.java

@@ -0,0 +1,90 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.attribute.FileTime;
+import java.util.Map;
+import java.util.TreeMap;
+import org.junit.jupiter.api.extension.AfterEachCallback;
+import org.junit.jupiter.api.extension.BeforeEachCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** Snapshots real disk boundaries around every SFTP test; target build products are excluded.
+ *******************************************************************************/
+public class NoDiskExtension implements BeforeEachCallback, AfterEachCallback
+{
+   private Map<String, FileTime> before;
+
+   /*******************************************************************************
+    ** Before each.
+    *******************************************************************************/
+   @Override
+   public void beforeEach(ExtensionContext context) throws Exception
+   {
+      Files.createDirectories(Path.of(System.getProperty("java.io.tmpdir")));
+      Files.createDirectories(Path.of(System.getProperty("user.home")));
+      before = snapshot();
+   }
+
+
+
+   /*******************************************************************************
+    ** After each.
+    *******************************************************************************/
+   @Override
+   public void afterEach(ExtensionContext context) throws Exception
+   {
+      Map<String, FileTime> after = snapshot();
+      assertThat(after.keySet()).as("Files created or removed outside the temporary area").isEqualTo(before.keySet());
+      after.forEach((path, time) -> assertThat(time).as("Modified: %s", path).isEqualTo(before.get(path)));
+      try(var files = Files.list(Path.of(System.getProperty("java.io.tmpdir"))))
+      {
+         assertThat(files.toList()).isEmpty();
+      }
+   }
+
+   private Map<String, FileTime> snapshot() throws Exception
+   {
+      Map<String, FileTime> result = new TreeMap<>();
+      Path working = Path.of("").toAbsolutePath();
+      try(var files = Files.walk(working))
+      {
+         for(Path file : files.filter(p -> !p.startsWith(working.resolve("target"))).toList())
+         {
+            result.put(file.toString(), Files.getLastModifiedTime(file));
+         }
+      }
+      try(var files = Files.list(Path.of(System.getProperty("user.home"))))
+      {
+         for(Path file : files.toList())
+         {
+            result.put(file.toString(), Files.getLastModifiedTime(file));
+         }
+      }
+      return (result);
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/SftpIntegrationTest.java

@@ -0,0 +1,297 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+
+import java.security.KeyPair;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.filegateway.sftp.CidrMatcher;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.sftp.client.SftpClient;
+import org.apache.sshd.sftp.client.SftpClientFactory;
+import org.apache.sshd.sftp.common.SftpConstants;
+import org.apache.sshd.sftp.common.SftpException;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+
+/*******************************************************************************
+ ** Real protocol acceptance tests for listing, authentication, auditing and no-disk operation.
+ *******************************************************************************/
+@ExtendWith(NoDiskExtension.class)
+class SftpIntegrationTest
+{
+   /*******************************************************************************
+    ** Pair.
+    *******************************************************************************/
+   private KeyPair pair() throws Exception
+   {
+      return (KeyUtils.generateKeyPair("ssh-ed25519", 256));
+   }
+
+
+
+   /*******************************************************************************
+    ** Names.
+    *******************************************************************************/
+   private List<String> names(SftpClient client, String path) throws Exception
+   {
+      List<String> result = new ArrayList<>();
+      for(SftpClient.DirEntry entry : client.readDir(path))
+      {
+         if(!entry.getFilename().equals(".") && !entry.getFilename().equals(".."))
+         {
+            result.add(entry.getFilename());
+         }
+      }
+      return (result);
+   }
+
+
+
+   /*******************************************************************************
+    ** Permission filtered listings and audit.
+    *******************************************************************************/
+   @Test
+   void permissionFilteredListingsAndAudit() throws Exception
+   {
+      try(var harness = new FileGatewayTestServer())
+      {
+         KeyPair a = pair();
+         KeyPair b = pair();
+         harness.key(harness.account("alice", "A"), a);
+         harness.key(harness.account("bob", "B"), b);
+         assertThat(harness.saved("savedBulkLoadProfile", "alpha", "Orders In").getErrors()).isEmpty();
+         assertThat(harness.saved("savedView", "alpha", "All Orders").getErrors()).isEmpty();
+         assertThat(harness.saved("savedReport", "alpha", "Daily Orders").getErrors()).isEmpty();
+         int tokens = UiApiInProcessSessions.size();
+         try(var sa = harness.login("alice", a); var sb = harness.login("bob", b);
+             var ca = SftpClientFactory.instance().createSftpClient(sa); var cb = SftpClientFactory.instance().createSftpClient(sb))
+         {
+            assertThat(names(ca, "/")).contains("testapp");
+            assertThat(names(ca, "/testapp")).containsExactlyInAnyOrder("alpha", "beta");
+            assertThat(names(cb, "/testapp")).containsExactly("beta");
+            assertThatThrownBy(() -> cb.stat("/testapp/alpha")).isInstanceOfSatisfying(SftpException.class,
+               e -> assertThat(e.getStatus()).isEqualTo(SftpConstants.SSH_FX_NO_SUCH_FILE));
+            assertThat(names(ca, "/testapp/alpha")).contains("upload", "views", "reports", "results", "archive");
+            assertThat(names(ca, "/testapp/beta")).doesNotContain("upload");
+            assertThat(names(ca, "/testapp/alpha/upload")).containsExactly("orders-in");
+            assertThat(names(ca, "/testapp/alpha/upload/orders-in")).isEmpty();
+            assertThat(names(ca, "/testapp/alpha/views")).containsExactlyInAnyOrder("all-orders.csv", "all-orders.xlsx");
+            assertThat(names(ca, "/testapp/alpha/reports")).contains("daily-orders.csv", "daily-orders.xlsx");
+            assertThat(ca.stat("/testapp/alpha/views/all-orders.csv").getSize()).isZero();
+            assertThatThrownBy(() -> ca.read("/testapp/alpha/views/all-orders.csv")).isInstanceOf(SftpException.class);
+            assertThatThrownBy(() -> ca.write("/testapp/alpha/test.csv")).isInstanceOf(SftpException.class);
+            assertThatThrownBy(() -> ca.setStat("/testapp/alpha/views/all-orders.csv", new SftpClient.Attributes().modifyTime(0))).isInstanceOfSatisfying(SftpException.class,
+               e -> assertThat(e.getStatus()).isEqualTo(SftpConstants.SSH_FX_PERMISSION_DENIED));
+            assertThatThrownBy(() -> names(cb, "/testapp/alpha")).isInstanceOf(java.io.UncheckedIOException.class).hasCauseInstanceOf(SftpException.class);
+            assertThatThrownBy(() -> ca.mkdir("/testapp/alpha/new")).isInstanceOf(SftpException.class);
+            assertThatThrownBy(() -> ca.remove("/testapp/alpha/views/all-orders.csv")).isInstanceOf(SftpException.class);
+            assertThatThrownBy(() -> ca.rename("/testapp/alpha/views/all-orders.csv", "/testapp/alpha/views/new.csv")).isInstanceOf(SftpException.class);
+            assertThat(harness.runtime.getInvoker().findOperation(HttpMethod.GET, "/metaData")).isPresent();
+            assertThat(harness.runtime.getInvoker().findOperation(HttpMethod.POST, "/table/{tableName}/query")).isPresent();
+            assertThat(harness.runtime.getInvoker().getClientChannel()).isEqualTo("sftp");
+         }
+         assertThat(harness.calls).isNotEmpty().allSatisfy(call ->
+         {
+            String template = call.path().substring("/qqq/v1".length()).replaceAll("/table/[^/]+/query", "/table/{tableName}/query");
+            assertThat(harness.runtime.getInvoker().findOperation(HttpMethod.valueOf(call.method()), template)).isPresent();
+            assertThat(call.channel()).startsWith("sftp/");
+            assertThat(call.invokedBy()).isEqualTo("file-gateway");
+         });
+         harness.runtime.stop();
+         assertThat(UiApiInProcessSessions.size()).isEqualTo(tokens);
+         var connections = harness.audit.query("file_gateway_connection_log", new QQueryFilter());
+         assertThat(connections).hasSize(2).allSatisfy(row ->
+         {
+            assertThat(row.getValueString("clientChannel")).isEqualTo("sftp");
+            assertThat(row.getValueString("outcome")).isEqualTo("CONNECTED");
+            assertThat(row.getValue("disconnectedAt")).isNotNull();
+         });
+         var transfers = harness.audit.query("file_gateway_transfer_log", new QQueryFilter());
+         assertThat(transfers).anySatisfy(row ->
+         {
+            assertThat(row.getValueString("operation")).isEqualTo("LIST");
+            assertThat(row.getValueString("outcome")).isEqualTo("OK");
+         }).anySatisfy(row -> assertThat(row.getValueString("outcome")).isEqualTo("DENIED"))
+            .allSatisfy(row -> assertThat(row.getValueString("clientChannel")).startsWith("sftp/"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Refuses invalid credentials with audit.
+    *******************************************************************************/
+   @ParameterizedTest
+   @ValueSource(strings = {"unknown", "revoked", "expiredKey", "disabled", "expiredAccount", "ip", "permission"})
+   void refusesInvalidCredentialsWithAudit(String scenario) throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var key = pair();
+         var account = h.account("alice", scenario.equals("permission") ? "NONE" : "A");
+         var stored = h.key(account, key);
+         String outcome = "AUTH_FAILED";
+         switch(scenario)
+         {
+            case "revoked" -> h.audit.update("file_gateway_ssh_key", new QRecord().withValue("id", stored.getValue("id")).withValue("revokedAt", Instant.now()));
+            case "expiredKey" ->
+            {
+               h.audit.update("file_gateway_ssh_key", new QRecord().withValue("id", stored.getValue("id")).withValue("expiresAt", Instant.now().minusSeconds(1)));
+               outcome = "EXPIRED";
+            }
+            case "disabled" ->
+            {
+               h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("status", "DISABLED"));
+               outcome = "DISABLED";
+            }
+            case "expiredAccount" ->
+            {
+               h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("expiresAt", Instant.now().minusSeconds(1)));
+               outcome = "EXPIRED";
+            }
+            case "ip" ->
+            {
+               h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("allowedCidrs", "192.0.2.0/24"));
+               outcome = "IP_DENIED";
+            }
+         }
+         KeyPair offered = scenario.equals("unknown") ? pair() : key;
+         assertThatThrownBy(() -> h.login("alice", offered)).isInstanceOf(Exception.class);
+         assertThat(h.audit.query("file_gateway_connection_log", new QQueryFilter()).getFirst().getValueString("outcome")).isEqualTo(outcome);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Lockout and session limit.
+    *******************************************************************************/
+   @Test
+   void lockoutAndSessionLimit() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData.ofOrWithNew(h.instance)
+            .withSink(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(RecordingAlerts.class));
+         RecordingAlerts.signals.clear();
+         var account = h.account("alice", "A");
+         var good = pair();
+         h.key(account, good);
+         var bad = pair();
+         for(int i = 0; i < 5; i++)
+         {
+            assertThatThrownBy(() -> h.login("alice", bad)).isInstanceOf(Exception.class);
+         }
+         assertThatThrownBy(() -> h.login("alice", good)).isInstanceOf(Exception.class);
+         assertThat(RecordingAlerts.signals).containsExactly("filegateway.lockout");
+         var rows = h.audit.query("file_gateway_connection_log", new QQueryFilter());
+         assertThat(rows).anySatisfy(row -> assertThat(row.getValueString("outcome")).isEqualTo("LOCKED_OUT"));
+         assertThat(h.audit.query("file_gateway_account", new QQueryFilter()).getFirst().getValue("lockedUntil")).isNotNull();
+         h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("lockedUntil", null));
+         try(var unlocked = h.login("alice", good))
+         {
+            assertThat(unlocked.isAuthenticated()).isTrue();
+         }
+      }
+      try(var h = new FileGatewayTestServer())
+      {
+         var account = h.account("alice", "A");
+         h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("maxConcurrentSessions", 1));
+         var good = pair();
+         h.key(account, good);
+         try(var session = h.login("alice", good))
+         {
+            assertThatThrownBy(() -> h.login("alice", good)).isInstanceOf(Exception.class);
+            assertThat(h.audit.query("file_gateway_connection_log", new QQueryFilter())).anySatisfy(row -> assertThat(row.getValueString("outcome")).isEqualTo("LIMIT"));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Public key only and live revocation.
+    *******************************************************************************/
+   @Test
+   void publicKeyOnlyAndLiveRevocation() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var account = h.account("alice", "A");
+         var pair = pair();
+         var key = h.key(account, pair);
+         var server = h.runtime.getProtocol().getServer();
+         assertThat(server.getPasswordAuthenticator()).isNull();
+         assertThat(server.getKeyboardInteractiveAuthenticator()).isNull();
+         assertThat(server.getUserAuthFactories()).extracting(factory -> factory.getName()).containsExactly("publickey");
+         assertThat(server.getCipherFactories()).noneSatisfy(factory -> assertThat(factory.getName()).contains("cbc"));
+         assertThat(server.getMacFactories()).allSatisfy(factory -> assertThat(factory.getName()).contains("sha2"));
+         assertThat(server.getSignatureFactories()).noneSatisfy(factory -> assertThat(factory.getName()).isEqualTo("ssh-rsa"));
+         try(var password = h.client.connect("alice", "127.0.0.1", h.runtime.getProtocol().getPort()).verify(Duration.ofSeconds(5)).getSession())
+         {
+            password.addPasswordIdentity("not-a-password");
+            assertThatThrownBy(() -> password.auth().verify(Duration.ofSeconds(5))).isInstanceOf(Exception.class);
+         }
+         try(var session = h.login("alice", pair))
+         {
+            h.audit.update("file_gateway_ssh_key", new QRecord().withValue("id", key.getValue("id")).withValue("revokedAt", Instant.now()));
+            h.runtime.getProtocol().getAuthenticator().poll();
+            session.waitFor(java.util.Set.of(org.apache.sshd.client.session.ClientSession.ClientSessionEvent.CLOSED), Duration.ofSeconds(5));
+            assertThat(session.isClosed()).isTrue();
+         }
+      }
+      assertThat(CidrMatcher.matches("::1", "::1/128")).isTrue();
+      assertThat(CidrMatcher.matches("127.0.0.1", "::1/128")).isFalse();
+      assertThat(CidrMatcher.matches("127.0.0.1", "localhost/32")).isFalse();
+   }
+
+
+
+   /*******************************************************************************
+    ** Exercise the core alert facade without adding any dependency on qbit-alerts.
+    *******************************************************************************/
+   public static class RecordingAlerts extends com.kingsrook.qqq.backend.core.alerts.LoggingAlertSink
+   {
+      static final List<String> signals = new java.util.concurrent.CopyOnWriteArrayList<>();
+
+      /*******************************************************************************
+       ** Record alert type names only, without account or key material.
+       *******************************************************************************/
+      @Override
+      public void raise(com.kingsrook.qqq.backend.core.alerts.AlertSignal signal)
+      {
+         signals.add(signal.getAlertTypeName());
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/TestAlertSink.java

@@ -0,0 +1,62 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.time.Instant;
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+import com.kingsrook.qqq.backend.core.alerts.AlertResolution;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+
+/*******************************************************************************
+ ** Captures real core notification dispatch.
+ *******************************************************************************/
+public class TestAlertSink implements AlertSinkInterface
+{
+   public static final List<NotificationSignal> notifications = new CopyOnWriteArrayList<>();
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void raise(AlertSignal signal)
+   {
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void resolve(AlertResolution resolution)
+   {
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void heartbeat(String key, Instant instant)
+   {
+   }
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public void notify(NotificationSignal signal)
+   {
+      notifications.add(signal);
+   }
+}

src/test/java/com/kingsrook/qbits/filegateway/UploadIntegrationTest.java

@@ -0,0 +1,595 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.filegateway;
+
+import java.nio.charset.StandardCharsets;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.ZoneOffset;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import com.kingsrook.qbits.filegateway.api.FileAcceptance;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumer;
+import com.kingsrook.qbits.filegateway.api.FileGatewayConsumersMetaData;
+import com.kingsrook.qbits.filegateway.api.FileGatewayScannerInterface;
+import com.kingsrook.qbits.filegateway.api.InboundFile;
+import com.kingsrook.qbits.filegateway.metadata.FileGatewayProcesses;
+import com.kingsrook.qbits.filegateway.upload.UploadJobWorker;
+import com.kingsrook.qbits.filegateway.upload.UploadJobs;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import org.apache.sshd.common.config.keys.KeyUtils;
+import org.apache.sshd.sftp.client.SftpClient;
+import org.apache.sshd.sftp.client.SftpClientFactory;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+@ExtendWith(NoDiskExtension.class)
+/*******************************************************************************
+ ** Test fixture operation.
+ *******************************************************************************/
+class UploadIntegrationTest
+{
+   static final String FOLDER = "/testapp/alpha/upload/orders-in/";
+   static final byte[] CSV = "name\nAlice\nBob\n".getBytes(StandardCharsets.UTF_8);
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   static QRecord profile(FileGatewayTestServer h) throws Exception
+   {
+      QRecord profile = h.saved("savedBulkLoadProfile", "alpha", "Orders In");
+      h.audit.system(() -> new UpdateAction().execute(new UpdateInput("savedBulkLoadProfile").withRecords(List.of(new QRecord()
+         .withValue("id", profile.getValue("id")).withValue("mappingJson", "{\"version\":\"v1\",\"layout\":\"FLAT\",\"hasHeaderRow\":true,\"fieldList\":[{\"fieldName\":\"name\",\"headerName\":\"name\",\"columnIndex\":null,\"defaultValue\":null}]}")))));
+      return (profile);
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void bulkInsertAndArtifacts() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         h.key(account, key);
+         h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("notifyOnSuccess", true));
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            try(var out = sftp.write(FOLDER + "orders.csv"))
+            {
+               out.write(CSV);
+            }
+            var jobs = h.audit.query("file_gateway_upload_job", new QQueryFilter());
+            assertThat(jobs).hasSize(1);
+            assertThat(jobs.getFirst().getValueString("status")).isEqualTo("QUEUED");
+            assertThat(h.runtime.getWorker().runOnce()).isTrue();
+            QRecord job = h.audit.query("file_gateway_upload_job", new QQueryFilter()).getFirst();
+            assertThat(job.getValueString("status")).withFailMessage("Job values: %s", job.getValues()).isEqualTo("SUCCEEDED");
+            var rows = h.audit.system(() -> new QueryAction().execute(new QueryInput("alpha")).getRecords());
+            assertThat(rows).hasSize(2);
+            assertThat(TestAlertSink.notifications).hasSize(1);
+            assertThat(TestAlertSink.notifications.getFirst().getAudience().getValues()).containsExactly("A");
+            var audits = h.audit.system(() -> new QueryAction().execute(new QueryInput("audit")).getRecords());
+            assertThat(audits).anySatisfy(row -> assertThat(row.getValueString("clientChannel")).withFailMessage("%s", row.getValues()).startsWith("sftp/"));
+            assertThat(job.getValueInteger("insertedCount")).isEqualTo(2);
+            try(var in = sftp.read(FOLDER + "orders.csv.done"))
+            {
+               assertThat(new String(in.readAllBytes(), StandardCharsets.UTF_8)).contains("2", "SUCCEEDED");
+            }
+            String date = job.getValueInstant("createDate").atZone(ZoneOffset.UTC).toLocalDate().toString();
+            try(var in = sftp.read("/testapp/alpha/archive/" + date + "/" + job.getValue("id") + "-orders.csv"))
+            {
+               assertThat(in.readAllBytes()).isEqualTo(CSV);
+            }
+            try(var in = sftp.read("/testapp/alpha/results/" + job.getValue("id") + "-orders.csv.results.csv"))
+            {
+               assertThat(in.readAllBytes()).isEqualTo(h.store.objects.get(job.getValueString("resultsStorageKey")));
+            }
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   static List<QRecord> jobs(FileGatewayTestServer h) throws Exception
+   {
+      return (h.audit.query("file_gateway_upload_job", new QQueryFilter()));
+   }
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   static void put(SftpClient sftp, String name, byte[] bytes) throws Exception
+   {
+      try(var out = sftp.write(FOLDER + name))
+      {
+         out.write(bytes);
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void temporaryRenameDedupeAndAcknowledgement() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            put(sftp, "orders.csv.filepart", CSV);
+            assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("RECEIVING");
+            assertThat(h.runtime.getWorker().runOnce()).isFalse();
+            sftp.rename(FOLDER + "orders.csv.filepart", FOLDER + "orders.csv");
+            assertThat(jobs(h)).hasSize(1);
+            assertThat(h.runtime.getWorker().runOnce()).isTrue();
+            assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("SUCCEEDED");
+            put(sftp, "orders.csv", CSV);
+            assertThat(jobs(h)).hasSize(2);
+            assertThat(jobs(h).getLast().getValueString("status")).isEqualTo("DUPLICATE");
+            assertThat(h.runtime.getWorker().runOnce()).isFalse();
+            try(var in = sftp.read(FOLDER + "orders.csv.done"))
+            {
+               assertThat(new String(in.readAllBytes(), StandardCharsets.UTF_8)).contains("Skipped");
+            }
+            assertThat(h.audit.system(() -> new QueryAction().execute(new QueryInput("alpha")).getRecords())).hasSize(2);
+            sftp.remove(FOLDER + "orders.csv.done");
+            assertThat(jobs(h).getLast().getValue("acknowledgedAt")).isNotNull();
+            String result = "/testapp/alpha/results/" + jobs(h).getFirst().getValue("id") + "-orders.csv.results.csv";
+            sftp.remove(result);
+            assertThatThrownBy(() -> sftp.stat(result)).isInstanceOf(Exception.class);
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void badRowAndMissingValueMappings() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         var profile = profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            h.audit.system(() -> new UpdateAction().execute(new UpdateInput("savedBulkLoadProfile").withRecords(List.of(new QRecord()
+               .withValue("id", profile.getValue("id")).withValue("mappingJson", "{\"version\":\"v1\",\"layout\":\"FLAT\",\"hasHeaderRow\":true,\"fieldList\":[{\"fieldName\":\"name\",\"headerName\":\"name\",\"columnIndex\":null,\"defaultValue\":null},{\"fieldName\":\"quantity\",\"headerName\":\"quantity\",\"columnIndex\":null,\"defaultValue\":null}]}")))));
+            put(sftp, "bad.csv", "name,quantity\nAlice,1\nBob,bad\n".getBytes(StandardCharsets.UTF_8));
+            h.runtime.getWorker().runOnce();
+            var job = jobs(h).getFirst();
+            assertThat(job.getValueString("status")).withFailMessage("%s", job.getValues()).isEqualTo("PARTIAL");
+            assertThat(job.getValueInteger("insertedCount")).isEqualTo(1);
+            assertThat(job.getValueInteger("errorCount")).isEqualTo(1);
+            assertThat(new String(h.store.objects.get(job.getValueString("resultsStorageKey")), StandardCharsets.UTF_8)).contains("Quantity", "3");
+            assertThat(TestAlertSink.notifications).hasSize(1);
+            h.audit.system(() -> new UpdateAction().execute(new UpdateInput("savedBulkLoadProfile").withRecords(List.of(new QRecord()
+               .withValue("id", profile.getValue("id")).withValue("mappingJson", "{\"version\":\"v1\",\"layout\":\"FLAT\",\"hasHeaderRow\":true,\"fieldList\":[{\"fieldName\":\"name\",\"doValueMapping\":true}]}")))));
+            put(sftp, "mapping.csv", CSV);
+            h.runtime.getWorker().runOnce();
+            assertThat(jobs(h).getLast().getValueString("status")).isEqualTo("FAILED");
+            assertThat(jobs(h).getLast().getValueString("errorMessage")).contains("this profile needs value mappings saved in it");
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void fileAndDailyLimitsDeletePartialObjects() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         h.key(account, key);
+         h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("maxFileSizeMb", 1));
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            assertThatThrownBy(() -> put(sftp, "large.csv", new byte[1024 * 1024 + 1])).isInstanceOf(Exception.class);
+            var job = jobs(h).getFirst();
+            assertThat(job.getValueString("status")).isEqualTo("REJECTED");
+            assertThat(h.store.objects).doesNotContainKey(job.getValueString("storageKey"));
+            assertThat(h.audit.query("file_gateway_transfer_log", new QQueryFilter())).anySatisfy(log -> assertThat(log.getValueString("outcome")).isEqualTo("LIMIT"));
+            h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id")).withValue("maxUploadMbPerDay", 1));
+            assertThatThrownBy(() -> put(sftp, "quota.csv", new byte[1024 * 1024])).isInstanceOf(Exception.class);
+            assertThat(jobs(h).getLast().getValueString("errorMessage")).contains("Daily upload quota");
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void permissionDenialCreatesNoJob() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         FileGatewayTestServer.GRANTS.put("A", Set.of("file_gateway.connect", "alpha.read"));
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            var names = new ArrayList<String>();
+            for(var entry : sftp.readDir("/testapp/alpha"))
+            {
+               names.add(entry.getFilename());
+            }
+            assertThat(names).doesNotContain("upload");
+            assertThatThrownBy(() -> put(sftp, "denied.csv", CSV)).isInstanceOf(Exception.class);
+            assertThat(jobs(h)).isEmpty();
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public static class Scanner implements FileGatewayScannerInterface
+   {
+      static Result next;
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public Result scan(InboundFile file) throws Exception
+      {
+         try(var in = file.openStream())
+         {
+            assertThat(in.readAllBytes()).isEqualTo(CSV);
+         }
+         return (next);
+      }
+   }
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public static class Consumer implements FileGatewayConsumer
+   {
+      static int count;
+      static FileAcceptance next;
+      static InboundFile received;
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public String getName()
+      {
+         return ("test-consumer");
+      }
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public FileAcceptance accept(InboundFile file) throws Exception
+      {
+         count++;
+         received = file;
+         assertThat(QContext.getQSession().getClientChannel()).startsWith("sftp/");
+         assertThat(QContext.getQSession().hasPermission("alpha.insert")).isTrue();
+         try(var in = file.openStream())
+         {
+            assertThat(in.readAllBytes()).isEqualTo(CSV);
+         }
+         return (next);
+      }
+   }
+
+   /*******************************************************************************
+    ** Test fixture operation.
+    *******************************************************************************/
+   public static class MutableClock extends Clock
+   {
+      Instant now = Instant.parse("2026-10-14T03:00:00Z");
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public Instant instant()
+      {
+         return (now);
+      }
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public ZoneId getZone()
+      {
+         return (ZoneOffset.UTC);
+      }
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      public Clock withZone(ZoneId zone)
+      {
+         return (this);
+      }
+      /*******************************************************************************
+       ** Test fixture operation.
+       *******************************************************************************/
+      void advance(Duration by)
+      {
+         now = now.plus(by);
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void scanAndConsumerRetriesAfterDisconnect() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         profile(h);
+         h.config.withScanner(new QCodeReference(Scanner.class));
+         FileGatewayConsumersMetaData.ofOrWithNew(h.instance).withConsumer(new QCodeReference(Consumer.class));
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("alice", "A");
+         h.key(account, key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            put(sftp, "in.csv", CSV);
+         }
+         QRecord job = jobs(h).getFirst();
+         h.audit.update("file_gateway_upload_job", new QRecord().withValue("id", job.getValue("id")).withValue("consumerName", "test-consumer"));
+         Scanner.next = FileGatewayScannerInterface.Result.ERROR;
+         Consumer.count = 0;
+         h.runtime.getWorker().runOnce();
+         assertThat(jobs(h).getFirst().getValueInstant("nextAttemptAt")).isEqualTo(clock.instant().plusSeconds(60));
+         assertThat(h.runtime.getWorker().runOnce()).isFalse();
+         clock.advance(Duration.ofMinutes(1));
+         Scanner.next = FileGatewayScannerInterface.Result.CLEAN;
+         Consumer.next = FileAcceptance.retryLater("Busy", Duration.ofSeconds(2));
+         h.runtime.getWorker().runOnce();
+         assertThat(Consumer.count).isEqualTo(1);
+         assertThat(Consumer.received.sha256()).isEqualTo(job.getValueString("sha256"));
+         assertThat(Consumer.received.remoteAddress()).isEqualTo("127.0.0.1");
+         assertThat(Consumer.received.username()).isEqualTo("alice");
+         assertThat(Consumer.received.endpointName()).isNull();
+         assertThat(jobs(h).getFirst().getValueInstant("nextAttemptAt")).isEqualTo(clock.instant().plusSeconds(300));
+         clock.advance(Duration.ofMinutes(5));
+         Consumer.next = FileAcceptance.accepted("accepted-123");
+         h.runtime.getWorker().runOnce();
+         assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("SUCCEEDED");
+         assertThat(jobs(h).getFirst().getValueString("consumerReference")).isEqualTo("accepted-123");
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void quarantineUnknownConsumerAndRetryAction() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            put(sftp, "infected.csv", CSV);
+            String original = jobs(h).getFirst().getValueString("storageKey");
+            h.config.withScanner(new QCodeReference(Scanner.class));
+            Scanner.next = FileGatewayScannerInterface.Result.INFECTED;
+            h.runtime.getWorker().runOnce();
+            var rejected = jobs(h).getFirst();
+            assertThat(rejected.getValueString("status")).isEqualTo("REJECTED");
+            assertThat(h.store.objects).doesNotContainKey(original);
+            assertThat(h.store.objects.get(rejected.getValueString("storageKey"))).isEqualTo(CSV);
+            assertThat(rejected.getValueString("storageKey")).startsWith("file-gateway/quarantine/");
+            h.config.withScanner(null);
+            put(sftp, "unknown.csv", CSV);
+            var job = jobs(h).getLast();
+            h.audit.update("file_gateway_upload_job", new QRecord().withValue("id", job.getValue("id")).withValue("consumerName", "missing"));
+            h.runtime.getWorker().runOnce();
+            job = jobs(h).getLast();
+            assertThat(job.getValueString("status")).isEqualTo("REJECTED");
+            assertThat(job.getValueString("errorMessage")).contains("Unknown consumer");
+            QContext.init(h.instance, new QSession().withPermissions(Set.of("file_gateway.admin")));
+            var input = new RunBackendStepInput().withProcessName(FileGatewayProcesses.RETRY_UPLOAD).withRecords(List.of(job));
+            var review = new FileGatewayProcesses.Review();
+            var output = new RunBackendStepOutput();
+            review.runOnePage(input, output);
+            assertThat(review.getProcessSummary(output, false)).hasSize(1);
+            new FileGatewayProcesses.Apply().runOnePage(input, new RunBackendStepOutput());
+            assertThat(jobs(h).getLast().getValueString("status")).isEqualTo("QUEUED");
+            assertThat(jobs(h).getLast().getValueInteger("attempts")).isZero();
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void exclusiveClaimsStaleRecoveryAndStop() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            put(sftp, "claim.csv", CSV);
+         }
+         var job = jobs(h).getFirst();
+         var first = new UploadJobWorker(h.instance, h.runtime.getInvoker(), h.config, h.store, clock, "pod-a");
+         var second = new UploadJobWorker(h.instance, h.runtime.getInvoker(), h.config, h.store, clock, "pod-b");
+         var a = CompletableFuture.supplyAsync(() ->
+         {
+            try
+            {
+               return (first.claim(job));
+            }
+            catch(Exception e)
+            {
+               throw new RuntimeException(e);
+            }
+         }
+         );
+         var b = CompletableFuture.supplyAsync(() ->
+         {
+            try
+            {
+               return (second.claim(job));
+            }
+            catch(Exception e)
+            {
+               throw new RuntimeException(e);
+            }
+         }
+         );
+         assertThat((a.get() == null ? 0 : 1) + (b.get() == null ? 0 : 1)).isEqualTo(1);
+         clock.advance(Duration.ofMinutes(31));
+         second.requeueStale();
+         assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("QUEUED");
+         assertThat(first.claim(jobs(h).getFirst())).isNotNull();
+         first.close();
+         assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("QUEUED");
+         assertThat(first.runOnce()).isFalse();
+         second.close();
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void reorderedWritesRangesAndMissingBlocks() throws Exception
+   {
+      try(var h = new FileGatewayTestServer())
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         h.key(h.account("alice", "A"), key);
+         try(var ssh = h.login("alice", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            try(var handle = sftp.open(FOLDER + "reordered.csv", SftpClient.OpenMode.Write, SftpClient.OpenMode.Create, SftpClient.OpenMode.Truncate))
+            {
+               sftp.write(handle, 5, CSV, 5, CSV.length - 5);
+               sftp.write(handle, 0, CSV, 0, 5);
+            }
+            assertThat(h.store.objects.get(jobs(h).getFirst().getValueString("storageKey"))).isEqualTo(CSV);
+            h.runtime.getWorker().runOnce();
+            var job = jobs(h).getFirst();
+            String date = job.getValueInstant("createDate").atZone(ZoneOffset.UTC).toLocalDate().toString();
+            String path = "/testapp/alpha/archive/" + date + "/" + job.getValue("id") + "-reordered.csv";
+            assertThat(sftp.stat(path).getSize()).isEqualTo(CSV.length);
+            byte[] part = new byte[4];
+            try(var handle = sftp.open(path, SftpClient.OpenMode.Read))
+            {
+               assertThat(sftp.read(handle, 6, part)).isEqualTo(4);
+               assertThat(part).isEqualTo(java.util.Arrays.copyOfRange(CSV, 6, 10));
+            }
+            assertThat(h.audit.query("file_gateway_transfer_log", new QQueryFilter())).anySatisfy(log ->
+            {
+               assertThat(log.getValueString("operation")).isEqualTo("DOWNLOAD");
+               assertThat(log.getValueLong("bytes")).isEqualTo(4);
+            });
+            var hole = sftp.open(FOLDER + "hole.csv", SftpClient.OpenMode.Write, SftpClient.OpenMode.Create);
+            sftp.write(hole, 5, CSV, 5, CSV.length - 5);
+            assertThatThrownBy(hole::close).isInstanceOf(Exception.class);
+            assertThat(jobs(h).getLast().getValueString("status")).isEqualTo("ABORTED");
+            assertThat(h.store.objects).doesNotContainKey(jobs(h).getLast().getValueString("storageKey"));
+            h.config.withUploadReorderWindowMb(1);
+            try(var far = sftp.open(FOLDER + "far.csv", SftpClient.OpenMode.Write, SftpClient.OpenMode.Create))
+            {
+               assertThatThrownBy(() -> sftp.write(far, 2 * 1024 * 1024, CSV, 0, 1)).isInstanceOf(Exception.class);
+            }
+            assertThat(jobs(h).getLast().getValueString("status")).isEqualTo("ABORTED");
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify the upload contract through the real SFTP stack.
+    *******************************************************************************/
+   @Test
+   void retriesExhaustAndServiceOwnerIsNotified() throws Exception
+   {
+      var clock = new MutableClock();
+      try(var h = new FileGatewayTestServer(clock))
+      {
+         profile(h);
+         var key = KeyUtils.generateKeyPair("ssh-ed25519", 256);
+         var account = h.account("service", "A");
+         h.audit.update("file_gateway_account", new QRecord().withValue("id", account.getValue("id"))
+            .withValue("accountType", "SERVICE").withValue("ownerUserId", "OWNER").withValue("rootMode", "FULL_TREE"));
+         h.key(account, key);
+         try(var ssh = h.login("service", key); var sftp = SftpClientFactory.instance().createSftpClient(ssh))
+         {
+            put(sftp, "retry.csv", CSV);
+         }
+         FileGatewayConsumersMetaData.ofOrWithNew(h.instance).withConsumer(new QCodeReference(Consumer.class));
+         Consumer.next = FileAcceptance.retryLater("Busy", null);
+         Consumer.count = 0;
+         h.audit.update("file_gateway_upload_job", new QRecord().withValue("id", jobs(h).getFirst().getValue("id")).withValue("consumerName", "test-consumer"));
+         long[] delays = {60, 300, 900, 3600};
+         for(int attempt = 0; attempt < 5; attempt++)
+         {
+            assertThat(h.runtime.getWorker().runOnce()).isTrue();
+            if(attempt < 4)
+            {
+               assertThat(jobs(h).getFirst().getValueInstant("nextAttemptAt")).isEqualTo(clock.instant().plusSeconds(delays[attempt]));
+               clock.advance(Duration.ofSeconds(delays[attempt]));
+            }
+         }
+         assertThat(jobs(h).getFirst().getValueString("status")).isEqualTo("FAILED");
+         assertThat(Consumer.count).isEqualTo(5);
+         assertThat(TestAlertSink.notifications).hasSize(1);
+         assertThat(TestAlertSink.notifications.getFirst().getAudience().getValues()).containsExactly("OWNER");
+         assertThat(TestAlertSink.notifications.getFirst().getLink().getTableName()).isEqualTo("file_gateway_upload_job");
+         assertThat(h.runtime.getWorker().runOnce()).isFalse();
+      }
+   }
+
+}

src/test/resources/log4j2.xml

@@ -0,0 +1,11 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Configuration status="WARN">
+   <Appenders>
+      <Console name="Console" target="SYSTEM_OUT">
+         <PatternLayout pattern="%level %logger{1}: %msg%n"/>
+      </Console>
+   </Appenders>
+   <Loggers>
+      <Root level="WARN"><AppenderRef ref="Console"/></Root>
+   </Loggers>
+</Configuration>