From 7a5e42495dbbeac61ed1695e1c5556d60346acae Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 09:18:38 -0500 Subject: [PATCH 1/3] build: pin QQQ 4.1.0-RC.7 (first release with QRun-IO/qqq#1170) Replaces the 4.1.0-SNAPSHOT stand-in (guardrail G7) and drops the private FG-SNAPSHOT build notes. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 11 +++++------ CHANGELOG.md | 4 ++-- CLAUDE.md | 11 +++++------ README.md | 8 ++++---- docs/DESIGN.md | 10 +++++----- pom.xml | 4 ++-- 6 files changed, 23 insertions(+), 25 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a25cc23..a878485 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,13 +14,12 @@ code and QInstance. Design: `docs/DESIGN.md`. ## Build commands ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes +./mvnw compile +./mvnw test +./mvnw verify # Checkstyle; JaCoCo: 70% instructions, 90% classes ``` -Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell. -Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement. +Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`. ## Architecture @@ -92,7 +91,7 @@ directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must rem ## Dependencies -Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-SNAPSHOT` until 4.1.0-RC.6. +Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-RC.7` (first release with QRun-IO/qqq#1170); never a SNAPSHOT. Only allow-listed qbit API artifacts (`qbit-object-storage` `api` package) may be depended on. Notifications go through core `QAlerts.notify`, never qbit-alerts classes. diff --git a/CHANGELOG.md b/CHANGELOG.md index 6339a1c..b0ed14e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,8 +45,8 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Compatibility -- Local invoker integration builds use `4.1.0-FG-SNAPSHOT`; CI retains the POM's - `4.1.0-SNAPSHOT` default. RDBMS remains a host-provided runtime dependency. +- Builds on QQQ `4.1.0-RC.7` (the first release with QRun-IO/qqq#1170). RDBMS remains a + host-provided runtime dependency. - Alert type `filegateway.lockout` uses the dotted syntax required by core validation. - Object-storage RC.2 controls S3 part size through its connection configuration. - The Preferences "File transfer" section waits for QRun-IO/qqq#1162. diff --git a/CLAUDE.md b/CLAUDE.md index 252b732..5698088 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,13 +14,12 @@ code and QInstance. Design: `docs/DESIGN.md`. ## Build commands ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes +./mvnw compile +./mvnw test +./mvnw verify # Checkstyle; JaCoCo: 70% instructions, 90% classes ``` -Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell. -Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement. +Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`. ## Architecture @@ -92,7 +91,7 @@ directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must rem ## Dependencies -Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-SNAPSHOT` until 4.1.0-RC.6. +Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-RC.7` (first release with QRun-IO/qqq#1170); never a SNAPSHOT. Only allow-listed qbit API artifacts (`qbit-object-storage` `api` package) may be depended on. Notifications go through core `QAlerts.notify`, never qbit-alerts classes. diff --git a/README.md b/README.md index 1c85923..6f8a094 100644 --- a/README.md +++ b/README.md @@ -17,8 +17,8 @@ backend, qbit-object-storage 0.1.0-RC.2, and core saved bulk-load profile, saved report metadata. The gateway's RDBMS module dependency is test-scoped. Maven 3.9.11 is pinned by the wrapper. -Until the invoker core PR merges, local builds use the private `4.1.0-FG-SNAPSHOT` artifacts. -The POM default remains `4.1.0-SNAPSHOT` for CI, to be replaced by 4.1.0-RC.6 when released. +QQQ is pinned to `4.1.0-RC.7`, the first release with the UiApiInvoker in-process sessions, +streams and multipart (QRun-IO/qqq#1170). ## Install @@ -325,13 +325,13 @@ port 22 only (no passive range: v1 is SFTP only) with `externalTrafficPolicy: Lo ## Build and acceptance tests ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify +./mvnw verify ``` PowerShell: ```powershell -./mvnw.cmd -o '-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT' verify +./mvnw.cmd verify ``` The tests use H2 from the adopter DDL, a strict UI API server, generated in-memory SSH keys and diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 7be5a01..13e3675 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -250,8 +250,8 @@ in-process session token; there is never a service session. (`ProcessSpecUtilsV1.addUploadedFiles`). ### Shipping -§3.1–3.3 go in one core PR to `QRun-IO/qqq` `develop`, for 4.1.0-RC.6. Until then the qbit builds -on `4.1.0-SNAPSHOT`, with the pom marked `STAND-IN(removeBy=4.1.0-RC.6)`. +§3.1–3.3 shipped in QRun-IO/qqq#1170 (merged 2026-10-14), first released in QQQ 4.1.0-RC.7, which +the qbit pins. ### 3.4 Permissions for automated sessions (qbit-authentik) `AuthentikGroupPermissionMapper` builds permissions from token claims, and an automated session @@ -678,10 +678,10 @@ All tests run in the qbit's build. `Class.forName` of another qbit. The gateway's own `api` package is the published contract. - **G2.** Server tests run in strict mode. - **G3.** No extension is registered: the gateway is a UI API client. -- **G4.** Stand-ins are marked `STAND-IN(core=..., removeBy=...)`: the 4.1.0-SNAPSHOT baseline - until RC.6, and the preferences section until #1162. +- **G4.** No stand-ins remain: the core baseline is a release, and the preferences section uses + core's API from QRun-IO/qqq#1162. - **G5.** No HTML widgets. -- **G7.** Pin 4.1.0-RC.6 as soon as it is cut. Until then, develop builds on the SNAPSHOT. +- **G7.** Pinned to QQQ 4.1.0-RC.7; never a SNAPSHOT. ## 14. Phases - **(a)** The SFTP server, the virtual filesystem, and auth: diff --git a/pom.xml b/pom.xml index 20ca951..e9b5084 100644 --- a/pom.xml +++ b/pom.xml @@ -238,8 +238,8 @@ true - - 4.1.0-SNAPSHOT + + 4.1.0-RC.7 -- 2.54.0 From e6ff62181bc275a9b8695a1831266caa62799e88 Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 09:29:31 -0500 Subject: [PATCH 2/3] feat(preferences): the "File transfer" section (SSH keys self-service) A user-preferences section (QRun-IO/qqq#1162) for anyone with file_gateway.connect: the SFTP username, host and host-key fingerprint, and the user's own keys. "Add key" creates the user's account on first use and inserts the key through the table (validation and audits apply); "Revoke" sets revoked_at; labels and expiry are editable. The provider resolves the user only from the request context and touches only that user's account and keys. Duplicate fingerprints are rejected up front. 99 tests (11 new); JaCoCo 92.6% instructions, 97.7% classes. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 18 +- CHANGELOG.md | 12 +- CLAUDE.md | 20 +- README.md | 34 +- docs/DESIGN.md | 32 +- .../filegateway/FileGatewayQBitConfig.java | 22 + .../filegateway/FileGatewayQBitProducer.java | 1 + .../metadata/SshKeyCustomizer.java | 25 +- .../FileTransferPreferencesProvider.java | 411 ++++++++++++++++++ .../FileTransferPreferencesTest.java | 404 +++++++++++++++++ 10 files changed, 945 insertions(+), 34 deletions(-) create mode 100644 src/main/java/com/kingsrook/qbits/filegateway/preferences/FileTransferPreferencesProvider.java create mode 100644 src/test/java/com/kingsrook/qbits/filegateway/FileTransferPreferencesTest.java diff --git a/AGENTS.md b/AGENTS.md index a25cc23..8142a67 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,13 +14,13 @@ code and QInstance. Design: `docs/DESIGN.md`. ## Build commands ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes +./mvnw -o compile +./mvnw -o test +./mvnw -o verify # Checkstyle; JaCoCo: 70% instructions, 90% classes ``` -Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell. -Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement. +Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`. +Use the POM default `4.1.0-SNAPSHOT` from current QQQ develop, including preferences #1162 and invoker #1170. ## Architecture @@ -69,8 +69,12 @@ Phase (e) adds endpoints and chroot (`AliasCustomizer` validates paths), `outbox decrypt/encrypt, keys in memory), `maintenance/` (daily retention process and record-action effects), `FileGatewayHome` (typed STATISTICS and TABLE widgets only; guardrail G5), `FileGatewayIntegrations` (ESB, Quick Search, API lists) and `file_gateway_server` heartbeats. -The Preferences "File transfer" section waits for QRun-IO/qqq#1162. The k8s example is in -`docs/k8s/`. +The `file-transfer` Preferences section requires only `file_gateway.connect`. Its provider uses +core's trusted scope with the original session, derives identity only from the request context, +and manages the caller's unrevoked USER-account keys through normal table actions. Add key +creates an account on first use; Revoke sets `revoked_at`; PATCH edits labels and expiry. +Optional `publicHostName` advertises the hostname; port/fingerprint use the latest heartbeat. +The k8s example is in `docs/k8s/`. Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension snapshots the working tree (excluding target) and isolated test home, and asserts the test temp diff --git a/CHANGELOG.md b/CHANGELOG.md index 6339a1c..b48785b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,11 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Added +- File transfer Preferences with connect-permission visibility, read-only connection details, + optional public hostname, self-service Add key/Revoke actions and editable key labels/expiry. +- First-key USER account creation with configured defaults and unique usernames; transactional + ownership checks, friendly duplicate-key validation, core/HTTP tests and real SFTP acceptance. + - Phase (c) generated saved-view and saved-report downloads through the user-scoped UI API, CSV/XLSX/JSON account formats, a bounded rolling read window and HTTP stream cleanup. - Generated-file and UTC daily-download quotas, persisted byte charges under account locks, @@ -45,12 +50,11 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Compatibility -- Local invoker integration builds use `4.1.0-FG-SNAPSHOT`; CI retains the POM's - `4.1.0-SNAPSHOT` default. RDBMS remains a host-provided runtime dependency. +- Local and CI builds use current QQQ develop via the POM default `4.1.0-SNAPSHOT`, + including preferences #1162 and invoker #1170. RDBMS remains host-provided at runtime. - Alert type `filegateway.lockout` uses the dotted syntax required by core validation. - Object-storage RC.2 controls S3 part size through its connection configuration. -- The Preferences "File transfer" section waits for QRun-IO/qqq#1162. -- Private core #1170 sends multipart process values without re-uploading the original file. +- Core #1170 sends multipart process values without re-uploading the original file. - FieldSecurityLock requires core permission enforcement enabled. Saved-report rendering still uses a temporary file inside core; fully disk-free report generation needs a core change. The report folder is hidden without the render process's configured storage table. diff --git a/CLAUDE.md b/CLAUDE.md index 252b732..8142a67 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,4 +1,4 @@ -# CLAUDE.md +# AGENTS.md Guidance for coding agents working in this repository. @@ -14,13 +14,13 @@ code and QInstance. Design: `docs/DESIGN.md`. ## Build commands ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes +./mvnw -o compile +./mvnw -o test +./mvnw -o verify # Checkstyle; JaCoCo: 70% instructions, 90% classes ``` -Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell. -Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement. +Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`. +Use the POM default `4.1.0-SNAPSHOT` from current QQQ develop, including preferences #1162 and invoker #1170. ## Architecture @@ -69,8 +69,12 @@ Phase (e) adds endpoints and chroot (`AliasCustomizer` validates paths), `outbox decrypt/encrypt, keys in memory), `maintenance/` (daily retention process and record-action effects), `FileGatewayHome` (typed STATISTICS and TABLE widgets only; guardrail G5), `FileGatewayIntegrations` (ESB, Quick Search, API lists) and `file_gateway_server` heartbeats. -The Preferences "File transfer" section waits for QRun-IO/qqq#1162. The k8s example is in -`docs/k8s/`. +The `file-transfer` Preferences section requires only `file_gateway.connect`. Its provider uses +core's trusted scope with the original session, derives identity only from the request context, +and manages the caller's unrevoked USER-account keys through normal table actions. Add key +creates an account on first use; Revoke sets `revoked_at`; PATCH edits labels and expiry. +Optional `publicHostName` advertises the hostname; port/fingerprint use the latest heartbeat. +The k8s example is in `docs/k8s/`. Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension snapshots the working tree (excluding target) and isolated test home, and asserts the test temp diff --git a/README.md b/README.md index 1c85923..4f8bf78 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ virtual folder tree. Application metadata and saved objects are read through the Phases (a) through (c) implement authentication, listing/stat, account administration, uploads into saved bulk-load profiles, background consumer delivery, readable sidecars/results/archive and -saved-view/report downloads. Endpoint routing/chroot, outbox delivery, preferences and the app -home follow in later phases. No native filesystem, host-key file, home directory or spool is -used by the gateway. See [the approved design](docs/DESIGN.md). +saved-view/report downloads. Phase (e) adds endpoint routing/chroot, outbox delivery, PGP, +Preferences, retention and the app home. No native filesystem, host-key file, home directory +or spool is used by the gateway. See [the approved design](docs/DESIGN.md). ## Requirements @@ -17,8 +17,8 @@ backend, qbit-object-storage 0.1.0-RC.2, and core saved bulk-load profile, saved report metadata. The gateway's RDBMS module dependency is test-scoped. Maven 3.9.11 is pinned by the wrapper. -Until the invoker core PR merges, local builds use the private `4.1.0-FG-SNAPSHOT` artifacts. -The POM default remains `4.1.0-SNAPSHOT` for CI, to be replaced by 4.1.0-RC.6 when released. +Local and CI builds use the POM default `4.1.0-SNAPSHOT` from current QQQ develop, including +the preferences API (#1162) and invoker (#1170). Pin 4.1.0-RC.6 when released. ## Install @@ -68,6 +68,25 @@ least 3072 bits. Public-key validation runs on insert and update and computes it fingerprint, type and size. The host authentication module must assign the user's permissions when `createAutomatedSessionForUser` is called, including `file_gateway.connect`. +## Preferences + +Users with `file_gateway.connect` manage their own SSH keys in **Preferences → File transfer**. +They need no File Gateway admin or table permissions. **Add key** accepts an OpenSSH public key, +a required label and optional expiry. The first successful key creates a USER account with +FULL_TREE access, the configured account defaults and a unique login derived from the user ID. +The section displays that login, plus the port and host key fingerprint from the latest gateway +heartbeat. Set `.withPublicHostName("sftp.example.com")` on the shared config to display the +public hostname; it defaults to null and is omitted when unset. The heartbeat port is the gateway +listener port; deployments publishing a different external port must communicate that mapping. + +The keys list contains only the user's unrevoked keys. Users can edit labels and expiry dates, +or choose **Revoke**. Revoked keys cannot sign in; existing sessions close at the next revocation +poll (normally within 60 seconds). Fingerprints, key types and last-used times are read-only. +Ed25519, ECDSA and RSA of at least 3072 bits are supported. Duplicate keys are rejected with a +validation message. Normal table customizers and configured audits apply, attributed to the +signed-in user and client channel. Administrators manage service accounts and other users' keys +through the standard account and SSH-key tables. + ## Tables and permissions All seven tables are registered, with camelCase QQQ fields mapped to snake_case database columns. @@ -243,6 +262,7 @@ present without enabling those features. | Options | Defaults | |---|---| | `backendName`, `sftpEnabled`, `sftpPort` | `rdbms`, `false`, `2222` | +| `publicHostName` | null (omit host from Preferences) | | `hostKeyEnvVarName`, `allowGeneratedHostKey` | `FILE_GATEWAY_SSH_HOST_KEY`, `false` | | `lockoutThreshold`, `lockoutWindow`, `lockoutDuration` | 5, 15 minutes, 30 minutes | | `maxConnectionsPerIpPerMinute` | 30 | @@ -325,13 +345,13 @@ port 22 only (no passive range: v1 is SFTP only) with `externalTrafficPolicy: Lo ## Build and acceptance tests ```bash -./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify +./mvnw -o verify ``` PowerShell: ```powershell -./mvnw.cmd -o '-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT' verify +./mvnw.cmd -o verify ``` The tests use H2 from the adopter DDL, a strict UI API server, generated in-memory SSH keys and diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 7be5a01..aa1b752 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -265,13 +265,30 @@ has no token. - **Who needs it.** It is a qbit-authentik PR. The gateway works without it under MOCK auth and in tests, but kof22 needs it before going live. -### 3.5 Preferences (QRun-IO/qqq#1162) -The gateway registers a "File transfer" section with the provider `FileTransferPreferencesProvider`. +### 3.5 Preferences +The gateway registers `file-transfer` ("File transfer", order 60) with +`FileTransferPreferencesProvider`, using core's user-preferences API. Only users with +`file_gateway.connect` see it; no admin-app or backing-table permissions are needed. -- **Scalar fields:** the SFTP username (read-only), "Add a public key" (TEXT) and "Key label". -- **The `keys` list:** label, fingerprint (read-only), last used (read-only), expires and revoked. -- **Behaviour.** The first key creates the user's `USER` account. Revoking a key sets `revoked_at`. -- **Until #1162 is in the baseline,** admins add keys on the account record. +- **Read-only connection details:** the account username, plus port and host key fingerprint from + the most recent `file_gateway_server` heartbeat. The optional `publicHostName` config supplies the host; when + unset, the host field is omitted. Before the first key, username is blank and the keys list + explains that adding a key creates the account. +- **The `keys` list:** only the caller's unrevoked USER-account keys. Label and expiry are editable + through PATCH; fingerprint, key type and last-used time are read-only. Public-key text is never + returned in preference values. +- **Add key (`addKey`, LIST action):** requires OpenSSH public-key text and a label, with optional + expiry. The first successful addition creates a USER account with FULL_TREE and configured + account defaults. Its login is derived from the session user ID, normalized to lowercase + `[a-z0-9._-]`, at most 64 characters; short IDs receive a `user-` prefix. Collisions receive + `-2`, `-3`, etc. Subsequent keys reuse that account. The result names the computed fingerprint. +- **Revoke (`revokeKey`, ROW action):** confirms revocation, sets `revoked_at`, and removes the + key from the list. New logins fail immediately; existing sessions end on the next gateway poll. +- **Persistence:** ordinary table actions retain `SshKeyCustomizer` validation and configured + audits under the original user/client channel. Weak and duplicate keys return actionable 400 + messages. Failed first-key inserts roll back account creation. Mutations lock and recheck + ownership; another user's key is not a setting here. Identity comes only from core's request + context, never from the request body. Service-account keys remain administrator-managed. ## 4. Upload 1. **Receive.** The SFTP write streams straight to object storage: @@ -569,6 +586,7 @@ FileGatewayConsumersMetaData.ofOrWithNew(qInstance).withConsumer(new QCodeRefere - `sftpEnabled` (default false; true only in the gateway deployment, from `FILE_GATEWAY_SFTP_ENABLED`); - `sftpPort` (2222); + - `publicHostName` (null; optional public hostname shown in Preferences); - `hostKeyEnvVarName` (`FILE_GATEWAY_SSH_HOST_KEY`); - `allowGeneratedHostKey` (false). - **Storage and buffers:** @@ -679,7 +697,7 @@ All tests run in the qbit's build. - **G2.** Server tests run in strict mode. - **G3.** No extension is registered: the gateway is a UI API client. - **G4.** Stand-ins are marked `STAND-IN(core=..., removeBy=...)`: the 4.1.0-SNAPSHOT baseline - until RC.6, and the preferences section until #1162. + until RC.6. The preferences section uses the implemented core API. - **G5.** No HTML widgets. - **G7.** Pin 4.1.0-RC.6 as soon as it is cut. Until then, develop builds on the SNAPSHOT. diff --git a/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitConfig.java b/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitConfig.java index 57b2f2c..2d009e5 100644 --- a/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitConfig.java +++ b/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitConfig.java @@ -39,6 +39,7 @@ public class FileGatewayQBitConfig implements QBitConfig private String backendName = "rdbms"; private boolean sftpEnabled = false; private int sftpPort = 2222; + private String publicHostName = null; private String hostKeyEnvVarName = "FILE_GATEWAY_SSH_HOST_KEY"; private boolean allowGeneratedHostKey = false; private String fileGatewayBucketAlias = "file-gateway"; @@ -123,6 +124,27 @@ public class FileGatewayQBitConfig implements QBitConfig + /******************************************************************************* + ** Public SFTP hostname advertised in Preferences, when configured. + *******************************************************************************/ + public String getPublicHostName() + { + return (publicHostName); + } + + + + /******************************************************************************* + ** Set the public SFTP hostname; null omits it from Preferences. + *******************************************************************************/ + public FileGatewayQBitConfig withPublicHostName(String publicHostName) + { + this.publicHostName = publicHostName; + return (this); + } + + + /******************************************************************************* ** Fluent setter for sftpPort. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducer.java b/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducer.java index 7fcf70b..ffdf517 100644 --- a/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/filegateway/FileGatewayQBitProducer.java @@ -171,6 +171,7 @@ public class FileGatewayQBitProducer implements QBitMetaDataProducer preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecords) + public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecords) throws QException { + Set fingerprints = new HashSet<>(); for(QRecord record : records) { QRecord old = oldRecords.orElse(List.of()).stream().filter(candidate -> Objects.equals(candidate.getValue("id"), record.getValue("id"))).findFirst().orElse(null); @@ -69,6 +79,19 @@ public class SshKeyCustomizer implements TableCustomizerInterface catch(Exception e) { record.addError(new BadInputStatusMessage("Use an OpenSSH Ed25519, ECDSA, or RSA key of at least 3072 bits")); + continue; + } + // Core's unique-key check precedes this computed fingerprint. + QQueryFilter filter = new QQueryFilter().withCriteria("fingerprint", QCriteriaOperator.EQUALS, record.getValue("fingerprint")); + if(old != null) + { + filter.withCriteria("id", QCriteriaOperator.NOT_EQUALS, old.getValue("id")); + } + var transaction = input instanceof InsertInput insert ? insert.getTransaction() : ((UpdateInput) input).getTransaction(); + if(!fingerprints.add(record.getValueString("fingerprint")) || !new QueryAction().execute(new QueryInput(FileGatewayTables.SSH_KEY) + .withFilter(filter.withLimit(1)).withTransaction(transaction)).getRecords().isEmpty()) + { + record.addError(new BadInputStatusMessage("Fingerprint: this SSH key is already registered. Use a different key.")); } } return (records); diff --git a/src/main/java/com/kingsrook/qbits/filegateway/preferences/FileTransferPreferencesProvider.java b/src/main/java/com/kingsrook/qbits/filegateway/preferences/FileTransferPreferencesProvider.java new file mode 100644 index 0000000..bc409bb --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/filegateway/preferences/FileTransferPreferencesProvider.java @@ -0,0 +1,411 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.filegateway.preferences; + + +import java.io.Serializable; +import java.time.Instant; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig; +import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables; +import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions; +import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; +import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent; +import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionRequest; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionResult; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionScope; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceChange; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceListMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceProviderInterface; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRequestContext; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRow; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRowChange; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceSectionMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceValues; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferencesMetaData; +import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage; + + +/******************************************************************************* + ** Self-service SSH keys. Core supplies the trusted scope and retains the user + ** and client channel; this provider never switches to a system session. + *******************************************************************************/ +public class FileTransferPreferencesProvider implements UserPreferenceProviderInterface +{ + public static final String SECTION_NAME = "file-transfer"; + private static final List KEY_FIELDS = List.of("label", "fingerprint", "keyType", "lastUsedAt", "expiresAt"); + + + + /******************************************************************************* + ** Register without an app scope: connect permission is sufficient. + *******************************************************************************/ + public static void register(QInstance instance, FileGatewayQBitConfig config) + { + UserPreferenceSectionMetaData section = new UserPreferenceSectionMetaData() + .withName(SECTION_NAME).withLabel("File transfer").withIcon(new QIcon("swap_vert")).withOrder(60) + .withDescription("Sign in to the SFTP file gateway with your SSH keys.") + .withPermissionName(FileGatewayPermissions.CONNECT).withProvider(new QCodeReference(FileTransferPreferencesProvider.class)) + .withField(new QFieldMetaData("username", QFieldType.STRING).withLabel("Username").withIsEditable(false) + .withHelpContent(new QHelpContent("Adding your first key creates your SFTP account and username."))) + .withField(new QFieldMetaData("port", QFieldType.INTEGER).withLabel("Port").withIsEditable(false)) + .withField(new QFieldMetaData("hostKeyFingerprint", QFieldType.STRING).withLabel("Host key fingerprint").withIsEditable(false)) + .withList(new UserPreferenceListMetaData().withName("keys").withLabel("SSH keys").withRowLabel("Key") + .withDescription("Adding your first key creates your SFTP account.") + .withField(new QFieldMetaData("label", QFieldType.STRING).withLabel("Label").withIsRequired(true).withMaxLength(1024)) + .withField(new QFieldMetaData("fingerprint", QFieldType.STRING).withLabel("Fingerprint").withIsEditable(false)) + .withField(new QFieldMetaData("keyType", QFieldType.STRING).withLabel("Key type").withIsEditable(false)) + .withField(new QFieldMetaData("lastUsedAt", QFieldType.DATE_TIME).withLabel("Last used").withIsEditable(false)) + .withField(new QFieldMetaData("expiresAt", QFieldType.DATE_TIME).withLabel("Expires at"))) + .withAction(new UserPreferenceActionMetaData().withName("addKey").withLabel("Add key") + .withScope(UserPreferenceActionScope.LIST).withListName("keys").withOrder(10) + .withInputField(new QFieldMetaData("publicKey", QFieldType.TEXT).withLabel("Public key").withIsRequired(true)) + .withInputField(new QFieldMetaData("label", QFieldType.STRING).withLabel("Label").withIsRequired(true).withMaxLength(1024)) + .withInputField(new QFieldMetaData("expiresAt", QFieldType.DATE_TIME).withLabel("Expires at"))) + .withAction(new UserPreferenceActionMetaData().withName("revokeKey").withLabel("Revoke") + .withScope(UserPreferenceActionScope.ROW).withListName("keys").withOrder(20).withIsDestructive(true) + .withConfirmationText("Revoke this SSH key? It will no longer be able to sign in.")); + if(config.getPublicHostName() != null && !config.getPublicHostName().isBlank()) + { + section.withField(new QFieldMetaData("host", QFieldType.STRING).withLabel("Host").withIsEditable(false).withDefaultValue(config.getPublicHostName())); + } + UserPreferencesMetaData.ofOrWithNew(instance).withSection(section); + } + + + + /******************************************************************************* + ** Publish only the user's active key metadata, never the public-key text. + *******************************************************************************/ + @Override + public UserPreferenceValues load(UserPreferenceRequestContext context) throws QException + { + QRecord account = account(context, null); + UserPreferenceValues result = new UserPreferenceValues(); + result.getValues().put("username", account == null ? "" : account.getValue("username")); + List servers = query(FileGatewayTables.SERVER, new QQueryFilter() + .withOrderBy(new QFilterOrderBy("lastSeenAt", false)).withOrderBy(new QFilterOrderBy("id", false)).withLimit(1), null); + if(!servers.isEmpty()) + { + result.getValues().put("port", servers.getFirst().getValue("port")); + result.getValues().put("hostKeyFingerprint", servers.getFirst().getValue("hostKeyFingerprint")); + } + List rows = new ArrayList<>(); + if(account != null) + { + for(QRecord key : query(FileGatewayTables.SSH_KEY, keysFilter(account), null)) + { + Map values = new LinkedHashMap<>(); + KEY_FIELDS.forEach(field -> values.put(field, key.getValue(field))); + rows.add(new UserPreferenceRow().withKey(key.getValueString("id")).withLabel(key.getValueString("label")).withValues(values)); + } + } + result.getLists().put("keys", rows); + return (result); + } + + + + /******************************************************************************* + ** Core validates types; reject blank labels as well as missing ones. + *******************************************************************************/ + @Override + public List validate(UserPreferenceRequestContext context, UserPreferenceChange change) + { + List errors = new ArrayList<>(); + for(UserPreferenceRowChange row : change.getLists().getOrDefault("keys", List.of())) + { + if(row.getValues().containsKey("label") && (row.getValues().get("label") == null || row.getValues().get("label").toString().isBlank())) + { + errors.add("Label: is required"); + } + } + return (errors); + } + + + + /******************************************************************************* + ** Apply partial changes atomically, checking ownership again under row locks. + *******************************************************************************/ + @Override + public void store(UserPreferenceRequestContext context, UserPreferenceChange change) throws QException + { + try(var tx = QBackendTransaction.openFor(new QueryInput(FileGatewayTables.ACCOUNT))) + { + try + { + QRecord account = lockedAccount(context, tx); + for(UserPreferenceRowChange row : change.getLists().getOrDefault("keys", List.of())) + { + QRecord key = lockedKey(account, row.getKey(), tx); + QRecord update = new QRecord().withValue("id", key.getValue("id")); + for(String field : List.of("label", "expiresAt")) + { + if(row.getValues().containsKey(field)) + { + update.setValue(field, row.getValues().get(field)); + } + } + update(FileGatewayTables.SSH_KEY, update, tx); + } + tx.commit(); + } + catch(QException | RuntimeException e) + { + tx.rollback(); + throw e; + } + } + } + + + + /******************************************************************************* + ** Add through the normal key customizer, or revoke an owned, unrevoked row. + *******************************************************************************/ + @Override + public UserPreferenceActionResult runAction(UserPreferenceRequestContext context, UserPreferenceActionRequest request) throws QException + { + if(!List.of("addKey", "revokeKey").contains(request.getActionName())) + { + throw (new QBadRequestException("This action isn't available.")); + } + if("addKey".equals(request.getActionName())) + { + List errors = new ArrayList<>(); + for(String field : List.of("publicKey", "label")) + { + if(request.getValues().get(field) == null || request.getValues().get(field).toString().isBlank()) + { + errors.add(("publicKey".equals(field) ? "Public key" : "Label") + ": is required"); + } + } + if(!errors.isEmpty()) + { + throw (new QBadRequestException(String.join("\n", errors))); + } + } + try(var tx = QBackendTransaction.openFor(new QueryInput(FileGatewayTables.ACCOUNT))) + { + try + { + QRecord account = lockedAccount(context, tx); + UserPreferenceActionResult result; + if("addKey".equals(request.getActionName())) + { + if(account == null) + { + account = insert(FileGatewayTables.ACCOUNT, new QRecord().withValue("username", availableUsername(context.getUserId(), tx)) + .withValue("userId", context.getUserId()).withValue("accountType", "USER").withValue("rootMode", "FULL_TREE"), tx); + } + QRecord key = insert(FileGatewayTables.SSH_KEY, new QRecord().withValue("accountId", account.getValue("id")) + .withValue("publicKey", request.getValues().get("publicKey")).withValue("label", request.getValues().get("label")) + .withValue("expiresAt", request.getValues().get("expiresAt")), tx); + result = new UserPreferenceActionResult().withMessage("Added key " + key.getValueString("fingerprint") + ".").withFocusRowKey(key.getValueString("id")); + } + else + { + QRecord key = lockedKey(account, request.getRowKey(), tx); + update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id")).withValue("revokedAt", Instant.now()), tx); + result = new UserPreferenceActionResult().withMessage("Key revoked."); + } + tx.commit(); + return (result); + } + catch(QException | RuntimeException e) + { + tx.rollback(); + throw e; + } + } + } + + + + /******************************************************************************* + ** Use only the session identity; service accounts stay in the admin tables. + *******************************************************************************/ + private QRecord account(UserPreferenceRequestContext context, QBackendTransaction tx) throws QException + { + List rows = query(FileGatewayTables.ACCOUNT, new QQueryFilter() + .withCriteria("userId", QCriteriaOperator.EQUALS, context.getUserId()) + .withCriteria("accountType", QCriteriaOperator.EQUALS, "USER").withOrderBy(new QFilterOrderBy("id", true)).withLimit(1), tx); + return (rows.isEmpty() ? null : rows.getFirst()); + } + + + + /******************************************************************************* + ** A transactional timestamp update acquires the same lock used by gateway auth. + ** Re-read after the lock in case an administrator changed ownership meanwhile. + *******************************************************************************/ + private QRecord lockedAccount(UserPreferenceRequestContext context, QBackendTransaction tx) throws QException + { + QRecord account = account(context, tx); + if(account != null) + { + update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("modifyDate", Instant.now()), tx); + QRecord current = account(context, tx); + if(current == null || !Objects.equals(current.getValue("id"), account.getValue("id"))) + { + throw (new QBadRequestException("Account: isn't a setting here")); + } + return (current); + } + return (null); + } + + + + /******************************************************************************* + ** Scope every key lookup to the caller's account and unrevoked rows. + *******************************************************************************/ + private QQueryFilter keysFilter(QRecord account) + { + return (new QQueryFilter().withCriteria("accountId", QCriteriaOperator.EQUALS, account.getValue("id")) + .withCriteria("revokedAt", QCriteriaOperator.IS_BLANK).withOrderBy(new QFilterOrderBy("id", true))); + } + + + + /******************************************************************************* + ** Check ownership both before and after locking the key against reassignment. + *******************************************************************************/ + private QRecord lockedKey(QRecord account, String rowKey, QBackendTransaction tx) throws QException + { + QRecord key = ownKey(account, rowKey, tx); + update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id")).withValue("modifyDate", Instant.now()), tx); + return (ownKey(account, rowKey, tx)); + } + + + + /******************************************************************************* + ** Do not interpret an arbitrary caller-supplied key as an unrestricted ID. + *******************************************************************************/ + private QRecord ownKey(QRecord account, String rowKey, QBackendTransaction tx) throws QException + { + if(account != null) + { + for(QRecord key : query(FileGatewayTables.SSH_KEY, keysFilter(account), tx)) + { + if(key.getValueString("id").equals(rowKey)) + { + return (key); + } + } + } + throw (new QBadRequestException("Key: isn't a setting here")); + } + + + + /******************************************************************************* + ** Normalize opaque user IDs to the account validator's 3-64 character syntax. + ** Truncate before suffixing, so even a 64-character collision stays valid. + *******************************************************************************/ + private String availableUsername(String userId, QBackendTransaction tx) throws QException + { + String base = userId.toLowerCase(Locale.ROOT).replaceAll("[^a-z0-9._-]", "-"); + if(base.length() < 3) + { + base = "user-" + base; + } + base = base.substring(0, Math.min(64, base.length())); + String username = base; + int suffix = 2; + while(!query(FileGatewayTables.ACCOUNT, new QQueryFilter().withCriteria("username", QCriteriaOperator.EQUALS, username).withLimit(1), tx).isEmpty()) + { + String ending = "-" + suffix++; + username = base.substring(0, Math.min(base.length(), 64 - ending.length())) + ending; + } + return (username); + } + + + + /******************************************************************************* + ** Standard actions retain customizers, configured audits and the caller's session. + *******************************************************************************/ + private List query(String table, QQueryFilter filter, QBackendTransaction tx) throws QException + { + return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withTransaction(tx)).getRecords()); + } + + + + /******************************************************************************* + ** Preserve domain validation messages while withholding internal database errors. + *******************************************************************************/ + private QRecord checked(QRecord record) throws QException + { + if(record.getErrors() != null && !record.getErrors().isEmpty()) + { + List messages = record.getErrors().stream().map(error -> error instanceof BadInputStatusMessage ? error.getMessage() : "Could not save this setting.").toList(); + throw (new QBadRequestException(String.join("\n", messages))); + } + return (record); + } + + + + /******************************************************************************* + ** Insert via ordinary table validation, including fingerprint uniqueness. + *******************************************************************************/ + private QRecord insert(String table, QRecord record, QBackendTransaction tx) throws QException + { + return (checked(new InsertAction().execute(new InsertInput(table).withRecord(record).withTransaction(tx)).getRecords().getFirst())); + } + + + + /******************************************************************************* + ** Persist only supplied fields, with the original user's audit attribution. + *******************************************************************************/ + private void update(String table, QRecord record, QBackendTransaction tx) throws QException + { + checked(new UpdateAction().execute(new UpdateInput(table).withRecords(List.of(record)).withTransaction(tx)).getRecords().getFirst()); + } +} diff --git a/src/test/java/com/kingsrook/qbits/filegateway/FileTransferPreferencesTest.java b/src/test/java/com/kingsrook/qbits/filegateway/FileTransferPreferencesTest.java new file mode 100644 index 0000000..c4ef059 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/filegateway/FileTransferPreferencesTest.java @@ -0,0 +1,404 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.filegateway; + + +import java.security.KeyPair; +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables; +import com.kingsrook.qbits.filegateway.preferences.FileTransferPreferencesProvider; +import com.kingsrook.qqq.backend.core.actions.preferences.UserPreferencesAction; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel; +import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionResult; +import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferencesMetaData; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher; +import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiCredentials; +import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions; +import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker; +import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest; +import com.kingsrook.qqq.openapi.model.HttpMethod; +import org.apache.sshd.common.config.keys.KeyUtils; +import org.apache.sshd.common.config.keys.PublicKeyEntry; +import org.apache.sshd.common.digest.BuiltinDigests; +import org.apache.sshd.sftp.client.SftpClientFactory; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + + +/******************************************************************************* + ** Self-service acceptance through core, strict HTTP routes and real SSH clients. + *******************************************************************************/ +@ExtendWith(NoDiskExtension.class) +class FileTransferPreferencesTest +{ + private static final String SECTION = FileTransferPreferencesProvider.SECTION_NAME; + private final UserPreferencesAction preferences = new UserPreferencesAction(); + + + + /******************************************************************************* + ** Connect permission alone reveals the section, independently of the admin app. + *******************************************************************************/ + @Test + void visibilityAndEmptyAccount() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + user(h, "A"); + assertThat(preferences.listSections()).hasSize(1); + var section = preferences.getSection(SECTION); + assertThat(section.getSection().getPermissionName()).isEqualTo("file_gateway.connect"); + assertThat(section.getSection().getAppName()).isNull(); + assertThat(section.getSection().getOrder()).isEqualTo(60); + assertThat(section.getValues().getValues().get("username")).isEqualTo(""); + assertThat(section.getValues().getLists().get("keys")).isEmpty(); + assertThat(section.getSection().getLists().getFirst().getDescription()).contains("first key creates"); + assertThat(section.getSection().getFields()).noneMatch(field -> "host".equals(field.getName())); + assertThat(section.getValues().getValues().get("port")).isEqualTo(h.runtime.getProtocol().getPort()); + assertThat(section.getValues().getValues().get("hostKeyFingerprint")).isNotNull(); + assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).isEmpty(); + user(h, "NONE"); + assertThat(preferences.listSections()).isEmpty(); + assertThatThrownBy(() -> preferences.getSection(SECTION)).isInstanceOf(QNotFoundException.class); + assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of())).isInstanceOf(QNotFoundException.class); + assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of())).isInstanceOf(QNotFoundException.class); + } + } + + + + /******************************************************************************* + ** Host is optional; heartbeat data comes from the most recent server. + *******************************************************************************/ + @Test + void publicHostAndLatestHeartbeat() throws Exception + { + try(var h = new FileGatewayTestServer(Clock.systemUTC(), instance -> { }, 0, false, false)) + { + assertThat(h.config.getPublicHostName()).isNull(); + UserPreferencesMetaData.of(h.instance).getSections().clear(); + FileTransferPreferencesProvider.register(h.instance, h.config.withPublicHostName("sftp.example.test")); + user(h, "A"); + assertThat(preferences.getSection(SECTION).getValues().getValues()).containsEntry("host", "sftp.example.test").containsEntry("port", null); + h.audit.insert(FileGatewayTables.SERVER, new QRecord().withValue("podName", "new").withValue("port", 2223) + .withValue("lastSeenAt", Instant.now()).withValue("hostKeyFingerprint", "SHA256:new")); + h.audit.insert(FileGatewayTables.SERVER, new QRecord().withValue("podName", "old").withValue("port", 2222) + .withValue("lastSeenAt", Instant.now().minusSeconds(60)).withValue("hostKeyFingerprint", "SHA256:old")); + assertThat(preferences.getSection(SECTION).getValues().getValues()).containsEntry("port", 2223).containsEntry("hostKeyFingerprint", "SHA256:new"); + } + } + + + + /******************************************************************************* + ** Creation uses schema defaults, reuses the account and preserves audit identity. + *******************************************************************************/ + @Test + void createsAndReusesAccountWithAudits() throws Exception + { + try(var h = new FileGatewayTestServer(Clock.systemUTC(), instance -> + { + for(String table : List.of(FileGatewayTables.ACCOUNT, FileGatewayTables.SSH_KEY)) + { + instance.getTable(table).withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.RECORD)); + } + instance.getTable(FileGatewayTables.ACCOUNT).getField("maxFileSizeMb").setDefaultValue(321); + instance.setEnforcePermissionsInCoreActions(true); + })) + { + user(h, "A"); + var key = pair(); + var first = add(key, "Laptop"); + assertThat(first.getMessage()).contains(KeyUtils.getFingerPrint(BuiltinDigests.sha256, key.getPublic())); + assertThat(first.getSection().getValues().getValues().get("username")).isEqualTo("user-a"); + assertThat(first.getSection().getValues().getLists().get("keys").getFirst().getValues()).containsEntry("keyType", "ssh-ed25519").doesNotContainKey("publicKey"); + add(pair(), "Backup"); + assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).singleElement().satisfies(account -> + { + assertThat(account.getValueString("userId")).isEqualTo("A"); + assertThat(account.getValueString("accountType")).isEqualTo("USER"); + assertThat(account.getValueString("rootMode")).isEqualTo("FULL_TREE"); + assertThat(account.getValueString("status")).isEqualTo("ACTIVE"); + assertThat(account.getValueInteger("maxFileSizeMb")).isEqualTo(321); + assertThat(account.getValueInteger("maxConcurrentSessions")).isEqualTo(4); + }); + assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).hasSize(2); + var audits = h.audit.system(() -> new QueryAction().execute(new QueryInput("audit")).getRecords()); + var auditTables = h.audit.system(() -> new QueryAction().execute(new QueryInput("auditTable")).getRecords()); + var auditUsers = h.audit.system(() -> new QueryAction().execute(new QueryInput("auditUser")).getRecords()); + Object tableId = auditTables.stream().filter(row -> FileGatewayTables.SSH_KEY.equals(row.getValueString("name"))).findFirst().orElseThrow().getValue("id"); + Object userId = auditUsers.stream().filter(row -> "Test user A".equals(row.getValueString("name"))).findFirst().orElseThrow().getValue("id"); + assertThat(audits).anySatisfy(audit -> + { + assertThat(audit.getValue("auditTableId")).isEqualTo(tableId); + assertThat(audit.getValue("auditUserId")).isEqualTo(userId); + assertThat(audit.getValueString("clientChannel")).isEqualTo("next-ui/preferences-test"); + }); + assertThat(QContext.getQSession().getUser().getIdReference()).isEqualTo("A"); + } + } + + + + /******************************************************************************* + ** Invalid first keys roll back the account; duplicates retain table messages. + *******************************************************************************/ + @Test + void rejectsWeakDuplicateAndMissingKeys() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + user(h, "A"); + assertThatThrownBy(() -> add(KeyUtils.generateKeyPair("ssh-rsa", 2048), "Weak")) + .isInstanceOf(QBadRequestException.class).hasMessageContaining("at least 3072 bits"); + assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).isEmpty(); + assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of())) + .isInstanceOf(QBadRequestException.class).hasMessageContaining("Public key: is required").hasMessageContaining("Label: is required"); + var key = pair(); + add(key, "First"); + assertThatThrownBy(() -> add(key, "Duplicate")).isInstanceOf(QBadRequestException.class).hasMessageContaining("Fingerprint"); + user(h, "B"); + assertThatThrownBy(() -> add(key, "Other user duplicate")).isInstanceOf(QBadRequestException.class).hasMessageContaining("Fingerprint"); + assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).hasSize(1); + assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter())).hasSize(1); + } + } + + + + /******************************************************************************* + ** Normalization, truncation and suffixes obey the account validator. + *******************************************************************************/ + @ParameterizedTest + @ValueSource(strings = {"Alice@EXAMPLE.COM", "A", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}) + void uniquifiesUsername(String id) throws Exception + { + try(var h = new FileGatewayTestServer()) + { + user(h, id); + String base = id.toLowerCase(java.util.Locale.ROOT).replaceAll("[^a-z0-9._-]", "-"); + base = base.length() < 3 ? "user-" + base : base.substring(0, Math.min(64, base.length())); + h.account(base, "B"); + String second = base.substring(0, Math.min(62, base.length())) + "-2"; + h.account(second, "B"); + var added = add(pair(), "Key"); + assertThat(added.getSection().getValues().getValues().get("username")).isEqualTo(base.substring(0, Math.min(62, base.length())) + "-3"); + } + } + + + + /******************************************************************************* + ** Lists and mutations exclude other users, revoked keys and service accounts. + *******************************************************************************/ + @Test + void isolatesKeysAndRejectsIdentityInjection() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + var other = h.key(h.account("bob", "B"), pair()); + var account = h.account("alice", "A"); + var revoked = h.key(account, pair()); + h.audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", revoked.getValue("id")).withValue("revokedAt", Instant.now())); + var service = h.account("service", "A"); + h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", service.getValue("id")).withValue("accountType", "SERVICE")); + var serviceKey = h.key(service, pair()); + user(h, "A"); + var own = add(pair(), "Own"); + assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).extracting(row -> row.getKey()).containsExactly(own.getFocusRowKey()); + for(QRecord excluded : List.of(other, revoked, serviceKey)) + { + assertThatThrownBy(() -> preferences.runAction(SECTION, "revokeKey", Map.of("rowKey", excluded.getValueString("id")))) + .isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> patch(excluded.getValueString("id"), Map.of("label", "Stolen"))) + .isInstanceOf(QBadRequestException.class).hasMessageContaining("isn't a setting here"); + } + assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of("userId", "B"))).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of("userId", "B"))).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> patch(own.getFocusRowKey(), Map.of("accountId", other.getValue("accountId")))) + .isInstanceOf(QBadRequestException.class); + assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter())).noneMatch(row -> "Stolen".equals(row.getValueString("label"))); + } + } + + + + /******************************************************************************* + ** PATCH changes only label/expiry, supports clearing expiry and rejects readonly fields. + *******************************************************************************/ + @Test + void editsOnlyMutableFields() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + user(h, "A"); + String key = add(pair(), "Laptop").getFocusRowKey(); + Instant expires = Instant.parse("2030-01-01T00:00:00Z"); + patch(key, Map.of("label", "Work laptop", "expiresAt", expires.toString())); + var values = preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues(); + assertThat(values).containsEntry("label", "Work laptop").containsEntry("expiresAt", expires); + Map clear = new HashMap<>(); + clear.put("expiresAt", null); + patch(key, clear); + assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues()).containsEntry("expiresAt", null); + for(String field : List.of("fingerprint", "keyType", "lastUsedAt")) + { + assertThatThrownBy(() -> patch(key, Map.of(field, "2030-01-01T00:00:00Z"))).isInstanceOf(QBadRequestException.class).hasMessageContaining("can't be changed"); + } + for(String field : List.of("username", "port", "hostKeyFingerprint")) + { + assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of("values", Map.of(field, "22")))) + .isInstanceOf(QBadRequestException.class).hasMessageContaining("can't be changed"); + } + assertThatThrownBy(() -> patch(key, Map.of("label", " "))).isInstanceOf(QBadRequestException.class).hasMessageContaining("Label: is required"); + assertThatThrownBy(() -> patch(key, Map.of("publicKey", "replacement"))).isInstanceOf(QBadRequestException.class); + } + } + + + + /******************************************************************************* + ** A self-service key signs in and lists SFTP; revocation closes and refuses it. + *******************************************************************************/ + @Test + void addedKeyLogsInAndRevokedKeyCannot() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + user(h, "A"); + var pair = pair(); + var added = add(pair, "Laptop"); + String username = added.getSection().getValues().getValues().get("username").toString(); + try(var session = h.login(username, pair); var client = SftpClientFactory.instance().createSftpClient(session)) + { + assertThat(client.readDir("/")).anyMatch(entry -> "testapp".equals(entry.getFilename())); + assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues().get("lastUsedAt")).isNotNull(); + assertThat(preferences.runAction(SECTION, "revokeKey", Map.of("rowKey", added.getFocusRowKey())).getMessage()).isEqualTo("Key revoked."); + h.runtime.getProtocol().getAuthenticator().poll(); + session.waitFor(Set.of(org.apache.sshd.client.session.ClientSession.ClientSessionEvent.CLOSED), Duration.ofSeconds(5)); + assertThat(session.isClosed()).isTrue(); + } + assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter()).getFirst().getValue("revokedAt")).isNotNull(); + assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).isEmpty(); + assertThatThrownBy(() -> h.login(username, pair)).isInstanceOf(Exception.class); + } + } + + + + /******************************************************************************* + ** The strict UI route exposes actions and translates domain failures to 400. + *******************************************************************************/ + @Test + void preferencesThroughStrictUiApi() throws Exception + { + try(var h = new FileGatewayTestServer()) + { + QSession session = user(h, "A"); + String token = UiApiInProcessSessions.register(session, Duration.ofMinutes(1)); + try + { + var invoker = UiApiInvoker.forServer(h.http, "next-ui"); + var get = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath("/preferences") + .withCredentials(UiApiCredentials.ofInProcessSession(token))); + assertThat(get.getStatus()).isEqualTo(200); + assertThat(get.getJson().toString()).contains("file-transfer", "addKey", "revokeKey"); + var added = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/preferences/{sectionName}/actions/{actionName}") + .withPathParam("sectionName", SECTION).withPathParam("actionName", "addKey").withCredentials(UiApiCredentials.ofInProcessSession(token)) + .withJsonBody(Map.of("values", Map.of("publicKey", PublicKeyEntry.toString(pair().getPublic()), "label", "HTTP key")))); + assertThat(added.getStatus()).as(added.getBody()).isEqualTo(200); + assertThat(added.getJson().toString()).contains("SHA256:", "HTTP key"); + var invalid = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/preferences/{sectionName}/actions/{actionName}") + .withPathParam("sectionName", SECTION).withPathParam("actionName", "addKey").withCredentials(UiApiCredentials.ofInProcessSession(token)) + .withJsonBody(Map.of("values", Map.of("publicKey", "invalid", "label", "Invalid")))); + assertThat(invalid.getStatus()).isEqualTo(400); + assertThat(invalid.getBody()).contains("at least 3072 bits"); + } + finally + { + UiApiInProcessSessions.revoke(token); + } + } + } + + + + /******************************************************************************* + ** Set a real non-system session with only the user's normal permissions. + *******************************************************************************/ + private QSession user(FileGatewayTestServer h, String id) throws Exception + { + QSession session = h.audit.system(() -> new QAuthenticationModuleDispatcher().getQModule(h.instance.getAuthentication()).createAutomatedSessionForUser(h.instance, id)); + session.setPermissions(FileGatewayTestServer.GRANTS.getOrDefault(id, Set.of("file_gateway.connect"))); + session.setClientChannel("next-ui/preferences-test"); + QContext.init(h.instance, session); + return (session); + } + + + + /******************************************************************************* + ** Generate a supported public key entirely in memory. + *******************************************************************************/ + private KeyPair pair() throws Exception + { + return (KeyUtils.generateKeyPair("ssh-ed25519", 256)); + } + + + + /******************************************************************************* + ** Exercise registered action input validation rather than calling the provider. + *******************************************************************************/ + private UserPreferenceActionResult add(KeyPair key, String label) throws Exception + { + return (preferences.runAction(SECTION, "addKey", Map.of("values", Map.of("publicKey", PublicKeyEntry.toString(key.getPublic()), "label", label)))); + } + + + + /******************************************************************************* + ** Send the same partial row body as a preferences client. + *******************************************************************************/ + private void patch(String key, Map values) throws Exception + { + preferences.patchSection(SECTION, Map.of("lists", Map.of("keys", List.of(Map.of("key", key, "values", values))))); + } +} -- 2.54.0 From d0572402a99c5f942b262ed5583107a78971e29d Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 09:57:19 -0500 Subject: [PATCH 3/3] ci: re-run against the develop SNAPSHOT that includes QRun-IO/qqq#1162 Co-Authored-By: Claude Opus 5.5 -- 2.54.0