feat(preferences): the File transfer section (SSH keys self-service) #10
No files matched your search
@@ -14,13 +14,12 @@ code and QInstance. Design: `docs/DESIGN.md`.
|
||||
## Build commands
|
||||
|
||||
```bash
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes
|
||||
./mvnw compile
|
||||
./mvnw test
|
||||
./mvnw verify # Checkstyle; JaCoCo: 70% instructions, 90% classes
|
||||
```
|
||||
|
||||
Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell.
|
||||
Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement.
|
||||
Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -69,8 +68,12 @@ Phase (e) adds endpoints and chroot (`AliasCustomizer` validates paths), `outbox
|
||||
decrypt/encrypt, keys in memory), `maintenance/` (daily retention process and record-action
|
||||
effects), `FileGatewayHome` (typed STATISTICS and TABLE widgets only; guardrail G5),
|
||||
`FileGatewayIntegrations` (ESB, Quick Search, API lists) and `file_gateway_server` heartbeats.
|
||||
The Preferences "File transfer" section waits for QRun-IO/qqq#1162. The k8s example is in
|
||||
`docs/k8s/`.
|
||||
The `file-transfer` Preferences section requires only `file_gateway.connect`. Its provider uses
|
||||
core's trusted scope with the original session, derives identity only from the request context,
|
||||
and manages the caller's unrevoked USER-account keys through normal table actions. Add key
|
||||
creates an account on first use; Revoke sets `revoked_at`; PATCH edits labels and expiry.
|
||||
Optional `publicHostName` advertises the hostname; port/fingerprint use the latest heartbeat.
|
||||
The k8s example is in `docs/k8s/`.
|
||||
|
||||
Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension
|
||||
snapshots the working tree (excluding target) and isolated test home, and asserts the test temp
|
||||
@@ -92,7 +95,7 @@ directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must rem
|
||||
|
||||
## Dependencies
|
||||
|
||||
Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-SNAPSHOT` until 4.1.0-RC.6.
|
||||
Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-RC.7` (first release with QRun-IO/qqq#1170); never a SNAPSHOT.
|
||||
Only allow-listed qbit API artifacts (`qbit-object-storage` `api` package) may be depended on.
|
||||
Notifications go through core `QAlerts.notify`, never qbit-alerts classes.
|
||||
|
||||
|
||||
+8
-4
@@ -6,6 +6,11 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
|
||||
### Added
|
||||
|
||||
- File transfer Preferences with connect-permission visibility, read-only connection details,
|
||||
optional public hostname, self-service Add key/Revoke actions and editable key labels/expiry.
|
||||
- First-key USER account creation with configured defaults and unique usernames; transactional
|
||||
ownership checks, friendly duplicate-key validation, core/HTTP tests and real SFTP acceptance.
|
||||
|
||||
- Phase (c) generated saved-view and saved-report downloads through the user-scoped UI API,
|
||||
CSV/XLSX/JSON account formats, a bounded rolling read window and HTTP stream cleanup.
|
||||
- Generated-file and UTC daily-download quotas, persisted byte charges under account locks,
|
||||
@@ -45,12 +50,11 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
|
||||
### Compatibility
|
||||
|
||||
- Local invoker integration builds use `4.1.0-FG-SNAPSHOT`; CI retains the POM's
|
||||
`4.1.0-SNAPSHOT` default. RDBMS remains a host-provided runtime dependency.
|
||||
- Builds on QQQ `4.1.0-RC.7` (the first release with QRun-IO/qqq#1170). RDBMS remains a
|
||||
host-provided runtime dependency.
|
||||
- Alert type `filegateway.lockout` uses the dotted syntax required by core validation.
|
||||
- Object-storage RC.2 controls S3 part size through its connection configuration.
|
||||
- The Preferences "File transfer" section waits for QRun-IO/qqq#1162.
|
||||
- Private core #1170 sends multipart process values without re-uploading the original file.
|
||||
- Core #1170 sends multipart process values without re-uploading the original file.
|
||||
- FieldSecurityLock requires core permission enforcement enabled. Saved-report rendering still
|
||||
uses a temporary file inside core; fully disk-free report generation needs a core change.
|
||||
The report folder is hidden without the render process's configured storage table.
|
||||
@@ -1,4 +1,4 @@
|
||||
# CLAUDE.md
|
||||
# AGENTS.md
|
||||
|
||||
Guidance for coding agents working in this repository.
|
||||
|
||||
@@ -14,13 +14,12 @@ code and QInstance. Design: `docs/DESIGN.md`.
|
||||
## Build commands
|
||||
|
||||
```bash
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT compile
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT test
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify # Checkstyle; JaCoCo: 70% instructions, 90% classes
|
||||
./mvnw compile
|
||||
./mvnw test
|
||||
./mvnw verify # Checkstyle; JaCoCo: 70% instructions, 90% classes
|
||||
```
|
||||
|
||||
Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd` and quote the `-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT` argument in PowerShell.
|
||||
Keep the POM default at `4.1.0-SNAPSHOT` for CI; the private version prevents local artifact replacement.
|
||||
Use the Maven 3.9.11 wrapper (Java 21); Maven 3.10 cannot read the parent POM. On Windows use `./mvnw.cmd`.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -69,8 +68,12 @@ Phase (e) adds endpoints and chroot (`AliasCustomizer` validates paths), `outbox
|
||||
decrypt/encrypt, keys in memory), `maintenance/` (daily retention process and record-action
|
||||
effects), `FileGatewayHome` (typed STATISTICS and TABLE widgets only; guardrail G5),
|
||||
`FileGatewayIntegrations` (ESB, Quick Search, API lists) and `file_gateway_server` heartbeats.
|
||||
The Preferences "File transfer" section waits for QRun-IO/qqq#1162. The k8s example is in
|
||||
`docs/k8s/`.
|
||||
The `file-transfer` Preferences section requires only `file_gateway.connect`. Its provider uses
|
||||
core's trusted scope with the original session, derives identity only from the request context,
|
||||
and manages the caller's unrevoked USER-account keys through normal table actions. Add key
|
||||
creates an account on first use; Revoke sets `revoked_at`; PATCH edits labels and expiry.
|
||||
Optional `publicHostName` advertises the hostname; port/fingerprint use the latest heartbeat.
|
||||
The k8s example is in `docs/k8s/`.
|
||||
|
||||
Tests run a strict UI API server, H2 from the adopter DDL and real MINA clients. NoDiskExtension
|
||||
snapshots the working tree (excluding target) and isolated test home, and asserts the test temp
|
||||
@@ -92,7 +95,7 @@ directory is empty. The host supplies RDBMS; `qqq-backend-module-rdbms` must rem
|
||||
|
||||
## Dependencies
|
||||
|
||||
Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-SNAPSHOT` until 4.1.0-RC.6.
|
||||
Parent: `com.kingsrook:qbit-build-parent:2.0.0`. QQQ BOM: `4.1.0-RC.7` (first release with QRun-IO/qqq#1170); never a SNAPSHOT.
|
||||
Only allow-listed qbit API artifacts (`qbit-object-storage` `api` package) may be depended on.
|
||||
Notifications go through core `QAlerts.notify`, never qbit-alerts classes.
|
||||
|
||||
|
||||
@@ -6,9 +6,9 @@ virtual folder tree. Application metadata and saved objects are read through the
|
||||
|
||||
Phases (a) through (c) implement authentication, listing/stat, account administration, uploads into
|
||||
saved bulk-load profiles, background consumer delivery, readable sidecars/results/archive and
|
||||
saved-view/report downloads. Endpoint routing/chroot, outbox delivery, preferences and the app
|
||||
home follow in later phases. No native filesystem, host-key file, home directory or spool is
|
||||
used by the gateway. See [the approved design](docs/DESIGN.md).
|
||||
saved-view/report downloads. Phase (e) adds endpoint routing/chroot, outbox delivery, PGP,
|
||||
Preferences, retention and the app home. No native filesystem, host-key file, home directory
|
||||
or spool is used by the gateway. See [the approved design](docs/DESIGN.md).
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -17,8 +17,8 @@ backend, qbit-object-storage 0.1.0-RC.2, and core saved bulk-load profile, saved
|
||||
report metadata. The gateway's
|
||||
RDBMS module dependency is test-scoped. Maven 3.9.11 is pinned by the wrapper.
|
||||
|
||||
Until the invoker core PR merges, local builds use the private `4.1.0-FG-SNAPSHOT` artifacts.
|
||||
The POM default remains `4.1.0-SNAPSHOT` for CI, to be replaced by 4.1.0-RC.6 when released.
|
||||
QQQ is pinned to `4.1.0-RC.7`, the first release with the UiApiInvoker in-process sessions,
|
||||
streams and multipart (QRun-IO/qqq#1170).
|
||||
|
||||
## Install
|
||||
|
||||
@@ -68,6 +68,25 @@ least 3072 bits. Public-key validation runs on insert and update and computes it
|
||||
fingerprint, type and size. The host authentication module must assign the user's permissions
|
||||
when `createAutomatedSessionForUser` is called, including `file_gateway.connect`.
|
||||
|
||||
## Preferences
|
||||
|
||||
Users with `file_gateway.connect` manage their own SSH keys in **Preferences → File transfer**.
|
||||
They need no File Gateway admin or table permissions. **Add key** accepts an OpenSSH public key,
|
||||
a required label and optional expiry. The first successful key creates a USER account with
|
||||
FULL_TREE access, the configured account defaults and a unique login derived from the user ID.
|
||||
The section displays that login, plus the port and host key fingerprint from the latest gateway
|
||||
heartbeat. Set `.withPublicHostName("sftp.example.com")` on the shared config to display the
|
||||
public hostname; it defaults to null and is omitted when unset. The heartbeat port is the gateway
|
||||
listener port; deployments publishing a different external port must communicate that mapping.
|
||||
|
||||
The keys list contains only the user's unrevoked keys. Users can edit labels and expiry dates,
|
||||
or choose **Revoke**. Revoked keys cannot sign in; existing sessions close at the next revocation
|
||||
poll (normally within 60 seconds). Fingerprints, key types and last-used times are read-only.
|
||||
Ed25519, ECDSA and RSA of at least 3072 bits are supported. Duplicate keys are rejected with a
|
||||
validation message. Normal table customizers and configured audits apply, attributed to the
|
||||
signed-in user and client channel. Administrators manage service accounts and other users' keys
|
||||
through the standard account and SSH-key tables.
|
||||
|
||||
## Tables and permissions
|
||||
|
||||
All seven tables are registered, with camelCase QQQ fields mapped to snake_case database columns.
|
||||
@@ -243,6 +262,7 @@ present without enabling those features.
|
||||
| Options | Defaults |
|
||||
|---|---|
|
||||
| `backendName`, `sftpEnabled`, `sftpPort` | `rdbms`, `false`, `2222` |
|
||||
| `publicHostName` | null (omit host from Preferences) |
|
||||
| `hostKeyEnvVarName`, `allowGeneratedHostKey` | `FILE_GATEWAY_SSH_HOST_KEY`, `false` |
|
||||
| `lockoutThreshold`, `lockoutWindow`, `lockoutDuration` | 5, 15 minutes, 30 minutes |
|
||||
| `maxConnectionsPerIpPerMinute` | 30 |
|
||||
@@ -325,13 +345,13 @@ port 22 only (no passive range: v1 is SFTP only) with `externalTrafficPolicy: Lo
|
||||
## Build and acceptance tests
|
||||
|
||||
```bash
|
||||
./mvnw -o -Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT verify
|
||||
./mvnw verify
|
||||
```
|
||||
|
||||
PowerShell:
|
||||
|
||||
```powershell
|
||||
./mvnw.cmd -o '-Dqqq.snapshot.version=4.1.0-FG-SNAPSHOT' verify
|
||||
./mvnw.cmd verify
|
||||
```
|
||||
|
||||
The tests use H2 from the adopter DDL, a strict UI API server, generated in-memory SSH keys and
|
||||
|
||||
+29
-11
@@ -250,8 +250,8 @@ in-process session token; there is never a service session.
|
||||
(`ProcessSpecUtilsV1.addUploadedFiles`).
|
||||
|
||||
### Shipping
|
||||
§3.1–3.3 go in one core PR to `QRun-IO/qqq` `develop`, for 4.1.0-RC.6. Until then the qbit builds
|
||||
on `4.1.0-SNAPSHOT`, with the pom marked `STAND-IN(removeBy=4.1.0-RC.6)`.
|
||||
§3.1–3.3 shipped in QRun-IO/qqq#1170 (merged 2026-10-14), first released in QQQ 4.1.0-RC.7, which
|
||||
the qbit pins.
|
||||
|
||||
### 3.4 Permissions for automated sessions (qbit-authentik)
|
||||
`AuthentikGroupPermissionMapper` builds permissions from token claims, and an automated session
|
||||
@@ -265,13 +265,30 @@ has no token.
|
||||
- **Who needs it.** It is a qbit-authentik PR. The gateway works without it under MOCK auth and in
|
||||
tests, but kof22 needs it before going live.
|
||||
|
||||
### 3.5 Preferences (QRun-IO/qqq#1162)
|
||||
The gateway registers a "File transfer" section with the provider `FileTransferPreferencesProvider`.
|
||||
### 3.5 Preferences
|
||||
The gateway registers `file-transfer` ("File transfer", order 60) with
|
||||
`FileTransferPreferencesProvider`, using core's user-preferences API. Only users with
|
||||
`file_gateway.connect` see it; no admin-app or backing-table permissions are needed.
|
||||
|
||||
- **Scalar fields:** the SFTP username (read-only), "Add a public key" (TEXT) and "Key label".
|
||||
- **The `keys` list:** label, fingerprint (read-only), last used (read-only), expires and revoked.
|
||||
- **Behaviour.** The first key creates the user's `USER` account. Revoking a key sets `revoked_at`.
|
||||
- **Until #1162 is in the baseline,** admins add keys on the account record.
|
||||
- **Read-only connection details:** the account username, plus port and host key fingerprint from
|
||||
the most recent `file_gateway_server` heartbeat. The optional `publicHostName` config supplies the host; when
|
||||
unset, the host field is omitted. Before the first key, username is blank and the keys list
|
||||
explains that adding a key creates the account.
|
||||
- **The `keys` list:** only the caller's unrevoked USER-account keys. Label and expiry are editable
|
||||
through PATCH; fingerprint, key type and last-used time are read-only. Public-key text is never
|
||||
returned in preference values.
|
||||
- **Add key (`addKey`, LIST action):** requires OpenSSH public-key text and a label, with optional
|
||||
expiry. The first successful addition creates a USER account with FULL_TREE and configured
|
||||
account defaults. Its login is derived from the session user ID, normalized to lowercase
|
||||
`[a-z0-9._-]`, at most 64 characters; short IDs receive a `user-` prefix. Collisions receive
|
||||
`-2`, `-3`, etc. Subsequent keys reuse that account. The result names the computed fingerprint.
|
||||
- **Revoke (`revokeKey`, ROW action):** confirms revocation, sets `revoked_at`, and removes the
|
||||
key from the list. New logins fail immediately; existing sessions end on the next gateway poll.
|
||||
- **Persistence:** ordinary table actions retain `SshKeyCustomizer` validation and configured
|
||||
audits under the original user/client channel. Weak and duplicate keys return actionable 400
|
||||
messages. Failed first-key inserts roll back account creation. Mutations lock and recheck
|
||||
ownership; another user's key is not a setting here. Identity comes only from core's request
|
||||
context, never from the request body. Service-account keys remain administrator-managed.
|
||||
|
||||
## 4. Upload
|
||||
1. **Receive.** The SFTP write streams straight to object storage:
|
||||
@@ -569,6 +586,7 @@ FileGatewayConsumersMetaData.ofOrWithNew(qInstance).withConsumer(new QCodeRefere
|
||||
- `sftpEnabled` (default false; true only in the gateway deployment, from
|
||||
`FILE_GATEWAY_SFTP_ENABLED`);
|
||||
- `sftpPort` (2222);
|
||||
- `publicHostName` (null; optional public hostname shown in Preferences);
|
||||
- `hostKeyEnvVarName` (`FILE_GATEWAY_SSH_HOST_KEY`);
|
||||
- `allowGeneratedHostKey` (false).
|
||||
- **Storage and buffers:**
|
||||
@@ -678,10 +696,10 @@ All tests run in the qbit's build.
|
||||
`Class.forName` of another qbit. The gateway's own `api` package is the published contract.
|
||||
- **G2.** Server tests run in strict mode.
|
||||
- **G3.** No extension is registered: the gateway is a UI API client.
|
||||
- **G4.** Stand-ins are marked `STAND-IN(core=..., removeBy=...)`: the 4.1.0-SNAPSHOT baseline
|
||||
until RC.6, and the preferences section until #1162.
|
||||
- **G4.** No stand-ins remain: the core baseline is a release, and the preferences section uses
|
||||
core's API from QRun-IO/qqq#1162.
|
||||
- **G5.** No HTML widgets.
|
||||
- **G7.** Pin 4.1.0-RC.6 as soon as it is cut. Until then, develop builds on the SNAPSHOT.
|
||||
- **G7.** Pinned to QQQ 4.1.0-RC.7; never a SNAPSHOT.
|
||||
|
||||
## 14. Phases
|
||||
- **(a)** The SFTP server, the virtual filesystem, and auth:
|
||||
|
||||
@@ -238,8 +238,8 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<!-- STAND-IN(core=QRun-IO/qqq develop after v4.1.0-RC.5, removeBy=4.1.0-RC.6): UiApiInvoker client channels and QAlerts.notify are on develop only; pin the RC when it is cut -->
|
||||
<qqq.snapshot.version>4.1.0-SNAPSHOT</qqq.snapshot.version>
|
||||
<!-- QQQ 4.1.0-RC.7: the first released core with the UiApiInvoker in-process sessions, streams and multipart (QRun-IO/qqq#1170). Never a SNAPSHOT (guardrail G7). -->
|
||||
<qqq.snapshot.version>4.1.0-RC.7</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
@@ -39,6 +39,7 @@ public class FileGatewayQBitConfig implements QBitConfig
|
||||
private String backendName = "rdbms";
|
||||
private boolean sftpEnabled = false;
|
||||
private int sftpPort = 2222;
|
||||
private String publicHostName = null;
|
||||
private String hostKeyEnvVarName = "FILE_GATEWAY_SSH_HOST_KEY";
|
||||
private boolean allowGeneratedHostKey = false;
|
||||
private String fileGatewayBucketAlias = "file-gateway";
|
||||
@@ -123,6 +124,27 @@ public class FileGatewayQBitConfig implements QBitConfig
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Public SFTP hostname advertised in Preferences, when configured.
|
||||
*******************************************************************************/
|
||||
public String getPublicHostName()
|
||||
{
|
||||
return (publicHostName);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Set the public SFTP hostname; null omits it from Preferences.
|
||||
*******************************************************************************/
|
||||
public FileGatewayQBitConfig withPublicHostName(String publicHostName)
|
||||
{
|
||||
this.publicHostName = publicHostName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for sftpPort.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -171,6 +171,7 @@ public class FileGatewayQBitProducer implements QBitMetaDataProducer<FileGateway
|
||||
.withSourceQBitName(qBit.getName()).withSubjectTableName(FileGatewayTables.ACCOUNT)
|
||||
.withVisibilityPermission(FileGatewayPermissions.VIEW));
|
||||
output.addSelfToInstance(qInstance);
|
||||
com.kingsrook.qbits.filegateway.preferences.FileTransferPreferencesProvider.register(qInstance, qBitConfig);
|
||||
ClientChannelsMetaData channels = ClientChannelsMetaData.ofOrWithNew(qInstance)
|
||||
.withChannel("sftp", "SFTP file gateway").withDeniedPattern("sftp", "file_gateway_*")
|
||||
.withDeniedPattern("assistant", "file_gateway_*");
|
||||
|
||||
@@ -22,12 +22,21 @@ package com.kingsrook.qbits.filegateway.metadata;
|
||||
|
||||
|
||||
import java.security.PublicKey;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
import org.apache.sshd.common.config.keys.KeyUtils;
|
||||
@@ -47,8 +56,9 @@ public class SshKeyCustomizer implements TableCustomizerInterface
|
||||
** Parse and validate inserts and partial updates without logging key material.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecords)
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecords) throws QException
|
||||
{
|
||||
Set<String> fingerprints = new HashSet<>();
|
||||
for(QRecord record : records)
|
||||
{
|
||||
QRecord old = oldRecords.orElse(List.of()).stream().filter(candidate -> Objects.equals(candidate.getValue("id"), record.getValue("id"))).findFirst().orElse(null);
|
||||
@@ -69,6 +79,19 @@ public class SshKeyCustomizer implements TableCustomizerInterface
|
||||
catch(Exception e)
|
||||
{
|
||||
record.addError(new BadInputStatusMessage("Use an OpenSSH Ed25519, ECDSA, or RSA key of at least 3072 bits"));
|
||||
continue;
|
||||
}
|
||||
// Core's unique-key check precedes this computed fingerprint.
|
||||
QQueryFilter filter = new QQueryFilter().withCriteria("fingerprint", QCriteriaOperator.EQUALS, record.getValue("fingerprint"));
|
||||
if(old != null)
|
||||
{
|
||||
filter.withCriteria("id", QCriteriaOperator.NOT_EQUALS, old.getValue("id"));
|
||||
}
|
||||
var transaction = input instanceof InsertInput insert ? insert.getTransaction() : ((UpdateInput) input).getTransaction();
|
||||
if(!fingerprints.add(record.getValueString("fingerprint")) || !new QueryAction().execute(new QueryInput(FileGatewayTables.SSH_KEY)
|
||||
.withFilter(filter.withLimit(1)).withTransaction(transaction)).getRecords().isEmpty())
|
||||
{
|
||||
record.addError(new BadInputStatusMessage("Fingerprint: this SSH key is already registered. Use a different key."));
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
|
||||
+411
@@ -0,0 +1,411 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.filegateway.preferences;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
|
||||
import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
|
||||
import com.kingsrook.qbits.filegateway.permissions.FileGatewayPermissions;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionRequest;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionResult;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionScope;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceChange;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceListMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRequestContext;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRow;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceRowChange;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceSectionMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceValues;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferencesMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Self-service SSH keys. Core supplies the trusted scope and retains the user
|
||||
** and client channel; this provider never switches to a system session.
|
||||
*******************************************************************************/
|
||||
public class FileTransferPreferencesProvider implements UserPreferenceProviderInterface
|
||||
{
|
||||
public static final String SECTION_NAME = "file-transfer";
|
||||
private static final List<String> KEY_FIELDS = List.of("label", "fingerprint", "keyType", "lastUsedAt", "expiresAt");
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Register without an app scope: connect permission is sufficient.
|
||||
*******************************************************************************/
|
||||
public static void register(QInstance instance, FileGatewayQBitConfig config)
|
||||
{
|
||||
UserPreferenceSectionMetaData section = new UserPreferenceSectionMetaData()
|
||||
.withName(SECTION_NAME).withLabel("File transfer").withIcon(new QIcon("swap_vert")).withOrder(60)
|
||||
.withDescription("Sign in to the SFTP file gateway with your SSH keys.")
|
||||
.withPermissionName(FileGatewayPermissions.CONNECT).withProvider(new QCodeReference(FileTransferPreferencesProvider.class))
|
||||
.withField(new QFieldMetaData("username", QFieldType.STRING).withLabel("Username").withIsEditable(false)
|
||||
.withHelpContent(new QHelpContent("Adding your first key creates your SFTP account and username.")))
|
||||
.withField(new QFieldMetaData("port", QFieldType.INTEGER).withLabel("Port").withIsEditable(false))
|
||||
.withField(new QFieldMetaData("hostKeyFingerprint", QFieldType.STRING).withLabel("Host key fingerprint").withIsEditable(false))
|
||||
.withList(new UserPreferenceListMetaData().withName("keys").withLabel("SSH keys").withRowLabel("Key")
|
||||
.withDescription("Adding your first key creates your SFTP account.")
|
||||
.withField(new QFieldMetaData("label", QFieldType.STRING).withLabel("Label").withIsRequired(true).withMaxLength(1024))
|
||||
.withField(new QFieldMetaData("fingerprint", QFieldType.STRING).withLabel("Fingerprint").withIsEditable(false))
|
||||
.withField(new QFieldMetaData("keyType", QFieldType.STRING).withLabel("Key type").withIsEditable(false))
|
||||
.withField(new QFieldMetaData("lastUsedAt", QFieldType.DATE_TIME).withLabel("Last used").withIsEditable(false))
|
||||
.withField(new QFieldMetaData("expiresAt", QFieldType.DATE_TIME).withLabel("Expires at")))
|
||||
.withAction(new UserPreferenceActionMetaData().withName("addKey").withLabel("Add key")
|
||||
.withScope(UserPreferenceActionScope.LIST).withListName("keys").withOrder(10)
|
||||
.withInputField(new QFieldMetaData("publicKey", QFieldType.TEXT).withLabel("Public key").withIsRequired(true))
|
||||
.withInputField(new QFieldMetaData("label", QFieldType.STRING).withLabel("Label").withIsRequired(true).withMaxLength(1024))
|
||||
.withInputField(new QFieldMetaData("expiresAt", QFieldType.DATE_TIME).withLabel("Expires at")))
|
||||
.withAction(new UserPreferenceActionMetaData().withName("revokeKey").withLabel("Revoke")
|
||||
.withScope(UserPreferenceActionScope.ROW).withListName("keys").withOrder(20).withIsDestructive(true)
|
||||
.withConfirmationText("Revoke this SSH key? It will no longer be able to sign in."));
|
||||
if(config.getPublicHostName() != null && !config.getPublicHostName().isBlank())
|
||||
{
|
||||
section.withField(new QFieldMetaData("host", QFieldType.STRING).withLabel("Host").withIsEditable(false).withDefaultValue(config.getPublicHostName()));
|
||||
}
|
||||
UserPreferencesMetaData.ofOrWithNew(instance).withSection(section);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Publish only the user's active key metadata, never the public-key text.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public UserPreferenceValues load(UserPreferenceRequestContext context) throws QException
|
||||
{
|
||||
QRecord account = account(context, null);
|
||||
UserPreferenceValues result = new UserPreferenceValues();
|
||||
result.getValues().put("username", account == null ? "" : account.getValue("username"));
|
||||
List<QRecord> servers = query(FileGatewayTables.SERVER, new QQueryFilter()
|
||||
.withOrderBy(new QFilterOrderBy("lastSeenAt", false)).withOrderBy(new QFilterOrderBy("id", false)).withLimit(1), null);
|
||||
if(!servers.isEmpty())
|
||||
{
|
||||
result.getValues().put("port", servers.getFirst().getValue("port"));
|
||||
result.getValues().put("hostKeyFingerprint", servers.getFirst().getValue("hostKeyFingerprint"));
|
||||
}
|
||||
List<UserPreferenceRow> rows = new ArrayList<>();
|
||||
if(account != null)
|
||||
{
|
||||
for(QRecord key : query(FileGatewayTables.SSH_KEY, keysFilter(account), null))
|
||||
{
|
||||
Map<String, Serializable> values = new LinkedHashMap<>();
|
||||
KEY_FIELDS.forEach(field -> values.put(field, key.getValue(field)));
|
||||
rows.add(new UserPreferenceRow().withKey(key.getValueString("id")).withLabel(key.getValueString("label")).withValues(values));
|
||||
}
|
||||
}
|
||||
result.getLists().put("keys", rows);
|
||||
return (result);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Core validates types; reject blank labels as well as missing ones.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> validate(UserPreferenceRequestContext context, UserPreferenceChange change)
|
||||
{
|
||||
List<String> errors = new ArrayList<>();
|
||||
for(UserPreferenceRowChange row : change.getLists().getOrDefault("keys", List.of()))
|
||||
{
|
||||
if(row.getValues().containsKey("label") && (row.getValues().get("label") == null || row.getValues().get("label").toString().isBlank()))
|
||||
{
|
||||
errors.add("Label: is required");
|
||||
}
|
||||
}
|
||||
return (errors);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Apply partial changes atomically, checking ownership again under row locks.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void store(UserPreferenceRequestContext context, UserPreferenceChange change) throws QException
|
||||
{
|
||||
try(var tx = QBackendTransaction.openFor(new QueryInput(FileGatewayTables.ACCOUNT)))
|
||||
{
|
||||
try
|
||||
{
|
||||
QRecord account = lockedAccount(context, tx);
|
||||
for(UserPreferenceRowChange row : change.getLists().getOrDefault("keys", List.of()))
|
||||
{
|
||||
QRecord key = lockedKey(account, row.getKey(), tx);
|
||||
QRecord update = new QRecord().withValue("id", key.getValue("id"));
|
||||
for(String field : List.of("label", "expiresAt"))
|
||||
{
|
||||
if(row.getValues().containsKey(field))
|
||||
{
|
||||
update.setValue(field, row.getValues().get(field));
|
||||
}
|
||||
}
|
||||
update(FileGatewayTables.SSH_KEY, update, tx);
|
||||
}
|
||||
tx.commit();
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
tx.rollback();
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Add through the normal key customizer, or revoke an owned, unrevoked row.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public UserPreferenceActionResult runAction(UserPreferenceRequestContext context, UserPreferenceActionRequest request) throws QException
|
||||
{
|
||||
if(!List.of("addKey", "revokeKey").contains(request.getActionName()))
|
||||
{
|
||||
throw (new QBadRequestException("This action isn't available."));
|
||||
}
|
||||
if("addKey".equals(request.getActionName()))
|
||||
{
|
||||
List<String> errors = new ArrayList<>();
|
||||
for(String field : List.of("publicKey", "label"))
|
||||
{
|
||||
if(request.getValues().get(field) == null || request.getValues().get(field).toString().isBlank())
|
||||
{
|
||||
errors.add(("publicKey".equals(field) ? "Public key" : "Label") + ": is required");
|
||||
}
|
||||
}
|
||||
if(!errors.isEmpty())
|
||||
{
|
||||
throw (new QBadRequestException(String.join("\n", errors)));
|
||||
}
|
||||
}
|
||||
try(var tx = QBackendTransaction.openFor(new QueryInput(FileGatewayTables.ACCOUNT)))
|
||||
{
|
||||
try
|
||||
{
|
||||
QRecord account = lockedAccount(context, tx);
|
||||
UserPreferenceActionResult result;
|
||||
if("addKey".equals(request.getActionName()))
|
||||
{
|
||||
if(account == null)
|
||||
{
|
||||
account = insert(FileGatewayTables.ACCOUNT, new QRecord().withValue("username", availableUsername(context.getUserId(), tx))
|
||||
.withValue("userId", context.getUserId()).withValue("accountType", "USER").withValue("rootMode", "FULL_TREE"), tx);
|
||||
}
|
||||
QRecord key = insert(FileGatewayTables.SSH_KEY, new QRecord().withValue("accountId", account.getValue("id"))
|
||||
.withValue("publicKey", request.getValues().get("publicKey")).withValue("label", request.getValues().get("label"))
|
||||
.withValue("expiresAt", request.getValues().get("expiresAt")), tx);
|
||||
result = new UserPreferenceActionResult().withMessage("Added key " + key.getValueString("fingerprint") + ".").withFocusRowKey(key.getValueString("id"));
|
||||
}
|
||||
else
|
||||
{
|
||||
QRecord key = lockedKey(account, request.getRowKey(), tx);
|
||||
update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id")).withValue("revokedAt", Instant.now()), tx);
|
||||
result = new UserPreferenceActionResult().withMessage("Key revoked.");
|
||||
}
|
||||
tx.commit();
|
||||
return (result);
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
tx.rollback();
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Use only the session identity; service accounts stay in the admin tables.
|
||||
*******************************************************************************/
|
||||
private QRecord account(UserPreferenceRequestContext context, QBackendTransaction tx) throws QException
|
||||
{
|
||||
List<QRecord> rows = query(FileGatewayTables.ACCOUNT, new QQueryFilter()
|
||||
.withCriteria("userId", QCriteriaOperator.EQUALS, context.getUserId())
|
||||
.withCriteria("accountType", QCriteriaOperator.EQUALS, "USER").withOrderBy(new QFilterOrderBy("id", true)).withLimit(1), tx);
|
||||
return (rows.isEmpty() ? null : rows.getFirst());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A transactional timestamp update acquires the same lock used by gateway auth.
|
||||
** Re-read after the lock in case an administrator changed ownership meanwhile.
|
||||
*******************************************************************************/
|
||||
private QRecord lockedAccount(UserPreferenceRequestContext context, QBackendTransaction tx) throws QException
|
||||
{
|
||||
QRecord account = account(context, tx);
|
||||
if(account != null)
|
||||
{
|
||||
update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", account.getValue("id")).withValue("modifyDate", Instant.now()), tx);
|
||||
QRecord current = account(context, tx);
|
||||
if(current == null || !Objects.equals(current.getValue("id"), account.getValue("id")))
|
||||
{
|
||||
throw (new QBadRequestException("Account: isn't a setting here"));
|
||||
}
|
||||
return (current);
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Scope every key lookup to the caller's account and unrevoked rows.
|
||||
*******************************************************************************/
|
||||
private QQueryFilter keysFilter(QRecord account)
|
||||
{
|
||||
return (new QQueryFilter().withCriteria("accountId", QCriteriaOperator.EQUALS, account.getValue("id"))
|
||||
.withCriteria("revokedAt", QCriteriaOperator.IS_BLANK).withOrderBy(new QFilterOrderBy("id", true)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Check ownership both before and after locking the key against reassignment.
|
||||
*******************************************************************************/
|
||||
private QRecord lockedKey(QRecord account, String rowKey, QBackendTransaction tx) throws QException
|
||||
{
|
||||
QRecord key = ownKey(account, rowKey, tx);
|
||||
update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", key.getValue("id")).withValue("modifyDate", Instant.now()), tx);
|
||||
return (ownKey(account, rowKey, tx));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Do not interpret an arbitrary caller-supplied key as an unrestricted ID.
|
||||
*******************************************************************************/
|
||||
private QRecord ownKey(QRecord account, String rowKey, QBackendTransaction tx) throws QException
|
||||
{
|
||||
if(account != null)
|
||||
{
|
||||
for(QRecord key : query(FileGatewayTables.SSH_KEY, keysFilter(account), tx))
|
||||
{
|
||||
if(key.getValueString("id").equals(rowKey))
|
||||
{
|
||||
return (key);
|
||||
}
|
||||
}
|
||||
}
|
||||
throw (new QBadRequestException("Key: isn't a setting here"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Normalize opaque user IDs to the account validator's 3-64 character syntax.
|
||||
** Truncate before suffixing, so even a 64-character collision stays valid.
|
||||
*******************************************************************************/
|
||||
private String availableUsername(String userId, QBackendTransaction tx) throws QException
|
||||
{
|
||||
String base = userId.toLowerCase(Locale.ROOT).replaceAll("[^a-z0-9._-]", "-");
|
||||
if(base.length() < 3)
|
||||
{
|
||||
base = "user-" + base;
|
||||
}
|
||||
base = base.substring(0, Math.min(64, base.length()));
|
||||
String username = base;
|
||||
int suffix = 2;
|
||||
while(!query(FileGatewayTables.ACCOUNT, new QQueryFilter().withCriteria("username", QCriteriaOperator.EQUALS, username).withLimit(1), tx).isEmpty())
|
||||
{
|
||||
String ending = "-" + suffix++;
|
||||
username = base.substring(0, Math.min(base.length(), 64 - ending.length())) + ending;
|
||||
}
|
||||
return (username);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard actions retain customizers, configured audits and the caller's session.
|
||||
*******************************************************************************/
|
||||
private List<QRecord> query(String table, QQueryFilter filter, QBackendTransaction tx) throws QException
|
||||
{
|
||||
return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withTransaction(tx)).getRecords());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve domain validation messages while withholding internal database errors.
|
||||
*******************************************************************************/
|
||||
private QRecord checked(QRecord record) throws QException
|
||||
{
|
||||
if(record.getErrors() != null && !record.getErrors().isEmpty())
|
||||
{
|
||||
List<String> messages = record.getErrors().stream().map(error -> error instanceof BadInputStatusMessage ? error.getMessage() : "Could not save this setting.").toList();
|
||||
throw (new QBadRequestException(String.join("\n", messages)));
|
||||
}
|
||||
return (record);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert via ordinary table validation, including fingerprint uniqueness.
|
||||
*******************************************************************************/
|
||||
private QRecord insert(String table, QRecord record, QBackendTransaction tx) throws QException
|
||||
{
|
||||
return (checked(new InsertAction().execute(new InsertInput(table).withRecord(record).withTransaction(tx)).getRecords().getFirst()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Persist only supplied fields, with the original user's audit attribution.
|
||||
*******************************************************************************/
|
||||
private void update(String table, QRecord record, QBackendTransaction tx) throws QException
|
||||
{
|
||||
checked(new UpdateAction().execute(new UpdateInput(table).withRecords(List.of(record)).withTransaction(tx)).getRecords().getFirst());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,404 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.filegateway;
|
||||
|
||||
|
||||
import java.security.KeyPair;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.filegateway.metadata.FileGatewayTables;
|
||||
import com.kingsrook.qbits.filegateway.preferences.FileTransferPreferencesProvider;
|
||||
import com.kingsrook.qqq.backend.core.actions.preferences.UserPreferencesAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferenceActionResult;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.preferences.UserPreferencesMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
|
||||
import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiCredentials;
|
||||
import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInProcessSessions;
|
||||
import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
|
||||
import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
|
||||
import com.kingsrook.qqq.openapi.model.HttpMethod;
|
||||
import org.apache.sshd.common.config.keys.KeyUtils;
|
||||
import org.apache.sshd.common.config.keys.PublicKeyEntry;
|
||||
import org.apache.sshd.common.digest.BuiltinDigests;
|
||||
import org.apache.sshd.sftp.client.SftpClientFactory;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.extension.ExtendWith;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Self-service acceptance through core, strict HTTP routes and real SSH clients.
|
||||
*******************************************************************************/
|
||||
@ExtendWith(NoDiskExtension.class)
|
||||
class FileTransferPreferencesTest
|
||||
{
|
||||
private static final String SECTION = FileTransferPreferencesProvider.SECTION_NAME;
|
||||
private final UserPreferencesAction preferences = new UserPreferencesAction();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Connect permission alone reveals the section, independently of the admin app.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void visibilityAndEmptyAccount() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
user(h, "A");
|
||||
assertThat(preferences.listSections()).hasSize(1);
|
||||
var section = preferences.getSection(SECTION);
|
||||
assertThat(section.getSection().getPermissionName()).isEqualTo("file_gateway.connect");
|
||||
assertThat(section.getSection().getAppName()).isNull();
|
||||
assertThat(section.getSection().getOrder()).isEqualTo(60);
|
||||
assertThat(section.getValues().getValues().get("username")).isEqualTo("");
|
||||
assertThat(section.getValues().getLists().get("keys")).isEmpty();
|
||||
assertThat(section.getSection().getLists().getFirst().getDescription()).contains("first key creates");
|
||||
assertThat(section.getSection().getFields()).noneMatch(field -> "host".equals(field.getName()));
|
||||
assertThat(section.getValues().getValues().get("port")).isEqualTo(h.runtime.getProtocol().getPort());
|
||||
assertThat(section.getValues().getValues().get("hostKeyFingerprint")).isNotNull();
|
||||
assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).isEmpty();
|
||||
user(h, "NONE");
|
||||
assertThat(preferences.listSections()).isEmpty();
|
||||
assertThatThrownBy(() -> preferences.getSection(SECTION)).isInstanceOf(QNotFoundException.class);
|
||||
assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of())).isInstanceOf(QNotFoundException.class);
|
||||
assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of())).isInstanceOf(QNotFoundException.class);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Host is optional; heartbeat data comes from the most recent server.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void publicHostAndLatestHeartbeat() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer(Clock.systemUTC(), instance -> { }, 0, false, false))
|
||||
{
|
||||
assertThat(h.config.getPublicHostName()).isNull();
|
||||
UserPreferencesMetaData.of(h.instance).getSections().clear();
|
||||
FileTransferPreferencesProvider.register(h.instance, h.config.withPublicHostName("sftp.example.test"));
|
||||
user(h, "A");
|
||||
assertThat(preferences.getSection(SECTION).getValues().getValues()).containsEntry("host", "sftp.example.test").containsEntry("port", null);
|
||||
h.audit.insert(FileGatewayTables.SERVER, new QRecord().withValue("podName", "new").withValue("port", 2223)
|
||||
.withValue("lastSeenAt", Instant.now()).withValue("hostKeyFingerprint", "SHA256:new"));
|
||||
h.audit.insert(FileGatewayTables.SERVER, new QRecord().withValue("podName", "old").withValue("port", 2222)
|
||||
.withValue("lastSeenAt", Instant.now().minusSeconds(60)).withValue("hostKeyFingerprint", "SHA256:old"));
|
||||
assertThat(preferences.getSection(SECTION).getValues().getValues()).containsEntry("port", 2223).containsEntry("hostKeyFingerprint", "SHA256:new");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Creation uses schema defaults, reuses the account and preserves audit identity.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void createsAndReusesAccountWithAudits() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer(Clock.systemUTC(), instance ->
|
||||
{
|
||||
for(String table : List.of(FileGatewayTables.ACCOUNT, FileGatewayTables.SSH_KEY))
|
||||
{
|
||||
instance.getTable(table).withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.RECORD));
|
||||
}
|
||||
instance.getTable(FileGatewayTables.ACCOUNT).getField("maxFileSizeMb").setDefaultValue(321);
|
||||
instance.setEnforcePermissionsInCoreActions(true);
|
||||
}))
|
||||
{
|
||||
user(h, "A");
|
||||
var key = pair();
|
||||
var first = add(key, "Laptop");
|
||||
assertThat(first.getMessage()).contains(KeyUtils.getFingerPrint(BuiltinDigests.sha256, key.getPublic()));
|
||||
assertThat(first.getSection().getValues().getValues().get("username")).isEqualTo("user-a");
|
||||
assertThat(first.getSection().getValues().getLists().get("keys").getFirst().getValues()).containsEntry("keyType", "ssh-ed25519").doesNotContainKey("publicKey");
|
||||
add(pair(), "Backup");
|
||||
assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).singleElement().satisfies(account ->
|
||||
{
|
||||
assertThat(account.getValueString("userId")).isEqualTo("A");
|
||||
assertThat(account.getValueString("accountType")).isEqualTo("USER");
|
||||
assertThat(account.getValueString("rootMode")).isEqualTo("FULL_TREE");
|
||||
assertThat(account.getValueString("status")).isEqualTo("ACTIVE");
|
||||
assertThat(account.getValueInteger("maxFileSizeMb")).isEqualTo(321);
|
||||
assertThat(account.getValueInteger("maxConcurrentSessions")).isEqualTo(4);
|
||||
});
|
||||
assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).hasSize(2);
|
||||
var audits = h.audit.system(() -> new QueryAction().execute(new QueryInput("audit")).getRecords());
|
||||
var auditTables = h.audit.system(() -> new QueryAction().execute(new QueryInput("auditTable")).getRecords());
|
||||
var auditUsers = h.audit.system(() -> new QueryAction().execute(new QueryInput("auditUser")).getRecords());
|
||||
Object tableId = auditTables.stream().filter(row -> FileGatewayTables.SSH_KEY.equals(row.getValueString("name"))).findFirst().orElseThrow().getValue("id");
|
||||
Object userId = auditUsers.stream().filter(row -> "Test user A".equals(row.getValueString("name"))).findFirst().orElseThrow().getValue("id");
|
||||
assertThat(audits).anySatisfy(audit ->
|
||||
{
|
||||
assertThat(audit.getValue("auditTableId")).isEqualTo(tableId);
|
||||
assertThat(audit.getValue("auditUserId")).isEqualTo(userId);
|
||||
assertThat(audit.getValueString("clientChannel")).isEqualTo("next-ui/preferences-test");
|
||||
});
|
||||
assertThat(QContext.getQSession().getUser().getIdReference()).isEqualTo("A");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Invalid first keys roll back the account; duplicates retain table messages.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void rejectsWeakDuplicateAndMissingKeys() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
user(h, "A");
|
||||
assertThatThrownBy(() -> add(KeyUtils.generateKeyPair("ssh-rsa", 2048), "Weak"))
|
||||
.isInstanceOf(QBadRequestException.class).hasMessageContaining("at least 3072 bits");
|
||||
assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).isEmpty();
|
||||
assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of()))
|
||||
.isInstanceOf(QBadRequestException.class).hasMessageContaining("Public key: is required").hasMessageContaining("Label: is required");
|
||||
var key = pair();
|
||||
add(key, "First");
|
||||
assertThatThrownBy(() -> add(key, "Duplicate")).isInstanceOf(QBadRequestException.class).hasMessageContaining("Fingerprint");
|
||||
user(h, "B");
|
||||
assertThatThrownBy(() -> add(key, "Other user duplicate")).isInstanceOf(QBadRequestException.class).hasMessageContaining("Fingerprint");
|
||||
assertThat(h.audit.query(FileGatewayTables.ACCOUNT, new QQueryFilter())).hasSize(1);
|
||||
assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter())).hasSize(1);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Normalization, truncation and suffixes obey the account validator.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"Alice@EXAMPLE.COM", "A", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"})
|
||||
void uniquifiesUsername(String id) throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
user(h, id);
|
||||
String base = id.toLowerCase(java.util.Locale.ROOT).replaceAll("[^a-z0-9._-]", "-");
|
||||
base = base.length() < 3 ? "user-" + base : base.substring(0, Math.min(64, base.length()));
|
||||
h.account(base, "B");
|
||||
String second = base.substring(0, Math.min(62, base.length())) + "-2";
|
||||
h.account(second, "B");
|
||||
var added = add(pair(), "Key");
|
||||
assertThat(added.getSection().getValues().getValues().get("username")).isEqualTo(base.substring(0, Math.min(62, base.length())) + "-3");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Lists and mutations exclude other users, revoked keys and service accounts.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void isolatesKeysAndRejectsIdentityInjection() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
var other = h.key(h.account("bob", "B"), pair());
|
||||
var account = h.account("alice", "A");
|
||||
var revoked = h.key(account, pair());
|
||||
h.audit.update(FileGatewayTables.SSH_KEY, new QRecord().withValue("id", revoked.getValue("id")).withValue("revokedAt", Instant.now()));
|
||||
var service = h.account("service", "A");
|
||||
h.audit.update(FileGatewayTables.ACCOUNT, new QRecord().withValue("id", service.getValue("id")).withValue("accountType", "SERVICE"));
|
||||
var serviceKey = h.key(service, pair());
|
||||
user(h, "A");
|
||||
var own = add(pair(), "Own");
|
||||
assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).extracting(row -> row.getKey()).containsExactly(own.getFocusRowKey());
|
||||
for(QRecord excluded : List.of(other, revoked, serviceKey))
|
||||
{
|
||||
assertThatThrownBy(() -> preferences.runAction(SECTION, "revokeKey", Map.of("rowKey", excluded.getValueString("id"))))
|
||||
.isInstanceOf(QBadRequestException.class);
|
||||
assertThatThrownBy(() -> patch(excluded.getValueString("id"), Map.of("label", "Stolen")))
|
||||
.isInstanceOf(QBadRequestException.class).hasMessageContaining("isn't a setting here");
|
||||
}
|
||||
assertThatThrownBy(() -> preferences.runAction(SECTION, "addKey", Map.of("userId", "B"))).isInstanceOf(QBadRequestException.class);
|
||||
assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of("userId", "B"))).isInstanceOf(QBadRequestException.class);
|
||||
assertThatThrownBy(() -> patch(own.getFocusRowKey(), Map.of("accountId", other.getValue("accountId"))))
|
||||
.isInstanceOf(QBadRequestException.class);
|
||||
assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter())).noneMatch(row -> "Stolen".equals(row.getValueString("label")));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** PATCH changes only label/expiry, supports clearing expiry and rejects readonly fields.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void editsOnlyMutableFields() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
user(h, "A");
|
||||
String key = add(pair(), "Laptop").getFocusRowKey();
|
||||
Instant expires = Instant.parse("2030-01-01T00:00:00Z");
|
||||
patch(key, Map.of("label", "Work laptop", "expiresAt", expires.toString()));
|
||||
var values = preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues();
|
||||
assertThat(values).containsEntry("label", "Work laptop").containsEntry("expiresAt", expires);
|
||||
Map<String, Object> clear = new HashMap<>();
|
||||
clear.put("expiresAt", null);
|
||||
patch(key, clear);
|
||||
assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues()).containsEntry("expiresAt", null);
|
||||
for(String field : List.of("fingerprint", "keyType", "lastUsedAt"))
|
||||
{
|
||||
assertThatThrownBy(() -> patch(key, Map.of(field, "2030-01-01T00:00:00Z"))).isInstanceOf(QBadRequestException.class).hasMessageContaining("can't be changed");
|
||||
}
|
||||
for(String field : List.of("username", "port", "hostKeyFingerprint"))
|
||||
{
|
||||
assertThatThrownBy(() -> preferences.patchSection(SECTION, Map.of("values", Map.of(field, "22"))))
|
||||
.isInstanceOf(QBadRequestException.class).hasMessageContaining("can't be changed");
|
||||
}
|
||||
assertThatThrownBy(() -> patch(key, Map.of("label", " "))).isInstanceOf(QBadRequestException.class).hasMessageContaining("Label: is required");
|
||||
assertThatThrownBy(() -> patch(key, Map.of("publicKey", "replacement"))).isInstanceOf(QBadRequestException.class);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A self-service key signs in and lists SFTP; revocation closes and refuses it.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void addedKeyLogsInAndRevokedKeyCannot() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
user(h, "A");
|
||||
var pair = pair();
|
||||
var added = add(pair, "Laptop");
|
||||
String username = added.getSection().getValues().getValues().get("username").toString();
|
||||
try(var session = h.login(username, pair); var client = SftpClientFactory.instance().createSftpClient(session))
|
||||
{
|
||||
assertThat(client.readDir("/")).anyMatch(entry -> "testapp".equals(entry.getFilename()));
|
||||
assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys").getFirst().getValues().get("lastUsedAt")).isNotNull();
|
||||
assertThat(preferences.runAction(SECTION, "revokeKey", Map.of("rowKey", added.getFocusRowKey())).getMessage()).isEqualTo("Key revoked.");
|
||||
h.runtime.getProtocol().getAuthenticator().poll();
|
||||
session.waitFor(Set.of(org.apache.sshd.client.session.ClientSession.ClientSessionEvent.CLOSED), Duration.ofSeconds(5));
|
||||
assertThat(session.isClosed()).isTrue();
|
||||
}
|
||||
assertThat(h.audit.query(FileGatewayTables.SSH_KEY, new QQueryFilter()).getFirst().getValue("revokedAt")).isNotNull();
|
||||
assertThat(preferences.getSection(SECTION).getValues().getLists().get("keys")).isEmpty();
|
||||
assertThatThrownBy(() -> h.login(username, pair)).isInstanceOf(Exception.class);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The strict UI route exposes actions and translates domain failures to 400.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void preferencesThroughStrictUiApi() throws Exception
|
||||
{
|
||||
try(var h = new FileGatewayTestServer())
|
||||
{
|
||||
QSession session = user(h, "A");
|
||||
String token = UiApiInProcessSessions.register(session, Duration.ofMinutes(1));
|
||||
try
|
||||
{
|
||||
var invoker = UiApiInvoker.forServer(h.http, "next-ui");
|
||||
var get = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath("/preferences")
|
||||
.withCredentials(UiApiCredentials.ofInProcessSession(token)));
|
||||
assertThat(get.getStatus()).isEqualTo(200);
|
||||
assertThat(get.getJson().toString()).contains("file-transfer", "addKey", "revokeKey");
|
||||
var added = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/preferences/{sectionName}/actions/{actionName}")
|
||||
.withPathParam("sectionName", SECTION).withPathParam("actionName", "addKey").withCredentials(UiApiCredentials.ofInProcessSession(token))
|
||||
.withJsonBody(Map.of("values", Map.of("publicKey", PublicKeyEntry.toString(pair().getPublic()), "label", "HTTP key"))));
|
||||
assertThat(added.getStatus()).as(added.getBody()).isEqualTo(200);
|
||||
assertThat(added.getJson().toString()).contains("SHA256:", "HTTP key");
|
||||
var invalid = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/preferences/{sectionName}/actions/{actionName}")
|
||||
.withPathParam("sectionName", SECTION).withPathParam("actionName", "addKey").withCredentials(UiApiCredentials.ofInProcessSession(token))
|
||||
.withJsonBody(Map.of("values", Map.of("publicKey", "invalid", "label", "Invalid"))));
|
||||
assertThat(invalid.getStatus()).isEqualTo(400);
|
||||
assertThat(invalid.getBody()).contains("at least 3072 bits");
|
||||
}
|
||||
finally
|
||||
{
|
||||
UiApiInProcessSessions.revoke(token);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Set a real non-system session with only the user's normal permissions.
|
||||
*******************************************************************************/
|
||||
private QSession user(FileGatewayTestServer h, String id) throws Exception
|
||||
{
|
||||
QSession session = h.audit.system(() -> new QAuthenticationModuleDispatcher().getQModule(h.instance.getAuthentication()).createAutomatedSessionForUser(h.instance, id));
|
||||
session.setPermissions(FileGatewayTestServer.GRANTS.getOrDefault(id, Set.of("file_gateway.connect")));
|
||||
session.setClientChannel("next-ui/preferences-test");
|
||||
QContext.init(h.instance, session);
|
||||
return (session);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Generate a supported public key entirely in memory.
|
||||
*******************************************************************************/
|
||||
private KeyPair pair() throws Exception
|
||||
{
|
||||
return (KeyUtils.generateKeyPair("ssh-ed25519", 256));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Exercise registered action input validation rather than calling the provider.
|
||||
*******************************************************************************/
|
||||
private UserPreferenceActionResult add(KeyPair key, String label) throws Exception
|
||||
{
|
||||
return (preferences.runAction(SECTION, "addKey", Map.of("values", Map.of("publicKey", PublicKeyEntry.toString(key.getPublic()), "label", label))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Send the same partial row body as a preferences client.
|
||||
*******************************************************************************/
|
||||
private void patch(String key, Map<String, ?> values) throws Exception
|
||||
{
|
||||
preferences.patchSection(SECTION, Map.of("lists", Map.of("keys", List.of(Map.of("key", key, "values", values)))));
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user