feat(full): configure external QQQ profile (#5) #15

Merged
KofTwentyTwo merged 2 commits from feature/GH-5-full-profile into develop 2026-09-26 14:10:42 +00:00
12 changed files with 938 additions and 1 deletions

No files matched your search

+19
View File
@@ -10,3 +10,22 @@ java -jar qqq-all-app/target/qqq-all-app-4.1.0-SNAPSHOT.jar
The core app binds to `127.0.0.1` and serves the Next dashboard at `http://127.0.0.1:8080/` and health at `/health`. Set `QQQ_ALL_PORT` to change the port and `QQQ_ALL_DATA_DIR` to change the default `./data` directory. On first start it seeds customers and orders in H2, order lines in SQLite, and products as JSON files. Artemis runs in the same JVM; inserts and updates to `order` publish to the `orderEvents` topic and trigger `syncOrder`.
Core uses mock authentication. The `sessionId` cookie selects the two fixed demo identities: `aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa` for Admin and `dddddddd-dddd-4ddd-8ddd-dddddddddddd` for Demo User. Any other session ID uses Demo User. These identities are for local demonstrations; role-based permissions are wired in task C5.
## Full profile
Set `QQQ_ALL_PROFILE=full` to use external services. The full profile requires an explicit `QQQ_ALL_BIND_HOST`; use `127.0.0.1` for local-only access or deliberately set another interface when a reverse proxy needs it. `QQQ_ALL_PORT` defaults to `8080`. The full profile replaces core's mock authentication with Keycloak OIDC. It does not start an embedded broker.
Configure the following environment variables before starting the same jar:
| Service | Required variables | Optional variables (default) |
| --- | --- | --- |
| PostgreSQL | `QQQ_ALL_POSTGRES_HOST`, `QQQ_ALL_POSTGRES_DATABASE`, `QQQ_ALL_POSTGRES_USER`, `QQQ_ALL_POSTGRES_PASSWORD` | `QQQ_ALL_POSTGRES_PORT` (5432) |
| MySQL | `QQQ_ALL_MYSQL_HOST`, `QQQ_ALL_MYSQL_DATABASE`, `QQQ_ALL_MYSQL_USER`, `QQQ_ALL_MYSQL_PASSWORD` | `QQQ_ALL_MYSQL_PORT` (3306) |
| MongoDB replica set | `QQQ_ALL_MONGO_HOST`, `QQQ_ALL_MONGO_DATABASE`, `QQQ_ALL_MONGO_USER`, `QQQ_ALL_MONGO_PASSWORD`, `QQQ_ALL_MONGO_REPLICA_SET` | `QQQ_ALL_MONGO_PORT` (27017), `QQQ_ALL_MONGO_AUTH_DATABASE` (admin) |
| MinIO S3 | `QQQ_ALL_S3_ENDPOINT`, `QQQ_ALL_S3_BUCKET`, `QQQ_ALL_S3_ACCESS_KEY`, `QQQ_ALL_S3_SECRET_KEY` | `QQQ_ALL_S3_REGION` (us-east-1) |
| SFTP | `QQQ_ALL_SFTP_HOST`, `QQQ_ALL_SFTP_USER`, `QQQ_ALL_SFTP_PASSWORD` | `QQQ_ALL_SFTP_PORT` (22), `QQQ_ALL_SFTP_BASE_PATH` (/upload) |
| ESB | `QQQ_ALL_ARTEMIS_URL`, `QQQ_ALL_RABBITMQ_URL`, `QQQ_ALL_RABBITMQ_USER`, `QQQ_ALL_RABBITMQ_PASSWORD` | — |
| Mailpit SMTP | `QQQ_ALL_SMTP_HOST` | `QQQ_ALL_SMTP_PORT` (1025) |
| Keycloak OIDC | `QQQ_ALL_OIDC_BASE_URL`, `QQQ_ALL_OIDC_CLIENT_ID`, `QQQ_ALL_OIDC_CLIENT_SECRET` | `QQQ_ALL_OIDC_EXTERNAL_BASE_URL` (base URL), `QQQ_ALL_OIDC_SCOPES` (`openid email profile`) |
The full app adds sample tables on each external backend, `orderEvents` on Artemis, and `orderSyncEvents` on RabbitMQ. Keycloak realm roles resolve by name against the PostgreSQL `role` table from `qbit-user-role-permissions`, then become QQQ session permissions. Unknown roles receive no permissions. PostgreSQL must also contain the qbit permission tables and OAuth session and redirect-state tables. External database schema, demo roles, the MinIO bucket, and the service stack are provisioned by the forthcoming compose and seed work in C6; this C4 profile expects those resources to exist.
+5 -1
View File
@@ -31,12 +31,16 @@
<dependencies>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-core</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-module-rdbms</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-module-mongodb</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-module-sqlite</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-module-filesystem</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-middleware-javalin</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-middleware-health</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-esb</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-frontend-next</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-user-role-permissions</artifactId></dependency>
<dependency><groupId>org.postgresql</groupId><artifactId>postgresql</artifactId></dependency>
<dependency><groupId>com.mysql</groupId><artifactId>mysql-connector-j</artifactId></dependency>
<dependency><groupId>com.h2database</groupId><artifactId>h2</artifactId></dependency>
<dependency><groupId>org.xerial</groupId><artifactId>sqlite-jdbc</artifactId></dependency>
<dependency><groupId>org.apache.artemis</groupId><artifactId>artemis-jakarta-server</artifactId><version>2.57.0</version></dependency>
@@ -57,7 +61,7 @@
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.kingsrook.qqq.all.QqqAllApplication</mainClass>
<mainClass>com.kingsrook.qqq.all.ProfileLauncher</mainClass>
</transformer>
</transformers>
<filters>
@@ -0,0 +1,92 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.userrolepermissions.model.Role;
import com.kingsrook.qbits.userrolepermissions.utils.PermissionManager;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
import org.json.JSONArray;
import org.json.JSONObject;
/** Resolves Keycloak realm roles through qbit-user-role-permissions. */
public class FullOidcSessionCustomizer implements QAuthenticationModuleCustomizerInterface
{
@Override
public void customizeSession(QInstance instance, QSession session, Map<String, Object> context)
{
Set<String> roles = realmRoles(context.get("idToken") instanceof JSONObject idToken ? idToken : null);
if(roles.isEmpty() && context.get("jwtPayloadJsonObject") instanceof JSONObject accessToken)
{
roles = realmRoles(accessToken);
}
try
{
session.setPermissions(permissionsForRoles(roles));
}
catch(QException e)
{
throw new IllegalStateException("Could not resolve OIDC role permissions", e);
}
}
private static Set<String> realmRoles(JSONObject claims)
{
if(claims == null)
{
return Set.of();
}
JSONObject realmAccess = claims.optJSONObject("realm_access");
JSONArray values = realmAccess == null ? null : realmAccess.optJSONArray("roles");
if(values == null)
{
return Set.of();
}
Set<String> roles = new HashSet<>();
for(Integer i = 0; i < values.length(); i++)
{
if(values.opt(i) instanceof String role && !role.isBlank())
{
roles.add(role);
}
}
return roles;
}
protected Set<String> permissionsForRoles(Set<String> roles) throws QException
{
if(roles.isEmpty())
{
return Set.of();
}
QSession previousSession = QContext.getQSession();
try
{
QContext.setQSession(new QSystemUserSession());
Set<Integer> roleIds = new QueryAction().execute(new QueryInput(Role.TABLE_NAME)
.withFilter(new QQueryFilter().withCriteria("name", QCriteriaOperator.IN, roles)))
.getRecords().stream().map(record -> record.getValueInteger("id")).collect(Collectors.toSet());
return PermissionManager.getInstance().getEffectivePermissionsForRoles(roleIds);
}
finally
{
QContext.setQSession(previousSession);
}
}
}
@@ -0,0 +1,175 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.List;
import com.kingsrook.qbits.userrolepermissions.UserRolePermissionsQBitConfig;
import com.kingsrook.qbits.userrolepermissions.UserRolePermissionsQBitProducer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.AuthScope;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.OAuth2AuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.messaging.email.EmailMessagingProviderMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.ProvidedOrSuppliedTableConfig;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.modules.authentication.implementations.metadata.RedirectStateMetaDataProducer;
import com.kingsrook.qqq.backend.core.modules.authentication.implementations.metadata.UserSessionMetaDataProducer;
import com.kingsrook.qqq.backend.core.modules.authentication.implementations.model.UserSession;
import com.kingsrook.qqq.backend.module.filesystem.base.model.metadata.Cardinality;
import com.kingsrook.qqq.backend.module.filesystem.base.model.metadata.RecordFormat;
import com.kingsrook.qqq.backend.module.filesystem.s3.model.metadata.S3TableBackendDetails;
import com.kingsrook.qqq.backend.module.filesystem.sftp.model.metadata.SFTPBackendMetaData;
import com.kingsrook.qqq.backend.module.filesystem.sftp.model.metadata.SFTPTableBackendDetails;
import com.kingsrook.qqq.backend.module.mongodb.model.metadata.MongoDBBackendMetaData;
import com.kingsrook.qqq.backend.module.mongodb.model.metadata.MongoDBTableBackendDetails;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
import com.kingsrook.qqq.esb.model.EsbDestinationType;
import com.kingsrook.qqq.esb.model.EsbInstanceMetaData;
import com.kingsrook.qqq.esb.model.EsbProcessEvent;
import com.kingsrook.qqq.esb.model.EsbProcessMetaData;
import com.kingsrook.qqq.esb.model.EsbProcessPublication;
import com.kingsrook.qqq.esb.model.EsbProviderType;
import com.kingsrook.qqq.esb.model.QEsbDestinationMetaData;
import com.kingsrook.qqq.esb.model.QEsbProviderMetaData;
/** Extends the core demo with external services selected by the full profile. */
final class FullProfileApplication extends AbstractQQQApplication
{
private final Path dataDirectory;
private final FullProfileSettings settings;
FullProfileApplication(Path dataDirectory, FullProfileSettings settings)
{
this.dataDirectory = dataDirectory;
this.settings = settings;
}
@Override
public QInstance defineQInstance() throws QException
{
QInstance instance = new QqqAllApplication(dataDirectory, settings.get("ARTEMIS_URL")).defineQInstance();
addBackends(instance);
addTables(instance);
addAuthentication(instance);
addEsb(instance);
EmailMessagingProviderMetaData mail = new EmailMessagingProviderMetaData()
.withSmtpServer(settings.get("SMTP_HOST"))
.withSmtpPort(settings.getOrDefault("SMTP_PORT", "1025"));
mail.setName("mailpit");
instance.addMessagingProvider(mail);
return instance;
}
private void addBackends(QInstance instance)
{
instance.addBackend(new RDBMSBackendMetaData().withName("postgres").withVendor("postgres")
.withHostName(settings.get("POSTGRES_HOST")).withPort(settings.port("POSTGRES_PORT", "5432"))
.withDatabaseName(settings.get("POSTGRES_DATABASE"))
.withUsername(settings.get("POSTGRES_USER")).withPassword(settings.get("POSTGRES_PASSWORD")));
instance.addBackend(new RDBMSBackendMetaData().withName("mysql").withVendor("mysql")
.withHostName(settings.get("MYSQL_HOST")).withPort(settings.port("MYSQL_PORT", "3306"))
.withDatabaseName(settings.get("MYSQL_DATABASE"))
.withUsername(settings.get("MYSQL_USER")).withPassword(settings.get("MYSQL_PASSWORD")));
instance.addBackend(new MongoDBBackendMetaData().withName("mongo")
.withHost(settings.get("MONGO_HOST")).withPort(settings.port("MONGO_PORT", "27017"))
.withDatabaseName(settings.get("MONGO_DATABASE"))
.withUsername(settings.get("MONGO_USER")).withPassword(settings.get("MONGO_PASSWORD"))
.withAuthSourceDatabase(settings.getOrDefault("MONGO_AUTH_DATABASE", "admin"))
.withUrlSuffix("replicaSet=" + settings.get("MONGO_REPLICA_SET"))
.withTransactionsSupported(true));
instance.addBackend(new MinioBackendMetaData(settings.get("S3_ENDPOINT")).withName("s3")
.withBucketName(settings.get("S3_BUCKET"))
.withAccessKey(settings.get("S3_ACCESS_KEY")).withSecretKey(settings.get("S3_SECRET_KEY"))
.withRegion(settings.getOrDefault("S3_REGION", "us-east-1")));
instance.addBackend(new SFTPBackendMetaData().withName("sftp")
.withHostName(settings.get("SFTP_HOST")).withPort(settings.port("SFTP_PORT", "22"))
.withUsername(settings.get("SFTP_USER")).withPassword(settings.get("SFTP_PASSWORD"))
.withBasePath(settings.getOrDefault("SFTP_BASE_PATH", "/upload")));
}
private void addTables(QInstance instance)
{
instance.addTable(new QTableMetaData().withName("warehouseCustomer").withLabel("Warehouse Customer")
.withBackendName("postgres").withPrimaryKeyField("id")
.withBackendDetails(new RDBMSTableBackendDetails().withTableName("warehouse_customer"))
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING)));
instance.addTable(new QTableMetaData().withName("supplierOrder").withLabel("Supplier Order")
.withBackendName("mysql").withPrimaryKeyField("id")
.withBackendDetails(new RDBMSTableBackendDetails().withTableName("supplier_order"))
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("reference", QFieldType.STRING)));
instance.addTable(new QTableMetaData().withName("shipment").withLabel("Shipment")
.withBackendName("mongo").withPrimaryKeyField("id")
.withBackendDetails(new MongoDBTableBackendDetails().withTableName("shipment"))
.withField(new QFieldMetaData("id", QFieldType.STRING).withBackendName("_id"))
.withField(new QFieldMetaData("tracking", QFieldType.STRING)));
instance.addTable(new QTableMetaData().withName("document").withLabel("Document")
.withBackendName("s3").withPrimaryKeyField("id")
.withBackendDetails(new S3TableBackendDetails().withBasePath("documents")
.withRecordFormat(RecordFormat.JSON).withCardinality(Cardinality.MANY).withGlob("*.json"))
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING)));
instance.addTable(new QTableMetaData().withName("importFile").withLabel("Import File")
.withBackendName("sftp").withPrimaryKeyField("id")
.withBackendDetails(new SFTPTableBackendDetails().withBasePath("imports")
.withRecordFormat(RecordFormat.JSON).withCardinality(Cardinality.MANY).withGlob("*.json"))
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING)));
}
private void addAuthentication(QInstance instance) throws QException
{
instance.addTable(new UserSessionMetaDataProducer("postgres").produce(instance));
instance.addTable(new RedirectStateMetaDataProducer("postgres").produce(instance));
UserRolePermissionsQBitConfig qbitConfig = new UserRolePermissionsQBitConfig()
.withUserTableConfig(ProvidedOrSuppliedTableConfig.provideTableUsingBackendNamed("postgres"))
.withTableMetaDataCustomizer((i, table) ->
{
table.setBackendName("postgres");
table.setBackendDetails(new RDBMSTableBackendDetails()
.withTableName(QInstanceEnricher.inferBackendName(table.getName())));
QInstanceEnricher.setInferredFieldBackendNames(table);
return table;
});
new UserRolePermissionsQBitProducer().withUserRolePermissionsQBitConfig(qbitConfig).produce(instance);
instance.registerAuthenticationProvider(AuthScope.instanceDefault(), new OAuth2AuthenticationMetaData()
.withBaseUrl(settings.get("OIDC_BASE_URL"))
.withExternalBaseUrl(settings.getOrDefault("OIDC_EXTERNAL_BASE_URL", settings.get("OIDC_BASE_URL")))
.withClientId(settings.get("OIDC_CLIENT_ID"))
.withClientSecret(settings.get("OIDC_CLIENT_SECRET"))
.withScopes(settings.getOrDefault("OIDC_SCOPES", "openid email profile"))
.withUserSessionTableName(UserSession.TABLE_NAME)
.withRedirectStateTableName(RedirectStateMetaDataProducer.TABLE_NAME)
.withCustomizer(new QCodeReference(FullOidcSessionCustomizer.class))
.withName("keycloak"));
}
private void addEsb(QInstance instance)
{
EsbInstanceMetaData.of(instance)
.withProvider(new QEsbProviderMetaData().withName("rabbitmq")
.withType(EsbProviderType.RABBITMQ).withUrl(settings.get("RABBITMQ_URL"))
.withUsername(settings.get("RABBITMQ_USER")).withPassword(settings.get("RABBITMQ_PASSWORD")))
.withDestination(new QEsbDestinationMetaData().withName("orderSyncEvents")
.withType(EsbDestinationType.QUEUE).withProviderName("rabbitmq")
.withDestinationName("qqq.all.orderSyncEvents"));
EsbProcessMetaData.ofOrWithNew(instance.getProcess("syncOrder"))
.withPublication(new EsbProcessPublication().withDestinationName("orderSyncEvents")
.withEvents(List.of(EsbProcessEvent.COMPLETED)));
}
}
@@ -0,0 +1,64 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.util.List;
import java.util.Map;
/** Validated environment settings for the external services in the full profile. */
final class FullProfileSettings
{
private static final List<String> REQUIRED = List.of(
"BIND_HOST",
"POSTGRES_HOST", "POSTGRES_DATABASE", "POSTGRES_USER", "POSTGRES_PASSWORD",
"MYSQL_HOST", "MYSQL_DATABASE", "MYSQL_USER", "MYSQL_PASSWORD",
"MONGO_HOST", "MONGO_DATABASE", "MONGO_USER", "MONGO_PASSWORD", "MONGO_REPLICA_SET",
"S3_ENDPOINT", "S3_BUCKET", "S3_ACCESS_KEY", "S3_SECRET_KEY",
"SFTP_HOST", "SFTP_USER", "SFTP_PASSWORD",
"ARTEMIS_URL", "RABBITMQ_URL", "RABBITMQ_USER", "RABBITMQ_PASSWORD",
"SMTP_HOST", "OIDC_BASE_URL", "OIDC_CLIENT_ID", "OIDC_CLIENT_SECRET");
private final Map<String, String> values;
private FullProfileSettings(Map<String, String> values)
{
this.values = Map.copyOf(values);
}
static FullProfileSettings from(Map<String, String> environment)
{
for(String name : REQUIRED)
{
if(environment.get("QQQ_ALL_" + name) == null || environment.get("QQQ_ALL_" + name).isBlank())
{
throw new IllegalArgumentException("Missing required environment variable QQQ_ALL_" + name);
}
}
return new FullProfileSettings(environment);
}
String get(String name)
{
return values.get("QQQ_ALL_" + name);
}
String getOrDefault(String name, String defaultValue)
{
String value = get(name);
return value == null || value.isBlank() ? defaultValue : value;
}
Integer port(String name, String defaultValue)
{
Integer port = Integer.valueOf(getOrDefault(name, defaultValue));
if(port < 1 || port > 65535)
{
throw new IllegalArgumentException("Invalid port in QQQ_ALL_" + name);
}
return port;
}
}
@@ -0,0 +1,26 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import com.kingsrook.qqq.backend.module.filesystem.s3.model.metadata.S3BackendMetaData;
/** S3 metadata with an explicit endpoint for MinIO. */
public final class MinioBackendMetaData extends S3BackendMetaData
{
private final String endpoint;
public MinioBackendMetaData(String endpoint)
{
this.endpoint = endpoint;
setBackendType(MinioBackendModule.class);
}
public String getEndpoint()
{
return endpoint;
}
}
@@ -0,0 +1,138 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.kingsrook.qqq.backend.core.actions.interfaces.CountInterface;
import com.kingsrook.qqq.backend.core.actions.interfaces.DeleteInterface;
import com.kingsrook.qqq.backend.core.actions.interfaces.InsertInterface;
import com.kingsrook.qqq.backend.core.actions.interfaces.QStorageInterface;
import com.kingsrook.qqq.backend.core.actions.interfaces.QueryInterface;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
import com.kingsrook.qqq.backend.module.filesystem.base.actions.AbstractBaseFilesystemAction;
import com.kingsrook.qqq.backend.module.filesystem.s3.S3BackendModule;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.AbstractS3Action;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.S3CountAction;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.S3DeleteAction;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.S3InsertAction;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.S3QueryAction;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.S3StorageAction;
/** Routes every S3 operation to the configured MinIO endpoint. */
public final class MinioBackendModule extends S3BackendModule
{
static
{
QBackendModuleDispatcher.registerBackendModule(new MinioBackendModule());
}
@Override
public String getBackendType()
{
return "minio";
}
@Override
public Class<? extends QBackendMetaData> getBackendMetaDataClass()
{
return MinioBackendMetaData.class;
}
private static AmazonS3 client(QBackendMetaData metadata)
{
MinioBackendMetaData minio = (MinioBackendMetaData) metadata;
return AmazonS3ClientBuilder.standard()
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(minio.getEndpoint(), minio.getRegion()))
.withPathStyleAccessEnabled(true)
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(minio.getAccessKey(), minio.getSecretKey())))
.build();
}
@Override
public AbstractBaseFilesystemAction<S3ObjectSummary> getActionBase()
{
return new AbstractS3Action()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
@Override
public QueryInterface getQueryInterface()
{
return new S3QueryAction()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
@Override
public InsertInterface getInsertInterface()
{
return new S3InsertAction()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
@Override
public CountInterface getCountInterface()
{
return new S3CountAction()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
@Override
public DeleteInterface getDeleteInterface()
{
return new S3DeleteAction()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
@Override
public QStorageInterface getStorageInterface()
{
return new S3StorageAction()
{
@Override
protected AmazonS3 buildAmazonS3ClientFromBackendMetaData(QBackendMetaData metadata)
{
return client(metadata);
}
};
}
}
@@ -0,0 +1,54 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.Map;
import com.kingsrook.qqq.middleware.javalin.QApplicationLauncher;
import com.kingsrook.qqq.middleware.javalin.QApplicationLauncherConfig;
/** Selects the standalone or externally backed reference application. */
public final class ProfileLauncher
{
private ProfileLauncher()
{
}
public static void main(String[] args) throws Exception
{
launch(System.getenv());
}
static String profile(Map<String, String> environment)
{
String selected = environment.getOrDefault("QQQ_ALL_PROFILE", "core");
if(!"core".equals(selected) && !"full".equals(selected))
{
throw new IllegalArgumentException("QQQ_ALL_PROFILE must be core or full");
}
return selected;
}
static void launch(Map<String, String> environment) throws Exception
{
String selected = profile(environment);
Path dataDirectory = Path.of(environment.getOrDefault("QQQ_ALL_DATA_DIR", "./data"));
Integer port = Integer.valueOf(environment.getOrDefault("QQQ_ALL_PORT", "8080"));
if("core".equals(selected))
{
CoreRuntime.launch(dataDirectory, port);
return;
}
FullProfileSettings settings = FullProfileSettings.from(environment);
CoreData.seed(dataDirectory);
QApplicationLauncher.run(new FullProfileApplication(dataDirectory, settings),
new QApplicationLauncherConfig().withRegisterShutdownHook(true)
.withServerCustomizer(server -> server.withPort(port)
.withJavalinConfigCustomizer(config -> config.jetty.host = settings.get("BIND_HOST"))));
}
}
@@ -0,0 +1,104 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.userrolepermissions.model.Role;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.AuthScope;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
/** Keycloak realm roles must flow through the role permission resolver on login and resume. */
class FullOidcSessionCustomizerTest
{
@Test
void resolvesRolesBeforeAnAuthenticationSessionExists()
{
QInstance instance = new QInstance();
instance.registerAuthenticationProvider(AuthScope.instanceDefault(),
new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK));
instance.addBackend(new QBackendMetaData().withName("memory")
.withBackendType(MemoryBackendModule.class));
instance.addTable(new QTableMetaData().withName(Role.TABLE_NAME).withBackendName("memory")
.withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING)));
QContext.init(instance, null);
try
{
QSession session = new QSession();
new FullOidcSessionCustomizer().customizeSession(instance, session,
Map.of("idToken", new JSONObject("{\"realm_access\":{\"roles\":[\"viewer\"]}}")));
assertEquals(Set.of(), session.getPermissions());
assertNull(QContext.getQSession());
}
finally
{
QContext.clear();
}
}
@Test
void mapsIdTokenRolesToSessionPermissions()
{
RecordingCustomizer customizer = new RecordingCustomizer();
QSession session = new QSession();
customizer.customizeSession(null, session, Map.of("idToken", new JSONObject("{\"realm_access\":{\"roles\":[\"admin\",\"viewer\"]}}")));
assertEquals(Set.of("admin", "viewer"), customizer.requestedRoles);
assertEquals(Set.of("customer.read"), session.getPermissions());
}
@Test
void usesAccessTokenRolesOnSessionResumeAndDoesNotGrantUnknownRoles()
{
RecordingCustomizer customizer = new RecordingCustomizer();
QSession session = new QSession();
customizer.customizeSession(null, session, Map.of("jwtPayloadJsonObject", new JSONObject("{\"realm_access\":{\"roles\":[\"viewer\"]}}")));
assertEquals(Set.of("viewer"), customizer.requestedRoles);
QSession noRoles = new QSession();
customizer.customizeSession(null, noRoles, Map.of("jwtPayloadJsonObject", new JSONObject("{}")));
assertEquals(Set.of(), noRoles.getPermissions());
}
@Test
void usesAccessTokenRolesWhenIdTokenHasNoRealmRoles()
{
RecordingCustomizer customizer = new RecordingCustomizer();
QSession session = new QSession();
customizer.customizeSession(null, session, Map.of(
"idToken", new JSONObject("{\"email\":\"demo@example.test\"}"),
"jwtPayloadJsonObject", new JSONObject("{\"realm_access\":{\"roles\":[\"viewer\"]}}")));
assertEquals(Set.of("viewer"), customizer.requestedRoles);
}
private static class RecordingCustomizer extends FullOidcSessionCustomizer
{
private Set<String> requestedRoles;
@Override
protected Set<String> permissionsForRoles(Set<String> roles)
{
requestedRoles = roles;
return roles.isEmpty() ? Set.of() : Set.of("customer.read");
}
}
}
@@ -0,0 +1,106 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Map;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.OAuth2AuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.messaging.email.EmailMessagingProviderMetaData;
import com.kingsrook.qqq.backend.module.mongodb.model.metadata.MongoDBBackendMetaData;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import com.kingsrook.qqq.esb.model.EsbInstanceMetaData;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** Full profile wiring tests that require no external services. */
class FullProfileTest
{
@TempDir
Path dataDirectory;
@Test
void rejectsMissingFullProfileCredentialsBeforeStartup()
{
assertThrows(IllegalArgumentException.class, () -> FullProfileSettings.from(Map.of("QQQ_ALL_PROFILE", "full")));
}
@Test
void wiresEveryFullBackendBrokerMailAndOidcFromEnvironment() throws Exception
{
FullProfileSettings settings = FullProfileSettings.from(environment());
QInstance instance = new FullProfileApplication(dataDirectory, settings).defineQInstance();
new QInstanceValidator().validate(instance);
assertEquals("postgres.example.test", ((RDBMSBackendMetaData) instance.getBackend("postgres")).getHostName());
assertEquals("mysql.example.test", ((RDBMSBackendMetaData) instance.getBackend("mysql")).getHostName());
assertEquals("replicaSet=rs0", ((MongoDBBackendMetaData) instance.getBackend("mongo")).getUrlSuffix());
assertEquals("postgres", instance.getTable("warehouseCustomer").getBackendName());
assertEquals("mysql", instance.getTable("supplierOrder").getBackendName());
assertEquals("mongo", instance.getTable("shipment").getBackendName());
assertEquals("http://minio.example.test:9000",
((MinioBackendMetaData) instance.getBackend("s3")).getEndpoint());
assertEquals("s3", instance.getTable("document").getBackendName());
assertEquals("sftp", instance.getTable("importFile").getBackendName());
EsbInstanceMetaData esb = EsbInstanceMetaData.of(instance);
assertEquals("tcp://artemis.example.test:61616", esb.getProvider("artemis").getUrl());
assertEquals("amqp://rabbit.example.test:5672/%2F", esb.getProvider("rabbitmq").getUrl());
assertEquals("artemis", esb.getDestination("orderEvents").getProviderName());
assertEquals("rabbitmq", esb.getDestination("orderSyncEvents").getProviderName());
OAuth2AuthenticationMetaData auth = (OAuth2AuthenticationMetaData) instance.getAuthentication();
assertEquals(QAuthenticationType.OAUTH2, auth.getType());
assertEquals("https://keycloak.example.test/realms/demo", auth.getBaseUrl());
assertNotNull(instance.getTable(auth.getUserSessionTableName()));
assertNotNull(instance.getTable(auth.getRedirectStateTableName()));
assertEquals("mailpit.example.test", ((EmailMessagingProviderMetaData) instance.getMessagingProvider("mailpit")).getSmtpServer());
assertTrue(instance.getQBits().values().stream().anyMatch(qbit -> "user-role-permissions".equals(qbit.getArtifactId())));
}
static Map<String, String> environment()
{
Map<String, String> values = new HashMap<>();
values.put("QQQ_ALL_BIND_HOST", "127.0.0.1");
values.put("QQQ_ALL_POSTGRES_HOST", "postgres.example.test");
values.put("QQQ_ALL_POSTGRES_DATABASE", "qqq");
values.put("QQQ_ALL_POSTGRES_USER", "fixture");
values.put("QQQ_ALL_POSTGRES_PASSWORD", "fixture");
values.put("QQQ_ALL_MYSQL_HOST", "mysql.example.test");
values.put("QQQ_ALL_MYSQL_DATABASE", "qqq");
values.put("QQQ_ALL_MYSQL_USER", "fixture");
values.put("QQQ_ALL_MYSQL_PASSWORD", "fixture");
values.put("QQQ_ALL_MONGO_HOST", "mongo.example.test");
values.put("QQQ_ALL_MONGO_DATABASE", "qqq");
values.put("QQQ_ALL_MONGO_USER", "fixture");
values.put("QQQ_ALL_MONGO_PASSWORD", "fixture");
values.put("QQQ_ALL_MONGO_REPLICA_SET", "rs0");
values.put("QQQ_ALL_S3_BUCKET", "qqq-demo");
values.put("QQQ_ALL_S3_ENDPOINT", "http://minio.example.test:9000");
values.put("QQQ_ALL_S3_ACCESS_KEY", "fixture");
values.put("QQQ_ALL_S3_SECRET_KEY", "fixture");
values.put("QQQ_ALL_SFTP_HOST", "sftp.example.test");
values.put("QQQ_ALL_SFTP_USER", "fixture");
values.put("QQQ_ALL_SFTP_PASSWORD", "fixture");
values.put("QQQ_ALL_ARTEMIS_URL", "tcp://artemis.example.test:61616");
values.put("QQQ_ALL_RABBITMQ_URL", "amqp://rabbit.example.test:5672/%2F");
values.put("QQQ_ALL_RABBITMQ_USER", "fixture");
values.put("QQQ_ALL_RABBITMQ_PASSWORD", "fixture");
values.put("QQQ_ALL_SMTP_HOST", "mailpit.example.test");
values.put("QQQ_ALL_OIDC_BASE_URL", "https://keycloak.example.test/realms/demo");
values.put("QQQ_ALL_OIDC_CLIENT_ID", "qqq-all");
values.put("QQQ_ALL_OIDC_CLIENT_SECRET", "fixture");
return values;
}
}
@@ -0,0 +1,112 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicReference;
import com.sun.net.httpserver.HttpServer;
import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
import com.kingsrook.qqq.backend.module.filesystem.s3.actions.AbstractS3Action;
import com.kingsrook.qqq.backend.module.filesystem.s3.utils.S3Utils;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertTrue;
class MinioBackendTest
{
@Test
void usesConfiguredEndpointForAllFilesystemActions() throws Exception
{
MinioBackendMetaData backend = new MinioBackendMetaData("http://minio.example.test:9000");
backend.setName("s3");
backend.setBucketName("qqq-demo");
backend.setAccessKey("fixture");
backend.setSecretKey("fixture");
backend.setRegion("us-east-1");
MinioBackendModule module = assertInstanceOf(MinioBackendModule.class,
new QBackendModuleDispatcher().getQBackendModule(backend));
assertUrl((AbstractS3Action) module.getActionBase(), backend);
assertUrl((AbstractS3Action) module.getQueryInterface(), backend);
assertUrl((AbstractS3Action) module.getInsertInterface(), backend);
assertUrl((AbstractS3Action) module.getCountInterface(), backend);
assertUrl((AbstractS3Action) module.getDeleteInterface(), backend);
assertUrl((AbstractS3Action) module.getStorageInterface(), backend);
}
private static void assertUrl(AbstractS3Action action, MinioBackendMetaData backend) throws Exception
{
action.preAction(backend);
S3Utils utils = utils(action);
assertEquals("http://minio.example.test:9000/qqq-demo/example.json",
utils.getAmazonS3().getUrl("qqq-demo", "example.json").toString());
}
@Test
void queryAndWriteUseLocalS3Endpoint() throws Exception
{
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
AtomicReference<String> written = new AtomicReference<>();
server.createContext("/", exchange ->
{
byte[] response;
if("PUT".equals(exchange.getRequestMethod()))
{
written.set(exchange.getRequestURI().getPath() + ":" +
new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8));
exchange.getResponseHeaders().add("ETag", "\"fixture-etag\"");
response = new byte[0];
}
else
{
String xml = "<ListBucketResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\">"
+ "<Name>qqq-demo</Name><IsTruncated>false</IsTruncated>"
+ "<Contents><Key>documents/example.json</Key><Size>2</Size>"
+ "<LastModified>2026-01-01T00:00:00.000Z</LastModified></Contents></ListBucketResult>";
response = xml.getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().add("Content-Type", "application/xml");
}
exchange.sendResponseHeaders(200, response.length);
exchange.getResponseBody().write(response);
exchange.close();
});
server.start();
try
{
MinioBackendMetaData backend = new MinioBackendMetaData("http://127.0.0.1:" + server.getAddress().getPort());
backend.setName("s3");
backend.setBucketName("qqq-demo");
backend.setAccessKey("fixture");
backend.setSecretKey("fixture");
backend.setRegion("us-east-1");
MinioBackendModule module = new MinioBackendModule();
AbstractS3Action query = (AbstractS3Action) module.getQueryInterface();
query.preAction(backend);
assertEquals("documents/example.json", utils(query).getAmazonS3()
.listObjectsV2("qqq-demo").getObjectSummaries().getFirst().getKey());
AbstractS3Action insert = (AbstractS3Action) module.getInsertInterface();
insert.preAction(backend);
utils(insert).writeFile("qqq-demo", "documents/new.json", "{}".getBytes(StandardCharsets.UTF_8), "application/json");
assertTrue(written.get().startsWith("/qqq-demo/documents/new.json:"));
assertTrue(written.get().contains("\r\n{}\r\n"));
}
finally
{
server.stop(0);
}
}
private static S3Utils utils(AbstractS3Action action) throws Exception
{
var accessor = AbstractS3Action.class.getDeclaredMethod("getS3Utils");
accessor.setAccessible(true);
return (S3Utils) accessor.invoke(action);
}
}
@@ -0,0 +1,43 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.util.HashMap;
import java.util.Map;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
/** The executable profile selector must choose the right startup path. */
class ProfileLauncherTest
{
@Test
void selectsCoreByDefaultAndFullWhenRequested()
{
assertEquals("core", ProfileLauncher.profile(Map.of()));
assertEquals("full", ProfileLauncher.profile(Map.of("QQQ_ALL_PROFILE", "full")));
assertThrows(IllegalArgumentException.class,
() -> ProfileLauncher.profile(Map.of("QQQ_ALL_PROFILE", "typo")));
}
@Test
void fullFailsForMissingConfigurationBeforeOpeningServices()
{
assertThrows(IllegalArgumentException.class,
() -> ProfileLauncher.launch(Map.of("QQQ_ALL_PROFILE", "full")));
}
@Test
void fullRequiresAnExplicitHttpBindAddress()
{
Map<String, String> environment = new HashMap<>(FullProfileTest.environment());
environment.remove("QQQ_ALL_BIND_HOST");
assertEquals("Missing required environment variable QQQ_ALL_BIND_HOST",
assertThrows(IllegalArgumentException.class,
() -> FullProfileSettings.from(environment)).getMessage());
}
}