Compare commits

...
Author SHA1 Message Date
james.maes 76b661326e Add .circleci/config.yml 2026-09-26 01:09:01 -05:00
james.maes d45ee4e4c5 Merge pull request #11 from QRun-IO/feature/GH-1-plan-and-design
docs: qqq-all design and implementation plan (#1)
2026-09-25 19:48:15 -05:00
james.maesandClaude Opus 5.5 a5c3651239 docs: apply round-2 review suggestions
Apache-2.0 headers for new qqq-all files, launcher getter note, explicit
qbit version pins, C7 broker round trip, C4/C5 sequenced, ESB task 14
in C3 dependencies.

Part of #1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 19:47:46 -05:00
james.maesandClaude Opus 5.5 730434fa5c docs: resolve plan review findings
Real verification path for qbits on 4.1.0-SNAPSHOT (opt-in profile),
design wording aligned to the 4.0.0 parent, launcher reuses the
server-built instance, waves split so no task waits on its own wave,
licensing left unchanged pending an owner decision.

Part of #1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 19:31:29 -05:00
james.maesandClaude Opus 5.5 9192ac5778 docs: add qqq-all design and implementation plan
Part of #1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 19:15:09 -05:00
3 changed files with 200 additions and 0 deletions

No files matched your search

+31
View File
@@ -0,0 +1,31 @@
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/reference/configuration-reference/
version: 2.1
# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/guides/orchestrate/jobs-steps/#jobs-overview & https://circleci.com/docs/reference/configuration-reference/#jobs
jobs:
say-hello:
# Specify the execution environment. You can specify an image from Docker Hub or use one of our convenience images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/guides/execution-managed/executor-intro/ & https://circleci.com/docs/reference/configuration-reference/#executor-job
docker:
# Specify the version you desire here
# See: https://circleci.com/developer/images/image/cimg/base
- image: cimg/base:current
# Add steps to the job
# See: https://circleci.com/docs/guides/orchestrate/jobs-steps/#steps-overview & https://circleci.com/docs/reference/configuration-reference/#steps
steps:
# Checkout the code as the first step.
- checkout
- run:
name: "Say hello"
command: "echo Hello, World!"
# Orchestrate jobs using workflows
# See: https://circleci.com/docs/guides/orchestrate/workflows/ & https://circleci.com/docs/reference/configuration-reference/#workflows
workflows:
say-hello-workflow: # This is the name of the workflow, feel free to change it to better match your workflow.
# Inside the workflow, you define the jobs you want to run.
jobs:
- say-hello
+33
View File
@@ -0,0 +1,33 @@
# qqq-all — Design (approved 2026-09-25)
## Goal
One repo that gives a fully running QQQ app with every feature configured and working out of the box. New public repo `QRun-IO/qqq-all`, Apache-2.0. Version tracks qqq (4.1.x); builds on 4.1.0-SNAPSHOT until 4.1 ships.
## What's in it
- **Stack BOM** — one tested version set: qqq modules, `qqq-esb`, `qqq-frontend-next`, the included qbits, JDBC drivers, broker clients.
- **Reference app** — every backend, qbit, and the ESB wired, with seeded demo data.
- **Compose file** with two profiles.
- **Image** on GHCR; CI boots the stack and runs smoke tests plus the ESB conformance suite.
## Profiles
- **`core`** — no containers, `java -jar`: H2, SQLite, local filesystem, embedded Artemis, mock auth.
- **`full`** — `docker compose --profile full up`: Postgres, MySQL, MongoDB (replica set), MinIO (S3), SFTP, OpenSearch, ActiveMQ Artemis, RabbitMQ, Mailpit, Keycloak (demo admin and user accounts).
## Qbits (v1)
- **In:** quick-search, user-role-permissions, customizable-table-views, standard-process-trace, webhooks, workflows, geo-data, sftp-data-integration.
- **Deferred:** crm (being rehabilitated for the Business Platform), wms (large), session-store (alpha), middleware-mcp (no auth), easypost / custom-apps / worm-audit (external keys or stubs).
- Each included qbit is re-pinned to `qbit-build-parent` 2.0.0 (qqq 4.0.0) and verified against 4.1.0-SNAPSHOT (one PR per qbit repo); it moves to a 4.1 parent at 4.1 GA. Quick-search also gets its index-drift fix and repo cleanup.
## Core prerequisites (qqq 4.1)
- **Launcher** — one entry point that starts the web server, scheduler, and ESB when configured. Every app gets it, not just qqq-all.
- **Fail-fast option** — a metadata producer that throws stops startup instead of logging a warning.
## Demo
- A small orders/customers domain that touches every feature: a table per backend, quick search on customers, order events on the ESB triggering a process, a workflow, webhooks, geo data, an SFTP import.
- Admin sees everything, including the ESB app; the demo user sees a permission-limited view.
## Limits
Local demo and reference only. Production still needs DNS, TLS, backups, monitoring, and HA brokers.
## Next
Approve → issues (qqq-all epic, 2 core issues in qqq, one re-pin issue per qbit) → agents, after ESB wave 1 lands.
+136
View File
@@ -0,0 +1,136 @@
# qqq-all Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. Each task is one GitHub issue and one PR. Work test-first; list the commands you ran and their results in the PR.
**Goal:** A public repo that runs QQQ with every feature configured and working out of the box, published once qqq 4.1 ships with the ESB and the Next dashboard as default.
**Design:** `docs/design.md` (approved 2026-09-25).
**Depends on:** ESB epic QRun-IO/qqq#739 and Next-as-default QRun-IO/qqq#649, both in qqq 4.1.
**Licensing:** do not change LICENSE files or source headers in any repo. Header/license alignment (several repos still carry AGPL headers and LICENSE files) is a separate owner decision.
## Global Constraints
- Java 21. qqq `4.1.0-SNAPSHOT` until 4.1 GA, then `4.1.x`. qqq-all version tracks qqq.
- Repos use gitflow: branch from and PR to `develop` (qqq-frontend-next: `main`). Never force-push; never delete branches you didn't create.
- Maven: never `mvn install` while other agents build; use `-pl … -am verify`.
- QQQ code style and checkstyle of the target repo. New files in `qqq-all` use the Apache-2.0 header; in other repos copy a neighboring file's header.
- No secrets in the repo. Demo credentials live only in `compose.yaml` and the Keycloak realm file, are marked demo-only, and are overridable by env.
- Releases, tags, and Maven Central or GHCR publishing of release versions: ask James first.
## Tasks
### Core prerequisites (QRun-IO/qqq, ship in 4.1)
**A1. Application launcher.** One entry point that starts everything configured.
- `qqq-backend-core`: `QRuntimeServiceInterface { String getName(); void start(QInstance); void stop(); }`, `QInstance.withRuntimeService(QCodeReference)` / `getRuntimeServices()`.
- `qqq-middleware-javalin`: `QApplicationLauncher.run(AbstractQQQApplication, QApplicationLauncherConfig)`. It creates `QApplicationJavalinServer` for the app and reuses the instance that server builds (never a second instance; add a `getQInstance()` getter to the server if it lacks one), then starts:
- `QApplicationJavalinServer`;
- `QScheduleManager` when the instance has schedules or scheduled jobs;
- each registered runtime service.
A JVM shutdown hook stops them in reverse order.
- `qqq-esb`: `EsbInstanceMetaData.enrich` registers `QEsbRuntime` as a runtime service. This part waits for ESB task 10 (QRun-IO/qqq#748).
- Tests: start order, reverse stop order, a service failing to start stops the ones already started and fails the launch, and the scheduler is skipped when nothing is scheduled.
**A2. Fail-fast metadata producers.** With `QInstance.withFailOnMetaDataProducerError(true)` (or system property `qqq.metaData.failOnProducerError=true`), `MetaDataProducerHelper` throws wherever it currently logs a warning and drops a producer (evaluating a candidate producer class during discovery, and executing a producer). The default is unchanged. Tests cover both modes.
### Qbit re-pins (one per repo)
**B1–B8.** Repos: `qbit-quick-search`, `qbit-user-role-permissions`, `qbit-customizable-table-views`, `qbit-standard-process-trace`, `qbit-webhooks`, `qbit-workflows`, `qbit-geo-data`, `qbit-sftp-data-integration`.
Each:
1. If `main` has commits not in `develop`, open a back-merge PR (`main` → `develop`) first. Resolve conflicts keeping develop's features and main's fixes; no force-push.
2. Parent → `com.kingsrook:qbit-build-parent:2.0.0` (imports `qqq-bom-pom` 4.0.0). Remove child `qqq-bom-pom` imports that shadow the parent. Java 21. Leave LICENSE files and headers as they are.
3. Build and test green on the parent's qqq 4.0.0. Also add an opt-in Maven profile `qqq-snapshot` that imports `qqq-bom-pom:${qqq.snapshot.version}` (default `4.1.0-SNAPSHOT`) ahead of the parent's BOM and adds the Central snapshots repository (`https://central.sonatype.com/repository/maven-snapshots/`); `-Pqqq-snapshot` must also build and test green. (The parent has no qqq-version property to override.) The move to a 4.1 parent happens at 4.1 GA (task C9).
4. Integration tests that need Docker fail, not skip, when `CI=true`.
5. Remove explicit qqq module `<version>`s in the child pom so the BOM (or the `qqq-snapshot` profile) controls them — `qbit-workflows` pins them explicitly on develop.
**B1 extra (quick-search):**
- Fix index drift: update and delete re-index or remove documents, and add a reconcile process that rebuilds the index from the source table.
- Remove the stray root `com/` directory and `.claude/settings.local.json`, and add `.claude/settings.local.json` to `.gitignore`.
### qqq-all repo (QRun-IO/qqq-all)
**C1. Bootstrap.**
- README, NOTICE, CODE_OF_CONDUCT, CONTRIBUTING, SECURITY (adapted from qqq), `.github` issue and PR templates, `.gitignore`.
- CircleCI using `qqq-orb`: build plus unit tests on PRs.
- Maven reactor: parent `pom.xml` with modules `qqq-all-bom` and `qqq-all-app`.
**C2. Stack BOM (`qqq-all-bom`).**
- Imports `qqq-bom-pom`.
- Manages: `qqq-esb`, `qqq-frontend-next`, the 8 qbits, JDBC drivers (PostgreSQL, MySQL, SQLite, H2), the ActiveMQ Artemis Jakarta client, and `rabbitmq-jms` 3.9.0.
- A test resolves every managed artifact.
**C3. Reference app, `core` profile (`qqq-all-app`).**
- `QqqAllApplication` starts through `QApplicationLauncher` (A1) with fail-fast producers on (A2).
- Demo domain: customers, orders, order lines, products, with seed data. Backends: H2, SQLite, and the local filesystem. Embedded Artemis for the ESB. Mock auth with admin and demo users.
- ESB: `order` publishes to topic `orderEvents`, and process `syncOrder` is triggered from it.
- Serves the Next dashboard.
- Tests: boots with `java -jar`, health returns 200, every table queries, and the ESB round trip works.
**C4. `full` profile wiring.**
- Env-driven backends: PostgreSQL, MySQL, MongoDB (replica set), S3 (MinIO), SFTP.
- ESB on both Artemis and RabbitMQ (one destination each).
- SMTP to Mailpit.
- OIDC through Keycloak (the OAuth2 module), with roles mapped to permissions via `qbit-user-role-permissions`.
- Profile selected by `QQQ_ALL_PROFILE=core|full`.
**C5. Qbits wired.** All 8 qbits are wired into the app and used by the demo:
- quick search on customers (OpenSearch in `full`, disabled with a notice in `core`);
- a table view, a process trace, a webhook, a workflow, geo data on addresses, and an SFTP import.
**C6. Compose and image.**
- `compose.yaml` with profiles `core` and `full`, healthchecks, and `.env.example`.
- Keycloak realm JSON (demo admin and user). Mongo replica-set init. MinIO bucket init. SFTP (atmoz/sftp). OpenSearch single-node. Artemis with Jolokia. RabbitMQ management.
- `Dockerfile`: Java 21 JRE, non-root user, healthcheck.
- CI publishes `ghcr.io/qrun-io/qqq-all:develop` (public) on develop merges.
**C7. CI smoke and conformance.**
- CI boots `core` via `java -jar` and `full` via `docker compose --profile full up -d --wait`.
- Assertions: health 200; the Next UI is served; each backend's table queries; quick search finds a seeded customer; the ESB round trip works on both brokers; OIDC login works for the demo users.
- The ESB round trip runs against both compose brokers (the qqq-esb conformance suite itself starts its own containers and stays in qqq CI).
**C8. Docs.**
- README quick start (one command per profile) and a feature tour mapping each QQQ feature to where the demo shows it.
- Limits: local demo and reference only; production needs DNS, TLS, backups, monitoring, HA brokers.
**C9. Publish (after qqq 4.1 GA with ESB and Next default; ask James first).**
- Release a `qbit-build-parent` that imports the 4.1 BOM, re-pin the 8 qbits to it, and release them.
- Pin qqq 4.1.x.
- Tag `4.1.0`.
- Publish the image and `qqq-all-bom` release.
- GitHub release with `compose.yaml`.
## Issues
Epic: QRun-IO/qqq-all#1.
| Task | Issue | Task | Issue |
|---|---|---|---|
| A1 | QRun-IO/qqq#762 | C1 | QRun-IO/qqq-all#2 |
| A2 | QRun-IO/qqq#763 | C2 | #3 |
| B1 quick-search | QRun-IO/qbit-quick-search#6 | C3 | #4 |
| B2 user-role-permissions | QRun-IO/qqq#764 | C4 | #5 |
| B3 customizable-table-views | #765 (qqq) | C5 | #6 |
| B4 standard-process-trace | #766 (qqq) | C6 | #7 |
| B5 webhooks | #767 (qqq) | C7 | #8 |
| B6 workflows | #768 (qqq) | C8 | #9 |
| B7 geo-data | #769 (qqq) | C9 | #10 |
| B8 sftp-data-integration | #770 (qqq) | | |
Most qbit repos have issues disabled, so their tasks are tracked in QRun-IO/qqq.
## Waves
| Wave | Tasks | Needs |
|---|---|---|
| Q1 | A2, B1–B8, C1 | — |
| Q2 | C2 | B1–B8 merged, C1 |
| Q3 | A1 | ESB task 10 (qqq#748) merged |
| Q4 | C3 | A1, A2, C2; ESB tasks 8, 10, 14 (qqq#746, #748, #751) |
| Q5 | C4 | C3 |
| Q6 | C5 | C4 |
| Q7 | C6, C8 | C5 |
| Q8 | C7 | C6; ESB epic complete |
| Q9 | C9 | qqq 4.1 GA (ESB + Next default); James's go-ahead |