Compare commits

..
2 Commits
Author SHA1 Message Date
james.maesandClaude Fable 5.2 42dff1ff39 docs(realtime): describe printing on several application nodes
maven / build (push) Successful in 4m20s
build / maven (push) Successful in 4m20s
REALTIME.md replaces the single-node boundary with what is shared and where
(jobs and the claim, the connection record, the claim gate, wakeups), what
happens when the shared bus loses a wakeup, the delivery guarantee, the four
columns a host that owns its schema must add, and what is and is not verified.
README, OPERATIONS and the changelog follow.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 16:47:33 -06:00
james.maesandClaude Fable 5.2 4aa63ae1d1 feat(realtime): print on several nodes: shared wakeups, a durable connection record and a claim gate any node can answer
maven / build (push) Successful in 3m37s
build / maven (push) Successful in 3m37s
A device holds one realtime connection, on one node. Until now the wakeup
listeners, the connection ownership and the claim gate lived in that JVM, so a
job accepted on another node never reached the device, and a claim that
reached another node was refused with 409.

- PrintingRealtimeBus joins the application's QQQ realtime bus (in memory on
  one node, the configured shared bus such as Redis on several). A wakeup is
  delivered to this JVM directly and handed to the shared bus by a bounded
  background worker, so the committing caller never waits for the transport.
  The route provider joins at start; hosts need no new wiring.
- The connection is recorded on the device row (connection id, node, ready
  flag, lease). Every statement is a guarded update: the newest connection
  takes the row, only the named connection can renew, mark or clear it. The
  claim gate accepts a claim on a node without the socket when the row shows
  a live, ready connection and the device has applied the current desired
  state and settings.
- On every heartbeat (PING) the connection renews its lease and asks the
  database whether anything was committed that it has not been told; a
  wakeup the bus lost costs at most one heartbeat interval. A connection
  replaced through another node closes on the bus message, or on its next
  frame when the bus is down.
- The QUEUED to CLAIMED update carries a status guard in addition to the
  device row lock.
- A database failure during the socket upgrade answers 503 (retryable), not
  403, which a client treats as "ask a person to sign in".

Tests: two services with separate buses and route providers over one database
and one application bus (wake, claim over HTTP on the other node, events on
either node, bus down, replacement), the bus on its own, and the claim,
acknowledgement and connection record on PostgreSQL 17 (Testcontainers).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 16:39:22 -06:00
33 changed files with 1976 additions and 3024 deletions

No files matched your search

+24 -179
View File
@@ -2,186 +2,31 @@
## Unreleased (1.2.0)
Print jobs have a real creation date-time. No wire or contract change (extension 2.4.0, wire
v1, subprotocol unchanged); connectors need no update. **Schema change: a host that owns the
printing schema must add one column before deploying this version** (below).
Several application nodes. No wire or contract change (extension 2.4.0, wire v1, subprotocol
unchanged); connectors need no update. Until now the wakeup listeners, the connection
ownership and the claim gate lived in one JVM, so with two nodes a job accepted on the node
without the device's socket was never announced, and a claim that reached that node was
refused with 409.
- `printing_job.created_at` is ISO-8601 text and its QQQ field is a string, so QQQ could not
apply a relative date filter ("created in the last day"), a date sort or a date bucket to
print jobs. The table gains `created_time`, a `TIMESTAMP` without time zone holding UTC
(the QQQ date-time convention), written by the same `INSERT` from the same instant as
`created_at`. It is the read-only QQQ field **Created** (`created_time`, `DATE_TIME`) and
replaces the text field in the Print jobs default columns and Summary section.
- `created_at` is unchanged: the queue order, history cursors, admission seeding, the Printing
home and the `createdAt` value of the job API still read it. Its QQQ field stays in Details,
relabelled **Created at (ISO text)**.
- `initialize()` adds the column (`ADD COLUMN IF NOT EXISTS`), backfills every existing row
from `created_at` on each start (so a row written by an older node during a rolling restart
is repaired) and creates `printing_job_created_time_idx (tenant_id, created_time, id)`.
- A host that owns the schema (it never calls `initialize()`) appends the equivalent to its
own migrations. For PostgreSQL, in one transaction:
```sql
ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS created_time TIMESTAMP WITHOUT TIME ZONE;
UPDATE printing_job SET created_time = (created_at::timestamptz AT TIME ZONE 'UTC') WHERE created_time IS NULL;
ALTER TABLE printing_job ALTER COLUMN created_time SET DEFAULT (now() AT TIME ZONE 'UTC');
ALTER TABLE printing_job ALTER COLUMN created_time SET NOT NULL;
CREATE INDEX IF NOT EXISTS printing_job_created_time_idx ON printing_job(tenant_id, created_time, id);
```
The default and `NOT NULL` are optional hardening for a rolling deployment: a node still on
an older library omits the column, and the default gives its rows the database clock in UTC
instead of a null that date filters would skip. The library itself needs only the nullable
column and always writes it.
- On a schema without the column this version fails loudly and stores nothing: every
submission is rolled back with the database's missing-column error for `created_time`, and
QQQ cannot read the Print jobs table. Apply the schema step first; the previous library
keeps working on the new schema.
### Delete print stations and printers; "Print station"; grouped reads
No wire or contract change (extension 2.4.0, wire v1, subprotocol unchanged); connectors need
no update. **Schema change: a host that owns the printing schema applies the step below
before deploying this version.**
- **Names.** The computer that runs the print client is a **Print station** in every label,
action, preview, home page text and process message (it was "Print device", which read like
the printer). **Printer** is unchanged. The words live in `PrintingNames`; a different
choice is one edit there. Table, field and process names (`printing_device`,
`printingEnrollDevice`, ...), the HTTP API texts and the agent protocol are unchanged.
Changed labels a host may have help or tests for: tables **Print stations** and **Print
station control history**; actions **Enroll a print station**, **Disconnect print
stations**, **Reconnect print stations**, **Revoke print station access**, **Remove print
station**; the enrollment field `deviceName` is **Computer name**.
- **Delete print stations** (`printingDeleteDevices`) and **Delete printers**
(`printingDeleteSelectedPrinters`), on the list selection and on the record screen. Each
asks for a confirmation, then shows what will happen (printers that go with a station,
queued jobs canceled, jobs closed as outcome unknown, finished jobs kept) and names every
record it will not delete, with the reason. Service: `PrintingService.deleteDevice(deviceId,
actor)`, `deletePrinter(printerRowId, actor)`, and the read-only `deviceDeletionPlan` /
`printerDeletionPlan`.
- A station is revoked through the existing REVOKE control (credential rotated,
disconnected, pending commands closed) and, in the same transaction (RC.4, below; it was
a separate step before): its queued jobs become
`FAILED` ("Canceled before dispatch: its print station was deleted"); jobs it was sent
without a reported result (`CLAIMED`, `RECEIVED`, `SUBMITTING`) become `OUTCOME_UNKNOWN`;
each of those gets a `printing_event` row (`event_origin` `SERVER_DELETION`) naming the
actor; its control receipts, its printers and the station row are deleted.
- A printer: its queued jobs are canceled and its sent-but-unreported jobs closed the same
way; the printer row is deleted; its station stays in service.
- Refused, per record: a station (or a printer on a station) that holds a live credential,
shows as ONLINE or PAUSED and has jobs in `CLAIMED`, `RECEIVED` or `SUBMITTING`. Wait, or
use Disconnect or Revoke access first. A record that is gone or not the caller's is
reported and skipped. The other records of the selection are still deleted.
- Jobs are never deleted. `printing_job` gains `device_name`, `computer_name`,
`printer_name` and `queue_name`: the names the station and printer had when the job was
submitted (written by the same `INSERT`; existing jobs are filled from the current rows;
a delete fills any job that still lacks them). They are the read-only fields **Print
station**, **Computer name**, **Printer** and **Windows queue name** on Print jobs, and
the first two replace the record links in the default columns. `device_id`, `printer_id`
and `printer_row_id` keep the identifiers the deleted records had; they are no longer
references that must resolve, so the foreign key from `printing_job.device_id` to
`printing_device` is dropped.
- A computer enrolled again is a new station with a new identifier. A queue that a station
still shares is reported by its next inventory and comes back as a new printer, exposed
and not approved. Old jobs are not attached to either.
- Everything a deleted station sends afterwards (heartbeat, inventory, claim, content,
event, control poll, control receipt) and every management action on it is answered
`404 Device not found`; its token no longer authenticates (`401`). **Print again** on one
of its jobs is refused with the reason.
- New table `printing_deletion`, read-only as **Deletion history** in the Printing app: one
row per deleted station and per deleted printer with what it was (kind, name, computer
name, queue name, facility, the identifiers it had), who deleted it (`actor`), when
(`deleted_time`, a UTC timestamp) and what went with it (printers, queued jobs canceled,
jobs closed as unknown, jobs kept). It has no reference to the deleted rows.
- New permission **`printing_delete`** (`PrintingDeletePermission`). Both actions require
`printing_delete` **and** `printing_admin`; `printing_admin` alone does not delete and is
not offered the actions. A host grants it where it grants the others.
- **Remove printers**, **Remove print station** and **Revoke print station access** are
unchanged (same process names, `printing_admin`). Remove still archives (`removed_at`,
record kept); a removed record can be deleted.
- **A grouped read is a read.** `PrintingReadPermission` let a query, a get and a count
through and refused everything else as a write, so an aggregate ("print jobs by format")
was refused for everybody, administrators included, wherever permissions are enforced. It
now accepts `AggregateInput` and `ExportInput` with exactly the permission a query needs
(`printing_operator` or `printing_admin` on stations and printers, `printing_admin` on the
other tables). Inserts, updates and deletes through table actions stay refused.
- Schema step for a host that owns the schema (PostgreSQL, one transaction):
```sql
ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS device_name VARCHAR(1024);
ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS computer_name VARCHAR(1024);
ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS printer_name VARCHAR(1024);
ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS queue_name VARCHAR(1024);
UPDATE printing_job j SET device_name = d.name, computer_name = d.computer_name
FROM printing_device d WHERE d.id = j.device_id AND j.device_name IS NULL;
UPDATE printing_job j SET printer_name = p.name, queue_name = p.queue_name
FROM printing_printer p WHERE p.id = j.printer_row_id AND j.printer_name IS NULL;
ALTER TABLE printing_job DROP CONSTRAINT IF EXISTS printing_job_device_id_fkey;
CREATE TABLE IF NOT EXISTS printing_deletion (id VARCHAR(36) PRIMARY KEY, tenant_id VARCHAR(200) NOT NULL,
kind VARCHAR(20) NOT NULL, record_id VARCHAR(36) NOT NULL, name VARCHAR(1024) NOT NULL,
computer_name VARCHAR(1024), queue_name VARCHAR(1024), facility_id VARCHAR(200), device_id VARCHAR(36),
actor VARCHAR(200) NOT NULL, deleted_time TIMESTAMP WITHOUT TIME ZONE NOT NULL,
printer_count INTEGER DEFAULT 0 NOT NULL, cancelled_job_count INTEGER DEFAULT 0 NOT NULL,
unknown_job_count INTEGER DEFAULT 0 NOT NULL, kept_job_count INTEGER DEFAULT 0 NOT NULL);
CREATE INDEX IF NOT EXISTS printing_deletion_tenant_idx ON printing_deletion(tenant_id, deleted_time, id);
```
`printing_job_device_id_fkey` is the name PostgreSQL gives the inline `REFERENCES` of the
original `CREATE TABLE`; a schema that named it otherwise drops that name. `initialize()`
does all of this itself and finds the foreign key by metadata.
- Without the step this version fails loudly: every submission is rolled back with the
database's missing-column error for `device_name`, and QQQ cannot read Print jobs. If only
the foreign key is left in place, deleting a station that has jobs is refused (409, naming
the step) before anything is revoked. The previous library keeps working on the new schema;
jobs it writes get their names when their station or printer is deleted.
### 1.2.0-RC.4: a station delete decides its refusal under the lock
No wire, contract or schema change; the schema step above is the same.
- **Fixed: deleting a station could close a job as `OUTCOME_UNKNOWN` that the station had
just been sent and was about to print.** `deleteDevice` decided "connected and printing"
from a read without a lock, revoked the station, and only then locked and read again; a
revoked station never counts as connected, so the second read could not refuse. A station
that was idle at the first read and claimed a job before the revocation was deleted with
that job in its hands.
- The delete is now one transaction. It takes the tenant admission lock and then the
station's row lock (the order the other paths use; a claim, a job event and a heartbeat
take the station's row lock before they change anything), decides the refusal there,
applies the REVOKE control on the same connection, closes the jobs, deletes the rows and
commits once. A claim either commits before the delete locks, and the delete is refused
(409, the station and its job untouched), or waits for the delete and is then answered
`404 Device not found`: the job was canceled by the delete and was never sent.
- A delete that is refused or fails changes nothing. Before, a delete whose transaction
failed left the station revoked; now the revocation rolls back with the rest and the
delete can simply be run again. A station that an earlier version left revoked, or that
was revoked with **Revoke print station access**, is deleted without a second revocation.
- The station's connection is woken once, after the commit. The revocation of a delete no
longer goes through the public `PrintingService.deviceAction`; a host that wraps that
method to observe revocations sees a delete in `printing_deletion` instead.
- `deletePrinter` did not have the window (it decides under the station's row lock and
revokes nothing) and is unchanged.
- **HTTP API text restored.** RC.3 changed one text an HTTP route returns, contrary to the
note above: approving or unapproving a removed printer (`PUT
/printing/v1/admin/devices/{deviceId}/printers/{printerId}`, 409) answers "Printer was
removed; share it from the device again before changing its approval" again. The other
text RC.3 changed ("Revoke the print station's access before removing it") is raised only
by `removeDevice`, which no HTTP or agent route calls, and stays.
- `initialize()` fills `device_name` / `printer_name` only on jobs whose station or printer
still exists. Before, jobs of a deleted station or printer that have no name were rewritten on
every start, and where such a job had a queue name but no printer name the queue name was
set to null. They are still read on every start; there is no index on the name columns.
- Deleting a station or a printer writes the names only onto its jobs that lack one, so a
long job history is not rewritten while the tenant's admission lock is held.
- Tests added for: the claim that lands just before the delete locks (station and printer)
and the claim in flight that the delete waits for; a printer refused while it prints on a
connected station; an already revoked station; a failed transaction rolled back and
retried; the foreign-key violation (`23503`) named as the missing schema step; the final
step refusing a record that passed the review; the review naming a record that went away
or conflicts; a grouped read of print jobs counting the caller's tenant only. They run on
the in-memory database of the test suite; the build has no PostgreSQL driver or container
support, so the delete and the schema step are not exercised on PostgreSQL here.
- Commit wakeups are handed to the application's QQQ realtime bus (in memory on one node; the
host's shared bus, such as Redis, on several) as well as to this JVM. The realtime route
provider joins the bus at start; hosts add no wiring. The committing request never waits for
the transport.
- A device's one connection is recorded on its row (`realtime_connection_id`, `_node`, `_ready`,
`_expires_at`; four additive columns, added by `initialize()` or by the host's own migration)
with guarded updates. The claim gate on a node without the socket accepts a claim when the
row shows a live, ready connection and the device has applied the current desired state and
settings.
- On every client heartbeat the connection renews its lease and asks the database whether
anything was committed that it has not been told, so a wakeup the shared bus lost is late by
at most one heartbeat interval (20 seconds by default). An idle heartbeat sends no frame.
- A device that connects through another node replaces its previous connection there.
- The QUEUED to CLAIMED update is guarded by the status, in addition to the device row lock.
- The socket upgrade answers 503, not 403, when the database could not answer during
authorization, so a client retries instead of asking a person to sign in.
- Tests: two services with their own buses and route providers over one database and one
application bus, and the claim, the acknowledgement and the connection record on
PostgreSQL 17 (Testcontainers; Docker required to run the suite).
## 1.1.2
+6 -10
View File
@@ -92,14 +92,12 @@ After the artifact is available in your configured repository, add:
</dependency>
```
Supply an existing JDBC DataSource/pool and a QQQ RDBMS backend pointing to that same database. Initial schema is created by `initialize()`; the implementation uses portable H2/PostgreSQL SQL, but only H2 has been tested so far.
Supply an existing JDBC DataSource/pool and a QQQ RDBMS backend pointing to that same database. Initial schema is created by `initialize()`; the implementation uses portable H2/PostgreSQL SQL. H2 carries the full suite; the claim, the acknowledgement, the realtime connection record and `initialize()` itself also run on PostgreSQL 17 in `PrintingClaimPostgresTest` (Docker required).
Device and printer names, descriptions and locations are edited in the Windows client and synchronized through heartbeat/inventory. QQQ's native record forms and selectors show the effective friendly name; the physical computer/queue name remains visible in Details. These synchronized fields are read-only in QQQ so two editors do not overwrite each other. Approval remains a separate server control. Friendly names/locations allow 512 characters, descriptions 4096, and physical names/job titles 1024. Clearing a friendly name restores the technical name without changing IDs or routing.
`initialize()` adds profile columns, backfills physical names, widens existing name/title columns and creates dispatch/history indexes without deleting records. The existing `name` column becomes the effective display label; `computer_name`/`queue_name` retain technical names and `display_name` holds the optional override. Back up the database and run schema initialization during a maintenance/deployment window; column widening and index creation can lock populated tables.
Print jobs carry their creation instant twice: `created_at` is the ISO-8601 text the queue, history cursors and job API use, and `created_time` is the same instant as a UTC `TIMESTAMP` (no time zone, the QQQ convention), exposed as the date-time field **Created** so native filters ("created in the last day"), sorting and date buckets work. `initialize()` adds the column, backfills existing rows from `created_at` and creates `printing_job_created_time_idx (tenant_id, created_time, id)`. A host that owns the schema must add the column before deploying this version (see the [changelog](CHANGELOG.md) for the statements): without it every submission fails with the database's missing-column error and stores nothing, and the Print jobs table cannot be read.
Job history uses `GET /printing/v1/admin/jobs?limit=200&before=<nextCursor>` with opaque keyset cursors and a maximum 200 rows per page. Claims read one candidate, pending inspection reads 50, and all these queries project metadata without fetching stored documents. Native dashboard lists already cap attention rows at 10 per category and recent work at 25. Dispatch/history indexes support growing histories; the synthetic 10,000-row regression test checks bounded pages and claim correctness, **not** a certified 10,000-print/day throughput measurement. Driver speed, document size, database hardware and retention policy still determine capacity. Document cleanup follows the configured retention policy and host schedule; the audit does not run a live purge or delete job history.
```java
@@ -119,17 +117,15 @@ Configure a dedicated private printing bucket for the customer/environment throu
Enrollment uses a named facility selector. Set `withFacilityPossibleValueSourceName("your_existing_facility_source")` to reuse an already registered host source and its record security. Without it, the standalone choice is **Default facility**. The backend validates the selection under the current user's permissions before issuing an enrollment code; typing or forging an unknown/foreign-tenant ID cannot bypass that check. Existing stored facility IDs are preserved; configure their original source to resolve their names.
For native QQQ screens, grant `printing_admin` and populate the session's `printing_tenant` security key with the tenant ID; grant `printing_delete` as well to whoever may delete print stations and printers. A **print station** is the computer that runs the print client (table `printing_device`); a **printer** is a Windows queue it shares. The Printing home shows print station health, queue status and work needing attention, with Print stations, Printers, Print jobs, Print station control history and Deletion history beneath it. It nests under an existing `admin` app when available. Processes are dashboard/table/record actions, not navigation entries. Secret/token and document columns are excluded from metadata. Table mutations are rejected; use the service/processes instead.
For native QQQ screens, grant `printing_admin` and populate the session's `printing_tenant` security key with the tenant ID. The Printing home shows device health, queue status and work needing attention, with Devices, Printers and Print jobs beneath it. It nests under an existing `admin` app when available. Processes are dashboard/table/record actions, not navigation entries. Secret/token and document columns are excluded from metadata. Table mutations are rejected; use the service/processes instead.
Typical setup: run **Enroll a print station** from Printing home or Print stations, enroll the Windows connector, and select local queues to expose in its companion/tray. Use **Set printer approval** to choose a named print station and one of its printers. Test-print forms offer only exposed, approved printers supporting the chosen format. Device and printer choices respect the current user's tenant access.
Typical setup: run **Enroll a print device** from Printing home or Print devices, enroll the Windows connector, and select local queues to expose in its companion/tray. Use **Set printer approval** to choose a named device and one of its printers. Test-print forms offer only exposed, approved printers supporting the chosen format. Device and printer choices respect the current user's tenant access.
For label printers, configure the machine's **Windows Printing Defaults** for the actual stock size and print method before testing PDFs through the service. A test profile selects document size and ZPL resolution; it does not change the Windows driver or printer's media configuration. Our 300 dpi Zebra required 4 × 6-inch stock and Thermal Direct instead of its previous 4.09 × 2-inch Thermal Transfer defaults. After correcting those settings and resuming the printer, the user confirmed the built-in PDF label, alongside the already confirmed ZPL label and laser PDF. Barcode scanning and measured alignment remain separate checks; `SPOOLED` alone is not proof of physical output.
On Printers, select one or more rows or open a printer record to **Approve printers**, **Disable printers**, **Remove printers**, **Print a test page**, or **Print a test label**. Standard QQQ review/result screens show the selected scope; preview does not print. Each selected printer is read again under the current user's permissions before execution. Jobs display the printer and print station names they were submitted under (stored on the job, so they read the same after a rename or a delete); approval, availability, format and status fields have named choices.
On Printers, select one or more rows or open a printer record to **Approve printers**, **Disable printers**, **Remove printers**, **Print a test page**, or **Print a test label**. Standard QQQ review/result screens show the selected scope; preview does not print. Each selected printer is read again under the current user's permissions before execution. Jobs display the related printer and device names; approval, availability, format and status fields have named choices.
Deleting is separate from removing. **Delete printers** and **Delete print stations** (on a selection or a record; they need `printing_delete` as well as `printing_admin`) take the record out of every list for good. A station is revoked first, so its computer must be enrolled again and then is a new station; its printers go with it. Queued jobs are canceled, jobs already sent without a reported result are closed as outcome unknown, and finished jobs stay in Print jobs with the names they were printed under. A record is refused, by name, while its station is connected and printing. A printer that its station still shares comes back from the next inventory as a new printer awaiting approval. Every deletion is recorded in **Deletion history** (who, what, when, what went with it). See the [walkthrough](docs/UI-WALKTHROUGH.md#delete-a-print-station-or-a-printer).
Removing is archiving, not deleting. **Remove printers** sets **Removed at** on the selected printers, clears their approval and exposure, and takes them out of the operator catalog, claims, the `PRINTERS` frame and every device/printer selector; their rows and job history stay. The Printers and Print stations tables have no server-side default filter, so add the filter **Removed at is empty** to the view you use day to day. If the device shares a removed queue again, the next inventory brings it back exposed but **not approved**, exactly like a newly discovered queue. **Remove print station** (an action that requires confirmation) revokes the station's access through the existing REVOKE control, then archives the station and all of its printers; an already revoked station is archived without a new command. A removed station cannot be restored; connect the computer again with a fresh enrollment, which creates a new station. A removed record can be deleted. Retention cleanup later hard-deletes removed printers and devices that no job references (see [storage and retention](docs/STORAGE-AND-RETENTION.md)).
Removing is archiving, not deleting. **Remove printers** sets **Removed at** on the selected printers, clears their approval and exposure, and takes them out of the operator catalog, claims, the `PRINTERS` frame and every device/printer selector; their rows and job history stay. The Printers and Print devices tables have no server-side default filter, so add the filter **Removed at is empty** to the view you use day to day. If the device shares a removed queue again, the next inventory brings it back exposed but **not approved**, exactly like a newly discovered queue. **Remove device** (a device action that requires confirmation) revokes the device's access through the existing REVOKE control, then archives the device and all of its printers; an already revoked device is archived without a new command. A removed device cannot be restored; connect the computer again with a fresh enrollment, which creates a new device. Retention cleanup later hard-deletes removed printers and devices that no job references (see [storage and retention](docs/STORAGE-AND-RETENTION.md)).
Metadata names follow QBit conventions: tables and data sources use `printing_*`, while processes/widgets use `printing...`. The initial namespace is fixed to `printing`. The QBit preserves the host's `QBrandingMetaData`; customer identity belongs in the host branding, not a custom QBit build.
@@ -157,7 +153,7 @@ For durable QQQ jobs register `PrintEnqueueEffectTarget` as an `IDEMPOTENT_EXTER
## Deployment boundaries
Jobs target an explicit device/printer. Logical destination routing and a distributed real-time bus belong to the integrating host. The release includes cancellation of queued jobs, explicit reprints, owned spool-cancellation commands, durable receipt reconciliation, S3 delivery storage and scheduled retention. Received/submitting jobs are not automatically offered again; `SPOOLED` never asserts physical delivery.
Jobs target an explicit device/printer. Logical destination routing belongs to the integrating host. An application running as several nodes shares printing through its database and its QQQ realtime bus; see [several application nodes](docs/REALTIME.md#several-application-nodes) (unreleased; 1.1.2 and earlier support one node). The release includes cancellation of queued jobs, explicit reprints, owned spool-cancellation commands, durable receipt reconciliation, S3 delivery storage and scheduled retention. Received/submitting jobs are not automatically offered again; `SPOOLED` never asserts physical delivery.
Job paper (`paperWidthMm` and `paperHeightMm`, both positive and at most 2000, never combined with `mediaProfile`) is supported by the matching Windows 1.0.1 connector. The built-in 4 × 1 and 4 × 2 PDF tests use 101.6 × 25.4 / 50.8 mm. Without explicit paper or a reported media selection, jobs follow the Windows queue default. See [media selection](docs/OPERATOR-CATALOG.md).
+1 -1
View File
@@ -184,4 +184,4 @@ cancellation request, not that paper was withdrawn or a physical print was undon
The stable 1.0.0 library and its matching release/registry bytes are published; see [release evidence](RELEASES.md). Local checks cover H2 persistence, native QQQ operations, real HTTP/WebSocket transport, S3-compatible storage and bounded job histories. User-confirmed physical output includes laser PDF and Zebra ZPL/PDF.
A deployment still supplies and validates its own database, S3/IAM, authentication mapping, supervision and printer drivers. PostgreSQL and multiple-node operation are not qualified by the local H2/MinIO fixture. The default real-time bus is in-process; consult [host integration](REALTIME.md) before scaling hosts. These are deployment boundaries, not pending 1.0 publication tasks.
A deployment still supplies and validates its own database, S3/IAM, authentication mapping, supervision and printer drivers. The local H2/MinIO fixture does not qualify a deployment's PostgreSQL or its several nodes. From the release after 1.1.2 the library itself supports several nodes over one database (wakeups on the application's QQQ realtime bus, the device's connection and the claim gate in the database, proven in-process and on PostgreSQL 17); two real nodes behind a load balancer remain the deployment's own qualification. Read [several application nodes](REALTIME.md#several-application-nodes) before scaling hosts. These are deployment boundaries, not pending 1.0 publication tasks.
+76 -12
View File
@@ -80,20 +80,84 @@ expansion for legacy submission). Raising a document limit above the running
transport ceiling requires a host restart or corresponding production ingress
configuration. Download streaming does not base64-expand documents.
## Host ownership and deployment boundary
## Several application nodes
Commit wakeups are emitted after SQL commit. A subscription exists before initial
state reconstruction, closing the empty-claim/reconnect notification race.
`PrintingRealtimeBus` is an explicit host-owned notification and connection
ownership seam. The default implementation is in-process; this source does **not**
provide a distributed broker or claim tested multi-node production support.
Unreleased (the next release from `main`); 1.1.2 and earlier support one node only.
Use one writer/application host with its configured service/bus. Multiple route
providers sharing that explicit in-process bus fence a device's prior connection.
A multi-node deployment needs a host-provided shared implementation preserving
both post-commit notifications and single-channel ownership, and its own
qualification. Separate independent service instances do not magically publish
notifications to one another.
An application may run as several nodes (pods) over one database, each with its
own `PrintingService` and its own `PrintingRealtimeRoutes`. A device still holds
one connection, on one node. Nothing about a connection that another node needs
is kept in memory:
| What | Where it lives | What another node does with it |
|---|---|---|
| Jobs, events, the claim | `printing_job`, `printing_event`; the claim runs under the device row lock and its QUEUED to CLAIMED update is guarded by the status | Claims, cancels and events for one device are serialized across nodes. A job is handed out by exactly one transition; every later claim replays the same assignment until the device records RECEIVED |
| Which connection holds the device | `printing_device.realtime_connection_id`, `realtime_connection_node`, `realtime_connection_ready`, `realtime_connection_expires_at` | The newest connection takes the row. Only the connection named on the row can renew it, mark it ready or clear it, so a late close never removes a replacement. A node that stopped without clearing its row is forgotten when the lease (the heartbeat timeout, 60 seconds by default) runs out |
| Whether a claim may proceed | the same row, plus `control_applied_revision` against `control_revision` and `realtime_applied_revision` against the settings revision | The node holding the socket answers from the connection itself, as before. Any other node accepts the claim when the row shows a live, ready connection and the device has applied the current desired state and settings; otherwise 409, as before |
| Commit wakeups | `PrintingRealtimeBus`: delivered to this JVM directly, and handed to the application's QQQ realtime bus (`RealtimeBus.get(instance)`) on channel `qbit-printing` | Each node's bus receives the wakeup and wakes the connection it holds. The message carries the tenant and device identifiers only. Browsers cannot subscribe to that channel: QQQ realtime streams accept only `instance`, `user`, `table` and `record` channels |
The route provider joins the application's realtime bus when it starts
(`PrintingRealtimeBus.join(instance)`), so a host adds no wiring. On one node
that bus is QQQ's in-memory bus and behaviour is unchanged. On several nodes
the host configures a shared realtime bus the way it does for the rest of QQQ
(for example qbit-redis `RedisQBitConfig.withRealtime`); without one, each
node's wakeups stay on that node and only the heartbeat reconciliation below
connects them. A service that publishes but has no route provider (a worker
process) joins explicitly with the same call.
**The shared bus is best effort, and printing does not depend on it.** A
wakeup is handed to the bus by a bounded background worker, so the request that
committed never waits for the transport and a transport failure never fails
it (`getForwardFailures()` counts wakeups that could not be handed over). On
every client heartbeat (`PING`, every `heartbeatIntervalSeconds`, 20 by
default, 5 at the least) the connection renews its lease and asks the database
whether anything was committed that it has not been told: a newer desired
state or settings revision than it last sent, a pending command, or a job a
claim would hand out now. If so it runs the ordinary wakeup. A wakeup the bus
lost is therefore late by at most one heartbeat interval, never lost; an idle
device costs a few indexed reads per heartbeat and receives no frame. When the
transport reports that it may have lost messages (its `resync` hint after a
reconnect or an overflow), every connection a node holds reconciles at once.
A device that connects through another node replaces its previous connection:
the row changes hands, the other node is told over the bus and closes the old
socket, and if the bus is down the old socket's next frame finds the row named
another connection and closes then.
Delivery guarantee: a job is delivered to its device at least once (the claim
replays until RECEIVED commits), every event is idempotent by its event id on
any node, and the server never hands out a job that has been received. The
client does not submit a job its journal already holds in a final state, and
reports an interrupted submission as `OUTCOME_UNKNOWN` instead of printing it
again. Together: never two submissions to the spooler for one job.
The socket upgrade answers 503 when the database could not answer during
authorization, so a client retries; 401 and 403 remain what they were (sign-in
required, denied), because a client stops and asks for a person on those.
Hosts that own their schema (no `initialize()` at run time) add the four
columns themselves before running this version; they are nullable or
defaulted, so the previous release keeps working on the new schema:
```sql
ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_id VARCHAR(36);
ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_node VARCHAR(200);
ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_ready INTEGER DEFAULT 0 NOT NULL;
ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_expires_at BIGINT;
```
`realtime_connection_node` (the node id, a pod name in Kubernetes) is shown on
the device record as "Realtime connection node". Lease times are epoch
milliseconds from each node's own clock; nodes are expected to keep ordinary
clock synchronization.
Verified: `PrintingMultiNodeTest` (two services, buses and route providers over
one database and one application bus: wake, claim over HTTP on the node
without the socket, events on either node, the bus losing or refusing every
message, replacement), `PrintingRealtimeBusTest`, and `PrintingClaimPostgresTest`
(racing claims, concurrent acknowledgements and the connection record on
PostgreSQL 17). Not verified: two real processes behind a load balancer, a
node dying mid-claim, Redis as the transport, and a physical printer.
The existing `/printing/v1` routes remain explicit legacy diagnostic compatibility.
They are not a real-time fallback and cannot authorize a native bound device.
+1 -1
View File
@@ -36,7 +36,7 @@ Printing administration exposes retention settings and a reviewed **Run cleanup*
When the host supplies a scheduler, maintenance runs every five minutes in bounded batches of at most 200. That permits a nominal 57,600 eligible documents per day before storage latency or failures; it is not measured production throughput. Changing a retention setting does not itself purge documents. Disabling cleanup preserves content. Without a configured scheduler there is no automatic job; the administration screen must show that condition.
**Delete printers** and **Delete print stations** delete the record at once and keep every job (see the [walkthrough](UI-WALKTHROUGH.md#delete-a-print-station-or-a-printer)); a deleted station's retained-byte counter stays until cleanup has removed its last stored document, then cleanup deletes the counter. The same run purges archived inventory. **Remove printers** and **Remove print station** never delete rows; they set `removed_at` (shown as **Removed at**) so the rows leave the default views, selectors, catalog, claims and the realtime `PRINTERS` frame while job history keeps resolving to them. After the document step, each cleanup run hard-deletes, within the remaining batch allowance, removed printers that no `printing_job` references (neither by `printer_row_id` nor by device and queue identity) and then removed devices that have no jobs and no printer rows left; a removed device's control receipts and resource accounting go with it. Anything a job references is never deleted, so a removed printer or device stays, hidden, for as long as its jobs exist. The purge counts toward the run's reported total and is logged with the printer and device counts; disabling cleanup pauses it together with document removal. Cleanup never deletes an active (non-removed) printer or device, and never deletes a job.
The same run purges archived inventory. **Remove printers** and **Remove device** never delete rows; they set `removed_at` (shown as **Removed at**) so the rows leave the default views, selectors, catalog, claims and the realtime `PRINTERS` frame while job history keeps resolving to them. After the document step, each cleanup run hard-deletes, within the remaining batch allowance, removed printers that no `printing_job` references (neither by `printer_row_id` nor by device and queue identity) and then removed devices that have no jobs and no printer rows left; a removed device's control receipts and resource accounting go with it. Anything a job references is never deleted, so a removed printer or device stays, hidden, for as long as its jobs exist. The purge counts toward the run's reported total and is logged with the printer and device counts; disabling cleanup pauses it together with document removal. Cleanup never deletes an active (non-removed) printer or device, and never deletes a job.
Only eligible successful terminal jobs lose their document content. Active, failed or uncertain work remains protected. Job metadata, document hashes, receipts and idempotency records remain available for auditing and duplicate prevention. Legacy completion times are recovered only from durable successful events; unknown completion dates remain protected. Existing inline database documents remain readable; the initial storage integration does not silently migrate old bytes to S3. Reprinting a job whose document has been removed returns an explicit unavailable-document result.
+5 -20
View File
@@ -10,36 +10,21 @@ Open **Printing** in the navigation. The home presents the current tenant and re
## Enroll a facility computer
Choose **Enroll a print station** from Printing home or the Print stations table. A print station is the computer that runs the print client. Use the resulting short-lived enrollment code in the Windows companion's Connection settings. Once enrolled, select the installed printer queues to expose in the companion. Exposure is opt-in and persists in the service.
Choose **Enroll a print device** from Printing home or the Print devices table. Use the resulting short-lived enrollment code in the Windows companion's Connection settings. Once enrolled, select the installed printer queues to expose in the companion. Exposure is opt-in and persists in the service.
Open **Printers** in QQQ. The operational columns show the printer name, named print station, status, approval, local exposure, and last contact. Approval and exposure are independent: a printer must be both approved in QQQ and exposed on the computer before new jobs can be dispatched.
Open **Printers** in QQQ. The operational columns show the printer name, named print device, status, approval, local exposure, and last contact. Approval and exposure are independent: a printer must be both approved in QQQ and exposed on the computer before new jobs can be dispatched.
![Filtered printer table](screenshots/printers.png)
## Act on one printer or a selection
Select one or more printer rows and open **Actions**, or open an individual printer record and use its actions. Available operations include **Approve printers**, **Disable printers**, **Remove printers**, **Delete printers**, **Print a test page**, and **Print a test label**. The standard QQQ review screen describes the selection; reviewing does not print. Submitting a test action queues one job per selected eligible printer.
Select one or more printer rows and open **Actions**, or open an individual printer record and use its actions. Available operations include **Approve printers**, **Disable printers**, **Remove printers**, **Print a test page**, and **Print a test label**. The standard QQQ review screen describes the selection; reviewing does not print. Submitting a test action queues one job per selected eligible printer.
**Remove printers** archives the selection: each printer gets a **Removed at** time, loses approval and exposure, and disappears from the operator catalog, the connector's approval frame and the device/printer choices in the setup and test-print forms. Its record and job history stay. The Printers table has no server-side default filter, so add **Removed at is empty** to the Printers view you keep (the screenshots above predate this field) to hide archived rows; clear it to audit them. If the Windows companion shares the same queue again, the next inventory brings the printer back as exposed but not approved, so approve it again before it can print.
## Remove a print station
## Remove a device
Open a Print stations record or select rows and choose **Remove print station**. The action requires the confirmation checkbox. It revokes the station's access permanently through the same control as **Revoke print station access**, then marks the station and all of its printers removed. The station drops out of the Print stations view once you add **Removed at is empty** there, the setup and test-print choices and every enrollment, claim and control path; its jobs remain in Print jobs and its control history keeps the REVOKE receipt. A removed station cannot be un-removed; enroll the computer again to create a new station. Retention cleanup eventually hard-deletes removed stations and printers that no job references. To take the record out of the list now, delete it.
## Delete a print station or a printer
Deleting needs the `printing_delete` permission in addition to `printing_admin`; without it the two actions are not offered.
Open a Print stations record, or select rows in Print stations, and choose **Delete print stations**. Tick the confirmation and continue. The review screen then says exactly what will happen for the selection: how many stations are deleted, how many printers go with them, how many queued jobs are canceled, how many jobs already sent to a station are closed as outcome unknown, and how many finished jobs keep their history. It also lists, by name, every record that will not be deleted and why. Nothing has changed yet. Submit to delete.
- The station's access is revoked first. The computer cannot connect as that station again; enroll it again to print, and it is then a new station.
- Its printers are deleted with it.
- Queued jobs become **Failed** with the message "Canceled before dispatch: its print station was deleted". Jobs that were already sent to the station and had not reported a result become **Outcome unknown**: check the printer before printing them again.
- Finished jobs stay in Print jobs. Their **Print station**, **Computer name**, **Printer** and **Windows queue name** columns show the names they were printed under.
- A station that is connected (status Online or Paused) and printing is refused. Wait for the jobs to finish, or use **Disconnect print stations** or **Revoke print station access** first.
- **Deletion history** in the Printing app records who deleted what and when.
**Delete printers** works the same way on Printers. The printer's station stays in service. If the station still shares that Windows queue, its next inventory reports it again and it comes back as a new printer that is exposed and not approved; stop sharing it in the print client on that computer to keep it away.
Open a Print devices record or select rows and choose **Remove device**. The action requires the confirmation checkbox. It revokes the device's access permanently through the same control as **Revoke device access**, then marks the device and all of its printers removed. The device drops out of the Print devices view (add **Removed at is empty** there too), the setup and test-print device choices and every enrollment, claim and control path; its jobs remain in Print jobs and its control history keeps the REVOKE receipt. A removed device cannot be un-removed; enroll the computer again to create a new device. Retention cleanup eventually hard-deletes removed devices and printers that no job references.
![Actions on the selected printer](screenshots/printer-actions.png)
+3
View File
@@ -35,6 +35,9 @@
<!-- H2 2.4.240 fixes close-time compaction data loss (upstream #4247 / #4249). -->
<dependency><groupId>com.h2database</groupId><artifactId>h2</artifactId><version>2.4.240</version><scope>runtime</scope></dependency>
<dependency><groupId>org.junit.jupiter</groupId><artifactId>junit-jupiter</artifactId><version>6.0.1</version><scope>test</scope></dependency>
<!-- Real PostgreSQL for the tests whose claim, lease and row-lock SQL must be proven on the production database (needs Docker). -->
<dependency><groupId>org.testcontainers</groupId><artifactId>testcontainers</artifactId><version>2.0.3</version><scope>test</scope></dependency>
<dependency><groupId>org.postgresql</groupId><artifactId>postgresql</artifactId><version>42.7.8</version><scope>test</scope></dependency>
</dependencies>
<profiles><profile><id>ui-demo</id><dependencies>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-frontend-next</artifactId><version>1.0.0-RC.38</version><scope>runtime</scope></dependency>
@@ -1,64 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import com.kingsrook.qqq.backend.core.actions.permissions.UseOtherPermissionNameCustomPermissionChecker;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import org.jspecify.annotations.Nullable;
/*******************************************************************************
** Deleting a print station or a printer needs its own grant, printing_delete,
** in addition to printing_admin: an administrator who may pause, approve and
** revoke does not delete records unless a host grants this permission too.
*******************************************************************************/
public class PrintingDeletePermission extends UseOtherPermissionNameCustomPermissionChecker
{
public static final String NAME = "printing_delete";
/***************************************************************************
** Expose the permission in QQQ's available-permission catalog.
***************************************************************************/
public PrintingDeletePermission()
{
withPermissionName(NAME);
}
/***************************************************************************
** Both grants are required; neither implies the other.
***************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, @Nullable MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(NAME) || !QContext.getQSession().hasPermission("printing_admin"))
{
throw new QPermissionDeniedException("Permission to delete print stations and printers is required");
}
}
}
@@ -1,431 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
import org.json.JSONObject;
import org.jspecify.annotations.Nullable;
/*******************************************************************************
** Delete print stations and printers from their lists and record screens.
**
** Both actions need printing_delete as well as printing_admin. They ask for a
** confirmation, then show exactly what will happen (printers that go with a
** station, queued jobs canceled, jobs closed as outcome unknown, job history
** kept) and which records are refused and why. Nothing changes before the
** final step; every record is read again there with the caller's access, and
** a record that cannot be deleted is reported by name without stopping the
** others. PrintingService.deleteDevice and deletePrinter do the work.
*******************************************************************************/
public final class PrintingDeleteProcesses
{
public static final String DEVICES = "printingDeleteDevices";
public static final String PRINTERS = "printingDeleteSelectedPrinters";
public static final String DEVICES_LABEL = "Delete " + PrintingNames.STATIONS_LOWER;
public static final String PRINTERS_LABEL = "Delete " + PrintingNames.PRINTERS_LOWER;
static final String CONFIRM = "confirmPrintingDelete";
static final String DEVICES_PREVIEW = "Each selected " + PrintingNames.STATION_LOWER + " is deleted together with its " + PrintingNames.PRINTERS_LOWER + ". Its access is revoked first, so the computer must be enrolled again before it can print."
+ " Queued jobs are canceled, and jobs already sent to it without a reported result are closed as outcome unknown. Finished print jobs are kept, with the " + PrintingNames.STATION_LOWER + " and "
+ PrintingNames.PRINTER_LOWER + " names they were printed under. This cannot be undone.";
static final String PRINTERS_PREVIEW = "Each selected " + PrintingNames.PRINTER_LOWER + " is deleted; its " + PrintingNames.STATION_LOWER + " stays in service. Queued jobs for it are canceled, and jobs already sent without a reported result are closed as outcome unknown."
+ " Finished print jobs are kept, with the " + PrintingNames.PRINTER_LOWER + " and " + PrintingNames.STATION_LOWER + " names they were printed under. A " + PrintingNames.PRINTER_LOWER + " that its " + PrintingNames.STATION_LOWER
+ " still shares comes back as a new " + PrintingNames.PRINTER_LOWER + " awaiting approval; stop sharing it in the print client to keep it away. This cannot be undone.";
/***************************************************************************
** Utility class.
***************************************************************************/
private PrintingDeleteProcesses()
{
}
/***************************************************************************
** One action on each table, for a selection and for one record.
***************************************************************************/
public static void register(QInstance instance)
{
add(instance, DEVICES, DEVICES_LABEL, PrintingTables.DEVICE, DEVICES_PREVIEW,
"I understand: the selected " + PrintingNames.STATIONS_LOWER + " and their " + PrintingNames.PRINTERS_LOWER + " are deleted, access is revoked and each computer must be enrolled again");
add(instance, PRINTERS, PRINTERS_LABEL, PrintingTables.PRINTER, PRINTERS_PREVIEW,
"I understand: the selected " + PrintingNames.PRINTERS_LOWER + " are deleted and their queued jobs are canceled");
}
/***************************************************************************
** The grant every step of both actions checks.
***************************************************************************/
public static QPermissionRules permissionRules()
{
return new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION).withCustomPermissionChecker(new QCodeReference(PrintingDeletePermission.class));
}
/***************************************************************************
** prepare, confirm, review (what will happen), execute, result.
***************************************************************************/
private static void add(QInstance instance, String name, String label, String table, String preview, String confirmation)
{
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, table);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, preview);
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(PrintingExtract.class, Transform.class, Load.class, values)
.withName(name).withLabel(label).withTableName(table).withMinInputRecords(1).withPermissionRules(permissionRules());
process.withStep(0, new QBackendStepMetaData().withName("prepareDelete").withCode(new QCodeReference(Prepare.class)));
process.withStep(1, new QFrontendStepMetaData().withName("confirmDelete").withLabel(label)
.withFormField(new QFieldMetaData(CONFIRM, QFieldType.BOOLEAN).withLabel(confirmation).withDefaultValue(false))
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM)));
instance.addProcess(process);
}
/***************************************************************************
** The signed-in user who holds both grants, in this printing tenant.
***************************************************************************/
static String actor(RunBackendStepInput input) throws QException
{
new PrintingDeletePermission().checkPermissionsThrowing(input, null);
if(!DEVICES.equals(input.getProcessName()) && !PRINTERS.equals(input.getProcessName()))
{
throw new QException("Unknown printing delete action");
}
return PrintingDeviceProcesses.actor(input);
}
/***************************************************************************
** The record as the caller may read it now, or null when it is gone or not
** the caller's to see.
***************************************************************************/
private static @Nullable QRecord current(String table, @Nullable String id) throws QException
{
if(id == null || id.isBlank())
{
return null;
}
GetInput get = new GetInput();
get.setTableName(table);
get.setPrimaryKey(id);
get.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
return new GetAction().execute(get).getRecord();
}
/***************************************************************************
** What a delete of this record would do, or the reason it is refused.
***************************************************************************/
private static JSONObject plan(RunBackendStepInput input, QRecord record) throws QException
{
try
{
PrintingService service = PrintingProcesses.currentService();
return DEVICES.equals(input.getProcessName()) ? service.deviceDeletionPlan(record.getValueString("id")) : service.printerDeletionPlan(record.getValueString("id"));
}
catch(PrintingException exception)
{
return new JSONObject().put("refusal", exception.getStatus() == 404 ? name(input, record) + " no longer exists" : name(input, record) + ": " + exception.getMessage());
}
catch(java.sql.SQLException exception)
{
throw new QException("Could not read what the delete would do", exception);
}
}
/***************************************************************************
** The words for one record in a message.
***************************************************************************/
private static String name(RunBackendStepInput input, QRecord record)
{
return (DEVICES.equals(input.getProcessName()) ? PrintingNames.STATION : PrintingNames.PRINTER) + " " + record.getValueString("name");
}
/***************************************************************************
** The line listing every record that is not deleted, with its reason.
***************************************************************************/
private static void addRefusals(List<ProcessSummaryLineInterface> lines, List<String> refusals, boolean past)
{
if(!refusals.isEmpty())
{
lines.add(new ProcessSummaryLine(Status.ERROR, refusals.size(), refusals.size() == 1 ? (past ? "record was not deleted" : "record will not be deleted") : (past ? "records were not deleted" : "records will not be deleted"))
.withBulletsOfText(new ArrayList<>(refusals)));
}
}
/***************************************************************************
** Keep the selection across the confirmation form.
***************************************************************************/
public static class Prepare implements BackendStep
{
/************************************************************************
** Nothing is read or changed here beyond the caller's authority.
************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
actor(input);
PrintingProcesses.keepSelection(input, output);
}
}
/***************************************************************************
** The review: counts what will happen and names what is refused. Read-only.
***************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final List<String> refusals = new ArrayList<>();
private final ProcessSummaryLine ready = new ProcessSummaryLine(Status.OK, 0, null);
private boolean devices;
private long printers;
private long queued;
private long inProgress;
private long kept;
/************************************************************************
** Only records that can be deleted now go on to the final step.
************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
actor(input);
devices = DEVICES.equals(input.getProcessName());
for(QRecord selected : input.getRecords())
{
QRecord record = current(devices ? PrintingTables.DEVICE : PrintingTables.PRINTER, selected.getValueString("id"));
if(record == null)
{
refusals.add("A selected record no longer exists or is not yours to delete");
continue;
}
JSONObject plan = plan(input, record);
if(plan.has("refusal"))
{
refusals.add(plan.getString("refusal"));
continue;
}
printers += plan.optLong("printers", 0);
queued += plan.getLong("queuedJobs");
inProgress += plan.getLong("jobsInProgress");
kept += plan.getLong("jobs") - plan.getLong("queuedJobs") - plan.getLong("jobsInProgress");
ready.incrementCountAndAddPrimaryKey(record.getValue("id"));
output.getRecords().add(record);
}
}
/************************************************************************
** The records refused in review, for the final step's result.
************************************************************************/
List<String> refusals()
{
return refusals;
}
/************************************************************************
** One line for each consequence that applies, then the refusals.
************************************************************************/
// QQQ ProcessSummaryProvider requires this concrete return type; retain binary/API compatibility.
@SuppressWarnings("NonApiType")
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
if(devices)
{
ready.withSingularFutureMessage(PrintingNames.STATION_LOWER + " will be deleted; its access is revoked and the computer must be enrolled again before it can print")
.withPluralFutureMessage(PrintingNames.STATIONS_LOWER + " will be deleted; their access is revoked and each computer must be enrolled again before it can print");
}
else
{
ready.withSingularFutureMessage(PrintingNames.PRINTER_LOWER + " will be deleted").withPluralFutureMessage(PrintingNames.PRINTERS_LOWER + " will be deleted");
}
ready.addSelfToListIfAnyCount(lines);
add(lines, Status.WARNING, printers, PrintingNames.PRINTER_LOWER + " will be deleted with its " + PrintingNames.STATION_LOWER, PrintingNames.PRINTERS_LOWER + " will be deleted with their " + PrintingNames.STATIONS_LOWER);
add(lines, Status.WARNING, queued, "queued job will be canceled", "queued jobs will be canceled");
add(lines, Status.WARNING, inProgress, "job already sent without a reported result will be closed as outcome unknown", "jobs already sent without a reported result will be closed as outcome unknown");
add(lines, Status.INFO, kept, "finished print job keeps its history and the names it was printed under", "finished print jobs keep their history and the names they were printed under");
addRefusals(lines, refusals, false);
return lines;
}
/************************************************************************
** A counted consequence, shown only when it applies.
************************************************************************/
private static void add(List<ProcessSummaryLineInterface> lines, Status status, long count, String singular, String plural)
{
if(count > 0)
{
lines.add(new ProcessSummaryLine(status, Math.toIntExact(count), count == 1 ? singular : plural));
}
}
}
/***************************************************************************
** The delete itself, after the confirmation and the review.
***************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final List<String> refusals = new ArrayList<>();
private final ProcessSummaryLine deleted = new ProcessSummaryLine(Status.OK, 0, null);
private boolean devices;
private long printers;
private long cancelled;
private long unknown;
private long kept;
/************************************************************************
** Each record is read again with the caller's access and deleted on its
** own; a refusal names the record and does not stop the others.
************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
String actor = actor(input);
if(!Boolean.TRUE.equals(input.getValueBoolean(CONFIRM)))
{
throw new QException("Confirm the delete before continuing");
}
devices = DEVICES.equals(input.getProcessName());
PrintingService service = PrintingProcesses.currentService();
for(QRecord selected : input.getRecords())
{
QRecord record = current(devices ? PrintingTables.DEVICE : PrintingTables.PRINTER, selected.getValueString("id"));
if(record == null)
{
refusals.add("A selected record no longer exists or is not yours to delete");
continue;
}
try
{
JSONObject result = devices ? service.deleteDevice(record.getValueString("id"), actor) : service.deletePrinter(record.getValueString("id"), actor);
printers += result.optLong("printers", 0);
cancelled += result.getLong("cancelledJobs");
unknown += result.getLong("unknownJobs");
kept += result.getLong("keptJobs") - result.getLong("cancelledJobs") - result.getLong("unknownJobs");
deleted.incrementCount();
output.getRecords().add(record);
}
catch(PrintingException exception)
{
refusals.add(exception.getStatus() == 409 ? exception.getMessage() : exception.getStatus() == 404 ? name(input, record) + " no longer exists" : name(input, record) + ": " + exception.getMessage());
}
catch(java.sql.SQLException exception)
{
throw new QException(name(input, record) + " could not be deleted; nothing was changed for it. Earlier records in this selection were deleted.", exception);
}
}
}
/************************************************************************
** What was deleted and what went with it, then every refusal, including
** the records the review already set aside.
************************************************************************/
// QQQ ProcessSummaryProvider requires this concrete return type; retain binary/API compatibility.
@SuppressWarnings("NonApiType")
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
if(devices)
{
deleted.withSingularPastMessage(PrintingNames.STATION_LOWER + " was deleted; its access is revoked").withPluralPastMessage(PrintingNames.STATIONS_LOWER + " were deleted; their access is revoked");
}
else
{
deleted.withSingularPastMessage(PrintingNames.PRINTER_LOWER + " was deleted").withPluralPastMessage(PrintingNames.PRINTERS_LOWER + " were deleted");
}
deleted.addSelfToListIfAnyCount(lines);
Transform.add(lines, Status.OK, printers, PrintingNames.PRINTER_LOWER + " was deleted with its " + PrintingNames.STATION_LOWER, PrintingNames.PRINTERS_LOWER + " were deleted with their " + PrintingNames.STATIONS_LOWER);
Transform.add(lines, Status.WARNING, cancelled, "queued job was canceled", "queued jobs were canceled");
Transform.add(lines, Status.WARNING, unknown, "job was closed as outcome unknown; check the printer for output before printing it again", "jobs were closed as outcome unknown; check the printer for output before printing them again");
Transform.add(lines, Status.INFO, kept, "finished print job keeps its history and the names it was printed under", "finished print jobs keep their history and the names they were printed under");
List<String> all = new ArrayList<>(refusals);
if(getTransformStep() instanceof Transform review)
{
all.addAll(review.refusals());
}
addRefusals(lines, all, true);
return lines;
}
}
}
@@ -102,41 +102,6 @@ final class PrintingDeviceControls
/** A native authorized queue snapshot must still match after taking the device lock. */
JSONObject action(String deviceId, JSONObject request, String actor, @Nullable Set<String> authorizedQueue) throws SQLException
{
Validated valid = validated(request, actor);
try(Connection connection = source.getConnection())
{
connection.setAutoCommit(false);
try
{
JSONObject response = apply(connection, deviceId, request, valid, actor, authorizedQueue);
connection.commit();
return response;
}
catch(Exception exception)
{
connection.rollback();
throw exception;
}
}
}
/** The same action inside a transaction its caller began and will commit or roll back; nothing is committed here. */
JSONObject action(Connection connection, String deviceId, JSONObject request, String actor) throws SQLException
{
return apply(connection, deviceId, request, validated(request, actor), actor, null);
}
/** The checked parts of one action request. */
private record Validated(String requestId, String action, @Nullable String jobId)
{
}
/** Strict input is refused before a connection or a lock is taken. */
private static Validated validated(JSONObject request, String actor)
{
fields(request, Set.of("requestId", "issuedAt", "action", "jobId"));
String requestId = text(request, "requestId", 128);
@@ -154,136 +119,141 @@ final class PrintingDeviceControls
{
throw new PrintingException(400, "A bounded authenticated actor is required");
}
return new Validated(requestId, action, jobId);
}
/** Lock the device, replay a retained receipt or apply and record the action; the caller owns the transaction. */
private JSONObject apply(Connection connection, String deviceId, JSONObject request, Validated valid, String actor, @Nullable Set<String> authorizedQueue) throws SQLException
{
String requestId = valid.requestId();
String action = valid.action();
String jobId = valid.jobId();
JSONObject device = lock(connection, deviceId, false);
List<JSONObject> old = query(connection, "SELECT * FROM printing_device_command WHERE tenant_id=? AND device_id=? AND request_id=?", tenant, deviceId, requestId);
if(!old.isEmpty())
try(Connection connection = source.getConnection())
{
if(!new JSONObject(old.getFirst().getString("request_json")).similar(request))
connection.setAutoCommit(false);
try
{
throw new PrintingException(409, "Request identity was already used for a different action");
}
return record(old.getFirst());
}
Instant issued;
try
{
issued = Instant.parse(utc(request, "issuedAt"));
}
catch(java.time.format.DateTimeParseException exception)
{
throw new PrintingException(400, "Invalid action time");
}
Instant current = Instant.now();
if(issued.isBefore(current.minusSeconds(900)))
{
throw new PrintingException(410, "Action request expired; review a new action explicitly");
}
if(issued.isAfter(current.plusSeconds(60)))
{
throw new PrintingException(400, "Action time is too far in the future");
}
if(device.getInt("enabled") != 1 || device.has("revoked_at"))
{
throw new PrintingException(409, "Revoked devices require a new local enrollment");
}
if(!REMOTE.contains(action) && !Set.of("CLEAR_QUEUE", "REVOKE").contains(action) && device.getLong("control_revision") >= 9007199254740991L)
{
throw new PrintingException(409, "Control revision capacity reached");
}
expire(connection, deviceId);
update(connection, "DELETE FROM printing_device_command WHERE id IN (SELECT id FROM printing_device_command WHERE tenant_id=? AND device_id=? AND status<>'PENDING' AND created_at<? AND expires_at<? ORDER BY created_at,id LIMIT 200)", tenant, deviceId, Instant.now().minusSeconds(7 * 86400).toString(), Instant.now().toString());
long history = count(connection, "SELECT COUNT(*) AS n FROM printing_device_command WHERE tenant_id=? AND device_id=?", tenant, deviceId);
if(history >= 1000 && !"REVOKE".equals(action))
{
throw new PrintingException(429, "Device control receipt capacity reached; receipts are retained and revocation remains available");
}
String id = UUID.randomUUID().toString();
String now = current.toString();
String expires = current.plusSeconds(900).toString();
String result = null;
Integer cancelled = null;
if(REMOTE.contains(action))
{
if(device.getInt("control_version") != 1 || !new JSONArray(device.getString("control_capabilities")).toList().contains(action))
{
throw new PrintingException(409, "CONTROL_UNSUPPORTED");
}
if(count(connection, "SELECT COUNT(*) AS n FROM printing_device_command WHERE tenant_id=? AND device_id=? AND status='PENDING'", tenant, deviceId) >= 100)
{
throw new PrintingException(429, "Too many pending device commands");
}
if(jobId != null)
{
List<JSONObject> jobs = query(connection, "SELECT id,status,spooler_job_id FROM printing_job WHERE tenant_id=? AND device_id=? AND id=?", tenant, deviceId, jobId);
if(jobs.isEmpty())
JSONObject device = lock(connection, deviceId, false);
List<JSONObject> old = query(connection, "SELECT * FROM printing_device_command WHERE tenant_id=? AND device_id=? AND request_id=?", tenant, deviceId, requestId);
if(!old.isEmpty())
{
throw new PrintingException(404, "Job not found");
if(!new JSONObject(old.getFirst().getString("request_json")).similar(request))
{
throw new PrintingException(409, "Request identity was already used for a different action");
}
connection.commit();
return record(old.getFirst());
}
if(!Set.of("SPOOLED", "SPOOLER_COMPLETED").contains(jobs.getFirst().getString("status")) || jobs.getFirst().optString("spooler_job_id").isBlank())
Instant issued;
try
{
throw new PrintingException(409, "Only an owned job with a recorded spool identity can be canceled");
issued = Instant.parse(utc(request, "issuedAt"));
}
}
}
else if("CLEAR_QUEUE".equals(action))
{
if(authorizedQueue != null)
{
Set<String> currentQueue = new java.util.HashSet<>();
for(JSONObject row : query(connection, "SELECT id FROM printing_job WHERE tenant_id=? AND device_id=? AND status='QUEUED' LIMIT 10001", tenant, deviceId))
catch(java.time.format.DateTimeParseException exception)
{
currentQueue.add(row.getString("id"));
throw new PrintingException(400, "Invalid action time");
}
if(!currentQueue.equals(authorizedQueue))
Instant current = Instant.now();
if(issued.isBefore(current.minusSeconds(900)))
{
throw new PrintingException(409, "The queue changed; review the current authorized jobs before retrying");
throw new PrintingException(410, "Action request expired; review a new action explicitly");
}
if(issued.isAfter(current.plusSeconds(60)))
{
throw new PrintingException(400, "Action time is too far in the future");
}
if(device.getInt("enabled") != 1 || device.has("revoked_at"))
{
throw new PrintingException(409, "Revoked devices require a new local enrollment");
}
if(!REMOTE.contains(action) && !Set.of("CLEAR_QUEUE", "REVOKE").contains(action) && device.getLong("control_revision") >= 9007199254740991L)
{
throw new PrintingException(409, "Control revision capacity reached");
}
expire(connection, deviceId);
update(connection, "DELETE FROM printing_device_command WHERE id IN (SELECT id FROM printing_device_command WHERE tenant_id=? AND device_id=? AND status<>'PENDING' AND created_at<? AND expires_at<? ORDER BY created_at,id LIMIT 200)", tenant, deviceId, Instant.now().minusSeconds(7 * 86400).toString(), Instant.now().toString());
long history = count(connection, "SELECT COUNT(*) AS n FROM printing_device_command WHERE tenant_id=? AND device_id=?", tenant, deviceId);
if(history >= 1000 && !"REVOKE".equals(action))
{
throw new PrintingException(429, "Device control receipt capacity reached; receipts are retained and revocation remains available");
}
String id = UUID.randomUUID().toString();
String now = current.toString();
String expires = current.plusSeconds(900).toString();
String result = null;
Integer cancelled = null;
if(REMOTE.contains(action))
{
if(device.getInt("control_version") != 1 || !new JSONArray(device.getString("control_capabilities")).toList().contains(action))
{
throw new PrintingException(409, "CONTROL_UNSUPPORTED");
}
if(count(connection, "SELECT COUNT(*) AS n FROM printing_device_command WHERE tenant_id=? AND device_id=? AND status='PENDING'", tenant, deviceId) >= 100)
{
throw new PrintingException(429, "Too many pending device commands");
}
if(jobId != null)
{
List<JSONObject> jobs = query(connection, "SELECT id,status,spooler_job_id FROM printing_job WHERE tenant_id=? AND device_id=? AND id=?", tenant, deviceId, jobId);
if(jobs.isEmpty())
{
throw new PrintingException(404, "Job not found");
}
if(!Set.of("SPOOLED", "SPOOLER_COMPLETED").contains(jobs.getFirst().getString("status")) || jobs.getFirst().optString("spooler_job_id").isBlank())
{
throw new PrintingException(409, "Only an owned job with a recorded spool identity can be canceled");
}
}
}
else if("CLEAR_QUEUE".equals(action))
{
if(authorizedQueue != null)
{
Set<String> currentQueue = new java.util.HashSet<>();
for(JSONObject row : query(connection, "SELECT id FROM printing_job WHERE tenant_id=? AND device_id=? AND status='QUEUED' LIMIT 10001", tenant, deviceId))
{
currentQueue.add(row.getString("id"));
}
if(!currentQueue.equals(authorizedQueue))
{
throw new PrintingException(409, "The queue changed; review the current authorized jobs before retrying");
}
}
cancelled = update(connection, "UPDATE printing_job SET status='FAILED',message=? WHERE tenant_id=? AND device_id=? AND status='QUEUED'", PrintingService.CANCELED_BEFORE_DISPATCH, tenant, deviceId);
result = "QUEUED_JOBS_CANCELLED";
}
else if("REVOKE".equals(action))
{
update(connection, "UPDATE printing_device SET enabled=0,token_hash=?,revoked_at=?,dispatch_paused=1,connection_state='DISCONNECTED',control_revision=? WHERE tenant_id=? AND id=?", UUID.randomUUID().toString(), now, Math.min(9007199254740991L, device.getLong("control_revision") + 1), tenant, deviceId);
update(connection, "UPDATE printing_device_command SET status='OUTCOME_UNKNOWN' WHERE tenant_id=? AND device_id=? AND status='PENDING'", tenant, deviceId);
result = "CREDENTIAL_REVOKED";
}
else
{
int paused = device.getInt("dispatch_paused");
String connected = device.getString("connection_state");
if("PAUSE_DISPATCH".equals(action))
{
paused = 1;
}
if("RESUME_DISPATCH".equals(action))
{
paused = 0;
}
if("DISCONNECT".equals(action))
{
connected = "DISCONNECTED";
}
if("RECONNECT".equals(action))
{
connected = "CONNECTED";
}
update(connection, "UPDATE printing_device SET dispatch_paused=?,connection_state=?,control_revision=control_revision+1 WHERE tenant_id=? AND id=?", paused, connected, tenant, deviceId);
result = "DESIRED_STATE_UPDATED";
}
update(connection, "INSERT INTO printing_device_command (id,tenant_id,device_id,request_id,action,job_id,actor,request_json,status,created_at,expires_at,result_code,cancelled_job_count) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)",
id, tenant, deviceId, requestId, action, jobId, actor, request.toString(), REMOTE.contains(action) ? "PENDING" : "SUCCEEDED", now, expires, result, cancelled);
JSONObject response = record(command(connection, deviceId, id));
connection.commit();
return response;
}
catch(Exception exception)
{
connection.rollback();
throw exception;
}
cancelled = update(connection, "UPDATE printing_job SET status='FAILED',message=? WHERE tenant_id=? AND device_id=? AND status='QUEUED'", PrintingService.CANCELED_BEFORE_DISPATCH, tenant, deviceId);
result = "QUEUED_JOBS_CANCELLED";
}
else if("REVOKE".equals(action))
{
update(connection, "UPDATE printing_device SET enabled=0,token_hash=?,revoked_at=?,dispatch_paused=1,connection_state='DISCONNECTED',control_revision=? WHERE tenant_id=? AND id=?", UUID.randomUUID().toString(), now, Math.min(9007199254740991L, device.getLong("control_revision") + 1), tenant, deviceId);
update(connection, "UPDATE printing_device_command SET status='OUTCOME_UNKNOWN' WHERE tenant_id=? AND device_id=? AND status='PENDING'", tenant, deviceId);
result = "CREDENTIAL_REVOKED";
}
else
{
int paused = device.getInt("dispatch_paused");
String connected = device.getString("connection_state");
if("PAUSE_DISPATCH".equals(action))
{
paused = 1;
}
if("RESUME_DISPATCH".equals(action))
{
paused = 0;
}
if("DISCONNECT".equals(action))
{
connected = "DISCONNECTED";
}
if("RECONNECT".equals(action))
{
connected = "CONNECTED";
}
update(connection, "UPDATE printing_device SET dispatch_paused=?,connection_state=?,control_revision=control_revision+1 WHERE tenant_id=? AND id=?", paused, connected, tenant, deviceId);
result = "DESIRED_STATE_UPDATED";
}
update(connection, "INSERT INTO printing_device_command (id,tenant_id,device_id,request_id,action,job_id,actor,request_json,status,created_at,expires_at,result_code,cancelled_job_count) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)",
id, tenant, deviceId, requestId, action, jobId, actor, request.toString(), REMOTE.contains(action) ? "PENDING" : "SUCCEEDED", now, expires, result, cancelled);
return record(command(connection, deviceId, id));
}
@@ -75,16 +75,15 @@ public final class PrintingDeviceProcesses
PAUSE("printingPauseDevices", "Pause printing", "PAUSE_DISPATCH", false, false,
"New work will stop being dispatched. Jobs already received by Windows may still print."),
RESUME("printingResumeDevices", "Resume printing", "RESUME_DISPATCH", false, false,
"Dispatch will resume for connected, authorized " + PrintingNames.STATIONS_LOWER + ". Administrative disconnect remains unchanged."),
DISCONNECT("printingDisconnectDevices", "Disconnect " + PrintingNames.STATIONS_LOWER, "DISCONNECT", false, false,
"Dispatch will resume for connected, authorized devices. Administrative disconnect remains unchanged."),
DISCONNECT("printingDisconnectDevices", "Disconnect devices", "DISCONNECT", false, false,
"Normal printing traffic will stop. The outbound control channel remains available for reconnect."),
RECONNECT("printingReconnectDevices", "Reconnect " + PrintingNames.STATIONS_LOWER, "RECONNECT", false, false,
RECONNECT("printingReconnectDevices", "Reconnect devices", "RECONNECT", false, false,
"Connected agents will be asked to resume normal traffic. An offline computer cannot be reached immediately."),
REVOKE("printingRevokeDevices", "Revoke " + PrintingNames.STATION_LOWER + " access", "REVOKE", true, false,
REVOKE("printingRevokeDevices", "Revoke device access", "REVOKE", true, false,
"Access will be revoked permanently. The old credential cannot be restored; local re-enrollment is required."),
REMOVE("printingRemoveDevices", "Remove " + PrintingNames.STATION_LOWER, "REVOKE", true, false,
"Access will be revoked permanently and the " + PrintingNames.STATION_LOWER + " and its " + PrintingNames.PRINTERS_LOWER + " will be hidden from the printing views; the records stay, marked removed. Job history is kept. A removed "
+ PrintingNames.STATION_LOWER + " cannot be restored; a new enrollment creates a new " + PrintingNames.STATION_LOWER + ". To take the record out of the list as well, use " + PrintingDeleteProcesses.DEVICES_LABEL + "."),
REMOVE("printingRemoveDevices", "Remove device", "REVOKE", true, false,
"Access will be revoked permanently and the device and its printers will be hidden from the printing views. Job history is kept. A removed device cannot be restored; a new enrollment creates a new device."),
REFRESH("printingRefreshDevices", "Refresh printer inventory", "REFRESH_INVENTORY", false, false,
"A bounded refresh request will wait for a compatible agent. Acceptance does not mean it has executed."),
CLEAR("printingClearDeviceQueues", "Cancel queued jobs", "CLEAR_QUEUE", true, false,
@@ -183,7 +182,7 @@ public final class PrintingDeviceProcesses
if(QContext.getQSession() == null || QContext.getQSession().getUser() == null
|| QContext.getQSession().getUser().getIdReference() == null || QContext.getQSession().getUser().getIdReference().isBlank())
{
throw new QException("Sign in with an authorized QQQ user before controlling a " + PrintingNames.STATION_LOWER);
throw new QException("Sign in with an authorized QQQ user before controlling a print device");
}
return QContext.getQSession().getUser().getIdReference();
}
@@ -203,7 +202,7 @@ public final class PrintingDeviceProcesses
QRecord record = new GetAction().execute(get).getRecord();
if(record == null)
{
throw new QException("Choose an accessible " + (action.job ? "print job" : PrintingNames.STATION_LOWER));
throw new QException("Choose an accessible print " + (action.job ? "job" : "device"));
}
if(action.job && "QUEUED".equals(record.getValueString("status")))
{
@@ -288,8 +287,8 @@ public final class PrintingDeviceProcesses
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine applied = new ProcessSummaryLine(Status.OK, 0, null)
.withSingularPastMessage("server control action was applied; what the " + PrintingNames.STATION_LOWER + " reports is shown separately")
.withPluralPastMessage("server control actions were applied; what each " + PrintingNames.STATION_LOWER + " reports is shown separately");
.withSingularPastMessage("server control action was applied; device observation is shown separately")
.withPluralPastMessage("server control actions were applied; device observations are shown separately");
private final ProcessSummaryLine pending = new ProcessSummaryLine(Status.WARNING, 0, null)
.withSingularPastMessage("agent request is pending; no remote execution is confirmed")
.withPluralPastMessage("agent requests are pending; no remote execution is confirmed");
@@ -84,11 +84,11 @@ public class PrintingHome extends AbstractWidgetRenderer
{
add(instance, app, "printingHomeHeader", "Printing operations", WidgetType.COMPOSITE, "header", 0, 12);
add(instance, app, "printingHomeAttention", "Needs attention", WidgetType.TABLE, "systemAttention", 10, 12);
add(instance, app, "printingHomeOffline", "Disconnected " + PrintingNames.STATIONS_LOWER, WidgetType.STATISTICS, "systemHealth", 20, 3);
add(instance, app, "printingHomePrinters", PrintingNames.PRINTERS + " needing attention", WidgetType.STATISTICS, "systemHealth", 21, 3);
add(instance, app, "printingHomeOffline", "Disconnected devices", WidgetType.STATISTICS, "systemHealth", 20, 3);
add(instance, app, "printingHomePrinters", "Printers needing attention", WidgetType.STATISTICS, "systemHealth", 21, 3);
add(instance, app, "printingHomeFailures", "Failed or uncertain jobs", WidgetType.STATISTICS, "systemHealth", 22, 3);
add(instance, app, "printingHomePending", "Pending print jobs", WidgetType.STATISTICS, "systemHealth", 23, 3);
add(instance, app, "printingHomeControls", "Pending " + PrintingNames.STATION_LOWER + " commands", WidgetType.STATISTICS, "systemHealth", 24, 12);
add(instance, app, "printingHomeControls", "Pending device commands", WidgetType.STATISTICS, "systemHealth", 24, 12);
add(instance, app, "printingHomeWork", "Pending and recent print work", WidgetType.TABLE, "queue", 30, 12);
}
@@ -134,7 +134,7 @@ public class PrintingHome extends AbstractWidgetRenderer
case "printingHomeFailures" -> statistic(JOB, failures(now), "Jobs created in last 24 hours · target 0 failed or uncertain");
case "printingHomePending" -> statistic(JOB, statuses(PENDING), "Jobs · now · target queue drains after printing");
case "printingHomeControls" -> statistic(PrintingTables.COMMAND, statuses(List.of("PENDING"))
.withCriteria("expires_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.toString()), PrintingNames.STATION + " commands · now · awaiting an agent receipt; not confirmed applied");
.withCriteria("expires_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.toString()), "Device commands · now · awaiting an agent receipt; not confirmed applied");
case "printingHomeWork" -> work(now);
default -> throw new QException("Unknown printing home widget");
};
@@ -149,9 +149,9 @@ public class PrintingHome extends AbstractWidgetRenderer
***************************************************************************/
private static CompositeWidgetData header() throws QException
{
CompositeWidgetData data = column().withBlock(text("Keep packing slips and shipping labels moving by resolving disconnected " + PrintingNames.STATIONS_LOWER + ", " + PrintingNames.PRINTER_LOWER + " issues and print failures."))
.withBlock(text("Scope: current tenant. " + PrintingNames.STATIONS + " and " + PrintingNames.PRINTERS_LOWER + ": now. Job exceptions/recent work: last 24 hours."))
.withBlock(text("Enroll a " + PrintingNames.STATION_LOWER).withLink(new BlockLink("/app/printingEnrollDevice/?returnTo=%2Fapp%2Fprinting%2F")))
CompositeWidgetData data = column().withBlock(text("Keep packing slips and shipping labels moving by resolving disconnected devices, printer issues and print failures."))
.withBlock(text("Scope: current tenant. Devices/printers: now. Job exceptions/recent work: last 24 hours."))
.withBlock(text("Enroll a print device").withLink(new BlockLink("/app/printingEnrollDevice/?returnTo=%2Fapp%2Fprinting%2F")))
.withBlock(text("Printing retention settings").withLink(new BlockLink("/app/printingRetentionSettings/?returnTo=%2Fapp%2Fprinting%2F")))
.withBlock(text("Printing connection and limits").withLink(new BlockLink("/app/printingRealtimeSettings/?returnTo=%2Fapp%2Fprinting%2F")))
.withBlock(text("Printing queue and storage limits").withLink(new BlockLink("/app/printingAdmissionSettings/?returnTo=%2Fapp%2Fprinting%2F")))
@@ -160,7 +160,7 @@ public class PrintingHome extends AbstractWidgetRenderer
QQueryFilter present = new QQueryFilter().withCriteria("removed_at", QCriteriaOperator.IS_BLANK);
if(count(DEVICE, present) == 0)
{
data.withBlock(text("No " + PrintingNames.STATIONS_LOWER + " are connected. Open " + PrintingNames.STATIONS + " and use Enroll a " + PrintingNames.STATION_LOWER + "; then connect the Windows agent and expose its " + PrintingNames.PRINTERS_LOWER + ".").withLink(new BlockLink(tableUrl(DEVICE, present))));
data.withBlock(text("No print devices are connected. Open Print devices and use Enroll a print device; then connect the Windows agent and expose its printers.").withLink(new BlockLink(tableUrl(DEVICE, present))));
}
return data;
}
@@ -181,25 +181,25 @@ public class PrintingHome extends AbstractWidgetRenderer
{
boolean disconnected = "DISCONNECTED".equals(device.getValueString("connection_state"));
rows.add(issue(DEVICE, device, device.getValueString("name"), disconnected ? "Administratively disconnected" : "No connection for " + freshnessSeconds + " seconds",
disconnected ? "Open the " + PrintingNames.STATION_LOWER + " and use Reconnect; review its observed control revision." : "Open the Windows agent and check its service, sign-in and outbound connection.", "last_seen_at"));
disconnected ? "Open the device and use Reconnect; review its observed control revision." : "Open the Windows agent and check its service, sign-in and outbound connection.", "last_seen_at"));
}
for(QRecord device : query(DEVICE, new QQueryFilter().withCriteria("enabled", QCriteriaOperator.EQUALS, 1)
.withCriteria("dispatch_paused", QCriteriaOperator.EQUALS, 1).withOrderBy(new QFilterOrderBy("name")).withLimit(10)))
{
rows.add(issue(DEVICE, device, device.getValueString("name"), "Dispatch paused", "Open the " + PrintingNames.STATION_LOWER + " queue, then use Resume printing when it is ready.", "control_last_poll_at"));
rows.add(issue(DEVICE, device, device.getValueString("name"), "Dispatch paused", "Open the device queue, then use Resume dispatch when it is ready.", "control_last_poll_at"));
}
for(QRecord command : query(PrintingTables.COMMAND, statuses(List.of("PENDING", "FAILED", "EXPIRED", "OUTCOME_UNKNOWN"))
.withCriteria("created_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minusSeconds(86400).toString())
.withOrderBy(new QFilterOrderBy("created_at", false)).withLimit(10)))
{
rows.add(issue(PrintingTables.COMMAND, command, command.getValueString("action"), command.getValueString("command_status"),
"Review command history and the " + PrintingNames.STATION_LOWER + " connection before submitting another request.", "created_at"));
"Review command history and the device connection before submitting another request.", "created_at"));
}
for(QRecord printer : query(PRINTER, printerIssues().withOrderBy(new QFilterOrderBy("last_seen_at", true)).withLimit(10)))
{
rows.add(issue(PRINTER, printer, printer.getValueString("name"), printer.getValueString("status"), "Check paper, paused/error state and whether this queue is still exposed by the agent.", "last_seen_at"));
}
return table(rows, "No issues in this scope. If no " + PrintingNames.STATIONS_LOWER + " are enrolled, start from " + PrintingNames.STATIONS + ".",
return table(rows, "No issues in this scope. If no devices are enrolled, start from Print devices.",
new TableData.Column("composite", "Issue / record", "issue", null, "left"),
new TableData.Column("default", "Next step", "action", null, "left"),
new TableData.Column("default", "Observed at", "observed", null, "left"));
@@ -287,7 +287,7 @@ public class PrintingHome extends AbstractWidgetRenderer
private static StatisticsData offlineStatistic(Instant now) throws QException
{
long seconds = freshnessSeconds();
return statistic(DEVICE, offline(now, seconds), "Enabled " + PrintingNames.STATIONS_LOWER + " · now · disconnected by an administrator or unseen for " + seconds + " seconds");
return statistic(DEVICE, offline(now, seconds), "Enabled devices · now · disconnected by an administrator or unseen for " + seconds + " seconds");
}
@@ -80,7 +80,7 @@ public class PrintingJobProcesses
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, retry
? "Each selected job will create a new print job. Previously spooled or uncertain output may print twice. Inspect the printer before continuing."
: "Only selected jobs still queued in QQQ will be canceled. Jobs already received by a " + PrintingNames.STATION_LOWER + " must be inspected in the Windows companion.");
: "Only selected jobs still queued in QQQ will be canceled. Jobs already received by a device must be inspected in the Windows companion.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(PrintingExtract.class, Transform.class, Load.class, values)
.withName(name).withLabel(retry ? "Print again" : "Cancel queued jobs").withTableName(PrintingTables.JOB).withMinInputRecords(1)
.withPermissionRules(PrintingProcesses.permissionRules());
@@ -243,10 +243,6 @@ public class PrintingJobProcesses
}
catch(java.sql.SQLException | PrintingException exception)
{
if(exception instanceof PrintingException refused && refused.getStatus() == 404 && "Device not found".equals(refused.getMessage()))
{
throw new QException("Job " + job.getValueString("title") + " cannot be processed: its " + PrintingNames.STATION_LOWER + " was deleted, removed or revoked. Print the document again from where it was printed.", exception);
}
throw new QException("Print job action failed: " + exception.getMessage(), exception);
}
}
@@ -1,54 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
/*******************************************************************************
** The words an administrator reads for the two things printing manages. Every
** label, action label, preview and message is built from these constants, so
** a different choice of name is one edit here. Table, field and process names
** (printing_device, printingEnrollDevice, ...) and the agent wire protocol do
** not change with them.
**
** - Station: the computer that runs the print client (table printing_device).
** - Printer: a physical printer, seen as a Windows print queue that a station
** shares (table printing_printer).
*******************************************************************************/
public final class PrintingNames
{
public static final String STATION = "Print station";
public static final String STATIONS = "Print stations";
public static final String STATION_LOWER = "print station";
public static final String STATIONS_LOWER = "print stations";
public static final String PRINTER = "Printer";
public static final String PRINTERS = "Printers";
public static final String PRINTER_LOWER = "printer";
public static final String PRINTERS_LOWER = "printers";
/***************************************************************************
** Constants only.
***************************************************************************/
private PrintingNames()
{
}
}
@@ -81,16 +81,15 @@ public class PrintingProcesses implements BackendStep
{
PrintingJobProcesses.register(instance);
registerProfiles(instance);
add(instance, "printingEnrollDevice", "Enroll a " + PrintingNames.STATION_LOWER, List.of(PrintingFacilities.field(instance), field("deviceName", "Computer name")));
add(instance, "printingEnablePrinter", "Set " + PrintingNames.PRINTER_LOWER + " approval", List.of(deviceField(), printerField(null), new QFieldMetaData("enabled", QFieldType.BOOLEAN).withLabel("Approved for printing").withIsRequired(true)));
add(instance, "printingEnrollDevice", "Enroll a print device", List.of(PrintingFacilities.field(instance), field("deviceName", "Device name")));
add(instance, "printingEnablePrinter", "Set printer approval", List.of(deviceField(), printerField(null), new QFieldMetaData("enabled", QFieldType.BOOLEAN).withLabel("Approved for printing").withIsRequired(true)));
add(instance, "printingTestZpl", "Print a test label", List.of(deviceField(), printerField("ZPL"), profileField(false)));
add(instance, "printingTestPdf", "Print a test page", List.of(deviceField(), printerField("PDF"), profileField(true)));
selected(instance, "printingTestSelectedPdf", "Print a test page", "Each selected PDF printer will receive one test page after you continue.");
selected(instance, "printingTestSelectedZpl", "Print a test label", "Each selected ZPL printer will receive one test label after you continue.");
selected(instance, "printingEnableSelectedPrinters", "Approve " + PrintingNames.PRINTERS_LOWER, "The selected " + PrintingNames.PRINTERS_LOWER + " will be approved for printing after you continue.");
selected(instance, "printingDisableSelectedPrinters", "Disable " + PrintingNames.PRINTERS_LOWER, "The selected " + PrintingNames.PRINTERS_LOWER + " will stop receiving new print jobs after you continue.");
selected(instance, "printingRemoveSelectedPrinters", "Remove " + PrintingNames.PRINTERS_LOWER, "The selected " + PrintingNames.PRINTERS_LOWER + " will be hidden from the operator catalog, the selectors and their " + PrintingNames.STATIONS_LOWER + " after you continue; the records stay, marked removed. Job history is kept; sharing a queue again from its "
+ PrintingNames.STATION_LOWER + " brings it back awaiting approval. To take the record out of the list as well, use " + PrintingDeleteProcesses.PRINTERS_LABEL + ".");
selected(instance, "printingEnableSelectedPrinters", "Approve printers", "The selected printers will be approved for printing after you continue.");
selected(instance, "printingDisableSelectedPrinters", "Disable printers", "The selected printers will stop receiving new print jobs after you continue.");
selected(instance, "printingRemoveSelectedPrinters", "Remove printers", "The selected printers will be removed from Printers, the operator catalog and their connectors after you continue. Job history is kept; sharing a queue again from its device brings it back awaiting approval.");
}
@@ -156,7 +155,7 @@ public class PrintingProcesses implements BackendStep
***************************************************************************/
private static QFieldMetaData deviceField()
{
return field("deviceId", PrintingNames.STATION).withPossibleValueSourceName(PrintingTables.DEVICE)
return field("deviceId", "Print device").withPossibleValueSourceName(PrintingTables.DEVICE)
.withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("removed_at", QCriteriaOperator.IS_BLANK)));
}
@@ -175,7 +174,7 @@ public class PrintingProcesses implements BackendStep
filter.withCriteria(new QFilterCriteria("available", QCriteriaOperator.EQUALS, 1));
filter.withCriteria(new QFilterCriteria("formats", QCriteriaOperator.CONTAINS, format));
}
return field("printerId", PrintingNames.PRINTER).withPossibleValueSourceName(PrintingTables.PRINTER)
return field("printerId", "Printer").withPossibleValueSourceName(PrintingTables.PRINTER)
.withPossibleValueSourceFilter(filter).withPossibleValueSourceDependsOn(List.of("deviceId"));
}
@@ -314,7 +313,7 @@ public class PrintingProcesses implements BackendStep
QRecord printer = new GetAction().execute(get).getRecord();
if(printer == null || (deviceId != null && !deviceId.equals(printer.getValueString("device_id"))))
{
throw new QException("Choose an accessible " + PrintingNames.PRINTER_LOWER + " belonging to the selected " + PrintingNames.STATION_LOWER);
throw new QException("Choose an accessible printer belonging to the selected device");
}
return printer;
}
@@ -79,7 +79,7 @@ public class PrintingQBitProducer implements QBitProducer
/***************************************************************************
** Register the station (device), printer, job, control and deletion tables and the Printing navigation app.
** Register device, printer and job tables and the Printing navigation app.
***************************************************************************/
@Override
public void produce(QInstance instance, String namespace) throws QException
@@ -96,7 +96,7 @@ public class PrintingQBitProducer implements QBitProducer
config.getService().configureStorage(instance, config.getStorageTableName());
instance.addQBit(new QBitMetaData().withGroupId("com.kingsrook.qbits").withArtifactId("qbit-printing").withVersion(version()).withNamespace("printing").withConfig(config));
instance.addSecurityKeyType(new QSecurityKeyType().withName("printing_tenant"));
QTableMetaData devices = table(PrintingTables.DEVICE, PrintingNames.STATIONS, List.of("id", "tenant_id", "facility_id", "name", "display_name", "computer_name", "description", "location", "agent_version", "last_seen_at", "removed_at"));
QTableMetaData devices = table(PrintingTables.DEVICE, "Print devices", List.of("id", "tenant_id", "facility_id", "name", "display_name", "computer_name", "description", "location", "agent_version", "last_seen_at", "removed_at"));
devices.withField(new QFieldMetaData("enabled", QFieldType.INTEGER).withLabel("Enabled").withPossibleValueSourceName("printing_enabled"));
devices.withField(new QFieldMetaData("dispatch_paused", QFieldType.INTEGER).withLabel("Printing paused").withPossibleValueSourceName("printing_paused"));
devices.withField(new QFieldMetaData("connection_state", QFieldType.STRING).withLabel("Requested connection").withPossibleValueSourceName("printing_connectionState"));
@@ -105,29 +105,21 @@ public class PrintingQBitProducer implements QBitProducer
devices.withField(new QFieldMetaData("control_version", QFieldType.INTEGER).withLabel("Agent control version"));
devices.withField(new QFieldMetaData("control_capabilities", QFieldType.STRING).withLabel("Agent control capabilities"));
devices.withField(new QFieldMetaData("control_last_poll_at", QFieldType.STRING).withLabel("Last control connection"));
devices.withField(new QFieldMetaData("realtime_connection_node", QFieldType.STRING).withLabel("Realtime connection node"));
devices.withField(new QFieldMetaData("revoked_at", QFieldType.STRING).withLabel("Access revoked at"));
devices.withVirtualField(new QVirtualFieldMetaData("device_status", QFieldType.STRING).withLabel(PrintingNames.STATION + " status").withPossibleValueSourceName("printing_deviceStatus").withIsQuerySelectable(true));
devices.withVirtualField(new QVirtualFieldMetaData("device_status", QFieldType.STRING).withLabel("Device status").withPossibleValueSourceName("printing_deviceStatus").withIsQuerySelectable(true));
devices.withVirtualField(new QVirtualFieldMetaData("control_status", QFieldType.STRING).withLabel("Control application").withPossibleValueSourceName("printing_controlApplication").withIsQuerySelectable(true));
devices.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(PrintingDeviceStatus.class));
devices.getField("facility_id").withLabel("Facility").withPossibleValueSourceName(PrintingFacilities.register(instance, config));
QTableMetaData printers = table(PrintingTables.PRINTER, PrintingNames.PRINTERS, List.of("id", "tenant_id", "device_id", "printer_id", "name", "display_name", "queue_name", "description", "location", "driver_name", "port_name", "status", "formats", "last_seen_at", "removed_at"));
QTableMetaData printers = table(PrintingTables.PRINTER, "Printers", List.of("id", "tenant_id", "device_id", "printer_id", "name", "display_name", "queue_name", "description", "location", "driver_name", "port_name", "status", "formats", "last_seen_at", "removed_at"));
printers.withField(new QFieldMetaData("enabled", QFieldType.INTEGER).withLabel("Approved for printing").withPossibleValueSourceName("printing_approved"));
printers.withField(new QFieldMetaData("available", QFieldType.INTEGER).withLabel("Exposed by " + PrintingNames.STATION_LOWER).withPossibleValueSourceName("printing_available"));
printers.withField(new QFieldMetaData("available", QFieldType.INTEGER).withLabel("Exposed by device").withPossibleValueSourceName("printing_available"));
printers.withField(new QFieldMetaData("media_profiles_json", QFieldType.STRING).withLabel("Reported queue default media"));
QTableMetaData jobs = table(PrintingTables.JOB, "Print jobs", List.of("id", "tenant_id", "device_id", "printer_row_id", "title", "format", "content_hash", "options_json", "idempotency_key", "created_at", "status", "message", "spooler_job_id",
"device_name", "computer_name", "printer_name", "queue_name"));
// What the station and printer were called when the job was submitted: the job reads the same after either is renamed or deleted.
jobs.getField("device_name").withLabel(PrintingNames.STATION).withMaxLength(1024);
jobs.getField("computer_name").withLabel("Computer name").withMaxLength(1024);
jobs.getField("printer_name").withLabel(PrintingNames.PRINTER).withMaxLength(1024);
jobs.getField("queue_name").withLabel("Windows queue name").withMaxLength(1024);
QTableMetaData jobs = table(PrintingTables.JOB, "Print jobs", List.of("id", "tenant_id", "device_id", "printer_row_id", "title", "format", "content_hash", "options_json", "idempotency_key", "created_at", "status", "message", "spooler_job_id"));
jobs.withField(new QFieldMetaData("content_length", QFieldType.INTEGER).withLabel("Document size (bytes)"));
// created_at stays the ISO-8601 text the queue SQL orders by; created_time is the same instant as a real date-time for relative filters, sorting and date buckets.
jobs.withField(new QFieldMetaData("created_time", QFieldType.DATE_TIME).withLabel("Created"));
jobs.getField("created_at").withLabel("Created at (ISO text)");
QTableMetaData commands = table(PrintingTables.COMMAND, PrintingNames.STATION + " control history", List.of("id", "tenant_id", "device_id", "request_id", "action", "job_id", "actor", "status", "created_at", "expires_at", "result_code"));
QTableMetaData commands = table(PrintingTables.COMMAND, "Device control history", List.of("id", "tenant_id", "device_id", "request_id", "action", "job_id", "actor", "status", "created_at", "expires_at", "result_code"));
commands.withField(new QFieldMetaData("cancelled_job_count", QFieldType.INTEGER).withLabel("Queued jobs canceled"));
commands.getField("device_id").withLabel(PrintingNames.STATION).withPossibleValueSourceName(PrintingTables.DEVICE);
commands.getField("device_id").withLabel("Print device").withPossibleValueSourceName(PrintingTables.DEVICE);
commands.getField("job_id").withLabel("Print job").withPossibleValueSourceName(PrintingTables.JOB);
commands.getField("action").withPossibleValueSourceName("printing_controlAction");
commands.getField("status").withLabel("Recorded status").withPossibleValueSourceName("printing_commandStatus");
@@ -135,7 +127,7 @@ public class PrintingQBitProducer implements QBitProducer
.withPossibleValueSourceName("printing_commandStatus").withIsQuerySelectable(true));
commands.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(PrintingCommandStatus.class));
commands.withRecordLabelFormatAndFields("%s", "action");
jobs.getField("printer_row_id").withLabel(PrintingNames.PRINTER + " record").withPossibleValueSourceName(PrintingTables.PRINTER);
jobs.getField("printer_row_id").withLabel("Printer").withPossibleValueSourceName(PrintingTables.PRINTER);
jobs.getField("format").withPossibleValueSourceName("printing_format");
jobs.getField("status").withPossibleValueSourceName("printing_jobStatus");
printers.getField("status").withPossibleValueSourceName("printing_printerStatus");
@@ -165,50 +157,30 @@ public class PrintingQBitProducer implements QBitProducer
printers.withRecordLabelFormatAndFields("%s", "name");
jobs.withRecordLabelFormatAndFields("%s", "title");
printers.getField("device_id").withPossibleValueSourceName(PrintingTables.DEVICE);
jobs.getField("device_id").withLabel(PrintingNames.STATION + " record").withPossibleValueSourceName(PrintingTables.DEVICE);
printers.getField("device_id").withLabel(PrintingNames.STATION);
jobs.getField("device_id").withLabel("Print device").withPossibleValueSourceName(PrintingTables.DEVICE);
printers.getField("device_id").withLabel("Print device");
instance.addPossibleValueSource(com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource.newForTable(PrintingTables.DEVICE));
instance.addPossibleValueSource(com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource.newForTable(PrintingTables.PRINTER));
instance.addPossibleValueSource(com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource.newForTable(PrintingTables.JOB));
layout(devices, List.of("name", "device_status", "control_status", "location", "facility_id", "last_seen_at"));
layout(printers, List.of("name", "location", "description", "device_id", "status", "enabled", "available", "last_seen_at"));
layout(jobs, List.of("title", "printer_name", "device_name", "status", "created_time", "format"));
QTableMetaData deletions = table(PrintingTables.DELETION, "Deletion history", List.of("id", "tenant_id", "kind", "name", "computer_name", "queue_name", "facility_id", "actor", "record_id", "device_id"));
deletions.withField(new QFieldMetaData("deleted_time", QFieldType.DATE_TIME).withLabel("Deleted"));
deletions.withField(new QFieldMetaData("printer_count", QFieldType.INTEGER).withLabel(PrintingNames.PRINTERS + " deleted with it"));
deletions.withField(new QFieldMetaData("cancelled_job_count", QFieldType.INTEGER).withLabel("Queued jobs canceled"));
deletions.withField(new QFieldMetaData("unknown_job_count", QFieldType.INTEGER).withLabel("Jobs closed as outcome unknown"));
deletions.withField(new QFieldMetaData("kept_job_count", QFieldType.INTEGER).withLabel("Print jobs kept"));
deletions.getField("kind").withLabel("What was deleted").withPossibleValueSourceName("printing_deletionKind");
deletions.getField("name").withMaxLength(1024);
deletions.getField("computer_name").withLabel("Computer name").withMaxLength(1024);
deletions.getField("queue_name").withLabel("Windows queue name").withMaxLength(1024);
deletions.getField("facility_id").withLabel("Facility").withPossibleValueSourceName(devices.getField("facility_id").getPossibleValueSourceName());
deletions.getField("actor").withLabel("Deleted by");
deletions.getField("record_id").withLabel("Identifier it had");
deletions.getField("device_id").withLabel("Identifier of its " + PrintingNames.STATION_LOWER);
deletions.withRecordLabelFormatAndFields("%s", "name");
instance.addPossibleValueSource(new QPossibleValueSource().withName("printing_deletionKind").withType(QPossibleValueSourceType.ENUM)
.withEnumValues(List.of(new QPossibleValue<>("DEVICE", PrintingNames.STATION), new QPossibleValue<>("PRINTER", PrintingNames.PRINTER))));
layout(deletions, List.of("kind", "name", "computer_name", "actor", "deleted_time", "printer_count", "cancelled_job_count", "unknown_job_count", "kept_job_count"));
layout(jobs, List.of("title", "device_id", "printer_row_id", "status", "created_at", "format"));
layout(commands, List.of("device_id", "action", "command_status", "created_at", "expires_at", "result_code", "cancelled_job_count"));
instance.addTable(devices);
instance.addTable(printers);
instance.addTable(jobs);
instance.addTable(commands);
instance.addTable(deletions);
related(instance, devices, printers, "device_id", PrintingNames.PRINTERS);
related(instance, devices, printers, "device_id", "Printers");
related(instance, devices, jobs, "device_id", "Jobs and queue");
related(instance, devices, commands, "device_id", "Control history");
related(instance, printers, jobs, "printer_row_id", "Jobs and queue");
QAppMetaData app = new QAppMetaData().withName("printing").withLabel("Printing").withIcon(new QIcon("print")).withChild(devices).withChild(printers).withChild(jobs).withChild(commands).withChild(deletions);
QAppMetaData app = new QAppMetaData().withName("printing").withLabel("Printing").withIcon(new QIcon("print")).withChild(devices).withChild(printers).withChild(jobs).withChild(commands);
app.withPermissionRules(PrintingProcesses.permissionRules());
PrintingProcesses.register(instance, app);
PrintingOperatorProcesses.register(instance);
PrintingListProcesses.register(instance);
PrintingRenderProcesses.register(instance);
PrintingDeviceProcesses.register(instance);
PrintingDeleteProcesses.register(instance);
PrintingUiApi.register(instance);
PrintingRetentionProcesses.register(instance, config);
PrintingRealtimeProcesses.register(instance, app);
@@ -26,8 +26,6 @@ import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.metadata.MetaDataInput;
import com.kingsrook.qqq.backend.core.model.actions.metadata.TableMetaDataInput;
import com.kingsrook.qqq.backend.core.model.actions.reporting.ExportInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
@@ -55,26 +53,12 @@ public class PrintingReadPermission extends PrintingAdminPermission
{
super.checkPermissionsThrowing(input, metadata);
}
if(input != null && !isRead(input))
if(input != null && !(input instanceof QueryInput) && !(input instanceof GetInput) && !(input instanceof CountInput)
&& !(input instanceof MetaDataInput) && !(input instanceof TableMetaDataInput))
{
throw new QPermissionDeniedException("Use the printing management service to change operational state");
}
}
/***************************************************************************
** The inputs that only read rows or describe the table. QQQ does not tell a
** custom checker whether it is being asked about a read or a write, so the
** input's own type decides: a query, a get, a count, an aggregate (a grouped
** read: counts and sums by a field) and an export (a query written to a file)
** read; anything else is treated as a write and refused.
***************************************************************************/
static boolean isRead(AbstractActionInput input)
{
return input instanceof QueryInput || input instanceof GetInput || input instanceof CountInput || input instanceof AggregateInput
|| input instanceof ExportInput || input instanceof MetaDataInput || input instanceof TableMetaDataInput;
}
}
@@ -21,16 +21,51 @@
package com.kingsrook.qbits.printing;
import java.io.Serializable;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
import java.util.UUID;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.BiConsumer;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBusInterface;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
import org.jspecify.annotations.Nullable;
/** Host-owned commit notification bridge. Callbacks only schedule bounded work. */
/***************************************************************************
** Host-owned commit notification bridge. Callbacks only schedule bounded work.
**
** A wakeup always reaches the listeners of this JVM directly. Once joined to
** the application's QQQ realtime bus (see join), it is also handed to that
** bus, so the node holding a device's connection is told whichever node
** committed the change. The shared bus is best effort: a wakeup it loses is
** recovered by the connection's own reconciliation against the database on
** each heartbeat, never by this class. A message carries a tenant and a
** device identifier only.
***************************************************************************/
public class PrintingRealtimeBus
{
/** Not a browser channel: QQQ realtime streams accept only instance, user, table and record channels. */
static final String SHARED_CHANNEL = "qbit-printing";
static final String SHARED_EVENT = "printing.changed";
private static final int MAX_QUEUED_FORWARDS = 1024;
private final CopyOnWriteArrayList<BiConsumer<String, String>> listeners = new CopyOnWriteArrayList<>();
private final java.util.concurrent.ConcurrentHashMap<String, Owner> owners = new java.util.concurrent.ConcurrentHashMap<>();
private final CopyOnWriteArrayList<Runnable> gapListeners = new CopyOnWriteArrayList<>();
private final ConcurrentHashMap<String, Owner> owners = new ConcurrentHashMap<>();
private final String origin = UUID.randomUUID().toString();
private final AtomicLong forwardFailures = new AtomicLong();
private @Nullable Link link;
private int joins;
/** Atomically fence the preceding connection across route providers sharing this host bus. */
@@ -65,6 +100,12 @@ public class PrintingRealtimeBus
}
/** The shared transport, this bus's subscription to it and the bounded worker that hands wakeups to it. */
private record Link(RealtimeBusInterface transport, AutoCloseable subscription, ExecutorService forwarder)
{
}
/** Register before reading the durable snapshot to avoid a reconnect lost wakeup. */
public AutoCloseable subscribe(BiConsumer<String, String> listener)
{
@@ -73,8 +114,135 @@ public class PrintingRealtimeBus
}
/***************************************************************************
** Be told when the shared bus reports that it may have lost messages (its
** resync hint after a reconnect or an overflow), so every local connection
** can reconcile against the database at once.
***************************************************************************/
public AutoCloseable subscribeGaps(Runnable listener)
{
gapListeners.add(listener);
return () -> gapListeners.remove(listener);
}
/***************************************************************************
** Join the application's QQQ realtime bus: in memory on one node, the
** configured shared bus (such as Redis) on several. The realtime route
** provider does this itself at start; a host calls it only for a service
** that publishes without a route provider. Joining again shares the one
** link; close every returned handle to leave. One bus joins one application.
***************************************************************************/
public synchronized AutoCloseable join(QInstance instance)
{
if(link == null)
{
RealtimeBusInterface transport = RealtimeBus.get(Objects.requireNonNull(instance));
ThreadPoolExecutor forwarder = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS, new ArrayBlockingQueue<>(MAX_QUEUED_FORWARDS),
Thread.ofVirtual().name("printing-realtime-forward").factory(), (task, executor) -> forwardFailures.incrementAndGet());
link = new Link(transport, transport.subscribe(this::received), forwarder);
}
joins++;
AtomicBoolean left = new AtomicBoolean();
return () ->
{
if(left.compareAndSet(false, true))
{
leave();
}
};
}
/** Drop the link when its last user leaves; queued wakeups are abandoned because the database remains authoritative. */
private synchronized void leave() throws Exception
{
Link current = link;
if(--joins > 0 || current == null)
{
return;
}
link = null;
current.forwarder().shutdownNow();
current.subscription().close();
}
/** Wakeups this bus could not hand to the shared bus (full queue or transport error); connections recover them by reconciliation. */
public long getForwardFailures()
{
return forwardFailures.get();
}
/** Publish only after commit. Shared-host implementations must preserve this nonblocking contract. */
public void publish(String tenant, @Nullable String device)
{
deliver(tenant, device);
Link current;
synchronized(this)
{
current = link;
}
if(current != null)
{
// The transport may wait on its network; the committing caller never does.
current.forwarder().execute(() -> forward(current.transport(), tenant, device));
}
}
/** Hand one wakeup to the shared bus, marked with this bus so its own echo is not delivered twice. */
private void forward(RealtimeBusInterface transport, String tenant, @Nullable String device)
{
try
{
Map<String, Serializable> data = new LinkedHashMap<>();
data.put("origin", origin);
data.put("tenant", tenant);
if(device != null)
{
data.put("device", device);
}
transport.publish(new RealtimeMessage().withChannel(SHARED_CHANNEL).withEvent(SHARED_EVENT).withData(data));
}
catch(RuntimeException exception)
{
forwardFailures.incrementAndGet();
}
}
/** Accept only this QBit's wakeups from other buses, and the transport's own lost-message hint. */
private void received(RealtimeMessage message)
{
if(message.getAudience() == null && "instance".equals(message.getChannel()) && "resync".equals(message.getEvent()))
{
for(Runnable listener : gapListeners)
{
try
{
listener.run();
}
catch(RuntimeException ignored)
{
// A stopped listener must not stop the others; each connection also reconciles on its own heartbeat.
}
}
return;
}
Map<String, Serializable> data = message.getData();
if(!SHARED_CHANNEL.equals(message.getChannel()) || !SHARED_EVENT.equals(message.getEvent()) || data == null
|| !(data.get("origin") instanceof String sender) || origin.equals(sender) || !(data.get("tenant") instanceof String tenant) || tenant.isBlank())
{
return;
}
deliver(tenant, data.get("device") instanceof String device && !device.isBlank() ? device : null);
}
/** Run this JVM's listeners; none of them performs I/O on the calling thread. */
private void deliver(String tenant, @Nullable String device)
{
for(BiConsumer<String, String> listener : listeners)
{
@@ -0,0 +1,128 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.SQLException;
import org.jspecify.annotations.Nullable;
/***************************************************************************
** The one realtime connection a device holds, recorded on its row so every
** node of an application reads the same answer. Each statement is a single
** guarded update: the newest connection takes the row, and only the
** connection named on the row can renew, mark or clear it. A node that
** stops without clearing its row is forgotten when the lease expires.
**
** Lease times are epoch milliseconds from the writing node's clock, compared
** with the reading node's clock; nodes are expected to keep ordinary clock
** synchronization, and the lease (the heartbeat timeout) is tens of seconds.
***************************************************************************/
final class PrintingRealtimeConnections
{
private final DataSource source;
private final String tenant;
/** Bind to the same immutable tenant and pool as the queue. */
PrintingRealtimeConnections(DataSource source, String tenant)
{
this.source = source;
this.tenant = tenant;
}
/** Additive, nullable or defaulted columns: a previous release keeps working on the new schema. */
static void initialize(Connection connection) throws SQLException
{
for(String sql : new String[] {
"ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_id VARCHAR(36)",
"ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_node VARCHAR(200)",
"ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_ready INTEGER DEFAULT 0 NOT NULL",
"ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS realtime_connection_expires_at BIGINT"
})
{
PrintingService.update(connection, sql);
}
}
/** The newest authenticated connection takes the device; it must apply state and settings again before it is ready. */
void acquire(String device, String connectionId, String node, long expiresAt) throws SQLException
{
try(Connection connection = source.getConnection())
{
PrintingService.update(connection, "UPDATE printing_device SET realtime_connection_id=?, realtime_connection_node=?, realtime_connection_ready=0, realtime_connection_expires_at=? WHERE tenant_id=? AND id=?",
connectionId, node.length() > 200 ? node.substring(0, 200) : node, expiresAt, tenant, device);
}
}
/** Extend the lease; false means another connection, on this node or another, has taken the device. */
boolean renew(String device, String connectionId, long expiresAt) throws SQLException
{
try(Connection connection = source.getConnection())
{
return PrintingService.update(connection, "UPDATE printing_device SET realtime_connection_expires_at=? WHERE tenant_id=? AND id=? AND realtime_connection_id=?", expiresAt, tenant, device, connectionId) == 1;
}
}
/** Record that this connection acknowledged everything it was sent; false means it was replaced first. */
boolean ready(String device, String connectionId, long expiresAt) throws SQLException
{
try(Connection connection = source.getConnection())
{
return PrintingService.update(connection, "UPDATE printing_device SET realtime_connection_ready=1, realtime_connection_expires_at=? WHERE tenant_id=? AND id=? AND realtime_connection_id=?", expiresAt, tenant, device, connectionId) == 1;
}
}
/** Clear only this connection's row; a replacement is never removed by a late close. */
void release(String device, String connectionId) throws SQLException
{
try(Connection connection = source.getConnection())
{
PrintingService.update(connection, "UPDATE printing_device SET realtime_connection_id=NULL, realtime_connection_node=NULL, realtime_connection_ready=0, realtime_connection_expires_at=NULL WHERE tenant_id=? AND id=? AND realtime_connection_id=?",
tenant, device, connectionId);
}
}
/***************************************************************************
** Whether a connection other than the caller's own holds the device, has
** an unexpired lease, and the device has acknowledged the current desired
** state and the current settings. This is the claim gate for a request
** that reached a node which does not hold the device's socket.
***************************************************************************/
boolean readyElsewhere(String device, @Nullable String localConnectionId, long now) throws SQLException
{
try(Connection connection = source.getConnection())
{
return !PrintingService.query(connection, "SELECT d.id FROM printing_device d JOIN printing_settings s ON s.tenant_id=d.tenant_id WHERE d.tenant_id=? AND d.id=?"
+ " AND d.enabled=1 AND d.revoked_at IS NULL AND d.removed_at IS NULL AND d.realtime_connection_id IS NOT NULL AND d.realtime_connection_id<>?"
+ " AND d.realtime_connection_ready=1 AND d.realtime_connection_expires_at>? AND d.control_applied_revision=d.control_revision AND d.realtime_applied_revision=s.realtime_revision",
tenant, device, localConnectionId == null ? "" : localConnectionId, now).isEmpty();
}
}
}
@@ -53,7 +53,7 @@ public class PrintingRealtimeProcesses implements BackendStep
new Setting("maxPdfMiB", "maxPdfBytes", "Maximum PDF size (MiB)", MEBIBYTE),
new Setting("maxZplMiB", "maxZplBytes", "Maximum ZPL size (MiB)", MEBIBYTE),
new Setting("maxCopies", "maxCopies", "Maximum copies per job", 1),
new Setting("maxStagedMiB", "maxStagedBytes", "Maximum staged documents on each " + PrintingNames.STATION_LOWER + " (MiB)", MEBIBYTE));
new Setting("maxStagedMiB", "maxStagedBytes", "Maximum staged documents on each device (MiB)", MEBIBYTE));
private static final List<Setting> TRANSFERS = List.of(
new Setting("maxConcurrentDownloads", "maxConcurrentDownloads", "Concurrent document downloads", 1),
new Setting("maxConcurrentPrints", "maxConcurrentPrints", "Concurrent printer submissions", 1),
@@ -76,16 +76,16 @@ public class PrintingRealtimeProcesses implements BackendStep
new Setting("maxQueuedFrames", "maxQueuedFrames", "Maximum queued control messages", 1),
new Setting("maxQueuedControlMiB", "maxQueuedFrameBytes", "Maximum queued control message data (MiB)", MEBIBYTE));
private static final List<Setting> ADMISSION_COUNTS = List.of(
new Setting("deviceOutstandingJobs", "deviceOutstandingJobs", "Outstanding jobs per " + PrintingNames.STATION_LOWER, 1),
new Setting("deviceOutstandingJobs", "deviceOutstandingJobs", "Outstanding jobs per device", 1),
new Setting("tenantOutstandingJobs", "tenantOutstandingJobs", "Outstanding jobs across this tenant", 1),
new Setting("deviceJobsPerHour", "deviceJobsPerHour", "Jobs allocated per " + PrintingNames.STATION_LOWER + " per hour", 1),
new Setting("deviceJobsPerHour", "deviceJobsPerHour", "Jobs allocated per device per hour", 1),
new Setting("tenantJobsPerHour", "tenantJobsPerHour", "Jobs allocated across this tenant per hour", 1),
new Setting("devicePrinters", "devicePrinters", PrintingNames.PRINTERS + " per " + PrintingNames.STATION_LOWER, 1),
new Setting("devicePrinters", "devicePrinters", "Printers per device", 1),
new Setting("tenantPrinters", "tenantPrinters", "Printers across this tenant", 1));
private static final List<Setting> ADMISSION_STORAGE = List.of(
new Setting("deviceOutstandingMiB", "deviceOutstandingBytes", "Outstanding document data per " + PrintingNames.STATION_LOWER + " (MiB)", MEBIBYTE),
new Setting("deviceOutstandingMiB", "deviceOutstandingBytes", "Outstanding document data per device (MiB)", MEBIBYTE),
new Setting("tenantOutstandingMiB", "tenantOutstandingBytes", "Outstanding document data across this tenant (MiB)", MEBIBYTE),
new Setting("deviceRetainedGiB", "deviceRetainedBytes", "Retained document data per " + PrintingNames.STATION_LOWER + " (GiB)", 1024 * MEBIBYTE),
new Setting("deviceRetainedGiB", "deviceRetainedBytes", "Retained document data per device (GiB)", 1024 * MEBIBYTE),
new Setting("tenantRetainedGiB", "tenantRetainedBytes", "Retained document data across this tenant (GiB)", 1024 * MEBIBYTE));
@@ -150,9 +150,9 @@ public class PrintingRealtimeProcesses implements BackendStep
output.addValue(setting.input(), BigDecimal.valueOf(policy.getLong(setting.policy())).divide(BigDecimal.valueOf(setting.unit())));
}
}
output.addValue("transport", "The " + PrintingNames.STATION_LOWER + " opens a two-way WebSocket to this server. Documents use separate HTTPS transfers; job and command polling is not used.");
output.addValue("transport", "The device opens a two-way WebSocket to this server. Documents use separate HTTPS transfers; job and command polling is not used.");
output.addValue("scope", admission ? "These limits apply to this tenant. Capacity is checked when jobs and printers are admitted. Changing a limit does not remove existing jobs, documents or history."
: "These settings apply to this tenant. Saved changes are sent to connected " + PrintingNames.STATIONS_LOWER + ". Unsupported " + PrintingNames.STATION_LOWER + " limits require an explicit configuration correction; they are not silently reduced.");
: "These settings apply to this tenant. Saved changes are sent to connected devices. Unsupported device limits require an explicit configuration correction; they are not silently reduced.");
output.addValue("revisionLabel", admission ? "Tenant admission policy" : "Policy revision " + policy.getLong("revision"));
}
else if("execute".equals(input.getStepName()))
@@ -167,7 +167,7 @@ public class PrintingRealtimeProcesses implements BackendStep
}
JSONObject saved = admission ? service.saveAdmissionSettings(requested) : service.saveRealtimeSettings(requested);
output.addValue("result", admission ? "Queue, rate, printer and storage limits saved for this tenant. Existing jobs and history were preserved."
: "Connection and document limits saved as policy revision " + saved.getLong("revision") + ". Connected " + PrintingNames.STATIONS_LOWER + " receive the update over their existing WebSocket; no documents or job history were removed.");
: "Connection and document limits saved as policy revision " + saved.getLong("revision") + ". Connected devices receive the update over their existing WebSocket; no documents or job history were removed.");
}
else
{
@@ -36,6 +36,7 @@ import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import com.kingsrook.qqq.backend.core.actions.profile.UserProfiles;
import com.kingsrook.qqq.backend.core.cluster.QNodeRoles;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
@@ -55,7 +56,18 @@ import org.json.JSONObject;
import org.jspecify.annotations.Nullable;
/** Current-user authenticated outbound channels. SQL is authoritative; notifications are only wakeups. */
/***************************************************************************
** Current-user authenticated outbound channels. SQL is authoritative;
** notifications are only wakeups.
**
** Several nodes of one application may each run this provider over the same
** database. A device holds one connection, on one node, and that connection
** is recorded on the device row (PrintingRealtimeConnections), so a claim
** that reaches another node is judged by the same durable state. Commit
** wakeups travel between nodes on the application's QQQ realtime bus, which
** this provider joins when it starts; each connection also reconciles with
** the database on every heartbeat, so a wakeup the bus lost is only late.
***************************************************************************/
public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterface, AutoCloseable
{
public static final String PATH = "/qqq/v1/qbit/printing/device/realtime";
@@ -66,7 +78,10 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
private final Map<String, Channel> channels = new ConcurrentHashMap<>();
private final ExecutorService workers = Executors.newVirtualThreadPerTaskExecutor();
private final AutoCloseable subscription;
private final AutoCloseable gapSubscription;
private final String node = QNodeRoles.nodeId();
private @MonotonicNonNull QInstance instance;
private @Nullable AutoCloseable sharedBus;
/** Register this provider alongside the host's ordinary native QQQ endpoints. */
@@ -81,6 +96,8 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
channels.values().stream().filter(channel -> device == null || device.equals(channel.device)).forEach(Channel::changed);
}
});
// The shared bus says it may have lost messages: every connection held here re-reads the database.
gapSubscription = service.getRealtimeBus().subscribeGaps(() -> channels.values().forEach(Channel::changed));
}
@@ -94,6 +111,7 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
@Override
public void acceptJavalinConfig(JavalinConfig config)
{
joinSharedBus();
config.routes.wsBeforeUpgrade(PATH, context ->
{
try
@@ -109,7 +127,8 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
}
catch(Exception exception)
{
throw new HttpResponseException(exception instanceof QAuthenticationException ? 401 : 403, "Realtime authentication or origin denied", Map.of());
int status = upgradeStatus(exception);
throw new HttpResponseException(status, status == 503 ? "Realtime authorization is temporarily unavailable" : "Realtime authentication or origin denied", Map.of());
}
finally
{
@@ -123,12 +142,16 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
String device = Objects.requireNonNull(context.attribute(DEVICE_ATTRIBUTE), "WebSocket connected without the authorized device attribute");
Channel next = new Channel(context, device, context.attribute("printingRealtimeCredential"));
next.policy = service.realtimePolicy();
// The device row names the one connection every node honors. Taking it here fences a connection another node still holds.
service.realtimeConnections().acquire(device, next.generation, node, next.leaseExpiry());
Channel previous = channels.put(device, next);
if(previous != null)
{
previous.stop("Connection replaced");
}
service.getRealtimeBus().acquire(service.getTenantId(), device, next.generation, () -> next.stop("Connection replaced"));
// Tell the other nodes now, so one still holding the previous socket of this device closes it without waiting for its next frame.
service.getRealtimeBus().publish(service.getTenantId(), device);
context.session.setMaxTextMessageSize(next.policy.get("maxFrameBytes"));
context.session.setMaxBinaryMessageSize(0);
context.session.setIdleTimeout(Duration.ofMillis(next.policy.get("webSocketHandshakeTimeoutMillis")));
@@ -175,13 +198,53 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
}
/** Gate native claims on acknowledged current settings. */
/***************************************************************************
** Join the application's QQQ realtime bus once the host has supplied its
** instance, so commit wakeups cross nodes. On one node that bus is in
** memory and nothing changes.
***************************************************************************/
private synchronized void joinSharedBus()
{
if(sharedBus == null && instance != null)
{
sharedBus = service.getRealtimeBus().join(instance);
}
}
/***************************************************************************
** Refuse an upgrade as retryable (503) when the database could not answer,
** as sign-in required (401) when the session is not accepted, and as
** denied (403) otherwise. A client stops and asks for a person on 401 and
** 403, so an outage must never be reported as either.
***************************************************************************/
static int upgradeStatus(Exception exception)
{
for(Throwable cause = exception; cause != null; cause = cause.getCause())
{
if(cause instanceof java.sql.SQLException)
{
return 503;
}
}
return exception instanceof QAuthenticationException ? 401 : 403;
}
/***************************************************************************
** Gate native claims on acknowledged current state and settings. The
** connection held here answers for itself. A claim that reached a node
** which does not hold the device's connection is accepted when the device
** row shows a live, ready connection on another node.
***************************************************************************/
private void requireReady(String device) throws java.sql.SQLException
{
Channel channel = channels.get(device);
if(channel == null || !channel.ready(service.getRealtimeSettings().getLong("revision")))
if(channel != null && channel.ready(service.getRealtimeSettings().getLong("revision")))
{
return;
}
if(!service.realtimeConnections().readyElsewhere(device, channel == null ? null : channel.generation, System.currentTimeMillis()))
{
throw new PrintingException(409, "An authenticated realtime connection must apply current state and settings before claiming work");
}
@@ -273,6 +336,17 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
try
{
subscription.close();
gapSubscription.close();
AutoCloseable joined;
synchronized(this)
{
joined = sharedBus;
sharedBus = null;
}
if(joined != null)
{
joined.close();
}
}
catch(Exception exception)
{
@@ -364,6 +438,63 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
}
/** The lease this connection writes on its device row: one heartbeat timeout from now, renewed on every heartbeat. */
private long leaseExpiry()
{
return System.currentTimeMillis() + policy.get("heartbeatTimeoutSeconds") * 1000;
}
/** Extend the device row lease. A connection that another node replaced learns it here and stops. */
private boolean renew() throws java.sql.SQLException
{
if(service.realtimeConnections().renew(device, generation, leaseExpiry()))
{
return true;
}
stop("Connection replaced");
return false;
}
/***************************************************************************
** Heartbeat reconciliation. Keep the lease, then ask the database whether
** anything was committed that this connection has not been told (a commit
** on another node whose wakeup never arrived). If so, run the ordinary
** wakeup; otherwise only printer approval is compared. An idle device
** costs a few indexed reads per heartbeat and receives no frame.
***************************************************************************/
private void reconcile() throws Exception
{
if(!renew())
{
return;
}
long state;
long settings;
synchronized(this)
{
state = sentState;
settings = sentSettings;
}
if(service.realtimePending(device, state, settings))
{
changed();
}
else
{
printers(true, false);
}
}
/** Everything sent on this connection has been acknowledged by the device. */
private synchronized boolean caughtUp()
{
return hello != null && appliedState == sentState && appliedSettings == sentSettings;
}
/** Read synchronized connection application state. */
@@ -448,6 +579,10 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
authorizeCurrent();
if(frame == null)
{
if(!renew())
{
return;
}
snapshot(false, helloPayload());
}
else
@@ -514,6 +649,12 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
appliedState = state;
appliedSettings = settings;
}
// Durable readiness lets a claim that reaches another node pass its gate; only the connection named on the row may set it.
if(caughtUp() && !service.realtimeConnections().ready(device, generation, leaseExpiry()))
{
stop("Connection replaced");
return;
}
changed();
}
case "JOB_EVENT" ->
@@ -543,6 +684,7 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
{
service.realtimeContact(device);
send("PONG", new JSONObject().put("serverTime", Instant.now().toString()));
reconcile();
return;
}
default -> throw new PrintingException(400, "Unsupported realtime message");
@@ -745,10 +887,14 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
/** Fence and remove this connection without affecting a replacement. */
private synchronized void stop(String reason)
private void stop(String reason)
{
if(!stopped)
synchronized(this)
{
if(stopped)
{
return;
}
stopped = true;
mailbox.clear();
queuedBytes = 0;
@@ -756,6 +902,15 @@ public final class PrintingRealtimeRoutes implements QJavalinRouteProviderInterf
service.getRealtimeBus().release(service.getTenantId(), device, generation);
socket.closeSession(1008, reason);
}
try
{
service.realtimeConnections().release(device, generation);
}
catch(java.sql.SQLException | RuntimeException exception)
{
// The row is cleared only by its own connection; one that could not clear it is forgotten when its lease expires.
System.getLogger(PrintingRealtimeRoutes.class.getName()).log(System.Logger.Level.WARNING, "Realtime connection record was not cleared: " + exception.getClass().getSimpleName());
}
}
}
@@ -189,7 +189,7 @@ public final class PrintingRecordProcesses
if(!Integer.valueOf(1).equals(printer.getValueInteger("enabled")) || !Integer.valueOf(1).equals(printer.getValueInteger("available"))
|| printer.getValueString("formats") == null || !new org.json.JSONArray(printer.getValueString("formats")).toList().contains("PDF"))
{
throw new QException("Printer " + printer.getValueString("name") + " must be exposed by its " + PrintingNames.STATION_LOWER + ", approved, and support PDF");
throw new QException("Printer " + printer.getValueString("name") + " must be exposed by its device, approved, and support PDF");
}
return printer;
}
@@ -177,7 +177,7 @@ public class PrintingRetentionProcesses implements BackendStep
else if("execute".equals(input.getStepName()) && (RUN_NOW.equals(input.getProcessName()) || scheduled))
{
int removed = service.runRetention();
output.addValue("result", "Removed " + removed + " eligible documents, abandoned uploads or purged removed " + PrintingNames.PRINTERS_LOWER + " and " + PrintingNames.STATIONS_LOWER + " that no job references. Job receipts and uncertain or active work were preserved. Cleanup processes one bounded batch per run.");
output.addValue("result", "Removed " + removed + " eligible documents, abandoned uploads or purged removed printers and devices that no job references. Job receipts and uncertain or active work were preserved. Cleanup processes one bounded batch per run.");
}
else
{
@@ -212,7 +212,7 @@ public class PrintingRetentionProcesses implements BackendStep
output.addValue("progress", "Eligible documents waiting: " + settings.optLong("eligibleBacklog", 0)
+ "; last run: " + (lastRun.isBlank() ? "Not run yet" : lastRun) + "; removed: " + settings.optInt("lastRunCount", 0)
+ (settings.optString("lastRunError", "").isBlank() ? "" : "; last run problem: " + settings.optString("lastRunError")));
output.addValue("safety", "Cleanup removes only old successful-terminal document content, and purges removed " + PrintingNames.PRINTERS_LOWER + " and " + PrintingNames.STATIONS_LOWER + " that no job references. Active, failed and uncertain jobs, receipt metadata and deduplication records remain. Removed documents cannot be reprinted from history. Continue to edit the policy or run the selected cleanup action.");
output.addValue("safety", "Cleanup removes only old successful-terminal document content, and purges removed printers and devices that no job references. Active, failed and uncertain jobs, receipt metadata and deduplication records remain. Removed documents cannot be reprinted from history. Continue to edit the policy or run the selected cleanup action.");
}
@@ -38,7 +38,6 @@ import java.util.Objects;
import java.util.Set;
import java.util.UUID;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.uber.nullaway.annotations.EnsuresNonNull;
import org.json.JSONArray;
import org.json.JSONObject;
@@ -54,9 +53,6 @@ public class PrintingService
{
static final String CANCELED_BEFORE_DISPATCH = "Canceled by operator before dispatch";
private static final QLogger LOG = QLogger.getLogger(PrintingService.class);
// The jobs of one printer: by its row, or by station and queue identity for a job written before printer_row_id existed. Parameters: printer row id, device id, printer_id.
private static final String PRINTER_JOBS = "(printer_row_id=? OR (printer_row_id IS NULL AND device_id=? AND printer_id=?))";
private static final String SCHEMA_STEP_MISSING = "Print jobs are still tied to their print station by a database foreign key (printing_job.device_id). Apply the schema step of this qbit-printing version, then delete again.";
private static final String JOB_COLUMNS = "id, tenant_id, device_id, printer_id, title, format, content_hash, content_length, options_json, idempotency_key, created_at, status, message, spooler_job_id, printer_row_id";
private final DataSource dataSource;
private final String tenantId;
@@ -123,6 +119,47 @@ public class PrintingService
}
/** The durable record of each device's one realtime connection, shared by every node using this database. */
PrintingRealtimeConnections realtimeConnections()
{
return new PrintingRealtimeConnections(dataSource, tenantId);
}
/***************************************************************************
** Whether the database holds something a connection has not been told:
** a newer desired state or settings revision than it last sent, a pending
** command, or a job a claim would hand out now. A connection asks on each
** heartbeat, so a commit wakeup that never arrived (it was published on
** another node while the shared bus was unavailable) costs at most one
** heartbeat interval. Read only; it takes no row lock.
***************************************************************************/
boolean realtimePending(String deviceId, long sentState, long sentSettings) throws SQLException
{
try(Connection connection = dataSource.getConnection())
{
List<JSONObject> rows = query(connection, "SELECT d.dispatch_paused, d.connection_state, d.control_capabilities, d.control_revision, s.realtime_revision FROM printing_device d JOIN printing_settings s ON s.tenant_id=d.tenant_id"
+ " WHERE d.tenant_id=? AND d.id=? AND d.enabled=1 AND d.revoked_at IS NULL AND d.removed_at IS NULL", tenantId, deviceId);
if(rows.isEmpty())
{
return false;
}
JSONObject control = rows.getFirst();
if(control.getLong("control_revision") != sentState || control.getLong("realtime_revision") != sentSettings)
{
return true;
}
if(!query(connection, "SELECT id FROM printing_device_command WHERE tenant_id=? AND device_id=? AND status='PENDING' AND expires_at>=? AND (action<>'REFRESH_INVENTORY' OR ?='CONNECTED') LIMIT 1",
tenantId, deviceId, Instant.now().toString(), control.getString("connection_state")).isEmpty())
{
return true;
}
JSONObject candidate = dispatchCandidate(connection, deviceId, null, control);
return candidate != null && mediaSupported(candidate, control);
}
}
/** A host may share this event bus across services using the same database. */
public PrintingService withRealtimeBus(PrintingRealtimeBus bus)
{
@@ -301,8 +338,6 @@ public class PrintingService
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS content_available INTEGER DEFAULT 1 NOT NULL",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS completed_at VARCHAR(40)",
"UPDATE printing_job SET completed_at=(SELECT MAX(e.created_at) FROM printing_event e WHERE e.job_id=printing_job.id AND e.status IN ('SPOOLED','SPOOLER_COMPLETED')) WHERE completed_at IS NULL AND status IN ('SPOOLED','SPOOLER_COMPLETED')",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS created_time TIMESTAMP",
"UPDATE printing_job SET created_time=CAST(REPLACE(REPLACE(created_at, 'T', ' '), 'Z', '') AS TIMESTAMP) WHERE created_time IS NULL",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS content_storage_table VARCHAR(200)",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS content_reference VARCHAR(500)",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS content_binding VARCHAR(64)",
@@ -327,17 +362,8 @@ public class PrintingService
"ALTER TABLE printing_printer ADD COLUMN IF NOT EXISTS location VARCHAR(512)",
"UPDATE printing_device SET computer_name=name WHERE computer_name IS NULL",
"UPDATE printing_printer SET queue_name=name WHERE queue_name IS NULL",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS device_name VARCHAR(1024)",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS computer_name VARCHAR(1024)",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS printer_name VARCHAR(1024)",
"ALTER TABLE printing_job ADD COLUMN IF NOT EXISTS queue_name VARCHAR(1024)",
"UPDATE printing_job SET device_name=(SELECT d.name FROM printing_device d WHERE d.id=printing_job.device_id), computer_name=(SELECT d.computer_name FROM printing_device d WHERE d.id=printing_job.device_id) WHERE device_name IS NULL AND EXISTS (SELECT 1 FROM printing_device d WHERE d.id=printing_job.device_id)",
"UPDATE printing_job SET printer_name=(SELECT p.name FROM printing_printer p WHERE p.id=printing_job.printer_row_id), queue_name=(SELECT p.queue_name FROM printing_printer p WHERE p.id=printing_job.printer_row_id) WHERE printer_name IS NULL AND EXISTS (SELECT 1 FROM printing_printer p WHERE p.id=printing_job.printer_row_id)",
"CREATE TABLE IF NOT EXISTS printing_deletion (id VARCHAR(36) PRIMARY KEY, tenant_id VARCHAR(200) NOT NULL, kind VARCHAR(20) NOT NULL, record_id VARCHAR(36) NOT NULL, name VARCHAR(1024) NOT NULL, computer_name VARCHAR(1024), queue_name VARCHAR(1024), facility_id VARCHAR(200), device_id VARCHAR(36), actor VARCHAR(200) NOT NULL, deleted_time TIMESTAMP NOT NULL, printer_count INTEGER DEFAULT 0 NOT NULL, cancelled_job_count INTEGER DEFAULT 0 NOT NULL, unknown_job_count INTEGER DEFAULT 0 NOT NULL, kept_job_count INTEGER DEFAULT 0 NOT NULL)",
"CREATE INDEX IF NOT EXISTS printing_deletion_tenant_idx ON printing_deletion(tenant_id, deleted_time, id)",
"CREATE INDEX IF NOT EXISTS printing_job_dispatch_idx ON printing_job(tenant_id, device_id, status, created_at, id)",
"CREATE INDEX IF NOT EXISTS printing_job_history_idx ON printing_job(tenant_id, created_at, id)",
"CREATE INDEX IF NOT EXISTS printing_job_created_time_idx ON printing_job(tenant_id, created_time, id)",
"CREATE INDEX IF NOT EXISTS printing_job_retention_completed_idx ON printing_job(tenant_id, content_available, status, completed_at, id)",
"CREATE INDEX IF NOT EXISTS printing_event_job_idx ON printing_event(job_id)",
"CREATE INDEX IF NOT EXISTS printing_device_tenant_idx ON printing_device(tenant_id, last_seen_at)",
@@ -355,8 +381,8 @@ public class PrintingService
}
PrintingAdmission.initialize(connection, tenantId);
PrintingDeviceControls.initialize(connection);
PrintingRealtimeConnections.initialize(connection);
update(connection, "ALTER TABLE printing_device ADD COLUMN IF NOT EXISTS bound_user_id VARCHAR(200)");
dropJobDeviceForeignKey(connection);
widenColumn(connection, "printing_device", "name", 1024);
widenColumn(connection, "printing_enrollment", "device_name", 1024);
widenColumn(connection, "printing_printer", "name", 1024);
@@ -372,62 +398,6 @@ public class PrintingService
/***************************************************************************
** A print job outlives the print station it was sent to: once a station can
** be deleted, printing_job.device_id is the identifier the station had, not
** a reference that must resolve. Drop the foreign key the first schema
** created (its name is database-generated, so it is found, not assumed).
***************************************************************************/
private static void dropJobDeviceForeignKey(Connection connection) throws SQLException
{
for(String constraint : jobDeviceForeignKeys(connection))
{
try(PreparedStatement statement = connection.prepareStatement("ALTER TABLE printing_job DROP CONSTRAINT \"" + constraint + "\""))
{
statement.execute();
}
}
}
/***************************************************************************
** The names of the foreign keys from printing_job to printing_device in the
** connection's schema; none once this version's schema step has run.
***************************************************************************/
private static Set<String> jobDeviceForeignKeys(Connection connection) throws SQLException
{
List<String> jobTables = new ArrayList<>();
try(ResultSet tables = connection.getMetaData().getTables(connection.getCatalog(), connection.getSchema(), "%", new String[] {"TABLE"}))
{
while(tables.next())
{
if("printing_job".equalsIgnoreCase(tables.getString("TABLE_NAME")))
{
jobTables.add(tables.getString("TABLE_NAME"));
}
}
}
Set<String> constraints = new java.util.LinkedHashSet<>();
for(String jobTable : jobTables)
{
try(ResultSet keys = connection.getMetaData().getImportedKeys(connection.getCatalog(), connection.getSchema(), jobTable))
{
while(keys.next())
{
String name = keys.getString("FK_NAME");
if("printing_device".equalsIgnoreCase(keys.getString("PKTABLE_NAME")) && name != null && name.matches("[A-Za-z0-9_$]+"))
{
constraints.add(name);
}
}
}
}
return constraints;
}
/***************************************************************************
** Widen fixed application columns without changing identifiers or content.
***************************************************************************/
@@ -922,7 +892,6 @@ public class PrintingService
{
if(!query(connection, "SELECT id FROM printing_printer WHERE device_id=? AND printer_id=?", deviceId, printerId).isEmpty())
{
// The HTTP printer approval route returns this text; an existing client reads it as it always was.
throw new PrintingException(409, "Printer was removed; share it from the device again before changing its approval");
}
throw new PrintingException(404, "Printer not found");
@@ -970,7 +939,7 @@ public class PrintingService
JSONObject device = requireDeviceRow(connection, deviceId);
if(!device.has("revoked_at"))
{
throw new PrintingException(409, "Revoke the " + PrintingNames.STATION_LOWER + "'s access before removing it");
throw new PrintingException(409, "Revoke the device's access before removing it");
}
String now = Instant.now().toString();
update(connection, "UPDATE printing_device SET removed_at=? WHERE tenant_id=? AND id=? AND removed_at IS NULL", now, tenantId, deviceId);
@@ -981,378 +950,6 @@ public class PrintingService
/***************************************************************************
** What deleting a print station would do, without changing anything: the
** printers that go with it, the queued jobs that would be canceled, the jobs
** already sent to it that would be closed as outcome unknown, the jobs whose
** history is kept, and "refusal" when it cannot be deleted now.
***************************************************************************/
public JSONObject deviceDeletionPlan(String deviceId) throws SQLException
{
try(Connection connection = dataSource.getConnection())
{
return deviceDeletionPlan(connection, deviceId, false);
}
}
/***************************************************************************
** What deleting one printer would do, without changing anything.
***************************************************************************/
public JSONObject printerDeletionPlan(String printerRowId) throws SQLException
{
try(Connection connection = dataSource.getConnection())
{
return printerDeletionPlan(connection, printerRowId);
}
}
/***************************************************************************
** Delete a print station and its printers for good.
**
** All of it is one transaction that takes the tenant admission lock and then
** the station's row lock, the lock a claim, a job event and a heartbeat take
** before they change anything. Nothing the station does can therefore land
** between the decision and the delete, and a delete that is refused or fails
** changes nothing: the station is not left revoked.
**
** 1. Refused (409) while the station is connected and has jobs it has been
** sent but not reported on: their result would be lost. This is decided
** under the locks, before anything is revoked.
** 2. Its credential is revoked through the REVOKE control, exactly as Revoke
** access does, so the computer is disconnected and cannot come back as
** this identity. An already revoked station skips this step.
** 3. Every job of the station gets the station and printer names it was
** printed under (if it does not have them yet); queued jobs become FAILED
** (canceled before dispatch); jobs sent to the station without a reported
** result become OUTCOME_UNKNOWN; each of those gets a job event naming the
** actor; the deletion is written to printing_deletion (one row for the
** station, one for each printer); the control receipts, the printers and
** the station row are deleted.
**
** Jobs are never deleted and keep device_id, printer_id and printer_row_id
** as the identifiers the station and printer had. A computer enrolled again
** is a new station with a new identifier. Anything that arrives later from
** the deleted station (heartbeat, claim, event, control poll or receipt) is
** answered "Device not found".
***************************************************************************/
public JSONObject deleteDevice(String deviceId, String actor) throws SQLException
{
requireActor(actor);
JSONObject result;
try(Connection connection = dataSource.getConnection())
{
// Schema metadata only, read before any lock is held.
boolean jobsReferenceStation = !jobDeviceForeignKeys(connection).isEmpty();
PrintingAdmission.initializeTenant(connection, tenantId);
connection.setAutoCommit(false);
try
{
PrintingAdmission.lock(connection, tenantId);
JSONObject plan = deviceDeletionPlan(connection, deviceId, true);
if(jobsReferenceStation && plan.getLong("jobs") > 0)
{
throw new PrintingException(409, SCHEMA_STEP_MISSING);
}
// The station row is locked: a claim or a job event waits for this transaction, so what is decided here still holds when the station is revoked and deleted below.
refuseIfPlanned(plan);
boolean revoked = plan.getBoolean("credentialLive");
if(revoked)
{
revokeBeforeDelete(connection, deviceId, actor);
}
Instant now = Instant.now();
String name = plan.getString("name");
String computer = plan.getString("computerName");
update(connection, "UPDATE printing_job SET device_name=COALESCE(device_name,?), computer_name=COALESCE(computer_name,?) WHERE tenant_id=? AND device_id=? AND (device_name IS NULL OR computer_name IS NULL)", name, computer, tenantId, deviceId);
List<JSONObject> printers = query(connection, "SELECT id, printer_id, name, queue_name FROM printing_printer WHERE tenant_id=? AND device_id=? ORDER BY name, id", tenantId, deviceId);
for(JSONObject printer : printers)
{
namePrinterJobs(connection, printer, deviceId);
}
int[] closed = closeOpenJobs(connection, "device_id=?", new Object[] {deviceId}, PrintingNames.STATION_LOWER, actor, now);
long kept = query(connection, "SELECT COUNT(*) AS n FROM printing_job WHERE tenant_id=? AND device_id=?", tenantId, deviceId).getFirst().getLong("n");
for(JSONObject printer : printers)
{
recordDeletion(connection, "PRINTER", printer.getString("id"), printer.getString("name"), computer, printer.optString("queue_name", printer.getString("name")), plan.optString("facilityId", null), deviceId, actor, now, 0, 0, 0, 0);
}
recordDeletion(connection, "DEVICE", deviceId, name, computer, null, plan.optString("facilityId", null), null, actor, now, printers.size(), closed[0], closed[1], kept);
update(connection, "DELETE FROM printing_device_command WHERE tenant_id=? AND device_id=?", tenantId, deviceId);
update(connection, "DELETE FROM printing_printer WHERE tenant_id=? AND device_id=?", tenantId, deviceId);
// The retained-byte counter stays while stored documents of this station's jobs are still charged to it; cleanup removes it afterwards.
update(connection, "DELETE FROM printing_resource_usage WHERE tenant_id=? AND device_id=? AND retained_bytes=0", tenantId, deviceId);
update(connection, "DELETE FROM printing_device WHERE tenant_id=? AND id=?", tenantId, deviceId);
connection.commit();
result = new JSONObject().put("deleted", true).put("deviceId", deviceId).put("name", name).put("printers", printers.size())
.put("cancelledJobs", closed[0]).put("unknownJobs", closed[1]).put("keptJobs", kept).put("credentialRevoked", revoked);
}
catch(Exception exception)
{
connection.rollback();
refuseIfSchemaStepMissing(exception);
throw exception;
}
}
LOG.info("A print station was deleted", logPair("tenantId", tenantId), logPair("deviceId", deviceId), logPair("actor", actor), logPair("printers", result.getInt("printers")),
logPair("cancelledJobs", result.getInt("cancelledJobs")), logPair("unknownJobs", result.getInt("unknownJobs")));
realtimeBus.publish(tenantId, deviceId);
return result;
}
/***************************************************************************
** Delete one printer for good, leaving its print station in service.
**
** Refused (409) while its station is connected and has jobs for this printer
** that it has been sent but not reported on. Otherwise, in one transaction:
** the printer's jobs get the printer and station names they were printed
** under (if missing); its queued jobs become FAILED (canceled before
** dispatch); jobs sent without a reported result become OUTCOME_UNKNOWN;
** the deletion is written to printing_deletion; the printer row is deleted.
**
** A queue the station still shares is reported again by its next inventory
** and comes back as a new printer, exposed and not approved.
***************************************************************************/
public JSONObject deletePrinter(String printerRowId, String actor) throws SQLException
{
requireActor(actor);
JSONObject result;
String deviceId;
try(Connection connection = dataSource.getConnection())
{
PrintingAdmission.initializeTenant(connection, tenantId);
connection.setAutoCommit(false);
try
{
PrintingAdmission.lock(connection, tenantId);
deviceId = printerDeletionPlan(connection, printerRowId).getString("deviceId");
query(connection, "SELECT id FROM printing_device WHERE tenant_id=? AND id=? FOR UPDATE", tenantId, deviceId);
JSONObject plan = printerDeletionPlan(connection, printerRowId);
refuseIfPlanned(plan);
Instant now = Instant.now();
JSONObject printer = new JSONObject().put("id", printerRowId).put("printer_id", plan.getString("printerKey")).put("name", plan.getString("name")).put("queue_name", plan.getString("queueName"));
update(connection, "UPDATE printing_job SET device_name=COALESCE(device_name,?), computer_name=COALESCE(computer_name,?) WHERE tenant_id=? AND " + PRINTER_JOBS, plan.getString("stationName"), plan.getString("computerName"), tenantId, printerRowId, deviceId, plan.getString("printerKey"));
namePrinterJobs(connection, printer, deviceId);
int[] closed = closeOpenJobs(connection, PRINTER_JOBS, new Object[] {printerRowId, deviceId, plan.getString("printerKey")}, PrintingNames.PRINTER_LOWER, actor, now);
long kept = query(connection, "SELECT COUNT(*) AS n FROM printing_job WHERE tenant_id=? AND " + PRINTER_JOBS, tenantId, printerRowId, deviceId, plan.getString("printerKey")).getFirst().getLong("n");
recordDeletion(connection, "PRINTER", printerRowId, plan.getString("name"), plan.getString("computerName"), plan.getString("queueName"), plan.optString("facilityId", null), deviceId, actor, now, 0, closed[0], closed[1], kept);
update(connection, "DELETE FROM printing_printer WHERE tenant_id=? AND id=?", tenantId, printerRowId);
connection.commit();
result = new JSONObject().put("deleted", true).put("printerId", printerRowId).put("deviceId", deviceId).put("name", plan.getString("name"))
.put("cancelledJobs", closed[0]).put("unknownJobs", closed[1]).put("keptJobs", kept);
}
catch(Exception exception)
{
connection.rollback();
throw exception;
}
}
LOG.info("A printer was deleted", logPair("tenantId", tenantId), logPair("printerRowId", printerRowId), logPair("deviceId", deviceId), logPair("actor", actor),
logPair("cancelledJobs", result.getInt("cancelledJobs")), logPair("unknownJobs", result.getInt("unknownJobs")));
realtimeBus.publish(tenantId, deviceId);
return result;
}
/***************************************************************************
** Revoke with the control the Revoke access action uses (credential rotated,
** disconnected, pending commands closed), inside the delete's transaction
** and under its station row lock: a delete that then fails rolls the
** revocation back with everything else. The connection is woken once, after
** the delete commits.
***************************************************************************/
private void revokeBeforeDelete(Connection connection, String deviceId, String actor) throws SQLException
{
new PrintingDeviceControls(dataSource, tenantId).action(connection, deviceId, new JSONObject().put("requestId", "delete-" + UUID.randomUUID()).put("issuedAt", Instant.now().toString()).put("action", "REVOKE"), actor);
}
/***************************************************************************
** Read (and with lock, hold) one station and count what a delete touches.
***************************************************************************/
private JSONObject deviceDeletionPlan(Connection connection, String deviceId, boolean lock) throws SQLException
{
List<JSONObject> rows = query(connection, "SELECT * FROM printing_device WHERE tenant_id=? AND id=?" + (lock ? " FOR UPDATE" : ""), tenantId, deviceId);
if(rows.isEmpty())
{
throw new PrintingException(404, "Device not found");
}
JSONObject device = rows.getFirst();
String name = device.getString("name");
long inProgress = jobCount(connection, "device_id=? AND status IN ('CLAIMED','RECEIVED','SUBMITTING')", deviceId);
JSONObject plan = new JSONObject().put("deviceId", deviceId).put("name", name).put("computerName", device.optString("computer_name", name)).put("facilityId", device.getString("facility_id"))
.put("printers", query(connection, "SELECT COUNT(*) AS n FROM printing_printer WHERE tenant_id=? AND device_id=?", tenantId, deviceId).getFirst().getLong("n"))
.put("queuedJobs", jobCount(connection, "device_id=? AND status='QUEUED'", deviceId)).put("jobsInProgress", inProgress).put("jobs", jobCount(connection, "device_id=?", deviceId))
.put("credentialLive", device.getInt("enabled") == 1 && !device.has("revoked_at"));
if(inProgress > 0 && reporting(device, Instant.now()))
{
plan.put("refusal", PrintingNames.STATION + " " + name + " is connected and is printing " + inProgress + (inProgress == 1 ? " job" : " jobs")
+ " right now. Wait until " + (inProgress == 1 ? "it finishes" : "they finish") + ", or use Disconnect or Revoke access first (the result is then recorded as unknown), and delete it again.");
}
return plan;
}
/***************************************************************************
** Read one printer with its station and count what a delete touches.
***************************************************************************/
private JSONObject printerDeletionPlan(Connection connection, String printerRowId) throws SQLException
{
List<JSONObject> rows = query(connection, "SELECT id, device_id, printer_id, name, queue_name FROM printing_printer WHERE tenant_id=? AND id=?", tenantId, printerRowId);
if(rows.isEmpty())
{
throw new PrintingException(404, "Printer not found");
}
JSONObject printer = rows.getFirst();
String deviceId = printer.getString("device_id");
String key = printer.getString("printer_id");
String name = printer.getString("name");
JSONObject device = query(connection, "SELECT * FROM printing_device WHERE tenant_id=? AND id=?", tenantId, deviceId).getFirst();
long inProgress = jobCount(connection, PRINTER_JOBS + " AND status IN ('CLAIMED','RECEIVED','SUBMITTING')", printerRowId, deviceId, key);
JSONObject plan = new JSONObject().put("printerId", printerRowId).put("deviceId", deviceId).put("printerKey", key).put("name", name).put("queueName", printer.optString("queue_name", name))
.put("stationName", device.getString("name")).put("computerName", device.optString("computer_name", device.getString("name"))).put("facilityId", device.getString("facility_id"))
.put("queuedJobs", jobCount(connection, PRINTER_JOBS + " AND status='QUEUED'", printerRowId, deviceId, key)).put("jobsInProgress", inProgress)
.put("jobs", jobCount(connection, PRINTER_JOBS, printerRowId, deviceId, key));
if(inProgress > 0 && reporting(device, Instant.now()))
{
plan.put("refusal", PrintingNames.PRINTER + " " + name + " is printing " + inProgress + (inProgress == 1 ? " job" : " jobs") + " right now on " + PrintingNames.STATION_LOWER + " " + device.getString("name")
+ ". Wait until " + (inProgress == 1 ? "it finishes" : "they finish") + ", or disconnect the " + PrintingNames.STATION_LOWER + " first (the result is then recorded as unknown), and delete it again.");
}
return plan;
}
/***************************************************************************
** Count this tenant's jobs matching a fixed predicate.
***************************************************************************/
private long jobCount(Connection connection, String predicate, Object... values) throws SQLException
{
Object[] scoped = new Object[values.length + 1];
scoped[0] = tenantId;
System.arraycopy(values, 0, scoped, 1, values.length);
return query(connection, "SELECT COUNT(*) AS n FROM printing_job WHERE tenant_id=? AND " + predicate, scoped).getFirst().getLong("n");
}
/***************************************************************************
** Can this station still report the result of a job it holds? Only one that
** holds a live credential and shows as ONLINE or PAUSED in the Status column
** (seen in the last 90 seconds and not disconnected by an administrator).
***************************************************************************/
private static boolean reporting(JSONObject device, Instant now)
{
if(device.getInt("enabled") != 1 || device.has("revoked_at"))
{
return false;
}
QRecord record = new QRecord().withValue("enabled", device.getInt("enabled")).withValue("connection_state", device.optString("connection_state", "CONNECTED"))
.withValue("last_seen_at", device.optString("last_seen_at", null)).withValue("control_last_poll_at", device.optString("control_last_poll_at", null))
.withValue("dispatch_paused", device.optInt("dispatch_paused", 0));
return Set.of("ONLINE", "PAUSED").contains(PrintingDeviceStatus.presence(record, now));
}
/***************************************************************************
** A plan that names a refusal is not carried out.
***************************************************************************/
private static void refuseIfPlanned(JSONObject plan)
{
if(plan.has("refusal"))
{
throw new PrintingException(409, plan.getString("refusal"));
}
}
/***************************************************************************
** A deletion is always attributed to a real, bounded actor.
***************************************************************************/
private static void requireActor(@Nullable String actor)
{
if(actor == null || actor.isBlank() || actor.length() > 200)
{
throw new PrintingException(400, "A bounded authenticated actor is required");
}
}
/***************************************************************************
** Give the printer's jobs the printer and queue names, where they have none.
***************************************************************************/
private void namePrinterJobs(Connection connection, JSONObject printer, String deviceId) throws SQLException
{
String name = printer.getString("name");
update(connection, "UPDATE printing_job SET printer_name=COALESCE(printer_name,?), queue_name=COALESCE(queue_name,?) WHERE tenant_id=? AND " + PRINTER_JOBS + " AND (printer_name IS NULL OR queue_name IS NULL)",
name, printer.optString("queue_name", name), tenantId, printer.getString("id"), deviceId, printer.getString("printer_id"));
}
/***************************************************************************
** Close work that can no longer complete: a queued job is canceled (FAILED),
** a job that was sent without a reported result becomes OUTCOME_UNKNOWN.
** Each gets a job event naming the actor. Returns {canceled, unknown}.
***************************************************************************/
private int[] closeOpenJobs(Connection connection, String predicate, Object[] values, String what, String actor, Instant now) throws SQLException
{
Object[] scoped = new Object[values.length + 1];
scoped[0] = tenantId;
System.arraycopy(values, 0, scoped, 1, values.length);
int[] closed = new int[2];
for(JSONObject job : query(connection, "SELECT id, status FROM printing_job WHERE tenant_id=? AND " + predicate + " AND status IN ('QUEUED','CLAIMED','RECEIVED','SUBMITTING') ORDER BY created_at, id FOR UPDATE", scoped))
{
boolean queued = "QUEUED".equals(job.getString("status"));
String status = queued ? "FAILED" : "OUTCOME_UNKNOWN";
String message = queued ? "Canceled before dispatch: its " + what + " was deleted" : "Its " + what + " was deleted before a result was reported; check the printer for output before printing again";
JSONObject event = new JSONObject().put("type", "DELETED").put("origin", "SERVER_DELETION").put("status", status).put("actor", actor).put("message", message);
update(connection, "INSERT INTO printing_event (id, job_id, status, event_json, created_at, event_origin) VALUES (?, ?, ?, ?, ?, 'SERVER_DELETION')", UUID.randomUUID().toString(), job.getString("id"), status, event.toString(), now.toString());
update(connection, "UPDATE printing_job SET status=?, message=? WHERE id=?", status, message, job.getString("id"));
closed[queued ? 0 : 1]++;
}
return closed;
}
/***************************************************************************
** The permanent record of a deletion: who deleted what, when, and what went
** with it. It has no reference to the deleted rows, only what they were.
***************************************************************************/
private void recordDeletion(Connection connection, String kind, String recordId, String name, @Nullable String computer, @Nullable String queue, @Nullable String facility, @Nullable String deviceId,
String actor, Instant now, int printers, int cancelled, int unknown, long kept) throws SQLException
{
update(connection, "INSERT INTO printing_deletion (id, tenant_id, kind, record_id, name, computer_name, queue_name, facility_id, device_id, actor, deleted_time, printer_count, cancelled_job_count, unknown_job_count, kept_job_count) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
UUID.randomUUID().toString(), tenantId, kind, recordId, name, computer, queue, facility, deviceId, actor, java.time.LocalDateTime.ofInstant(now, java.time.ZoneOffset.UTC), printers, cancelled, unknown, kept);
}
/***************************************************************************
** A host that owns its schema and has not applied this version's step still
** has the job-to-station foreign key: say so instead of a constraint error.
***************************************************************************/
private static void refuseIfSchemaStepMissing(Exception exception)
{
if(exception instanceof SQLException sql && "23503".equals(sql.getSQLState()))
{
throw new PrintingException(409, SCHEMA_STEP_MISSING);
}
}
/***************************************************************************
** Tenant ownership of a device row regardless of its revoked or removed state.
***************************************************************************/
@@ -1668,12 +1265,7 @@ public class PrintingService
configuredStorage().put(storageTableName, reference, format, content);
}
checkSubmission(guard);
// One instant, two columns: created_at is the ISO-8601 text the queue SQL and protocol compare; created_time is the UTC timestamp QQQ filters as a date-time.
Instant createdAt = Instant.now();
// The names the station and printer have now are stored on the job, so its history reads the same after either is renamed or deleted.
JSONObject station = query(connection, "SELECT name, computer_name FROM printing_device WHERE tenant_id=? AND id=?", tenantId, deviceId).getFirst();
update(connection, "INSERT INTO printing_job (id, tenant_id, device_id, printer_id, title, format, content_hash, content_base64, content_length, options_json, idempotency_key, created_at, created_time, status, printer_row_id, content_storage_table, content_reference, content_binding, device_name, computer_name, printer_name, queue_name) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'QUEUED', ?, ?, ?, ?, ?, ?, ?, ?)", id, tenantId, deviceId, printerId, title, format, digest, storageTableName == null ? Base64.getEncoder().encodeToString(content) : null, content.length, options.toString(), key, createdAt.toString(), java.time.LocalDateTime.ofInstant(createdAt, java.time.ZoneOffset.UTC), printers.getFirst().getString("id"), storageTableName, storageTableName == null ? null : reference, storageTableName == null ? null : binding,
station.getString("name"), station.optString("computer_name", station.getString("name")), printers.getFirst().getString("name"), printers.getFirst().optString("queue_name", printers.getFirst().getString("name")));
update(connection, "INSERT INTO printing_job (id, tenant_id, device_id, printer_id, title, format, content_hash, content_base64, content_length, options_json, idempotency_key, created_at, status, printer_row_id, content_storage_table, content_reference, content_binding) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'QUEUED', ?, ?, ?, ?)", id, tenantId, deviceId, printerId, title, format, digest, storageTableName == null ? Base64.getEncoder().encodeToString(content) : null, content.length, options.toString(), key, Instant.now().toString(), printers.getFirst().getString("id"), storageTableName, storageTableName == null ? null : reference, storageTableName == null ? null : binding);
if(bound != null)
{
String eventId = UUID.randomUUID().toString();
@@ -1916,6 +1508,26 @@ public class PrintingService
/** Native adapters authorize one exact job and printer before this locked dispatch transition. */
public JSONObject claimForJob(String deviceId, @Nullable String authorizedJobId) throws SQLException
{
return claimAttempt(deviceId, authorizedJobId).response();
}
/** One claim's wire response, and whether this call moved the job from QUEUED to CLAIMED (false for an empty queue or a replayed assignment). */
record ClaimAttempt(JSONObject response, boolean transitioned)
{
}
/***************************************************************************
** Claim under the device row lock, which serializes every claim, cancel
** and event for the device across all nodes sharing the database. The
** status guard on the update makes the QUEUED to CLAIMED transition
** happen once even for a writer that did not take that lock: a job is
** handed out by exactly one transition, and every later claim replays the
** same assignment until the device records RECEIVED.
***************************************************************************/
ClaimAttempt claimAttempt(String deviceId, @Nullable String authorizedJobId) throws SQLException
{
try(Connection connection = dataSource.getConnection())
{
@@ -1926,30 +1538,28 @@ public class PrintingService
query(connection, "SELECT id FROM printing_device WHERE id=? FOR UPDATE", deviceId);
requireDevice(connection, deviceId);
JSONObject control = query(connection, "SELECT dispatch_paused,connection_state,control_capabilities FROM printing_device WHERE tenant_id=? AND id=?", tenantId, deviceId).getFirst();
if(control.getInt("dispatch_paused") == 1 || !"CONNECTED".equals(control.getString("connection_state")))
JSONObject selected = dispatchCandidate(connection, deviceId, authorizedJobId, control);
if(selected == null)
{
connection.commit();
return new JSONObject().put("job", JSONObject.NULL);
return new ClaimAttempt(new JSONObject().put("job", JSONObject.NULL), false);
}
List<JSONObject> candidates = query(connection, "SELECT j." + JOB_COLUMNS.replace(", ", ", j.") + " FROM printing_job j JOIN printing_printer p ON p.device_id=j.device_id AND p.printer_id=j.printer_id WHERE j.tenant_id=? AND j.device_id=? AND p.enabled=1 AND p.available=1 AND p.removed_at IS NULL AND j.status IN ('QUEUED','CLAIMED') AND ((j.format='PDF' AND p.formats LIKE '%\"PDF\"%') OR (j.format='ZPL' AND p.formats LIKE '%\"ZPL\"%'))" + (authorizedJobId == null ? "" : " AND j.id=?") + " ORDER BY j.created_at, j.id LIMIT 1", authorizedJobId == null ? new Object[] { tenantId, deviceId } : new Object[] { tenantId, deviceId, authorizedJobId });
if(candidates.isEmpty())
{
connection.commit();
return new JSONObject().put("job", JSONObject.NULL);
}
JSONObject selected = candidates.getFirst();
if(new JSONObject(selected.getString("options_json")).has("mediaProfile")
&& !new JSONArray(control.optString("control_capabilities", "[]")).toList().contains("printer-media-v1"))
if(!mediaSupported(selected, control))
{
throw new PrintingException(409, "This job requires an agent that validates printer media profiles");
}
boolean transitioned = false;
if("QUEUED".equals(selected.getString("status")))
{
update(connection, "UPDATE printing_job SET status='CLAIMED' WHERE id=?", selected.getString("id"));
if(update(connection, "UPDATE printing_job SET status='CLAIMED' WHERE id=? AND tenant_id=? AND status='QUEUED'", selected.getString("id"), tenantId) != 1)
{
throw new PrintingException(409, "The job changed before it could be claimed");
}
selected.put("status", "CLAIMED");
transitioned = true;
}
connection.commit();
return new JSONObject().put("job", job(selected));
return new ClaimAttempt(new JSONObject().put("job", job(selected)), transitioned);
}
catch(Exception exception)
{
@@ -1960,6 +1570,32 @@ public class PrintingService
}
/***************************************************************************
** The one job a claim would hand this device now: the oldest QUEUED or
** CLAIMED job for an approved, exposed printer that accepts its format.
** Null while dispatch is paused, the device is asked to disconnect or
** nothing qualifies. The claim and the heartbeat reconciliation share it,
** so a connection is only told about work a claim would return.
***************************************************************************/
private @Nullable JSONObject dispatchCandidate(Connection connection, String deviceId, @Nullable String authorizedJobId, JSONObject control) throws SQLException
{
if(control.getInt("dispatch_paused") == 1 || !"CONNECTED".equals(control.getString("connection_state")))
{
return null;
}
List<JSONObject> candidates = query(connection, "SELECT j." + JOB_COLUMNS.replace(", ", ", j.") + " FROM printing_job j JOIN printing_printer p ON p.device_id=j.device_id AND p.printer_id=j.printer_id WHERE j.tenant_id=? AND j.device_id=? AND p.enabled=1 AND p.available=1 AND p.removed_at IS NULL AND j.status IN ('QUEUED','CLAIMED') AND ((j.format='PDF' AND p.formats LIKE '%\"PDF\"%') OR (j.format='ZPL' AND p.formats LIKE '%\"ZPL\"%'))" + (authorizedJobId == null ? "" : " AND j.id=?") + " ORDER BY j.created_at, j.id LIMIT 1", authorizedJobId == null ? new Object[] { tenantId, deviceId } : new Object[] { tenantId, deviceId, authorizedJobId });
return candidates.isEmpty() ? null : candidates.getFirst();
}
/** A job that names a media profile needs an agent that validates it. */
private static boolean mediaSupported(JSONObject job, JSONObject control)
{
return !new JSONObject(job.getString("options_json")).has("mediaProfile")
|| new JSONArray(control.optString("control_capabilities", "[]")).toList().contains("printer-media-v1");
}
/***************************************************************************
** Fetch content only for this device's assigned job.
@@ -2664,8 +2300,6 @@ public class PrintingService
update(connection, "DELETE FROM printing_resource_usage WHERE tenant_id=? AND device_id=?", tenantId, row.getString("id"));
devices += update(connection, "DELETE FROM printing_device WHERE tenant_id=? AND id=?", tenantId, row.getString("id"));
}
// A deleted station's retained-byte counter stays until its last stored document is purged; then nothing refers to it.
update(connection, "DELETE FROM printing_resource_usage WHERE tenant_id=? AND device_id<>'' AND retained_bytes=0 AND NOT EXISTS (SELECT 1 FROM printing_device d WHERE d.id=printing_resource_usage.device_id)", tenantId);
if(printers > 0 || devices > 0)
{
LOG.info("Purged removed printing rows that no job references", logPair("tenantId", tenantId), logPair("printers", printers), logPair("devices", devices));
@@ -30,7 +30,6 @@ public final class PrintingTables
public static final String PRINTER = "printing_printer";
public static final String JOB = "printing_job";
public static final String COMMAND = "printing_device_command";
public static final String DELETION = "printing_deletion";
@@ -0,0 +1,337 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.SQLException;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.Callable;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import org.json.JSONArray;
import org.json.JSONObject;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/***************************************************************************
** The claim, the acknowledgement and the connection record on real
** PostgreSQL, through two PrintingService instances with separate
** connections, as two nodes of one application use them. The row lock and
** the guarded updates are what make these safe across nodes, so they are
** proven on the production database and not on H2. Requires Docker.
***************************************************************************/
class PrintingClaimPostgresTest
{
private static final String PASSWORD = "printing-fixture-only";
private static GenericContainer<?> postgres;
private PrintingService first;
private PrintingService second;
private DriverDataSource source;
private String device;
/** An isolated database container is required; never use an application database. */
@BeforeAll
static void startPostgres()
{
postgres = new GenericContainer<>("postgres:17-alpine").withEnv("POSTGRES_PASSWORD", PASSWORD).withExposedPorts(5432)
.waitingFor(Wait.forLogMessage(".*database system is ready to accept connections.*", 2));
postgres.start();
}
/** Stop only the database owned by this fixture. */
@AfterAll
static void stopPostgres()
{
if(postgres != null)
{
postgres.stop();
}
}
/** A fresh schema, two services over it, one enrolled device with an approved queue. */
@BeforeEach
void setUp() throws Exception
{
source = new DriverDataSource("jdbc:postgresql://" + postgres.getHost() + ":" + postgres.getMappedPort(5432) + "/postgres", "postgres", PASSWORD);
try(Connection connection = source.getConnection(); var sql = connection.createStatement())
{
sql.execute("DROP SCHEMA public CASCADE; CREATE SCHEMA public");
}
String tenant = "tenant-" + UUID.randomUUID();
first = new PrintingService(source, tenant);
first.initialize();
second = new PrintingService(new DriverDataSource(source.url(), source.user(), source.password()), tenant);
JSONObject enrolled = first.enrollForUser(first.createEnrollment("default", "PostgreSQL fixture").put("deviceName", "PostgreSQL fixture").put("agentVersion", "test"), "operator");
device = enrolled.getString("deviceId");
first.inventory(device, new JSONArray().put(new JSONObject().put("printerId", "queue").put("name", "Synthetic queue").put("status", "READY").put("formats", new JSONArray().put("ZPL"))));
first.enablePrinter(device, "queue", true);
}
/** Many claims racing through both instances hand out one job, by one transition, and never the next job before the first is received. */
@Test
void racingClaimsThroughTwoInstancesTransitionEachJobOnce() throws Exception
{
try(ExecutorService pool = Executors.newFixedThreadPool(8))
{
for(int round = 0; round < 12; round++)
{
String job = first.submit(request("race-" + round)).getString("jobId");
String next = second.submit(request("race-next-" + round)).getString("jobId");
CountDownLatch start = new CountDownLatch(1);
List<Future<PrintingService.ClaimAttempt>> attempts = new ArrayList<>();
for(int index = 0; index < 8; index++)
{
PrintingService service = index % 2 == 0 ? first : second;
Callable<PrintingService.ClaimAttempt> claim = () ->
{
start.await();
return service.claimAttempt(device, null);
};
attempts.add(pool.submit(claim));
}
start.countDown();
int transitions = 0;
for(Future<PrintingService.ClaimAttempt> attempt : attempts)
{
PrintingService.ClaimAttempt result = attempt.get(30, TimeUnit.SECONDS);
assertEquals(job, result.response().getJSONObject("job").getString("jobId"), "Every racing claim is answered with the same, oldest job");
assertEquals("CLAIMED", result.response().getJSONObject("job").getString("status"));
transitions += result.transitioned() ? 1 : 0;
}
assertEquals(1, transitions, "Exactly one claim moved the job from QUEUED to CLAIMED");
assertEquals("QUEUED", first.getJob(next).getString("status"), "The next job is not handed out while the first is unacknowledged");
// The acknowledgement arrives through the other instance; the job is then never handed out again by either.
second.event(device, job, new JSONObject().put("eventId", "received-" + round).put("status", "RECEIVED"));
PrintingService.ClaimAttempt following = first.claimAttempt(device, null);
assertEquals(next, following.response().getJSONObject("job").getString("jobId"));
assertTrue(following.transitioned());
first.event(device, next, new JSONObject().put("eventId", "received-next-" + round).put("status", "RECEIVED"));
assertTrue(second.claim(device).isNull("job"));
assertFalse(first.claimAttempt(device, null).transitioned());
}
}
}
/** The same event delivered through both instances, even at the same instant, is recorded once; a different event under its id is refused. */
@Test
void acknowledgementThroughEitherInstanceIsRecordedOnce() throws Exception
{
String job = first.submit(request("acknowledge")).getString("jobId");
assertEquals(job, first.claim(device).getJSONObject("job").getString("jobId"));
JSONObject received = new JSONObject().put("eventId", "ack-received").put("status", "RECEIVED");
try(ExecutorService pool = Executors.newFixedThreadPool(6))
{
CountDownLatch start = new CountDownLatch(1);
List<Future<Boolean>> deliveries = new ArrayList<>();
for(int index = 0; index < 6; index++)
{
PrintingService service = index % 2 == 0 ? first : second;
Callable<Boolean> delivery = () ->
{
start.await();
service.event(device, job, new JSONObject(received.toString()));
return true;
};
deliveries.add(pool.submit(delivery));
}
start.countDown();
for(Future<Boolean> delivery : deliveries)
{
assertTrue(delivery.get(30, TimeUnit.SECONDS), "A replayed acknowledgement is accepted, never an error");
}
}
assertEquals(1, count("SELECT COUNT(*) FROM printing_event WHERE job_id=?", job));
assertEquals("RECEIVED", second.getJob(job).getString("status"));
assertEquals(409, assertThrows(PrintingException.class, () -> second.event(device, job, new JSONObject().put("eventId", "ack-received").put("status", "SPOOLED"))).getStatus());
second.event(device, job, new JSONObject().put("eventId", "ack-submitting").put("status", "SUBMITTING"));
first.event(device, job, new JSONObject().put("eventId", "ack-spooled").put("status", "SPOOLED").put("spoolerJobId", "17"));
second.event(device, job, new JSONObject().put("eventId", "ack-spooled").put("status", "SPOOLED").put("spoolerJobId", "17"));
assertEquals(409, assertThrows(PrintingException.class, () -> first.event(device, job, new JSONObject().put("eventId", "ack-late").put("status", "SUBMITTING"))).getStatus(),
"A job cannot move backwards through another instance");
assertEquals("SPOOLED", first.getJob(job).getString("status"));
assertEquals(3, count("SELECT COUNT(*) FROM printing_event WHERE job_id=?", job));
assertTrue(first.claim(device).isNull("job"));
}
/** The connection record: the newest connection takes the device, only the named connection can renew, mark or clear it, and readiness needs all of its parts. */
@Test
void connectionRecordIsGuardedAndReadinessNeedsEveryPart() throws Exception
{
PrintingRealtimeConnections onFirst = first.realtimeConnections();
PrintingRealtimeConnections onSecond = second.realtimeConnections();
long now = System.currentTimeMillis();
assertFalse(onSecond.readyElsewhere(device, null, now), "No connection");
onFirst.acquire(device, "connection-1", "node-a", now + 60000);
assertFalse(onSecond.readyElsewhere(device, null, now), "Connected, nothing applied");
first.applyRealtimeState(device, control(0), first.getRealtimeSettings().getLong("revision"));
assertTrue(onFirst.ready(device, "connection-1", now + 60000));
assertTrue(onSecond.readyElsewhere(device, null, now));
assertFalse(onFirst.readyElsewhere(device, "connection-1", now), "A node does not take its own connection for one elsewhere");
assertFalse(onSecond.readyElsewhere(device, null, now + 60001), "The lease ran out");
assertEquals("node-a", row().getString("realtime_connection_node"));
// Another node takes the device: the first connection can no longer renew, mark or clear the row.
onSecond.acquire(device, "connection-2", "node-b", now + 60000);
assertFalse(onFirst.renew(device, "connection-1", now + 120000));
assertFalse(onFirst.ready(device, "connection-1", now + 120000));
onFirst.release(device, "connection-1");
assertEquals("connection-2", row().getString("realtime_connection_id"));
assertEquals(0, row().getInt("realtime_connection_ready"), "A new connection must apply state again");
assertFalse(onFirst.readyElsewhere(device, null, now));
assertTrue(onSecond.ready(device, "connection-2", now + 60000));
assertTrue(onFirst.readyElsewhere(device, "connection-1", now));
assertTrue(onSecond.renew(device, "connection-2", now + 90000));
assertEquals(now + 90000, row().getLong("realtime_connection_expires_at"));
// Newer settings, or a newer desired state, close the gate on every node until the device applies them.
JSONObject policy = first.getRealtimeSettings();
policy.remove("revision");
first.saveRealtimeSettings(policy.put("maxCopies", 96));
assertFalse(onFirst.readyElsewhere(device, null, now));
second.applyRealtimeState(device, control(0), 1);
assertTrue(onFirst.readyElsewhere(device, null, now));
first.deviceAction(device, action("PAUSE_DISPATCH"), "operator");
assertFalse(onFirst.readyElsewhere(device, null, now));
second.applyRealtimeState(device, control(desiredRevision()), 1);
assertTrue(onFirst.readyElsewhere(device, null, now));
onSecond.release(device, "connection-2");
assertTrue(row().isNull("realtime_connection_id"));
assertTrue(row().isNull("realtime_connection_node"));
assertTrue(row().isNull("realtime_connection_expires_at"));
assertFalse(onFirst.readyElsewhere(device, null, now));
}
/** The heartbeat reconciliation announces exactly what a claim would hand out, and newer state, settings or commands. */
@Test
void pendingWorkMatchesWhatAClaimWouldReturn() throws Exception
{
long settings = first.getRealtimeSettings().getLong("revision");
first.deviceControlPoll(device, control(0));
assertFalse(second.realtimePending(device, 0, settings), "Idle");
String job = first.submit(request("pending")).getString("jobId");
assertTrue(second.realtimePending(device, 0, settings), "A queued job for an approved, exposed printer");
assertEquals(job, second.claim(device).getJSONObject("job").getString("jobId"));
assertTrue(first.realtimePending(device, 0, settings), "A claimed job is replayed until it is received");
first.event(device, job, new JSONObject().put("eventId", "pending-received").put("status", "RECEIVED"));
assertFalse(second.realtimePending(device, 0, settings), "A received job is not announced");
String waiting = first.submit(request("pending-paused")).getString("jobId");
first.deviceAction(device, action("PAUSE_DISPATCH"), "operator");
long paused = desiredRevision();
assertTrue(second.realtimePending(device, 0, settings), "A newer desired state than the connection last sent");
assertFalse(second.realtimePending(device, paused, settings), "Paused: a claim would return nothing, so nothing is announced");
first.deviceAction(device, action("RESUME_DISPATCH"), "operator");
long resumed = desiredRevision();
assertTrue(second.realtimePending(device, resumed, settings));
first.enablePrinter(device, "queue", false);
assertFalse(second.realtimePending(device, resumed, settings), "A job for a printer that is no longer approved is not claimable");
first.enablePrinter(device, "queue", true);
JSONObject policy = first.getRealtimeSettings();
policy.remove("revision");
first.saveRealtimeSettings(policy.put("maxCopies", 95));
first.cancel(device, waiting);
assertTrue(second.realtimePending(device, resumed, settings), "Newer settings than the connection last sent");
assertFalse(second.realtimePending(device, resumed, settings + 1));
first.deviceAction(device, action("REFRESH_INVENTORY"), "operator");
assertTrue(second.realtimePending(device, desiredRevision(), settings + 1), "A pending command");
assertFalse(second.realtimePending(UUID.randomUUID().toString(), 0, 0), "An unknown device has nothing pending");
}
/** Read the device row both instances consult. */
private JSONObject row() throws SQLException
{
try(Connection connection = source.getConnection())
{
return PrintingService.query(connection, "SELECT realtime_connection_id, realtime_connection_node, realtime_connection_ready, realtime_connection_expires_at FROM printing_device WHERE id=?", device).getFirst();
}
}
/** The desired-state revision an administrator's action left on the device. */
private long desiredRevision() throws SQLException
{
try(Connection connection = source.getConnection())
{
return PrintingService.query(connection, "SELECT control_revision FROM printing_device WHERE id=?", device).getFirst().getLong("control_revision");
}
}
/** Count rows for an assertion. */
private long count(String sql, Object... values) throws SQLException
{
try(Connection connection = source.getConnection())
{
return ((Number) PrintingService.query(connection, sql, values).getFirst().toMap().values().iterator().next()).longValue();
}
}
/** The control acknowledgement a connected device sends with its applied state. */
private static JSONObject control(long applied)
{
return new JSONObject().put("controlVersion", 1).put("agentVersion", "test").put("capabilities", new JSONArray().put("REFRESH_INVENTORY").put("CANCEL_SPOOL")).put("lastAppliedRevision", applied);
}
/** Stable synthetic submission identity; no physical printer exists in this fixture. */
private JSONObject request(String key)
{
return new JSONObject().put("deviceId", device).put("printerId", "queue").put("title", "Synthetic PostgreSQL")
.put("format", "ZPL").put("contentBase64", Base64.getEncoder().encodeToString("^XA^XZ".getBytes(StandardCharsets.US_ASCII))).put("idempotencyKey", key);
}
/** Native actions retain their existing temporal replay boundary. */
private static JSONObject action(String name)
{
return new JSONObject().put("action", name).put("requestId", UUID.randomUUID().toString()).put("issuedAt", Instant.now().toString());
}
}
File diff suppressed because it is too large. Load diff
@@ -141,7 +141,7 @@ class PrintingDeviceProcessesTest
{
var fixture = PrintingProcessesTest.fixture();
var process = fixture.instance().getProcess(PrintingDeviceProcesses.Action.REMOVE.process);
assertEquals("Remove print station", process.getLabel());
assertEquals("Remove device", process.getLabel());
assertTrue(process.getStep("confirmControl") instanceof com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData);
assertTrue(PrintingDeviceProcesses.Action.REMOVE.preview.contains("history is kept"));
String job = submit(fixture.service(), fixture.device());
@@ -120,7 +120,7 @@ class PrintingExtractTest
checked++;
}
}
assertEquals(18, checked);
assertEquals(16, checked);
}
@@ -191,7 +191,7 @@ class PrintingHomeTest
try
{
String header = JsonUtils.toJson(render(instance, "printingHomeHeader"));
assertTrue(header.contains("No print stations are connected"));
assertTrue(header.contains("No print devices are connected"));
assertTrue(header.contains("/app/printingEnrollDevice/?returnTo=%2Fapp%2Fprinting%2F"));
assertEquals(PrintingTables.DEVICE, instance.getProcess("printingEnrollDevice").getTableName());
assertEquals(0, instance.getProcess("printingEnrollDevice").getMinInputRecords());
@@ -1,364 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.time.Duration;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.temporal.ChronoUnit;
import java.util.Base64;
import java.util.List;
import java.util.Locale;
import java.util.UUID;
import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.DateBucket;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.DateBucketUnit;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.NowWithOffset;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QUser;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import org.json.JSONArray;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Print jobs carry their creation instant as a real UTC timestamp that QQQ
** filters, sorts and date-buckets, beside the ISO-8601 text the queue SQL uses.
*******************************************************************************/
class PrintingJobCreatedTimeTest
{
/***************************************************************************
** A submitted job stores one instant in both columns, the timestamp in UTC.
***************************************************************************/
@Test
void submittedJobStoresTheCreationInstantAsAUtcTimestamp() throws Exception
{
DriverDataSource source = source();
PrintingService service = new PrintingService(source, "tenant-a");
service.initialize();
String device = enroll(service);
Instant before = Instant.now();
String job = submit(service, device, "recent");
Instant after = Instant.now();
Instant text = Instant.parse(text(source, job));
Instant timestamp = timestamp(source, job).toInstant(ZoneOffset.UTC);
assertFalse(text.isBefore(before) || text.isAfter(after), text + " is outside " + before + " to " + after);
assertTrue(Duration.between(text, timestamp).abs().compareTo(Duration.ofNanos(1000)) <= 0, text + " and " + timestamp + " are different instants");
}
/***************************************************************************
** The job table exposes a DATE_TIME field in the default columns; the text stays available.
***************************************************************************/
@Test
void metadataExposesCreatedAsADateTimeField() throws Exception
{
QInstance instance = instance(source());
QFieldMetaData created = instance.getTable(PrintingTables.JOB).getField("created_time");
assertEquals(QFieldType.DATE_TIME, created.getType());
assertEquals("Created", created.getLabel());
assertEquals(QFieldType.STRING, instance.getTable(PrintingTables.JOB).getField("created_at").getType());
var columns = (PrintingQBitProducer.DefaultColumns) instance.getTable(PrintingTables.JOB).getSupplementalMetaData("nextUi");
assertTrue(columns.getDefaultColumns().contains("created_time"));
assertFalse(columns.getDefaultColumns().contains("created_at"));
}
/***************************************************************************
** "Created in the last day" selects, sorts and counts by format through QQQ.
***************************************************************************/
@Test
void relativeDateCriteriaSelectRecentJobsOnly() throws Exception
{
DriverDataSource source = source();
QInstance instance = instance(source);
PrintingService service = new PrintingService(source, "tenant-a");
String device = enroll(service);
String recent = submit(service, device, "recent");
String old = submit(service, device, "old");
age(source, old, Instant.now().minus(3, ChronoUnit.DAYS));
QContext.init(instance, viewer());
try
{
QQueryFilter lastDay = new QQueryFilter(new QFilterCriteria("created_time", QCriteriaOperator.GREATER_THAN_OR_EQUALS, NowWithOffset.minus(1, ChronoUnit.DAYS)));
List<QRecord> rows = new QueryAction().execute(new QueryInput().withTableName(PrintingTables.JOB).withFilter(lastDay)).getRecords();
assertEquals(List.of(recent), ids(rows));
Instant created = rows.getFirst().getValueInstant("created_time");
assertTrue(Duration.between(created, Instant.now()).abs().compareTo(Duration.ofMinutes(5)) < 0, created + " is not the current UTC instant");
QQueryFilter earlier = new QQueryFilter(new QFilterCriteria("created_time", QCriteriaOperator.LESS_THAN, NowWithOffset.minus(1, ChronoUnit.DAYS)));
assertEquals(List.of(old), ids(new QueryAction().execute(new QueryInput().withTableName(PrintingTables.JOB).withFilter(earlier)).getRecords()));
QQueryFilter newestFirst = new QQueryFilter().withOrderBy(new QFilterOrderBy("created_time", false));
assertEquals(List.of(recent, old), ids(new QueryAction().execute(new QueryInput().withTableName(PrintingTables.JOB).withFilter(newestFirst)).getRecords()));
Aggregate count = new Aggregate("id", AggregateOperator.COUNT);
GroupBy format = new GroupBy(QFieldType.STRING, "format");
List<AggregateResult> byFormat = new AggregateAction().execute(new AggregateInput(PrintingTables.JOB).withFilter(lastDay).withAggregate(count).withGroupBy(format)).getResults();
assertEquals(1, byFormat.size());
assertEquals("ZPL", byFormat.getFirst().getGroupByValue(format));
assertEquals(1, ValueUtils.getValueAsInteger(byFormat.getFirst().getAggregateValue(count)));
GroupBy day = new GroupBy(QFieldType.DATE_TIME, "created_time").withBucket(new DateBucket(DateBucketUnit.DAY));
List<AggregateResult> byDay = new AggregateAction().execute(new AggregateInput(PrintingTables.JOB).withAggregate(count).withGroupBy(day)).getResults();
assertEquals(2, byDay.size());
for(AggregateResult bucket : byDay)
{
assertNotNull(bucket.getGroupByValue(day));
assertEquals(1, ValueUtils.getValueAsInteger(bucket.getAggregateValue(count)));
}
}
finally
{
QContext.clear();
}
}
/***************************************************************************
** Rows written before the column existed receive their timestamp from the text.
***************************************************************************/
@Test
void initializeBackfillsExistingJobs() throws Exception
{
DriverDataSource source = source();
PrintingService service = new PrintingService(source, "tenant-a");
service.initialize();
String device = enroll(service);
String fractional = submit(service, device, "fractional");
String whole = submit(service, device, "whole");
dropCreatedTime(source);
execute(source, "UPDATE printing_job SET created_at=? WHERE id=?", "2026-01-02T03:04:05.123456Z", fractional);
execute(source, "UPDATE printing_job SET created_at=? WHERE id=?", "2025-12-31T23:59:59Z", whole);
service.initialize();
service.initialize();
assertEquals(LocalDateTime.parse("2026-01-02T03:04:05.123456"), timestamp(source, fractional));
assertEquals(LocalDateTime.parse("2025-12-31T23:59:59"), timestamp(source, whole));
assertNotNull(timestamp(source, submit(service, device, "after")));
}
/***************************************************************************
** A host schema without the column refuses submissions and reads; no job is stored without its timestamp.
***************************************************************************/
@Test
void schemaWithoutTheColumnFailsLoudly() throws Exception
{
DriverDataSource source = source();
QInstance instance = instance(source);
PrintingService service = new PrintingService(source, "tenant-a");
String device = enroll(service);
dropCreatedTime(source);
SQLException refused = assertThrows(SQLException.class, () -> submit(service, device, "refused"));
assertTrue(String.valueOf(refused.getMessage()).toLowerCase(Locale.ROOT).contains("created_time"), String.valueOf(refused.getMessage()));
try(Connection connection = source.getConnection(); PreparedStatement statement = connection.prepareStatement("SELECT COUNT(*) FROM printing_job"); ResultSet rows = statement.executeQuery())
{
assertTrue(rows.next());
assertEquals(0, rows.getInt(1));
}
QContext.init(instance, viewer());
try
{
assertThrows(QException.class, () -> new QueryAction().execute(new QueryInput().withTableName(PrintingTables.JOB)));
}
finally
{
QContext.clear();
}
}
/***************************************************************************
** Independent shared-connection H2 database for each test.
***************************************************************************/
private static DriverDataSource source()
{
return new DriverDataSource("jdbc:h2:mem:" + UUID.randomUUID() + ";DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE", "sa", "");
}
/***************************************************************************
** A real RDBMS instance binds the QBit and service to one isolated database.
***************************************************************************/
private static QInstance instance(DriverDataSource source) throws Exception
{
PrintingService service = new PrintingService(source, "tenant-a");
service.initialize();
QInstance instance = new QInstance();
String backend = "printing-test-" + UUID.randomUUID();
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK));
instance.addBackend(new RDBMSBackendMetaData().withName(backend).withVendor("h2").withHostName("mem").withDatabaseName("test").withUsername("sa").withPassword("").withJdbcUrl(source.url()));
new PrintingQBitProducer().withConfig(new PrintingQBitConfig().withBackendName(backend).withService(service)).produce(instance, "printing");
new QInstanceValidator().validate(instance);
return instance;
}
/***************************************************************************
** An ordinary scoped administrator, never the all-powerful system session.
***************************************************************************/
private static QSession viewer()
{
return new QSession().withUser(new QUser().withIdReference("viewer")).withSecurityKeyValue("printing_tenant", "tenant-a").withPermission("printing_admin");
}
/***************************************************************************
** Enroll one device with an approved synthetic label queue.
***************************************************************************/
private static String enroll(PrintingService service) throws Exception
{
JSONObject code = service.createEnrollment("warehouse", "station");
String device = service.enroll(code.put("deviceName", "station").put("agentVersion", "test")).getString("deviceId");
service.inventory(device, new JSONArray().put(new JSONObject().put("printerId", "labels").put("name", "Labels").put("status", "READY").put("formats", new JSONArray(List.of("ZPL")))));
service.enablePrinter(device, "labels", true);
return device;
}
/***************************************************************************
** Submit one synthetic label through the service; nothing leaves the test database.
***************************************************************************/
private static String submit(PrintingService service, String device, String key) throws Exception
{
return service.submit(new JSONObject().put("deviceId", device).put("printerId", "labels").put("title", key + " job").put("format", "ZPL")
.put("contentBase64", Base64.getEncoder().encodeToString("^XA^XZ".getBytes(StandardCharsets.UTF_8))).put("idempotencyKey", key)).getString("jobId");
}
/***************************************************************************
** Record identifiers in result order.
***************************************************************************/
private static List<String> ids(List<QRecord> rows)
{
return rows.stream().map(row -> row.getValueString("id")).toList();
}
/***************************************************************************
** Move the creation of one job to an earlier instant in both columns.
***************************************************************************/
private static void age(DriverDataSource source, String job, Instant created) throws SQLException
{
execute(source, "UPDATE printing_job SET created_at=?, created_time=? WHERE id=?", created.toString(), LocalDateTime.ofInstant(created, ZoneOffset.UTC), job);
}
/***************************************************************************
** Return the schema to its shape before the column was added.
***************************************************************************/
private static void dropCreatedTime(DriverDataSource source) throws SQLException
{
execute(source, "DROP INDEX printing_job_created_time_idx");
execute(source, "ALTER TABLE printing_job DROP COLUMN created_time");
}
/***************************************************************************
** The stored ISO-8601 text of one job.
***************************************************************************/
private static String text(DriverDataSource source, String job) throws SQLException
{
try(Connection connection = source.getConnection(); PreparedStatement statement = connection.prepareStatement("SELECT created_at FROM printing_job WHERE id=?"))
{
statement.setString(1, job);
try(ResultSet rows = statement.executeQuery())
{
assertTrue(rows.next());
return rows.getString(1);
}
}
}
/***************************************************************************
** The stored timestamp of one job, without any time zone conversion.
***************************************************************************/
private static LocalDateTime timestamp(DriverDataSource source, String job) throws SQLException
{
try(Connection connection = source.getConnection(); PreparedStatement statement = connection.prepareStatement("SELECT created_time FROM printing_job WHERE id=?"))
{
statement.setString(1, job);
try(ResultSet rows = statement.executeQuery())
{
assertTrue(rows.next());
return rows.getObject(1, LocalDateTime.class);
}
}
}
/***************************************************************************
** Run one statement with bound values.
***************************************************************************/
private static void execute(DriverDataSource source, String sql, Object... values) throws SQLException
{
try(Connection connection = source.getConnection())
{
PrintingService.update(connection, sql, values);
}
}
}
@@ -0,0 +1,565 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Consumer;
import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBusInterface;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMetaData;
import com.kingsrook.qqq.middleware.javalin.QApplicationLauncher;
import com.kingsrook.qqq.middleware.javalin.QApplicationLauncherConfig;
import io.javalin.Javalin;
import org.json.JSONArray;
import org.json.JSONObject;
import org.jspecify.annotations.Nullable;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/***************************************************************************
** Two nodes of one application in one JVM: each has its own PrintingService,
** its own notification bus and its own realtime route provider (so its own
** sockets), and they share one database and one QQQ realtime bus, as two
** pods do. Node A is the full QQQ server with the native HTTP routes; node B
** serves only the realtime socket. Sessions are real; jobs are synthetic and
** no printer exists.
***************************************************************************/
class PrintingMultiNodeTest
{
private static final String PREFIX = "/qqq/v1" + PrintingUiApi.PREFIX;
@TempDir
Path directory;
private @Nullable Nodes nodes;
/** Every test starts with a working shared bus. */
@BeforeEach
void resetBus()
{
SwitchableBus.reset();
}
/** Stop both nodes even when an assertion failed. */
@AfterEach
void stop()
{
if(nodes != null)
{
nodes.close();
}
}
/** A job accepted on the node that does not hold the device's socket wakes the device, and its claim and events are accepted on either node. */
@Test
void jobAcceptedOnOneNodeWakesTheDeviceAndIsClaimedThroughTheOther() throws Exception
{
Nodes cluster = start("wake");
try(HttpClient http = HttpClient.newHttpClient())
{
String cookie = login(http, cluster.baseA);
assertEquals(409, send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster).statusCode(), "No node holds a ready connection yet");
try(PrintingSocketFixture socket = PrintingSocketFixture.connect(http, cluster.baseB, cookie, cluster.device, cluster.proof))
{
socket.take("JOB_AVAILABLE");
socket.received.clear();
JSONObject row = cluster.deviceRow();
assertEquals(1, row.getInt("realtime_connection_ready"));
assertFalse(row.isNull("realtime_connection_id"));
assertFalse(row.getString("realtime_connection_node").isBlank());
// Accepted by node A, which holds no socket for this device: the wakeup crosses the shared bus to node B.
String job = cluster.serviceA.submit(request(cluster.device, "accepted-on-a")).getString("jobId");
socket.take("JOB_AVAILABLE");
assertTrue(SwitchableBus.PUBLISHED.get() > 0, "The wakeup was handed to the application's realtime bus");
// The claim reaches node A over HTTP. Its gate passes on the durable connection record, not on a socket of its own.
JSONObject claimed = json(send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster), 200).getJSONObject("job");
assertEquals(job, claimed.getString("jobId"));
assertEquals("CLAIMED", cluster.serviceB.getJob(job).getString("status"));
assertEquals(job, json(send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster), 200).getJSONObject("job").getString("jobId"),
"An unacknowledged claim replays the same assignment on any node");
// Acknowledged over HTTP on node A, then the identical event again over the socket on node B: one event, no error.
JSONObject received = new JSONObject().put("eventId", "multi-node-received").put("status", "RECEIVED");
json(send(http, cluster.baseA, "/device/jobs/" + job + "/events", "POST", received, cookie, cluster), 200);
socket.send("JOB_EVENT", "event-multi-node-received", new JSONObject().put("jobId", job).put("event", received));
assertEquals("event-multi-node-received", socket.take("ACK").getJSONObject("payload").getString("acknowledgedMessageId"));
assertEquals("RECEIVED", cluster.serviceA.getJob(job).getString("status"));
assertEquals(1, cluster.count("SELECT COUNT(*) FROM printing_event WHERE job_id=?", job));
assertTrue(json(send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster), 200).isNull("job"), "A received job is never handed out again");
socket.send("JOB_EVENT", "event-multi-node-spooled", new JSONObject().put("jobId", job).put("event", new JSONObject().put("eventId", "multi-node-spooled").put("status", "SPOOLED")));
socket.take("ACK");
json(send(http, cluster.baseA, "/device/jobs/" + job + "/events", "POST", new JSONObject().put("eventId", "multi-node-spooled").put("status", "SPOOLED"), cookie, cluster), 200);
assertEquals("SPOOLED", cluster.serviceA.getJob(job).getString("status"));
assertEquals(2, cluster.count("SELECT COUNT(*) FROM printing_event WHERE job_id=?", job));
}
awaitReleased(cluster);
assertEquals(409, send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster).statusCode(), "A closed connection is not ready on any node");
}
}
/** The gate on a node without the socket follows the device row: handshake, applied revisions and the lease. */
@Test
void claimGateOnAnotherNodeFollowsTheDurableConnectionRecord() throws Exception
{
Nodes cluster = start("gate");
try(HttpClient http = HttpClient.newHttpClient())
{
String cookie = login(http, cluster.baseA);
try(PrintingSocketFixture socket = PrintingSocketFixture.open(http, cluster.baseB, cookie, cluster.device, cluster.proof))
{
await(() -> !cluster.deviceRow().isNull("realtime_connection_id"));
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceA.requireRealtimeReady(cluster.device)).getStatus(), "Connected but nothing applied");
socket.initialize();
socket.take("JOB_AVAILABLE");
cluster.serviceA.requireRealtimeReady(cluster.device);
cluster.serviceB.requireRealtimeReady(cluster.device);
// An administrator on node A changes the settings: no node accepts a claim until the device applies them.
JSONObject policy = cluster.serviceA.getRealtimeSettings();
policy.remove("revision");
cluster.serviceA.saveRealtimeSettings(policy.put("maxCopies", 98));
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceA.requireRealtimeReady(cluster.device)).getStatus());
assertEquals(1, socket.take("SETTINGS").getJSONObject("payload").getLong("revision"));
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceB.requireRealtimeReady(cluster.device)).getStatus());
socket.apply(0, 1);
cluster.serviceA.requireRealtimeReady(cluster.device);
cluster.serviceB.requireRealtimeReady(cluster.device);
// Dispatch paused on node A: the desired state reaches the device on node B, and the gate waits for it to be applied.
cluster.serviceA.deviceAction(cluster.device, action("PAUSE_DISPATCH"), "operator");
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceA.requireRealtimeReady(cluster.device)).getStatus());
JSONObject desired = socket.take("DESIRED_STATE").getJSONObject("payload");
assertTrue(desired.getBoolean("dispatchPaused"));
socket.apply(desired.getLong("revision"), 1);
cluster.serviceA.requireRealtimeReady(cluster.device);
// A node that stopped without clearing its row is forgotten when the lease runs out; the socket's own node still answers for itself.
cluster.update("UPDATE printing_device SET realtime_connection_expires_at=? WHERE id=?", System.currentTimeMillis() - 1, cluster.device);
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceA.requireRealtimeReady(cluster.device)).getStatus());
cluster.serviceB.requireRealtimeReady(cluster.device);
socket.send("PING", "renew-lease", new JSONObject());
socket.take("PONG");
await(() -> cluster.deviceRow().getLong("realtime_connection_expires_at") > System.currentTimeMillis());
cluster.serviceA.requireRealtimeReady(cluster.device);
}
awaitReleased(cluster);
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceA.requireRealtimeReady(cluster.device)).getStatus());
assertEquals(409, assertThrows(PrintingException.class, () -> cluster.serviceB.requireRealtimeReady(cluster.device)).getStatus());
}
}
/** While the shared bus loses every message, the device's next heartbeat finds the work; an idle heartbeat stays silent. */
@Test
void workCommittedWhileTheBusIsDownIsFoundOnTheNextHeartbeat() throws Exception
{
Nodes cluster = start("bus-down");
try(HttpClient http = HttpClient.newHttpClient())
{
String cookie = login(http, cluster.baseA);
try(PrintingSocketFixture socket = PrintingSocketFixture.connect(http, cluster.baseB, cookie, cluster.device, cluster.proof))
{
socket.take("JOB_AVAILABLE");
socket.send("PING", "idle", new JSONObject());
socket.take("PONG");
assertTrue(drain(socket).isEmpty(), "A heartbeat with nothing pending sends nothing but its PONG");
for(SwitchableBus.Mode mode : List.of(SwitchableBus.Mode.DROP, SwitchableBus.Mode.THROW))
{
SwitchableBus.mode = mode;
long failures = cluster.serviceA.getRealtimeBus().getForwardFailures();
String job = cluster.serviceA.submit(request(cluster.device, "bus-" + mode)).getString("jobId");
assertTrue(drain(socket).isEmpty(), "The wakeup was lost with the bus (" + mode + ")");
assertEquals("QUEUED", cluster.serviceB.getJob(job).getString("status"));
if(mode == SwitchableBus.Mode.THROW)
{
await(() -> cluster.serviceA.getRealtimeBus().getForwardFailures() > failures);
}
socket.send("PING", "heartbeat-" + mode, new JSONObject());
socket.take("PONG");
socket.take("JOB_AVAILABLE");
assertEquals(job, json(send(http, cluster.baseA, "/device/jobs/claim", "POST", new JSONObject(), cookie, cluster), 200).getJSONObject("job").getString("jobId"));
json(send(http, cluster.baseA, "/device/jobs/" + job + "/events", "POST", new JSONObject().put("eventId", "received-" + mode).put("status", "RECEIVED"), cookie, cluster), 200);
socket.received.clear();
socket.send("PING", "after-" + mode, new JSONObject());
socket.take("PONG");
assertTrue(drain(socket).isEmpty(), "A received job is not announced again");
}
// A settings change committed on node A while the bus is down also arrives with the heartbeat.
SwitchableBus.mode = SwitchableBus.Mode.DROP;
JSONObject policy = cluster.serviceA.getRealtimeSettings();
policy.remove("revision");
cluster.serviceA.saveRealtimeSettings(policy.put("maxCopies", 97));
assertTrue(drain(socket).isEmpty());
socket.send("PING", "settings-heartbeat", new JSONObject());
socket.take("PONG");
assertEquals(1, socket.take("SETTINGS").getJSONObject("payload").getLong("revision"));
socket.apply(0, 1);
socket.take("JOB_AVAILABLE");
// When the transport reports a gap after it recovers, every connection reconciles at once, without waiting for a heartbeat.
String late = cluster.serviceA.submit(request(cluster.device, "before-recovery")).getString("jobId");
assertTrue(drain(socket).isEmpty());
SwitchableBus.mode = SwitchableBus.Mode.UP;
SwitchableBus.announceGap();
socket.take("JOB_AVAILABLE");
assertEquals(late, cluster.serviceB.claim(cluster.device).getJSONObject("job").getString("jobId"));
}
}
}
/** A device has one connection in the whole application: connecting through another node closes the first, with or without the bus. */
@Test
void aConnectionThroughAnotherNodeReplacesTheFirst() throws Exception
{
Nodes cluster = start("replace");
try(HttpClient http = HttpClient.newHttpClient())
{
String cookie = login(http, cluster.baseA);
try(PrintingSocketFixture onA = PrintingSocketFixture.connect(http, cluster.baseA, cookie, cluster.device, cluster.proof))
{
onA.take("JOB_AVAILABLE");
String first = cluster.deviceRow().getString("realtime_connection_id");
try(PrintingSocketFixture onB = PrintingSocketFixture.connect(http, cluster.baseB, cookie, cluster.device, cluster.proof))
{
assertEquals(1008, onA.closed.get(10, TimeUnit.SECONDS), "The bus told node A its connection was replaced");
onB.take("JOB_AVAILABLE");
String second = cluster.deviceRow().getString("realtime_connection_id");
assertFalse(first.equals(second));
cluster.serviceA.requireRealtimeReady(cluster.device);
await(() -> second.equals(cluster.deviceRow().optString("realtime_connection_id")));
// With the bus down node B is not told. Its connection learns on its next frame that the row names another connection.
SwitchableBus.mode = SwitchableBus.Mode.DROP;
try(PrintingSocketFixture backOnA = PrintingSocketFixture.connect(http, cluster.baseA, cookie, cluster.device, cluster.proof))
{
backOnA.take("JOB_AVAILABLE");
assertFalse(onB.closed.isDone(), "Node B has not been told");
onB.send("PING", "stale", new JSONObject());
assertEquals(1008, onB.closed.get(10, TimeUnit.SECONDS));
JSONObject row = cluster.deviceRow();
assertFalse(second.equals(row.getString("realtime_connection_id")), "The stale connection's close did not clear its replacement's row");
assertEquals(1, row.getInt("realtime_connection_ready"));
backOnA.send("PING", "current", new JSONObject());
backOnA.take("PONG");
}
}
}
}
}
/** A database failure during the upgrade is retryable; only a refused session or a denial asks for a person. */
@Test
void upgradeFailuresFromTheDatabaseAreRetryable()
{
assertEquals(503, PrintingRealtimeRoutes.upgradeStatus(new SQLException("connection refused")));
assertEquals(503, PrintingRealtimeRoutes.upgradeStatus(new QException("lookup failed", new SQLException("pool exhausted"))));
assertEquals(503, PrintingRealtimeRoutes.upgradeStatus(new QAuthenticationException("session lookup failed", new SQLException("failover"))));
assertEquals(401, PrintingRealtimeRoutes.upgradeStatus(new QAuthenticationException("Session not found")));
assertEquals(403, PrintingRealtimeRoutes.upgradeStatus(new QPermissionDeniedException("denied")));
assertEquals(403, PrintingRealtimeRoutes.upgradeStatus(new PrintingException(404, "Device not found")));
assertEquals(403, PrintingRealtimeRoutes.upgradeStatus(new IllegalArgumentException("bad origin")));
}
/** Start node A (QQQ server) and node B (socket only) on one database and one application bus, with one enrolled device and an approved queue. */
private Nodes start(String name) throws Exception
{
String jdbc = "jdbc:h2:file:" + directory.resolve(name).toString().replace('\\', '/');
PrintingLocalHost host = new SharedBusHost(new PrintingLocalSettings(jdbc, "sa", "", "demo", 0, "synthetic-management-token-long-enough", "operator", "synthetic-realtime-password-1234"));
AtomicReference<Javalin> app = new AtomicReference<>();
QApplicationLauncher launcher = QApplicationLauncher.run(host, new QApplicationLauncherConfig().withRegisterShutdownHook(false).withServerCustomizer(server ->
{
host.configure(server, false);
server.withJavalinConfigurationCustomizer(app::set);
}));
PrintingService serviceA = Objects.requireNonNull(launcher.getQInstance().getQBits().values().stream().map(value -> value.getConfig()).filter(PrintingQBitConfig.class::isInstance)
.map(PrintingQBitConfig.class::cast).findFirst().orElseThrow().getService(), "The printing service was not configured");
DriverDataSource source = new DriverDataSource(jdbc, "sa", "");
PrintingService serviceB = new PrintingService(source, "demo");
PrintingRealtimeRoutes routesB = new PrintingRealtimeRoutes(serviceB);
routesB.setQInstance(launcher.getQInstance());
Javalin other = Javalin.create(config ->
{
config.jetty.host = "127.0.0.1";
routesB.acceptJavalinConfig(config);
}).start(0);
JSONObject enrolled = serviceA.enrollForUser(serviceA.createEnrollment("default", "Multi-node fixture").put("deviceName", "Multi-node fixture").put("agentVersion", "test"), "operator");
String device = enrolled.getString("deviceId");
serviceA.inventory(device, new JSONArray().put(new JSONObject().put("printerId", "queue").put("name", "Synthetic queue").put("status", "READY").put("formats", new JSONArray().put("ZPL"))));
serviceA.enablePrinter(device, "queue", true);
nodes = new Nodes(launcher, other, serviceA, serviceB, source, "http://127.0.0.1:" + Objects.requireNonNull(app.get(), "Test HTTP server was not started").port(),
"http://127.0.0.1:" + other.port(), device, enrolled.getString("deviceToken"));
return nodes;
}
/** The two nodes, their services and the one device they share. */
private record Nodes(QApplicationLauncher launcher, Javalin other, PrintingService serviceA, PrintingService serviceB, DriverDataSource source, String baseA, String baseB, String device, String proof)
{
/** Read the device row both nodes consult. */
JSONObject deviceRow() throws SQLException
{
try(Connection connection = source.getConnection())
{
return PrintingService.query(connection, "SELECT realtime_connection_id, realtime_connection_node, realtime_connection_ready, realtime_connection_expires_at FROM printing_device WHERE id=?", device).getFirst();
}
}
/** Count rows for an assertion. */
long count(String sql, Object... values) throws SQLException
{
try(Connection connection = source.getConnection())
{
return ((Number) PrintingService.query(connection, sql, values).getFirst().toMap().values().iterator().next()).longValue();
}
}
/** Simulate what a stopped node leaves behind. */
void update(String sql, Object... values) throws SQLException
{
try(Connection connection = source.getConnection())
{
PrintingService.update(connection, sql, values);
}
}
/** Node B first, then the QQQ server. */
void close()
{
other.stop();
launcher.stop();
}
}
/** The same local host, with the application's realtime bus replaced by one the test can switch off. */
private static final class SharedBusHost extends PrintingLocalHost
{
/** Same settings as the ordinary local host. */
SharedBusHost(PrintingLocalSettings settings) throws Exception
{
super(settings);
}
@Override
public QInstance defineQInstance() throws QException
{
QInstance instance = super.defineQInstance();
instance.withSupplementalMetaData(RealtimeMetaData.of(instance).withBus(new QCodeReference(SwitchableBus.class)));
return instance;
}
}
/***************************************************************************
** Stands in for the shared transport (Redis in production): it delivers a
** published message to every subscriber, the publisher included, and can
** lose messages silently or fail, as an unreachable transport does.
***************************************************************************/
public static final class SwitchableBus implements RealtimeBusInterface
{
/** What the transport does with a published message. */
enum Mode
{
UP, DROP, THROW
}
static final AtomicInteger PUBLISHED = new AtomicInteger();
private static final List<Consumer<RealtimeMessage>> LISTENERS = new CopyOnWriteArrayList<>();
static volatile Mode mode = Mode.UP;
/** Forget the previous test's subscribers and outage. */
static void reset()
{
LISTENERS.clear();
PUBLISHED.set(0);
mode = Mode.UP;
}
/** What a real transport emits locally after it reconnects or overflows. */
static void announceGap()
{
LISTENERS.forEach(listener -> listener.accept(new RealtimeMessage().withChannel("instance").withEvent("resync")));
}
@Override
public void publish(RealtimeMessage message)
{
if(mode == Mode.THROW)
{
throw new IllegalStateException("The shared bus is unreachable");
}
if(mode == Mode.UP)
{
PUBLISHED.incrementAndGet();
LISTENERS.forEach(listener -> listener.accept(message));
}
}
@Override
public AutoCloseable subscribe(Consumer<RealtimeMessage> listener)
{
LISTENERS.add(listener);
return () -> LISTENERS.remove(listener);
}
}
/** Collect whatever the server pushes in a short bounded window; this is an absence check, not a polling loop. */
private static List<JSONObject> drain(PrintingSocketFixture socket) throws Exception
{
List<JSONObject> frames = new ArrayList<>();
JSONObject next = socket.received.poll(600, TimeUnit.MILLISECONDS);
while(next != null)
{
frames.add(next);
next = socket.received.poll(200, TimeUnit.MILLISECONDS);
}
return frames;
}
/** A condition another thread completes shortly. */
private static void await(Condition condition) throws Exception
{
long end = System.nanoTime() + TimeUnit.SECONDS.toNanos(10);
while(!condition.met())
{
assertTrue(System.nanoTime() < end, "Condition was not met in time");
Thread.sleep(20);
}
}
/** The server clears the device row when it sees the socket close. */
private static void awaitReleased(Nodes cluster) throws Exception
{
await(() -> cluster.deviceRow().isNull("realtime_connection_id"));
assertNull(cluster.deviceRow().opt("realtime_connection_node"));
assertEquals(0, cluster.deviceRow().getInt("realtime_connection_ready"));
}
/** A check that may read the database. */
@FunctionalInterface
private interface Condition
{
/** Whether the awaited state has been reached. */
boolean met() throws Exception;
}
/** A real sign-in supplies only its own temporary test session, valid on both nodes because sessions are rows. */
private static String login(HttpClient client, String base) throws Exception
{
HttpRequest request = HttpRequest.newBuilder(URI.create(base + "/qqq/v1/manageSession")).header("Origin", base)
.header("Authorization", "Basic " + Base64.getEncoder().encodeToString("operator:synthetic-realtime-password-1234".getBytes(StandardCharsets.UTF_8)))
.header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString("{}")).build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
assertEquals(200, response.statusCode());
return response.headers().allValues("set-cookie").stream().filter(value -> value.startsWith("sessionUUID=")).findFirst().orElseThrow().split(";", 2)[0];
}
/** A native device request as the Windows client sends it: session cookie plus device proof. */
private static HttpResponse<String> send(HttpClient client, String base, String path, String method, @Nullable JSONObject body, String cookie, Nodes cluster) throws Exception
{
HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(base + PREFIX + path)).header("Origin", base).header("X-QQQ-Client", "api/qqq-print-agent")
.header("Cookie", cookie).header(PrintingAgentAuthorization.DEVICE_HEADER, cluster.device).header(PrintingAgentAuthorization.TOKEN_HEADER, cluster.proof);
request.method(method, body == null ? HttpRequest.BodyPublishers.noBody() : HttpRequest.BodyPublishers.ofString(body.toString()));
if(body != null)
{
request.header("Content-Type", "application/json");
}
return client.send(request.build(), HttpResponse.BodyHandlers.ofString());
}
/** Require a status and parse the body. */
private static JSONObject json(HttpResponse<String> response, int status)
{
assertEquals(status, response.statusCode(), response.body());
assertNotNull(response.body());
return new JSONObject(response.body());
}
/** Stable synthetic submission identity; no physical printer exists in this fixture. */
private static JSONObject request(String device, String key)
{
return new JSONObject().put("deviceId", device).put("printerId", "queue").put("title", "Synthetic multi-node")
.put("format", "ZPL").put("contentBase64", Base64.getEncoder().encodeToString("^XA^XZ".getBytes(StandardCharsets.US_ASCII))).put("idempotencyKey", key);
}
/** Native actions retain their existing temporal replay boundary. */
private static JSONObject action(String name)
{
return new JSONObject().put("action", name).put("requestId", java.util.UUID.randomUUID().toString()).put("issuedAt", java.time.Instant.now().toString());
}
}
@@ -0,0 +1,211 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.printing;
import java.io.Serializable;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Consumer;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBusInterface;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** The notification bridge joined to an application's QQQ realtime bus: delivery, echo suppression, foreign traffic and a slow transport. */
class PrintingRealtimeBusTest
{
/** The shared fixture leaves a native context on the thread. */
@AfterEach
void clearContext()
{
QContext.clear();
}
/** Two buses joined to one application each hear a wakeup once, whichever of them published it. */
@Test
void wakeupsCrossJoinedBusesExactlyOnce() throws Exception
{
QInstance instance = PrintingProcessesTest.fixture().instance();
PrintingRealtimeBus first = new PrintingRealtimeBus();
PrintingRealtimeBus second = new PrintingRealtimeBus();
List<String> firstHeard = new CopyOnWriteArrayList<>();
List<String> secondHeard = new CopyOnWriteArrayList<>();
AtomicInteger gaps = new AtomicInteger();
first.subscribe((tenant, device) -> firstHeard.add(tenant + "/" + device));
second.subscribe((tenant, device) -> secondHeard.add(tenant + "/" + device));
second.subscribeGaps(gaps::incrementAndGet);
first.publish("tenant", "before-join");
assertEquals(List.of("tenant/before-join"), firstHeard, "An unjoined bus still serves its own JVM");
AutoCloseable joined = first.join(instance);
AutoCloseable joinedAgain = first.join(instance);
AutoCloseable other = second.join(instance);
try
{
first.publish("tenant", "device");
first.publish("tenant", null);
await(() -> secondHeard.size() == 2);
assertEquals(List.of("tenant/device", "tenant/null"), secondHeard);
second.publish("tenant", "reply");
await(() -> firstHeard.size() == 4);
// The transport echoes every message to its publisher; the publisher already delivered it locally and must not hear it twice.
Thread.sleep(200);
assertEquals(List.of("tenant/before-join", "tenant/device", "tenant/null", "tenant/reply"), firstHeard);
assertEquals(List.of("tenant/device", "tenant/null", "tenant/reply"), secondHeard);
// Other traffic on the application's bus, and malformed printing messages, are not wakeups.
RealtimeBusInterface transport = RealtimeBus.get(instance);
transport.publish(new RealtimeMessage().withChannel("table:printing_job").withEvent("record.updated"));
transport.publish(new RealtimeMessage().withChannel(PrintingRealtimeBus.SHARED_CHANNEL).withEvent("other"));
transport.publish(message(Map.of("tenant", "tenant")));
transport.publish(message(Map.of("origin", "elsewhere", "tenant", " ")));
transport.publish(message(Map.of("origin", "elsewhere")));
assertEquals(4, firstHeard.size());
assertEquals(3, secondHeard.size());
transport.publish(message(Map.of("origin", "elsewhere", "tenant", "tenant", "device", "external")));
assertEquals("tenant/external", firstHeard.getLast());
assertEquals("tenant/external", secondHeard.getLast());
// The transport's own lost-message hint reaches gap listeners, not wakeup listeners.
transport.publish(new RealtimeMessage().withChannel("instance").withEvent("resync"));
assertEquals(1, gaps.get());
assertEquals(5, firstHeard.size());
// Leaving is counted: the link stays while one user remains, and closing a handle twice counts once.
joinedAgain.close();
joinedAgain.close();
first.publish("tenant", "still-joined");
await(() -> secondHeard.contains("tenant/still-joined"));
joined.close();
first.publish("tenant", "after-leaving");
transport.publish(message(Map.of("origin", "elsewhere", "tenant", "tenant", "device", "to-the-joined")));
assertEquals("tenant/after-leaving", firstHeard.getLast(), "A bus that left hears only itself");
assertEquals("tenant/to-the-joined", secondHeard.getLast());
assertEquals(0, first.getForwardFailures());
}
finally
{
other.close();
RealtimeBus.close(instance);
}
}
/** The committing caller never waits for the transport, and wakeups the transport cannot take are counted, not thrown. */
@Test
void aBlockedOrFailingTransportNeverDelaysThePublisher() throws Exception
{
QInstance instance = PrintingProcessesTest.fixture().instance();
instance.withSupplementalMetaData(RealtimeMetaData.of(instance).withBus(new QCodeReference(BlockedBus.class)));
BlockedBus.release = new CountDownLatch(1);
PrintingRealtimeBus bus = new PrintingRealtimeBus();
AtomicInteger heard = new AtomicInteger();
bus.subscribe((tenant, device) -> heard.incrementAndGet());
AutoCloseable joined = bus.join(instance);
try
{
long started = System.nanoTime();
for(int index = 0; index < 1200; index++)
{
bus.publish("tenant", "device-" + index);
}
assertTrue(TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - started) < 5000, "Publishing waited for the transport");
assertEquals(1200, heard.get(), "Every wakeup reached this JVM's listeners");
assertTrue(bus.getForwardFailures() >= 1200 - 1024 - 1, "Wakeups beyond the bounded queue are counted as not forwarded");
BlockedBus.release.countDown();
long counted = bus.getForwardFailures();
await(() -> BlockedBus.FAILED.get() > 0);
await(() -> bus.getForwardFailures() > counted);
}
finally
{
BlockedBus.release.countDown();
joined.close();
RealtimeBus.close(instance);
}
}
/** A transport that blocks until released and then fails every publish. */
public static final class BlockedBus implements RealtimeBusInterface
{
static volatile CountDownLatch release = new CountDownLatch(0);
static final AtomicInteger FAILED = new AtomicInteger();
@Override
public void publish(RealtimeMessage message)
{
try
{
release.await();
}
catch(InterruptedException interrupted)
{
Thread.currentThread().interrupt();
}
FAILED.incrementAndGet();
throw new IllegalStateException("The shared bus is unreachable");
}
@Override
public AutoCloseable subscribe(Consumer<RealtimeMessage> listener)
{
return () ->
{
};
}
}
/** A printing wakeup as another node's bus would publish it. */
private static RealtimeMessage message(Map<String, String> values)
{
Map<String, Serializable> data = new java.util.LinkedHashMap<>(values);
return new RealtimeMessage().withChannel(PrintingRealtimeBus.SHARED_CHANNEL).withEvent(PrintingRealtimeBus.SHARED_EVENT).withData(data);
}
/** A condition the forwarding thread completes shortly. */
private static void await(java.util.function.BooleanSupplier condition) throws InterruptedException
{
long end = System.nanoTime() + TimeUnit.SECONDS.toNanos(10);
while(!condition.getAsBoolean())
{
assertTrue(System.nanoTime() < end, "Condition was not met in time");
Thread.sleep(10);
}
}
}