qbit-scripting/docs/PRODUCTION-CONNECTION.md

Connecting a QQQ application to external scripting services

Current work is local development. Use the existing local Docker/native tool workflow for fixed trusted development fixtures. This document describes the existing application connection APIs and the operator work needed when services are deployed in Kubernetes; it does not provision a cluster, require a local VM, or qualify a production deployment. Successful local examples do not satisfy the original requirement to execute fully untrusted author code safely.

The application can connect to the execution supervisor and cold compiler over persistent mutual TLS. It does not need Kubernetes API access, kubeconfig, service-account administration credentials, a Kubernetes SDK, or per-record pod creation. Operator deployment and secret delivery happen outside record execution. The current renderer bridge has a different interface and is described separately below.

Existing application API

PersistentWorkerProvider.Configuration is a Java record supplied by host composition. There is no generic production application JSON/YAML/environment loader in this QBit. The Java example uses actual constructors/setters; the host supplies its own existing secret/configuration integration. It is documentation, not a new runtime entrypoint.

Constructor field	Existing meaning and enforced bounds
sslContext	Host-owned client identity and server trust. Must support TLS 1.3 and present a client certificate. No trust-all fallback.
executionEndpoint	Required Endpoint(host, port). Host is a bounded DNS name/IPv4 literal; port 1–65535. It is not a URL, Kubernetes selector, Unix socket or credential. IPv6 literals are not accepted by the current host grammar.
compileEndpoint	Separate fixed cold endpoint, same Endpoint type. null makes compilation unavailable; it does not enable local compilation or an execution-endpoint fallback.
namespace	Exact application namespace checked against the authenticated client leaf-certificate fingerprint on both services. This is an application isolation label, not automatically the Kubernetes namespace.
profile, toolchain	Exact peer-agreed identifiers. Current compiler requires typescript-6.0.3-node-24; the shipped compiler-image verifier additionally pins Node 24.22.2 and TypeScript 6.0.3. A profile string alone does not configure or prove resource isolation.
qualificationId, requiredQualificationId	Required matching evidence IDs. Host ScriptingRuntime.withRequiredQualificationId must match too. These are operator trust inputs, not secrets and not automatic certification. Never reuse a trusted-lab identity as production evidence.
deadline	Whole transport exchange budget, positive and at most 120 seconds. Separate from the guest CPU/wall/resource caps.
executionConnections	1–64 application connections/admitted executions. Capacity is finite; misses or busy slots do not create a hot queue.
coldOperations	1–8 concurrent cold preparation/compilation operations.
entriesPerConnection, codeBytesPerConnection	1–256 entries, 1–64 MiB of code per connection. Must fit the server's independently configured limits.

Namespace/profile/toolchain/evidence identifiers use [A-Za-z0-9._:-]{1,128}. The optional second PersistentWorkerProvider(configuration, preparationCopies) constructor requires copies in 1..executionConnections; the compatible single-argument constructor uses one copy. Each copy occupies a distinct authenticated session/cache epoch. Connection capacity, aggregate JVM/native memory and all application replicas must be sized together; the example numbers are illustrative, not a throughput or latency claim.

SupervisorConnection enables TLS 1.3 and HTTPS-style hostname verification even though this is a framed TCP protocol, not HTTP. The server certificate SAN must match the exact configured service DNS name. Do not terminate TLS at an HTTP proxy that removes the client's identity: the supervisor/compiler require the actual client certificate and map its lowercase SHA-256 DER leaf fingerprint to a single application namespace. Multiple fingerprints can be declared during an operator-controlled certificate transition; there is no wildcard namespace rule or automatic certificate reload API.

Service-side configuration already implemented

Both entrypoints take one absolute path to an operator JSON file of at most 64 KiB. Unknown/missing keys fail closed. The example files contain deliberately invalid <...> placeholders for hashes and evidence; there is no interpolation engine. Replace them with reviewed deployment inputs before use, without changing the schema.

• SupervisorMain: exact JSON example.
• CompilerMain: exact JSON example.

Both accept bind, port, keyStore, trustStore, certificateNamespaces, profile, toolchain, evidence, requiredEvidence. bind is the server listen address; a literal avoids cold-start DNS dependence. keyStore and trustStore are absolute paths to PKCS12 files, each at most 1 MiB. certificateNamespaces contains 1–256 exact lowercase 64-hex leaf fingerprints mapped to namespace identifiers. Both enforce TLS 1.3 client authentication.

The existing entrypoints read passwords only from QQQ_SUPERVISOR_KEYSTORE_PASSWORD and QQQ_SUPERVISOR_TRUSTSTORE_PASSWORD, including the compiler. There are no corresponding JSON password fields and no password-file option. For Kubernetes, the operator can project PKCS12 files as read-only secret volumes and supply these exact environment variables through secret references such as env.valueFrom.secretKeyRef. Those references belong to the host deployment, not the QBit config parser or guest context. Keep the runtime/compiler tree separate from secret mounts. The app-side SSLContext is built by host code from its own mounted client keys/trust; these service-entrypoint env variables are not automatically read by PersistentWorkerProvider.

Additional supervisor keys:

Key	Existing requirement
runtimePath, runtimeSha256	Absolute fixed Wasm runtime path and exact lowercase SHA-256. Not author-provided bytecode.
nativePath, nativeSha256	Absolute fixed native library path and exact SHA-256. Pin the matching reviewed binary/OS/architecture closure.
slots, connections	1–64 engine slots; connections must be at least slots and at most 128.
coldOperations, entries, codeBytes	1–8, 1–256, 1–64 MiB respectively.
memoryBytes	64–256 MiB. Invocation memory must exactly match this runtime profile; smaller requested limits are not silently rounded.
idleMillis, fuel	100–60,000 ms; positive fuel at most 1,000,000,000. These are not substitutes for outer process/container enforcement.

Additional compiler keys:

Key	Existing requirement
nodePath, nodeSha256	Absolute fixed Node executable and exact SHA-256; no PATH lookup or author-selected command.
compilerRoot, manifestSha256	Absolute fixed image tree and exact compiler-manifest.json hash. The manifest uses qqq-typescript-image/1, Node 24.22.2, TypeScript 6.0.3, and the explicitly allowed compiler/TypeScript file closure. Extra files/symlinks fail verification.
heapMiB	64–512 MiB Node heap cap.
slots, connections	1–8 child-compile slots; 1–16 compiler connections.
cpuMillis, wallMillis	1–30,000 ms CPU; 100–60,000 ms wall. Source compile remains on this cold lane.

Mount pinned runtime/compiler inputs read-only: startup hashes cannot stop privileged post-check file replacement. The host/operator selects immutable image digests and the corresponding source/binary evidence; the connection API does not query a registry or deploy an image. Existing frame/session/sequence/request checks and RETIRED cleanup acknowledgment remain mandatory. See supervisor contract for exact cancellation, native cleanup and CPU accounting limits.

Stable signing, persistence and current authority

Configure ScriptingRuntime.withCoordinator(new BuildCoordinator(coordinatorId, key, toolchainId)), withWorker, withRequiredQualificationId, and withTransactionPolicy(new RdbmsScriptingTransactionPolicy()). The coordinator key must contain at least 32 bytes and must be held only in application secret-backed configuration; it is never sent to workers. Preserve the correct coordinator ID/key for existing immutable HMAC-signed artifacts across application restarts. There is no built-in key-ring/rotation migration or Kubernetes Secret loader: changing to an unrelated key makes existing verification fail, rather than silently resigning history.

The host's existing native backend is RDBMSBackendMetaData (com.kingsrook.qqq.backend.module.rdbms.model.metadata), configured with withName, withVendor, withJdbcUrl, withUsername and withPassword; ScriptingQBitConfig.withBackendName selects that registered backend. Resolve database credentials through host secrets, not a new scripting property. For MySQL, configure finite driver connectTimeout and socketTimeout, connectionTimeZone=UTC and forceConnectionTimeZoneToSession=true in the native JDBC/pool configuration; preserve authenticated TLS rather than copying the test fixtures' disabled TLS. Session lock timeout must be 1–5 seconds and UTC, with a writable non-autocommit transaction. MySQL qualification also requires InnoDB, exact binary text comparison/keys, innodb_flush_log_at_trx_commit=1, recovery-capable innodb_doublewrite, and sync_binlog=1 when binary logging is enabled. File H2 uses an actual jdbc:h2:file: backend with live WRITE_DELAY=0; memory H2 is test-only. These are requirements to configure/verify separately, not settings this QBit changes for the host.

The optional RDBMS adapter is mandatory for its supported durable deployments; absence refuses persistence/history, not just publication. It requalifies the actual transaction, mapped tables, identity keys and operation-ID capacity. Current support is qualified MySQL 8.4 and file H2 under the documented settings; Aurora/MariaDB/PostgreSQL are not implied compatible. Apply shipped migrations with a separate migration identity; application identity stays DML-only. Configure finite native JDBC connect/socket/query timeouts and bounded lock waits. The adapters do not create schemas or change server settings on the hot path. Native AFTER hooks additionally need the existing core durable-Jobs configuration/migrations, described in DURABLE-AFTER-HOOKS.md.

Keep ordinary invocation/broker authorization tied to the current user, current credential/channel ceilings, immutable publication and declared capabilities. Worker connection identity authorizes a transport namespace; it does not grant a source-record read or bypass tenant policy.

Per-node preparation and readiness

Configure WarmReconciliationConfig on ScriptingRuntime.withWarmReconciliation before producing the QBit. The native producer registers WarmRuntimeService automatically for maintenance-capable providers. It registers the scripting.warm health check with readiness inclusion, 10-second check interval, 5-second check timeout, failure threshold one and no probe-cache reuse.

Field	Existing bound/policy
principal	Declared native service-identity name, 1–100 characters from [A-Za-z0-9_.-]. Fresh ServiceIdentities session per pass/check; never a saved interactive session.
targetTables	Explicit nonempty set, at most 64 existing native table names; no scripting_ target names. No implicit all-table scope.
maxBindings, maxCodeBytes	1–1,000 active scoped bindings; 1–64 MiB total unique code. Overflow refuses readiness, not truncation.
interval, passBudget, maxAge	1–300 seconds; 1–120 seconds; 1–600 seconds. maxAge >= interval + passBudget.

Declare the principal through ServiceIdentitiesMetaData.ofOrWithNew(instance).withPrincipal(...) using QServicePrincipalMetaData. It needs scripting.warm, scripting.view and current authorized script-project/source-metadata visibility under the host's tenant/security-key/credential policy. Those two named grants alone do not override project access. Do not grant record data privileges simply to warm code. The example accepts this explicitly configured principal from the host rather than inventing an all-access identity.

The service owns one bounded coalesced fixed-delay maintenance future per QQQ instance. It verifies current binding/publication/build/HMAC identity before transferring immutable code, then rechecks it. Idle authenticated PING/PONG plus RETIRED confirms the same cache epoch without code, context, token or guest execution. Busy valid invocations are not evicted. Readiness checks all required exact prepared identities/copy counts/profile in one local snapshot, verifies age, then re-reads current pins. Missing principal, target, artifact, freshness, capacity or configuration means DOWN. TCP connection alone is insufficient. Readiness covers active bindings plus explicitly inventoried context publications in the configured scope; it does not enumerate every publication automatically.

User-initiated test/publish/bind already use the cold preparation SPI. An execute/save hook never connects, compiles, uploads or retries on a cache miss: it fails explicitly. Worker restart closes cache sessions; the cold reconciler reconnects and prepares the currently authorized scoped pins. Application restart must construct a fresh runtime/provider; reuse of a stopped instance is unsupported. Steady-state inventory work is linear; first-cold repeated pin validation can cost more, including quadratic metadata work. No p95/p99 or startup-time result follows from these configuration examples.

The host must actually run QQQ's native runtime-service lifecycle and expose its native readiness result through its existing deployment health integration. This module does not supply a new HTTP health endpoint, Kubernetes probe manifest, readiness gate controller or process manager. Cooperative pass cancellation cannot bound an unbounded JDBC/DNS/authentication call; configure those deadlines independently. Readiness should control admission without treating temporary cold recovery as a reason for an endless restart loop.

Optional ScriptingRuntime.withWarmPublicationSource(source) includes required unbound CONTEXT publications in the same cold lifecycle, for example immutable references maintained by an independently installed consumer adapter. The source receives configured target tables and the remaining combined pin limit under the fresh WARM + VIEW service identity. It must return all currently required authorized WarmPublicationSource.Pin(publicationId, artifactId, bundleSha256) values or throw, never silently truncate. The foundation revalidates current project/isolation, exact signed pins and successful exact-build CONTEXT testing. Configure the consuming adapter explicitly; otherwise null preserves hook-only behavior and unbound contexts are not automatically restored after restart. This addition is staged pending its root-owned verification; it does not qualify any deployment.

Renderer connections: local adapter and new remote transport

The original optional Generation Docker renderer bridge remains available. Its RendererConfiguration fields are:

dockerExecutable, image, imageId, qualificationId, requiredQualificationId, ownerId, stateDirectory, renderDeadline, maximumBytes, maximumPages, maximumConcurrent.

That constructor requires an existing absolute Docker executable, existing absolute non-symlink state directory, digest-pinned image (...@sha256:...), exact imageId (sha256:...), matching evidence IDs and bounded owner ID. Deadline is positive and at most30 seconds; output1–100 MiB; pages1–500; concurrency1–16 (compatible constructor defaults to one). It uses disposable Docker/runsc and the existing durable owned-container lifecycle/reaper. Its fields are not remote endpoint settings.

A new optional remote-renderer module now provides the service connection implementation. It adds qqq-remote-renderer-transport plus qbit-document-generation-remote-renderer, without a renderer engine in the application or a foundation dependency on transport. RemoteRendererBridge(RendererLimits, IsolatedRendererBridge.AssetReader, RemoteRendererClient.Configuration) reuses current asset authorization, immutable request/catalog checks and output hash/length/PDF/ZPL policy. Close the adapter during normal application shutdown.

The exact remote application settings are:

Existing API	Fields and contract
DeploymentIdentity	mode, namespace, profile, evidence, requiredEvidence, imageId. Explicit mode is TRUSTED_LAB or QUALIFIED_DEPLOYMENT; matching evidence and immutable sha256: image ID are required. Qualified clients refuse trusted-lab service mode; choosing qualified mode does not establish qualification.
RemoteRendererClient.Configuration	tls (SSLContext), endpoint (URI), address (already-resolved InetSocketAddress), identity, deadlineMillis (100–30000). Endpoint must be HTTPS /render, with no userinfo/query/fragment; address port must match. Certificate hostname verification uses the URI hostname independently of the resolved address. Resolve/refresh service DNS outside requests and recreate the operator configuration when routing changes.
RendererLimits	admissionIdentity, maximumBytes, maximumPages, maximumConcurrent. Keep a stable deployment identity and its immutable JVM concurrency limit across instances. Server slots/deployment enforce separate service/cluster capacity.

RendererServiceMain consumes one absolute operator JSON file; use its actual example schema. Required keys are bind, port, keyStore, trustStore, identity, certificateNamespaces, slots, deadlineMillis, maximumOutput, javaExecutable, classpath, fileHashes, and temporaryRoot. Supply the complete fixed ordered renderer JAR classpath and exact executable/JAR/native-provider hashes; there is no arbitrary author command. Passwords come from QQQ_RENDERER_KEYSTORE_PASSWORD and QQQ_RENDERER_TRUSTSTORE_PASSWORD, not JSON. Use host-owned secret references and immutable inputs as described in that module's README.

The private qqq-remote-renderer-1 transport is one TLS1.3 HTTP/1.1 POST /render per connection, with actual client-certificate fingerprint→namespace authorization, exact identity/exchange headers and service-generated retirement evidence. It has no redirects, automatic retry, compression, chunking, cookies or generic business REST endpoint. Direct TCP forwarding can preserve this contract; an ingress that changes framing or terminates client TLS is not implicitly supported. The application needs neither a Docker socket nor Kubernetes administrative credentials.

Current evidence and limits: parent reports the initial 41-test remote-module gate passed. Two deadline-edge corrections remain pending their verification at this documentation checkpoint. Do not infer final corrected-source/image/end-to-end approval from the baseline. The implementation is not claimed hostile-input or Kubernetes production-qualified. In particular, the service and renderer child can have the same UID/filesystem authority: filtering the child's environment does not prevent an escaped child from reading mounted service secrets. Production requires separately demonstrated credential/filesystem separation. ProcessHandle tracks the fixed child and observed descendants but is not atomic containment of arbitrary hostile forking processes; actual OS/cgroup containment, aggregate memory/CPU/PID/tmp limits and termination behavior require qualification. Image/evidence headers are operator pins, not self-attestation. Fresh per-request children provide no warm-pool or latency guarantee. The existing local Docker path is not silently replaced or weakened.

Continue local development with the existing local Docker/native tools and explicitly scoped trusted fixtures. Document and connect a production deployment only against its reviewed service/secret/containment evidence; no cluster provisioning or production mode was enabled by this documentation update.

Deployment boundary and remaining evidence

Operators provide service routing/DNS, actual TLS identities and rotation, immutable images/native libraries, resource/process isolation, egress restrictions, secret delivery, finite database/auth/DNS deadlines, capacity and graceful shutdown/forced termination policies. Service caches are connection-local; ordinary TCP routing must preserve established sessions. Every application replica must reconcile its own connection/copy capacity after restart. No cluster access or provisioning was performed for this document.

Current local trusted-fixture successes establish their stated source, transport, lifecycle and integration behavior only. Before admitting fully untrusted authors in production, qualify the exact image/native/Wasm/compiler/renderer tuple and deployment containment, resource cancellation/cleanup under load, restart/recovery behavior, authority isolation, durable storage and operational limits. Matching evidence strings do not perform that qualification. The proposed record-save overhead budget remains a target, not an accepted/measured production SLA.

Documentation checked against the actual source APIs on 2026-10-31. These examples contain no credentials or ready-to-use qualification identity. No runtime code, default, image, cluster, shared cache or installed artifact was changed.

Warm capacity integration update (verified source, 2026-11-01)

The optional Generation supplier now requires the declared maintenance principal's explicit document-generation.warm grant in addition to view. Its active/draft/unfinished-work inventory and batch-completion migration are documented in the optional bridge. Cold idle-session retirement and bounded authoring-use leases are described in WARM-LIFECYCLE.md. Keep configured capacity/headroom and finite JDBC/transport deadlines explicit. The source passed full Scripting227 and Generation focused67/Checkstyle (including H2 migration); actual MySQL migration delta qualification remains separate. Before adopting new application code, apply the appropriate additive migration once and retain NULL for existing batches. No backfill to true is authorized by this change. These gates do not establish production containment or deployment qualification.