Compare commits

..
Author SHA1 Message Date
dependabot[bot] 4213d62999 build(deps): Bump the github-actions group with 2 updates
Bumps the github-actions group with 2 updates: [QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml](https://github.com/qrun-io/qqq-orb) and [QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml](https://github.com/qrun-io/qqq-orb).


Updates `QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml` from c288a8864fe9054f68410e1d78f5863aea4c4d22 to 1c389210e6dfda1630d053c07ae4dae523dd5576
- [Release notes](https://github.com/qrun-io/qqq-orb/releases)
- [Changelog](https://github.com/QRun-IO/qqq-orb/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/qrun-io/qqq-orb/compare/c288a8864fe9054f68410e1d78f5863aea4c4d22...1c389210e6dfda1630d053c07ae4dae523dd5576)

Updates `QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml` from cc30f599e63c8e41629c99d23395bac1adb8bc44 to 1c389210e6dfda1630d053c07ae4dae523dd5576
- [Release notes](https://github.com/qrun-io/qqq-orb/releases)
- [Changelog](https://github.com/QRun-IO/qqq-orb/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/qrun-io/qqq-orb/compare/cc30f599e63c8e41629c99d23395bac1adb8bc44...1c389210e6dfda1630d053c07ae4dae523dd5576)

---
updated-dependencies:
- dependency-name: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml
  dependency-version: 1c389210e6dfda1630d053c07ae4dae523dd5576
  dependency-type: direct:production
  dependency-group: github-actions
- dependency-name: QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml
  dependency-version: 1c389210e6dfda1630d053c07ae4dae523dd5576
  dependency-type: direct:production
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-29 00:14:55 +00:00
james.maesandCodex 4fd2c30b07 ci: adopt managed orb 0.8 patch track (#28)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 15:26:36 -05:00
james.maesandCodex 65e63551f9 ci: adopt shared orb 0.7.18 (#27)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:43:20 -05:00
james.maesandCodex ccc0cc2842 ci: adopt shared orb 0.7.17 (#26)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:15:18 -05:00
james.maesandCodex 9178646423 ci: centralize branch verification and source scanning (#23)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:17:56 -05:00
james.maesandCodex 681598be57 ci: adopt central orb 0.7.12 and stable gates (#21)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 08:56:36 -05:00
james.maesandCodex ea9036509c ci: centralize rc release completion in qqq orb (#20)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:25:42 -05:00
james.maesandCodex 2a2ba58b06 ci: adopt shared orb 0.7.11 (#19)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:57:50 -05:00
james.maes 61e9c22563 Merge pull request #18 from QRun-IO/feature/central-ci
ci: adopt shared QQQ orb standard
2026-10-20 18:57:48 -05:00
james.maesandCodex 65922dd413 ci: adopt verified central QQQ orb 0.7.3
Co-Authored-By: Codex <noreply@openai.com>
2026-10-20 18:52:28 -05:00
james.maes bdaf35fc3e Merge pull request #17 from QRun-IO/feature/bounded-spike-history
fix(alerts): page spike history instead of holding 7 days of query snapshots
2026-10-16 15:54:56 -05:00
james.maesandClaude Opus 5.5 c49b479620 fix(alerts): page spike history instead of holding 7 days of query snapshots
SnapshotEvaluator.spikes loaded RuleContext.window("query_snapshot", 7)
as QRecords on every snapshot evaluation (every collection interval on
the worker). kof22 writes ~33,000 query snapshots a day, so by day 7 that
is ~230,000 records per evaluation.

history() now reads the window in BATCH_SIZE keyset pages (only OK and
PARTIAL snapshots, as before) and keeps per statement only the current
interval time, the latest capture time and earlier interval times in a
primitive double array. Median, twelve-sample baseline and the one-hour
absence rule are unchanged (SnapshotEvaluatorTest still passes).

SpikeHistoryBoundedTest counts every query snapshot QQQ materializes:
2,500 samples never exceed one 1,000-row page, failed snapshots and
samples outside the window are excluded, the current interval is found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 15:24:44 -05:00
james.maes de991509ca Merge pull request #16 from QRun-IO/feature/bounded-attribution-reads
fix(attribution): sum attribution in the database, page the daily rollup (prod OOM)
2026-10-16 15:15:33 -05:00
james.maesandClaude Opus 5.5 fd0116fbd0 fix(analyze): read attribution as database sums in the hourly analysis
dbPerfAnalyze (hourly, on the worker) loaded the full day of
db_perf_attribution through RuleContext.window and cached it for the
rules: ~570,000 rows. kof22 worker working set jumped from ~660 MiB to
~1.84 GiB at every :07-:08 run on 2026-10-16 (2 Gi limit, 1.5 GB heap).

RuleContext.window("attribution", days) now returns AggregateAction sums
(units, statements, rowsCount, errors, totalMs; max maxMs) grouped by
unitKind, unitName, tableName, operation: the only fields the rules
(OVER_AUDITED, HOT_JOB) read. AttributionBoundedReadTest proves no detail
rows are loaded and the totals are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 15:04:23 -05:00
james.maesandClaude Opus 5.5 744ff04ade fix(attribution): sum attribution in the database, page the daily rollup
The Attribution home widget queried every db_perf_attribution row of the
last 24 hours into QRecords and summed them in Java. kof22 writes about
30,000 attribution rows an hour, so one render loaded ~570,000 rows
(540 MB in PostgreSQL). On 2026-10-16 a single admin request logged
"Large number of records in QueryOutputList" at 50k, 100k and 250k rows
and the web pod died with OutOfMemoryError: Java heap space about 50 s
later, after slowing every other request with GC thrash.

- DbPerfWidgetRenderer.attribution() now uses AggregateAction: SUM of
  statements and totalMs grouped by unitKind, unitName, tableName,
  ordered by time and capped at 5,000 groups (prod has ~1,000).
- RollupService reads the day's attribution detail in 1,000-row keyset
  pages through the new DbPerfStore.forEachPage, so the nightly rollup
  on the worker is bounded by distinct keys instead of the day's rows.
- AttributionBoundedReadTest counts every detail record QQQ materializes
  (post-query customizer): the widget loads none, the rollup never holds
  more than BATCH_SIZE at once, and sums stay correct across pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:34:58 -05:00
james.maes fdcad364f1 Merge pull request #10 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:04:25 -05:00
james.maes 7c71e384c9 Merge pull request #12 from QRun-IO/fix/collector-failures
fix(collect): no stack overflow on huge statements; every failure closes the run as FAILED
2026-10-15 17:43:04 -05:00
james.maesandClaude Opus 5.5 b882e3d0f0 fix(collect): no stack overflow on huge statements; every failure closes the run
The normalizer folded IN and VALUES lists with nested-repetition regular
expressions, which threw StackOverflowError on kof22's 60 KB statements.
That Error escaped catch(Exception), so runs stayed RUNNING, nothing was
logged and no alert fired. List folding is now a linear token scan, the
collector catches Throwable (rethrowing OutOfMemoryError), logs class and
frame at ERROR, marks FAILED and alerts, and stale RUNNING runs are closed.
Verified against kof22's 4,462 real statement texts (under 1 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 17:40:01 -05:00
james.maes bb7e852a35 Merge pull request #11 from QRun-IO/fix/single-attribution-runtime
fix(attribution): one active attribution runtime per JVM
2026-10-15 17:09:55 -05:00
james.maesandClaude Opus 5.5 8a34ac4404 ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:19:54 -05:00
james.maesandClaude Opus 5.5 63d4c9669f fix(attribution): one active attribution runtime per JVM
Statement listeners are JVM-wide. Starting attribution on another
instance now stops the previous runtime's listener and flusher, so
repeated server starts (tests, hot swaps) never pin old instances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:11:08 -05:00
james.maes 1663f48073 Merge pull request #9 from QRun-IO/fix/table-icons
fix(nav): distinct icon per table; cuts 0.1.0-RC.3
2026-10-15 14:15:35 -05:00
james.maesandClaude Opus 5.5 0b517d7e16 fix(nav): a distinct icon per table and a storage app icon
Every db_perf_ table used the speed icon, which repeats in the nav and
clashes with web performance (kof22's nav icon uniqueness check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:12:56 -05:00
james.maes 1bc250c1b8 Merge pull request #8 from QRun-IO/fix/widget-columns
fix(home): at most four columns per widget table; cuts 0.1.0-RC.2
2026-10-15 13:25:03 -05:00
james.maesandClaude Opus 5.5 c9652fd541 fix(home): at most four columns per widget table, plain headers
Top queries, live activity, attribution, repeated statements and tables
fold secondary facts into a muted second line of the first cell, so the
home stays readable at every width (kof22's widget readability guard).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:22:12 -05:00
james.maes 8446ab1f0b Merge pull request #7 from QRun-IO/feature/attribution
feat: QQQ attribution and N+1 detection (QQQ 4.1.0-RC.13); cuts 0.1.0-RC.1
2026-10-15 12:55:23 -05:00
james.maesandClaude Opus 5.5 fe41490559 Merge develop (rule seed text) into feature/attribution; number the sample_rate changeset 34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:52:02 -05:00
james.maesandClaude Opus 5.5 34db3cdff6 Merge develop into feature/attribution; help for sampleRate, README and CHANGELOG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:21:26 -05:00
james.maes d61684b15e Merge pull request #6 from QRun-IO/fix/rule-seed-text
fix(rules): helpful rule descriptions and threshold labels; spec alert names
2026-10-15 12:18:59 -05:00
james.maesandClaude Opus 5.5 f0b544b074 fix(rules): helpful rule labels, descriptions and threshold labels; spec alert type names
Seeded suggestion rules get plain descriptions of what each finds and
what to do, and threshold labels without stray quotes. The spec now
names the alert types dbperf.* (core rejects underscores).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:15:51 -05:00
james.maes 890df96bc1 Merge pull request #5 from QRun-IO/feature/plans-and-home
feat: plan viewer, UI API extension, app home, help and docs
2026-10-15 12:12:51 -05:00
james.maesandClaude Opus 5.5 a571fe4c3a Merge develop (findings, alerts, health) into feature/plans-and-home; help for finding actions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:09:57 -05:00
james.maesandClaude Opus 5.5 f8d25d1f7c feat(ui): plan viewer, db-performance UI API extension, app home widgets, help bundle and docs
Query plan record action and POST explain: plain or generic EXPLAIN,
and EXPLAIN (ANALYZE, BUFFERS) for single SELECTs in a rolled-back
READ ONLY transaction with statement_timeout, behind explain_analyze,
always audited, ANALYZE plans and parameters never stored. UI API
extension /qqq/v1/qbit/db-performance (overview, activity with live
text only for view_live_text, explain, statements/lookup) with a
committed OpenAPI fragment and contract test. Eight full-width typed
home widgets. Data-driven help bundle with a coverage test. README,
CHANGELOG and UPGRADE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:06:37 -05:00
james.maes 1f14317e7e Merge pull request #4 from QRun-IO/feature/findings
feat: suggestions engine, finding actions, alerts and health
2026-10-15 11:54:19 -05:00
james.maesandClaude Opus 5.5 f218a1c820 feat(findings): suggestions engine, finding actions, alerts and health checks
Fourteen rules read their db_perf_rule row and produce deduplicated,
ranked findings with evidence and ready-to-commit Liquibase snippets
(never applied). Acknowledge, Mark resolved, Ignore and Reopen are
standard record actions. Collection failures, query spikes, connection
saturation and blocked chains raise and resolve keyed QAlerts; the
collector, stats access and connections register core health checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:51:46 -05:00
james.maesandClaude Opus 5.5 564ebf8d97 feat(attribution): QQQ statement attribution and N+1 detection
Registers a core StatementExecutionListener (QQQ 4.1.0-RC.13) that
aggregates statements per unit of work (route, scheduled job, process),
table, operation and fingerprint in bounded memory, detects repeated
statement shapes per unit, and flushes each window with one batched
insert per table from a daemon thread that never measures itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:44:26 -05:00
james.maes 4dde72733b Merge pull request #3 from QRun-IO/feature/foundation
feat: foundation (schema, Postgres collector, delta snapshots, rollup and retention)
2026-10-15 11:26:48 -05:00
james.maesandClaude Opus 5.5 b3814a8a95 test: H2 and PostgreSQL 17 tests for schema, foreign keys, collection, deltas, rollup and retention
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:23:04 -05:00
james.maesandClaude Opus 5.5 d91a02675c feat(collect): Postgres stats provider, SQL normalizer, delta snapshots, rollup and retention processes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:23:03 -05:00
james.maesandClaude Opus 5.5 99330035fe feat(schema): the 17 db_perf_ tables, changelog with seeded rules, metadata, permissions and app
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:23:03 -05:00
james.maesandClaude Opus 5.5 8b02386898 build: QQQ 4.1.0-RC.11, RDBMS, Liquibase and Testcontainers test deps, checkstyle in validate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:23:02 -05:00
james.maes 2ff64f4558 Merge pull request #2 from QRun-IO/feature/spec
docs: v1 specification
2026-10-15 10:28:38 -05:00
125 changed files with 28712 additions and 156 deletions

No files matched your search

+11 -57
View File
@@ -1,60 +1,14 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.9
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
qqq-orb: kingsrook/qqq-orb@0.8
workflows:
test_only:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
filters:
branches:
ignore: /(develop|release\/.*)/
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: snapshot
filters:
branches:
only: [develop]
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release_candidate
filters:
branches:
only: [/release\/.*/]
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [/^v\d+\.\d+\.\d+$/]
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: '0.8'
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /^v\d+\.\d+\.\d+$/
+23
View File
@@ -0,0 +1,23 @@
name: CodeQL
'on':
push:
branches:
- main
- develop
- release/**
- hotfix/**
pull_request: {}
schedule:
- cron: 41 16 * * 0
workflow_dispatch: {}
permissions:
contents: read
actions: read
security-events: write
packages: read
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
analyze:
uses: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@1c389210e6dfda1630d053c07ae4dae523dd5576
+62 -13
View File
@@ -1,25 +1,74 @@
# Cut the next release candidate (QRun-IO/qqq docs/QRUN-QBIT-REPO-STANDARD.md, C6):
# run this workflow by hand, or merge a develop PR labelled `release:rc`. It merges develop into
# the release line and the release/* push runs CircleCI publish_release_candidate.
name: Release candidate
on:
'on':
workflow_dispatch:
inputs:
dry_run:
description: Print the plan without merging or creating branches
description: Plan a cut without changing release state
type: boolean
default: false
finalize:
description: Retry finalization of an existing cut; do not cut another RC
type: boolean
default: false
recover:
description: Recover the same published RC after a deployment timeout; never
reupload
type: boolean
default: false
circle_build:
description: Original Circle publisher build number (recovery only)
type: string
default: ''
merge_sha:
description: Existing cut merge_sha (finalize retry only)
type: string
default: ''
bump_sha:
description: Existing cut bump_sha (finalize retry only)
type: string
default: ''
branch:
description: Existing cut branch (finalize retry only)
type: string
default: ''
version:
description: Existing cut version (finalize retry only)
type: string
default: ''
pull_request:
types: [closed]
branches: [develop]
types:
- closed
branches:
- develop
repository_dispatch:
types:
- qbit-rc-published
permissions:
contents: write
actions: read
checks: read
jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release:rc'))
uses: QRun-IO/.github/.github/workflows/qbit-release-rc.yml@main
if: github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.merged
== true && github.event.pull_request.base.ref == 'develop' && contains(github.event.pull_request.labels.*.name,
'release:rc'))
uses: QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml@1c389210e6dfda1630d053c07ae4dae523dd5576
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false
}}
finalize: ${{ github.event_name == 'repository_dispatch' || (github.event_name
== 'workflow_dispatch' && inputs.finalize) }}
recover: ${{ github.event_name == 'workflow_dispatch' && inputs.recover || false
}}
circle_build: ${{ github.event_name == 'workflow_dispatch' && inputs.circle_build
|| '' }}
merge_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.merge_sha
|| inputs.merge_sha || '' }}
bump_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.bump_sha
|| inputs.bump_sha || '' }}
branch: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.branch
|| inputs.branch || '' }}
version: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.version
|| inputs.version || '' }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
profile: qbit
orb_version: '0.8'
default_branch: develop
java_version: '21'
browsers: true
api_checks: false
docs_only: false
release_on_main: false
release_tag_policy: stable
test_branch_policy: non-release
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
release_completion: true
+3 -3
View File
@@ -4,7 +4,7 @@ This file provides guidance to Codex when working with code in this repository.
## Project Overview
**QBit DB Performance**: Database performance for QQQ apps: slow and expensive queries from pg_stat_statements with delta snapshots, table scan and index analysis, live activity and blocking chains, connection capacity, maintenance, QQQ query attribution with N+1 detection, and ranked suggestions, all as standard QQQ tables. Design: docs/SPEC.md in this repository. Implementation in progress.
**QBit DB Performance**: Database performance for QQQ apps in standard secured tables: PostgreSQL statement deltas, table and index analysis, live activity and blocking, connection capacity, audited read-only query plans, typed full-width dashboards, a versioned UI API and data-driven help. Collection and presentation are implemented; attribution, findings, alerts and health runtime integrations are developed separately. Design: docs/SPEC.md.
## Build Commands
@@ -21,7 +21,7 @@ This file provides guidance to Codex when working with code in this repository.
## Naming (owner rule)
- Tables are lowercase snake_case prefixed with the full qbit name: `db_perf_<name>`. The QQQ table name and the backend (SQL) name are identical; possible-value sources take the table name.
- Permissions use the full qbit name: `db_perf.<action>` (for example `db_perf.view`, `db_perf.admin`).
- Permissions use the full qbit name: `db_performance.<action>` (for example `db_performance.view`, `db_performance.manage_settings`).
- Qbit data lives in standard, visible QQQ tables, secured by permissions and record security locks, never hidden.
- Processes run as table or record actions, never in the nav.
@@ -35,7 +35,7 @@ This file provides guidance to Codex when working with code in this repository.
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.10` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.11` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Tests: JUnit 5, AssertJ.
## Release
+19 -3
View File
@@ -1,8 +1,24 @@
# Changelog
All notable changes to this project are documented here. The format follows Keep a Changelog; versions follow semantic versioning.
All notable changes are documented here. This project follows Keep a Changelog and semantic versioning.
## [0.1.0] - Unreleased
## [Unreleased]
### Fixed
- Query-spike alerts no longer hold seven days of query snapshots as records on every snapshot evaluation (~230,000 rows at kof22's rate). `SnapshotEvaluator` reads the window in 1,000-row keyset pages and keeps only each statement's current interval time, latest capture time and earlier interval times as primitive doubles; the spike rule (twelve prior samples, median, absence after an hour) is unchanged.
- The Attribution home panel no longer loads a day of attribution detail into the heap. It sums statements and time per unit and table in the database (at most 5,000 groups), so a render holds about a thousand aggregate rows instead of the roughly 570,000 detail rows (540 MB in PostgreSQL) that ran the kof22 web pod out of memory on 2026-10-16. The daily attribution rollup reads the day in 1,000-row keyset pages (`DbPerfStore.forEachPage`), so worker memory is bounded by distinct dimension keys, not by the day's row count. The hourly analysis (`dbPerfAnalyze`) reads attribution the same way: `RuleContext.window("attribution", ...)` returns database sums per unit, table and operation instead of every detail row (it filled the kof22 worker heap to ~1.8 GB every hour).
- Collection no longer stops at RUNNING on large production statement sets: list folding in the SQL normalizer is a linear token scan (the nested-repetition regular expressions overflowed the stack on 60 KB statements with long IN/VALUES lists). Any failure, including an `Error`, now closes the run as FAILED with the exception class, logs it at ERROR (class and frame only, never SQL text) and raises the collection alert; runs left RUNNING by a crash are closed as FAILED after three intervals.
- Only one runtime attributes per JVM: starting attribution on a newer instance (a hot swap, or repeated server starts in tests) stops the previous one, so old instances are never pinned by a forgotten listener and flusher.
- Each table has its own icon (the app icon is now `storage`), so nav entries are distinguishable and do not repeat icons used elsewhere.
- Home tables keep at most four columns with plain headers (readable at every width): secondary facts such as calls and mean time, the session's query text, and a unit's kind and top table move into a second line of the first cell.
### Added
- Repository scaffold: `DbPerfQBitConfig` and `DbPerfQBitProducer` (table prefix `db_perf_`, permissions `db_perf.*`), build, CI and repository standard files.
- QQQ attribution and N+1 detection: a core statement execution listener (QQQ 4.1.0-RC.13) aggregates database time per route, scheduled job, process, table and statement shape in bounded memory, records repeated statement shapes per unit of work, and each pod flushes one batched insert per table every `attribution_flush_seconds`. Hosts without QQQ's application launcher call `DbPerfRuntime.of(instance).startAttribution()` after startup.
- Secured database-performance tables, PostgreSQL statistics snapshots, daily rollups and configurable retention.
- Audited PLAN/GENERIC and read-only ANALYZE query viewer with SQL validation, timeout, rollback and transient parameter handling.
- Query plan statement action with readable plan text, cost, SQL Workbench link and a sanitized assistant prompt.
- UI API extension 1.0.0 with overview, live activity, explain and statement lookup, plus its committed OpenAPI contract.
- Eight full-width typed home panels with pagination, record links, guided empty states and permission-gated live activity.
- Maven-versioned QBIT help bundle, complete metadata coverage checks, rule threshold guidance and adoption documentation.
- PostgreSQL security acceptance tests and H2 API/widget fixtures.
+3 -3
View File
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code when working with code in this reposi
## Project Overview
**QBit DB Performance**: Database performance for QQQ apps: slow and expensive queries from pg_stat_statements with delta snapshots, table scan and index analysis, live activity and blocking chains, connection capacity, maintenance, QQQ query attribution with N+1 detection, and ranked suggestions, all as standard QQQ tables. Design: docs/SPEC.md in this repository. Implementation in progress.
**QBit DB Performance**: Database performance for QQQ apps in standard secured tables: PostgreSQL statement deltas, table and index analysis, live activity and blocking, connection capacity, audited read-only query plans, typed full-width dashboards, a versioned UI API and data-driven help. Collection and presentation are implemented; attribution, findings, alerts and health runtime integrations are developed separately. Design: docs/SPEC.md.
## Build Commands
@@ -21,7 +21,7 @@ This file provides guidance to Claude Code when working with code in this reposi
## Naming (owner rule)
- Tables are lowercase snake_case prefixed with the full qbit name: `db_perf_<name>`. The QQQ table name and the backend (SQL) name are identical; possible-value sources take the table name.
- Permissions use the full qbit name: `db_perf.<action>` (for example `db_perf.view`, `db_perf.admin`).
- Permissions use the full qbit name: `db_performance.<action>` (for example `db_performance.view`, `db_performance.manage_settings`).
- Qbit data lives in standard, visible QQQ tables, secured by permissions and record security locks, never hidden.
- Processes run as table or record actions, never in the nav.
@@ -35,7 +35,7 @@ This file provides guidance to Claude Code when working with code in this reposi
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.10` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.11` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Tests: JUnit 5, AssertJ.
## Release
+115 -16
View File
@@ -1,43 +1,142 @@
# QBit DB Performance
[![License](https://img.shields.io/badge/license-Apache%202.0-green.svg)](https://www.apache.org/licenses/LICENSE-2.0)
[![Java](https://img.shields.io/badge/java-21+-blue.svg)](https://adoptium.net/)
> Database performance for QQQ apps: slow and expensive queries from pg_stat_statements with delta snapshots, table scan and index analysis, live activity and blocking chains, connection capacity, maintenance, QQQ query attribution with N+1 detection, and ranked suggestions, all as standard QQQ tables.
**Status:** design: docs/SPEC.md in this repository; implementation in progress. Nothing is published yet.
Database performance for QQQ apps: PostgreSQL statement deltas, table and index statistics, live sessions and blocking, connection capacity, audited query plans, and typed home dashboards in standard secured QQQ tables. Approved design: [docs/SPEC.md](docs/SPEC.md). This branch implements collection and the presentation/plan-viewer phase; attribution listeners, findings evaluation, alerts and health are developed in a separate phase. No release has been published yet.
## Requirements
- Java 21.
- QQQ 4.1 (built against `4.1.0-RC.10`).
- Java 21 and QQQ **4.1.0-RC.13+** (the statement execution listener SPI used for QQQ attribution).
- PostgreSQL **13+**; **16+** for parameterized generic plans, with the pgJDBC driver (built against 42.7.14).
- `pg_stat_statements` preloaded, created in the monitored database, and readable by the application role.
- `pg_read_all_stats` granted to that role for complete statistics and other sessions' visibility.
- A host scheduler for automatic collection and rollups. Docker is required for the PostgreSQL acceptance tests.
## Install
Not yet published. Once released, add `com.kingsrook.qbits:qbit-db-performance` and produce the qbit into your `QInstance`:
Add the dependency when the first release is available (use your locally installed snapshot during development):
```xml
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-db-performance</artifactId>
<version>0.1.0-SNAPSHOT</version>
</dependency>
```
Register a QQQ `PostgreSQLBackendMetaData` backend and the scheduler first, then produce the qbit:
```java
new DbPerfQBitProducer()
.withQBitConfig(new DbPerfQBitConfig().withBackendName("rdbms"))
.withQBitConfig(new DbPerfQBitConfig()
.withBackendName("rdbms")
.withSchedulerName("scheduler")
.withParentAppName("operations"))
.produce(qInstance);
```
## Naming
The optional parent must already exist. Include the migration in the host's Liquibase master:
Tables are `db_perf_<name>` (lowercase snake_case, backend name equal to the table name). Permissions are `db_perf.<action>`.
```xml
<include file="db/liquibase/qbit-db-performance/db_performance.changelog.sql"/>
```
The changelog creates 17 tables and seeds 14 rules without overwriting administrator edits. The qbit never runs migrations itself. Declare the database role grant through your normal database provisioning:
```sql
GRANT pg_read_all_stats TO your_application_role;
```
Configure `shared_preload_libraries` with `pg_stat_statements` and create the extension in the monitored database through your normal deployment process. Missing access produces reduced-capability status and setup guidance.
With `schedulerName`, `dbPerfCollectSnapshot` runs every five minutes and `dbPerfRollupAndPurge` daily at 03:15 (UTC by default). Omitting it registers no schedules. Use **Collect now** on Snapshots for the first baseline and a second collection for deltas. Table and index statistics normally update hourly. The scheduled processes are durable-eligible when the host has durable jobs enabled.
UI hosts must include `qqq-middleware-javalin` (an optional dependency of this qbit). The producer registers UI API extension `db-performance` version `1.0.0` under `/qqq/v1/qbit/db-performance`. Core-only hosts can use tables and collection without middleware.
## Tables
All table names and SQL names are identical; all data is visible through standard QQQ tables.
| Tables | Purpose | Retention |
|---|---|---|
| `db_perf_settings`, `db_perf_rule` | Per-backend settings and rule thresholds | Indefinite |
| `db_perf_snapshot` | Collection status, interval totals and capacity | 30 days |
| `db_perf_statement` | Normalized query inventory and retained 24-hour totals | Stale after 365 days |
| `db_perf_query_snapshot`, `db_perf_query_daily` | Statement interval details and daily aggregates | 30 / 365 days |
| `db_perf_table_stat`, `db_perf_index_stat` | Current table/index inventory and maintenance | Stale after 365 days |
| `db_perf_table_snapshot`, `db_perf_table_daily` | Table history and growth | 30 / 365 days |
| `db_perf_activity_sample`, `db_perf_connection_sample` | Notable activity and connection counts | 30 days |
| `db_perf_attribution`, `db_perf_attribution_daily`, `db_perf_repeat_pattern` | QQQ units of work and repeated statements | 30 / 365 / 30 days |
| `db_perf_finding` | Ranked suggestions and evidence | Open retained; stale closed after 365 days |
| `db_perf_explain` | Every plan request and eligible saved plans | 30 days |
The home page stacks Setup, Database time, Top queries, Findings, Live activity, Connections, QQQ attribution and Tables at full width. Tables paginate; widgets refresh every 30–60 seconds. Statement records include Query Plans and snapshot/daily/finding child lists. Attribution and findings panels remain guided empty states until their producing runtime is installed and has collected data.
Snapshot totals include all observed statement deltas. Only the top configured details are retained, so statement totals and rollups can undercount queries outside that selection. The database-time share denominator includes all observed statement deltas. Reset counters start new baselines rather than creating spikes.
## Permissions
| Permission | Access |
|---|---|
| `db_performance.view` | App, read-only tables, widgets, overview, lookup and normalized activity |
| `db_performance.view_live_text` | Raw SQL in the transient activity API only; also requires view |
| `db_performance.explain` | PLAN/GENERIC requests and Query plan record action |
| `db_performance.explain_analyze` | Execute the guarded ANALYZE service; the record action also needs explain |
| `db_performance.manage_settings` | Settings/rule edits and Collect now |
| `db_performance.manage_findings` | Finding status actions supplied by the findings phase |
Hosts can change `permissionBaseName`; grant the corresponding complete names. Persistent collector and explain-audit writes run as the system user, retaining the requesting session's user ID for plans. Dashboard and API reads retain the viewer's session.
## Configuration
| Host option | Default / meaning |
|---|---|
| `backendName` | Required; monitored database and qbit storage backend |
| `schedulerName` | Optional; host-registered scheduler |
| `appName`, `parentAppName` | `dbPerformance`; optional existing parent app |
| `permissionBaseName` | `db_performance` |
| `attributionEnabled` | `true`; attribution listener integration is a separate phase |
| `sqlWorkbenchRunnerPath` | `/app/sqlWorkbenchRunner` |
| `podName` | `HOSTNAME`, or `local` |
| `snapshotCron`, `analyzeCron`, `rollupCron` | `0 */5 * * * ?`, `0 7 * * * ?`, `0 15 3 * * ?`; analysis schedule is owned by the findings phase |
| `timeZone` | `UTC` |
Runtime values live in Settings: 5-minute collection, hourly tables, 200 top statement details, 30-day detail retention, 365-day rollups, 10-second explain timeout, 30-second long-query and 60-second idle-transaction thresholds. Attribution and alert settings are documented in the shipped help bundle, as are every rule's threshold units and seeded values. Existing administrator settings are never overwritten on startup.
## Query plans and UI API
**Query plan** on one statement presents PLAN/ANALYZE and optional parameter values (one per line). The Result screen shows selectable plan text, cost, a SQL Workbench link and an assistant prompt. There is no action on the Query Plans audit table.
- `GET /overview?backend=`: setup status, latest snapshot, top ten queries and open findings.
- `GET /activity?backend=`: current provider sessions and transitive blocking chains; raw text is permission-gated.
- `POST /explain`: `{ "backend": "rdbms", "sql": "select * from orders where id = $1", "mode": "PLAN" }`. Supply either `statementId` or `sql`. For ANALYZE, add `"parameters": ["123"]` and obtain its separate permission.
- `GET /statements/lookup?backend=&sql=`: normalized fingerprint matches with `/app/<appName>/db_perf_statement/<id>` history paths.
Paths above are relative to `/qqq/v1/qbit/db-performance`. The [committed OpenAPI fragment](src/main/resources/openapi/qqq-v1/db-performance.yaml) is generated from endpoint specs and verified by a contract test. The server never calls an AI model.
## Security and PHI
Stored SQL is normalized and fingerprinted. Raw `pg_stat_activity` text is never persisted and appears only in the activity API with `view_live_text`; home widgets always use normalized text. SQL literals and parameter values must not be used as identifiers: normalization preserves quoted identifiers. Settings and rules are excluded from AI exposure.
PLAN executes EXPLAIN in a read-only transaction with a local timeout and rollback. Parameterized PLAN uses PostgreSQL 16+ GENERIC_PLAN. Only eligible literal-free PLAN/GENERIC plans are stored. A user's SQL must equal its own normalized form to qualify; SQL loaded from the normalized statement table also qualifies.
ANALYZE accepts one SELECT or a non-modifying WITH, rejects INTO and row locks, binds supplied strings as unknown JDBC parameters, and executes inside a read-only transaction. Functions are still constrained by PostgreSQL READ ONLY. The EXPLAIN command uses pgJDBC simple-query execution with JDBC-managed parameter quoting, so PostgreSQL can coerce unknown-typed EXECUTE arguments. The transaction always rolls back and the prepared statement is explicitly deallocated before the connection returns to its pool. SQLSTATE 57014 records TIMEOUT. The service stores only a sanitized audit, never ANALYZE plans or parameter values. Returned ANALYZE plans are transient and can contain values; the assistant prompt redacts literals in plan expressions. Review transient plans before copying them elsewhere.
## Adopting in another app
Follow the QQQ [adoption checklist](https://github.com/QRun-IO/qqq/blob/develop/docs/migration/adoption-checklist.adoc) once the first release is published.
1. Add compatible QQQ, PostgreSQL backend and qbit dependencies; include the changelog once.
2. Preload/create pg_stat_statements and grant the database statistics role through your provisioning.
3. Register the backend, optional parent app and scheduler, then produce the qbit.
4. Grant the exact permissions to intended operators; reserve ANALYZE and raw live text for appropriate roles.
5. Run Collect now twice, check Setup, and confirm the API extension in `/qqq/v1/openapi.yaml`.
6. Optionally configure SQL Workbench links; install attribution/findings runtime integrations when available. Review and commit proposed database changes through the host's migration workflow.
## Build
```bash
./mvnw verify
./mvnw -o verify
```
Always use the Maven wrapper (Maven 3.9.11); Maven 3.10 cannot read the parent POM.
On Windows use `./mvnw.cmd -o verify`. Always use the wrapper; Maven 3.10 cannot read the parent POM. Offline mode requires dependencies already cached. Verification runs main/test Checkstyle, JUnit including PostgreSQL 17 Testcontainers, and JaCoCo gates of 70% instructions and 90% classes. Docker must be accessible to the build process. Regenerate the API fragment with `./mvnw -o -Dtest=DbPerfApiContractTest -Ddbperf.openapi.update=true test`.
## License
Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
[Apache License 2.0](LICENSE).
+13
View File
@@ -0,0 +1,13 @@
# Upgrading QBit DB Performance
## First release
There is no previous published release. See [CHANGELOG.md](CHANGELOG.md) for additions and [README.md](README.md) for complete installation.
Use QQQ 4.1.0-RC.11 or newer, Java 21, and PostgreSQL 13 or newer (16+ for generic plans). Include `db/liquibase/qbit-db-performance/db_performance.changelog.sql` once in your application's Liquibase changelog. Run migrations through the host deployment before collecting statistics; the qbit does not apply them automatically.
Preload and create `pg_stat_statements`, then grant `pg_read_all_stats` to the application's database role through your normal provisioning. Without the role the collector reports reduced visibility. Preserve existing rule and settings edits; seeding is insert-only.
Register the producer and a host scheduler. Grant `db_performance.view` and the narrower operator permissions appropriate to each role; ANALYZE and raw live text require separate grants. Existing deployments using an early scaffold's `db_perf.*` grants must use the actual `db_performance.*` names (or the configured permission base).
Collect twice to establish useful deltas. Inspect Setup for access and schedule status. UI hosts need the optional `qqq-middleware-javalin` dependency so the four endpoints register in the standard UI API. The help bundle version is filtered from the Maven project version during the build.
+58
View File
@@ -0,0 +1,58 @@
# Attribution runtime (phase B2)
The producer registers `AttributionRuntimeService` with the QQQ application
launcher. It starts one daemon per application instance after initialization,
independently of scheduled-job leadership. Hosts without `QApplicationLauncher`
should call `DbPerfRuntime.of(instance).startAttribution()` after migrations and
instance validation, and `stopAttribution()` before disposing of the instance.
`attributionEnabled=false` prevents listener registration and background work.
Metadata production itself starts no threads, so existing metadata tests remain
isolated.
`AttributionFlusher` can also be constructed directly for deterministic tests:
`refreshSettings()`, register `getListener()` with `StatementExecutionListeners`,
and call `flushNow()`. Always close it to unregister. Production `start()` loads
settings, registers the listener, schedules flushing, and installs a JVM shutdown
hook. Closing stops scheduling, unregisters, and flushes once. Shutdown only
persists completed units; hosts should drain application requests first.
Statements never read settings from the database. A successful settings refresh
publishes an immutable cached copy; a failed refresh keeps the previous copy.
The initial copy disables recording until settings can be read. The configured
flush interval takes effect when the next pass is scheduled. Disabled attribution
still refreshes settings so administrators can re-enable it.
Units are sampled once and buffered until completion, allowing HTTP route names
to be finalized. A completed unit belongs to the window in which it ends; an
abandoned unit is merged when that worker sees a different unit. No-unit work is
sampled per statement and counted as one unit per statement. Sampling transitions
retain separate aggregates for each effective rate. Values are observed counts,
not extrapolated estimates. Daily rollups sum those counts across windows and
pods and take the maximum duration.
Each global map includes its overflow bucket within `attribution_max_keys`.
One slot is reserved for a universal `(other)` key (all dimensions collapse,
fingerprint is SHA-256 of `(other)`). Per-unit maps use the same bound with a
minimum of two slots. Collapsed per-unit shapes cannot establish N+1 evidence
and are excluded from repeat detection, avoiding false positives. Global repeat
overflow sums qualifying repetitions but counts each affected unit only once.
A mixed-rate overflow row has a null sample rate because there is no single
valid rate for it. The SQL normalization cache holds at most 5,000 raw SQL keys
in memory and clears on overflow; only normalized SQL is persisted, capped at
4,000 characters.
Window swaps and counter merges synchronize only in memory. Database work runs
under the system user on the flusher thread with attribution suppressed. Each
nonempty table receives one `InsertAction`, even above 1,000 rows; QQQ may page
the underlying SQL. Both tables are written in one transaction. Failed windows
are dropped, with rate-limited fixed-text logging, and never queued for retry.
The repeat-pattern migration adds `sample_rate` without changing prior
changesets. Existing retention already includes repeat patterns.
Build using the locally installed statement-listener API:
```powershell
.\mvnw.cmd -o verify '-Dqqq.snapshot.version=4.1.0-dbperf-SNAPSHOT'
```
The normal POM default is `4.1.0-RC.13`.
+4 -4
View File
@@ -230,11 +230,11 @@ batches, so a rerun after a crash never double counts. Each pass also writes its
`db_perf_snapshot` row (status, duration, error), which is the report even where the host runs
scheduled processes inline.
**Alerts** (core `QAlerts`; delivered by qbit-alerts when installed, logged otherwise; keyed, and
resolved when clear): `db_perf.collection_failed`, `db_perf.query_spike` (a statement's total time
**Alerts** (core `QAlerts`, type names use `dbperf.` because core rejects underscores in alert type names; delivered by qbit-alerts when installed, logged otherwise; keyed, and
resolved when clear): `dbperf.collection_failed`, `dbperf.query_spike` (a statement's total time
in an interval above `spike_factor` × its 7-day interval median and above a floor),
`db_perf.connection_saturation` (connections ≥ `connection_alert_percent` of `max_connections`),
`db_perf.blocked_chain` (a session blocked longer than `blocked_alert_seconds`).
`dbperf.connection_saturation` (connections ≥ `connection_alert_percent` of `max_connections`),
`dbperf.blocked_chain` (a session blocked longer than `blocked_alert_seconds`).
**Health** (core health SPI, Admin → Health): `db_performance.collector` (last successful snapshot
age vs interval), `db_performance.stats_access` (pg_stat_statements present, `pg_read_all_stats`
+105 -2
View File
@@ -22,6 +22,7 @@
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
<mockito.version>5.23.0</mockito.version>
</properties>
<parent>
@@ -67,6 +68,57 @@
</scm>
<dependencies>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>${mockito.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-middleware-javalin</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-postgres</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-rdbms</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.2.224</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<version>4.33.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.7.14</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>postgresql</artifactId>
<version>1.21.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>1.21.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-core</artifactId>
@@ -86,7 +138,58 @@
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes><exclude>help/**</exclude></excludes>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes><include>help/**</include></includes>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
<configuration>
<argLine>@{argLine} -javaagent:"${settings.localRepository}/org/mockito/mockito-core/${mockito.version}/mockito-core-${mockito.version}.jar"</argLine>
<includes>
<include>**/*Test.java</include>
<include>**/*IT.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>3.3.1</version>
<dependencies>
<dependency>
<groupId>com.puppycrawl.tools</groupId>
<artifactId>checkstyle</artifactId>
<version>10.12.6</version>
</dependency>
</dependencies>
<configuration>
<configLocation>checkstyle/config.xml</configLocation>
<headerLocation>checkstyle/license.txt</headerLocation>
<consoleOutput>true</consoleOutput>
<failOnViolation>true</failOnViolation>
<violationSeverity>warning</violationSeverity>
<includeTestSourceDirectory>true</includeTestSourceDirectory>
</configuration>
<executions>
<execution>
<id>validate</id>
<phase>validate</phase>
<goals><goal>check</goal></goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
@@ -138,14 +241,14 @@
<profiles>
<!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent -->
<!-- imports the released 4.1 BOM, this profile is active by default and -->
<!-- imports qqq-bom-pom 4.1.0-RC.10 ahead of the parent's 4.0.0 BOM. -->
<!-- imports qqq-bom-pom 4.1.0-RC.13 ahead of the parent's 4.0.0 BOM. -->
<profile>
<id>qqq-snapshot</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.10</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.13</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -0,0 +1,60 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.permissions.UseOtherPermissionNameCustomPermissionChecker;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
/*******************************************************************************
** Stable public permission names for host role configuration.
*******************************************************************************/
public final class DbPerfPermissions
{
public static final String VIEW = "db_performance.view";
public static final String VIEW_LIVE_TEXT = "db_performance.view_live_text";
public static final String EXPLAIN = "db_performance.explain";
public static final String EXPLAIN_ANALYZE = "db_performance.explain_analyze";
public static final String MANAGE_FINDINGS = "db_performance.manage_findings";
public static final String MANAGE_SETTINGS = "db_performance.manage_settings";
public static final List<String> ALL = List.of(VIEW, VIEW_LIVE_TEXT, EXPLAIN, EXPLAIN_ANALYZE, MANAGE_FINDINGS, MANAGE_SETTINGS);
/*******************************************************************************
** Constants only.
*******************************************************************************/
private DbPerfPermissions()
{
}
/*******************************************************************************
** Require an exact permission name, without a generated suffix.
*******************************************************************************/
public static QPermissionRules rules(String permission)
{
return (new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION).withPermissionBaseName(permission)
.withCustomPermissionChecker(UseOtherPermissionNameCustomPermissionChecker.build(permission)));
}
}
@@ -1,11 +1,15 @@
/*
* Copyright 2026 Kingsrook, LLC
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -20,37 +24,54 @@ package com.kingsrook.qbits.dbperformance;
import java.util.List;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
/*******************************************************************************
** Host-facing configuration for the DB Performance QBit.
**
** Tables are named db_perf_<name> (lowercase snake_case, backend name equal
** to the table name); permissions are db_perf.<action>.
** Host configuration; runtime thresholds live in the Settings table.
*******************************************************************************/
public class DbPerfQBitConfig implements QBitConfig
{
public static final String TABLE_NAME_PREFIX = "db_perf_";
public static final String PERMISSION_BASE_NAME = "db_perf";
public static final String TABLE_NAME_PREFIX = "db_perf_";
public static final String PERMISSION_BASE_NAME = "db_performance";
private String backendName;
private String schedulerName;
private String appName = "dbPerformance";
private String parentAppName;
private String permissionBaseName = "db_performance";
private Boolean attributionEnabled = true;
private String sqlWorkbenchRunnerPath = "/app/sqlWorkbenchRunner";
private String podName = System.getenv("HOSTNAME") == null ? "local" : System.getenv("HOSTNAME");
private String snapshotCron = "0 */5 * * * ?";
private String analyzeCron = "0 7 * * * ?";
private String rollupCron = "0 15 3 * * ?";
private String timeZone = "UTC";
/*******************************************************************************
** The backend must be named.
** Validate backend and application identity before registration.
*******************************************************************************/
@Override
public void validate(QInstance qInstance, List<String> errors)
{
assertCondition(StringUtils.hasContent(backendName), "backendName is required for the DB Performance QBit", errors);
assertCondition(backendName != null && qInstance.getBackend(backendName) instanceof RDBMSBackendMetaData, "backendName must name an existing RDBMS backend", errors);
assertCondition(appName != null && !appName.isBlank() && qInstance.getApp(appName) == null, "appName must be nonblank and unused", errors);
assertCondition(permissionBaseName != null && !permissionBaseName.isBlank(), "permissionBaseName is required", errors);
assertCondition(parentAppName == null || qInstance.getApp(parentAppName) != null, "parentAppName must name an existing app", errors);
try
{
java.time.ZoneId.of(timeZone);
}
catch(Exception e)
{
errors.add("timeZone must be a valid zone ID");
}
}
/*******************************************************************************
** Tables without their own backend use the configured backend.
** Default storage backend.
*******************************************************************************/
@Override
public String getDefaultBackendNameForTables()
@@ -59,9 +80,8 @@ public class DbPerfQBitConfig implements QBitConfig
}
/*******************************************************************************
** Getter for backendName
** Getter for backendName.
*******************************************************************************/
public String getBackendName()
{
@@ -69,9 +89,8 @@ public class DbPerfQBitConfig implements QBitConfig
}
/*******************************************************************************
** Setter for backendName
** Setter for backendName.
*******************************************************************************/
public void setBackendName(String backendName)
{
@@ -79,9 +98,8 @@ public class DbPerfQBitConfig implements QBitConfig
}
/*******************************************************************************
** Fluent setter for backendName
** Fluent setter for backendName.
*******************************************************************************/
public DbPerfQBitConfig withBackendName(String backendName)
{
@@ -89,4 +107,311 @@ public class DbPerfQBitConfig implements QBitConfig
return (this);
}
/*******************************************************************************
** Getter for schedulerName.
*******************************************************************************/
public String getSchedulerName()
{
return (this.schedulerName);
}
/*******************************************************************************
** Setter for schedulerName.
*******************************************************************************/
public void setSchedulerName(String schedulerName)
{
this.schedulerName = schedulerName;
}
/*******************************************************************************
** Fluent setter for schedulerName.
*******************************************************************************/
public DbPerfQBitConfig withSchedulerName(String schedulerName)
{
this.schedulerName = schedulerName;
return (this);
}
/*******************************************************************************
** Getter for appName.
*******************************************************************************/
public String getAppName()
{
return (this.appName);
}
/*******************************************************************************
** Setter for appName.
*******************************************************************************/
public void setAppName(String appName)
{
this.appName = appName;
}
/*******************************************************************************
** Fluent setter for appName.
*******************************************************************************/
public DbPerfQBitConfig withAppName(String appName)
{
this.appName = appName;
return (this);
}
/*******************************************************************************
** Getter for parentAppName.
*******************************************************************************/
public String getParentAppName()
{
return (this.parentAppName);
}
/*******************************************************************************
** Setter for parentAppName.
*******************************************************************************/
public void setParentAppName(String parentAppName)
{
this.parentAppName = parentAppName;
}
/*******************************************************************************
** Fluent setter for parentAppName.
*******************************************************************************/
public DbPerfQBitConfig withParentAppName(String parentAppName)
{
this.parentAppName = parentAppName;
return (this);
}
/*******************************************************************************
** Getter for permissionBaseName.
*******************************************************************************/
public String getPermissionBaseName()
{
return (this.permissionBaseName);
}
/*******************************************************************************
** Setter for permissionBaseName.
*******************************************************************************/
public void setPermissionBaseName(String permissionBaseName)
{
this.permissionBaseName = permissionBaseName;
}
/*******************************************************************************
** Fluent setter for permissionBaseName.
*******************************************************************************/
public DbPerfQBitConfig withPermissionBaseName(String permissionBaseName)
{
this.permissionBaseName = permissionBaseName;
return (this);
}
/*******************************************************************************
** Getter for attributionEnabled.
*******************************************************************************/
public Boolean getAttributionEnabled()
{
return (this.attributionEnabled);
}
/*******************************************************************************
** Setter for attributionEnabled.
*******************************************************************************/
public void setAttributionEnabled(Boolean attributionEnabled)
{
this.attributionEnabled = attributionEnabled;
}
/*******************************************************************************
** Fluent setter for attributionEnabled.
*******************************************************************************/
public DbPerfQBitConfig withAttributionEnabled(Boolean attributionEnabled)
{
this.attributionEnabled = attributionEnabled;
return (this);
}
/*******************************************************************************
** Getter for sqlWorkbenchRunnerPath.
*******************************************************************************/
public String getSqlWorkbenchRunnerPath()
{
return (this.sqlWorkbenchRunnerPath);
}
/*******************************************************************************
** Setter for sqlWorkbenchRunnerPath.
*******************************************************************************/
public void setSqlWorkbenchRunnerPath(String sqlWorkbenchRunnerPath)
{
this.sqlWorkbenchRunnerPath = sqlWorkbenchRunnerPath;
}
/*******************************************************************************
** Fluent setter for sqlWorkbenchRunnerPath.
*******************************************************************************/
public DbPerfQBitConfig withSqlWorkbenchRunnerPath(String sqlWorkbenchRunnerPath)
{
this.sqlWorkbenchRunnerPath = sqlWorkbenchRunnerPath;
return (this);
}
/*******************************************************************************
** Getter for podName.
*******************************************************************************/
public String getPodName()
{
return (this.podName);
}
/*******************************************************************************
** Setter for podName.
*******************************************************************************/
public void setPodName(String podName)
{
this.podName = podName;
}
/*******************************************************************************
** Fluent setter for podName.
*******************************************************************************/
public DbPerfQBitConfig withPodName(String podName)
{
this.podName = podName;
return (this);
}
/*******************************************************************************
** Getter for snapshotCron.
*******************************************************************************/
public String getSnapshotCron()
{
return (this.snapshotCron);
}
/*******************************************************************************
** Setter for snapshotCron.
*******************************************************************************/
public void setSnapshotCron(String snapshotCron)
{
this.snapshotCron = snapshotCron;
}
/*******************************************************************************
** Fluent setter for snapshotCron.
*******************************************************************************/
public DbPerfQBitConfig withSnapshotCron(String snapshotCron)
{
this.snapshotCron = snapshotCron;
return (this);
}
/*******************************************************************************
** Getter for analyzeCron.
*******************************************************************************/
public String getAnalyzeCron()
{
return (this.analyzeCron);
}
/*******************************************************************************
** Setter for analyzeCron.
*******************************************************************************/
public void setAnalyzeCron(String analyzeCron)
{
this.analyzeCron = analyzeCron;
}
/*******************************************************************************
** Fluent setter for analyzeCron.
*******************************************************************************/
public DbPerfQBitConfig withAnalyzeCron(String analyzeCron)
{
this.analyzeCron = analyzeCron;
return (this);
}
/*******************************************************************************
** Getter for rollupCron.
*******************************************************************************/
public String getRollupCron()
{
return (this.rollupCron);
}
/*******************************************************************************
** Setter for rollupCron.
*******************************************************************************/
public void setRollupCron(String rollupCron)
{
this.rollupCron = rollupCron;
}
/*******************************************************************************
** Fluent setter for rollupCron.
*******************************************************************************/
public DbPerfQBitConfig withRollupCron(String rollupCron)
{
this.rollupCron = rollupCron;
return (this);
}
/*******************************************************************************
** Getter for timeZone.
*******************************************************************************/
public String getTimeZone()
{
return (this.timeZone);
}
/*******************************************************************************
** Setter for timeZone.
*******************************************************************************/
public void setTimeZone(String timeZone)
{
this.timeZone = timeZone;
}
/*******************************************************************************
** Fluent setter for timeZone.
*******************************************************************************/
public DbPerfQBitConfig withTimeZone(String timeZone)
{
this.timeZone = timeZone;
return (this);
}
}
@@ -1,11 +1,15 @@
/*
* Copyright 2026 Kingsrook, LLC
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -17,6 +21,22 @@
package com.kingsrook.qbits.dbperformance;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.dbperformance.alerts.SnapshotEvaluator;
import com.kingsrook.qbits.dbperformance.api.DbPerformanceUiApi;
import com.kingsrook.qbits.dbperformance.attribution.AttributionRuntimeService;
import com.kingsrook.qbits.dbperformance.explain.QueryPlanProcess;
import com.kingsrook.qbits.dbperformance.health.DbPerfHealthChecks;
import com.kingsrook.qbits.dbperformance.metadata.DbPerfHelp;
import com.kingsrook.qbits.dbperformance.metadata.DbPerfMetaData;
import com.kingsrook.qbits.dbperformance.processes.DbPerfProcesses;
import com.kingsrook.qbits.dbperformance.widgets.DbPerfWidgets;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfigValidationException;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
@@ -100,4 +120,56 @@ public class DbPerfQBitProducer implements QBitMetaDataProducer<DbPerfQBitConfig
return (qBitConfig != null);
}
/*******************************************************************************
** Validate and register the Phase A metadata.
*******************************************************************************/
@Override
public MetaDataProducerMultiOutput produce(QInstance instance) throws QException
{
if(qBitConfig == null)
{
throw (new QException("DB Performance configuration is required"));
}
List<String> errors = new ArrayList<>();
qBitConfig.validate(instance, errors);
for(String table : DbPerfMetaData.TABLES)
{
if(instance.getTable(table) != null)
{
errors.add("Table already registered: " + table);
}
}
if(!errors.isEmpty())
{
throw (new QBitConfigValidationException(qBitConfig, errors));
}
instance.addQBit(getQBitMetaData());
DbPerfMetaData.addTo(instance, qBitConfig);
new DbPerfRuntime(instance, qBitConfig).addSelfToInstance(instance);
instance.withRuntimeService(new QCodeReference(AttributionRuntimeService.class));
DbPerfProcesses.addTo(instance, qBitConfig);
SnapshotEvaluator.register(instance, qBitConfig);
DbPerfHealthChecks.register(instance);
QueryPlanProcess.register(instance, qBitConfig);
registerUiApi(instance);
DbPerfWidgets.register(instance, qBitConfig);
DbPerfHelp.register(instance, qBitConfig);
return (new MetaDataProducerMultiOutput());
}
/*******************************************************************************
** Middleware is optional for hosts that use only the core tables and collector.
*******************************************************************************/
private static void registerUiApi(QInstance instance)
{
try
{
Class.forName("com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData");
DbPerformanceUiApi.register(instance);
}
catch(ClassNotFoundException e)
{
// Core-only hosts have no HTTP API to register.
}
}
}
@@ -0,0 +1,193 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.util.concurrent.atomic.AtomicReference;
import com.kingsrook.qbits.dbperformance.attribution.AttributionFlusher;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
/*******************************************************************************
** Per-instance configuration and service boundary; no global singleton or filesystem.
*******************************************************************************/
public class DbPerfRuntime implements QSupplementalInstanceMetaData
{
public static final String NAME = "dbPerformance";
private final QInstance instance;
private final DbPerfQBitConfig config;
private AttributionFlusher attributionFlusher;
/////////////////////////////////////////////////////////////////////////////
// statement listeners are JVM-wide: only one runtime attributes at a time //
// (a newer instance, e.g. after a hot swap or in tests, replaces the old) //
/////////////////////////////////////////////////////////////////////////////
private static final AtomicReference<DbPerfRuntime> ACTIVE_ATTRIBUTION = new AtomicReference<>();
/*******************************************************************************
** Runtime bound to a single host instance.
*******************************************************************************/
public DbPerfRuntime(QInstance instance, DbPerfQBitConfig config)
{
this.instance = instance;
this.config = config;
}
/*******************************************************************************
** Find the registered runtime.
*******************************************************************************/
public static DbPerfRuntime of(QInstance instance)
{
DbPerfRuntime runtime = QSupplementalInstanceMetaData.of(instance, NAME);
if(runtime == null)
{
throw (new IllegalStateException("Database Performance is not registered"));
}
return (runtime);
}
/*******************************************************************************
** Supplemental metadata name.
*******************************************************************************/
@Override
public String getName()
{
return (NAME);
}
/*******************************************************************************
** Register on the owning instance.
*******************************************************************************/
@Override
public void addSelfToInstance(QInstance instance)
{
instance.withSupplementalMetaData(this);
}
/*******************************************************************************
** Host configuration.
*******************************************************************************/
public DbPerfQBitConfig getConfig()
{
return (config);
}
/*******************************************************************************
** Owning instance.
*******************************************************************************/
public QInstance getInstance()
{
return (instance);
}
/*******************************************************************************
** Execute as the system user, always restoring the caller context.
*******************************************************************************/
public <T> T asSystem(Work<T> work) throws Exception
{
AtomicReference<T> result = new AtomicReference<>();
QContext.withTemporaryContext(new CapturedContext(instance, new QSystemUserSession(instance)), () -> result.set(work.run()));
return (result.get());
}
/*******************************************************************************
** Atomic QQQ action transaction, rolled back on failure.
*******************************************************************************/
public <T> T transaction(Work<T> work) throws Exception
{
return (asSystem(() ->
{
try(QBackendTransaction transaction = QBackendTransaction.openFor(new QueryInput("db_perf_snapshot")))
{
AtomicReference<T> result = new AtomicReference<>();
try
{
QContext.withTemporaryContext(new CapturedContext(instance, new QSystemUserSession(instance), transaction, null), () -> result.set(work.run()));
transaction.commit();
return (result.get());
}
catch(Exception e)
{
transaction.rollback();
throw (e);
}
}
}));
}
/*******************************************************************************
** Called by the launcher, or explicitly by hosts without a launcher.
*******************************************************************************/
public synchronized void startAttribution()
{
if(attributionFlusher == null && Boolean.TRUE.equals(config.getAttributionEnabled()))
{
DbPerfRuntime previous = ACTIVE_ATTRIBUTION.getAndSet(this);
if(previous != null && previous != this)
{
previous.stopAttribution();
}
attributionFlusher = new AttributionFlusher(this);
attributionFlusher.start();
}
}
/*******************************************************************************
** Stop the per-instance listener and its daemon; safe to call repeatedly.
*******************************************************************************/
public synchronized void stopAttribution()
{
ACTIVE_ATTRIBUTION.compareAndSet(this, null);
if(attributionFlusher != null)
{
attributionFlusher.close();
attributionFlusher = null;
}
}
/*******************************************************************************
** Checked service operation.
*******************************************************************************/
public interface Work<T>
{
/*******************************************************************************
** Execute the operation.
*******************************************************************************/
T run() throws Exception;
}
}
@@ -0,0 +1,301 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.alerts;
import java.time.Instant;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.analyze.RuleContext;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qqq.backend.core.alerts.AlertRecordRef;
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
** Optional QAlerts signals from committed observations. Alert delivery cannot
** turn a successfully collected snapshot into a failed collection.
*******************************************************************************/
public class SnapshotEvaluator
{
public static final String COLLECTION_FAILED = "dbperf.collection-failed";
public static final String QUERY_SPIKE = "dbperf.query-spike";
public static final String CONNECTION_SATURATION = "dbperf.connection-saturation";
public static final String BLOCKED_CHAIN = "dbperf.blocked-chain";
private static final QLogger LOG = QLogger.getLogger(SnapshotEvaluator.class);
private final DbPerfRuntime runtime;
/*******************************************************************************
** Instance-bound evaluator; the core resolves the optional alert sink.
*******************************************************************************/
public SnapshotEvaluator(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Alert types are discoverable even without an installed Alerts QBit.
*******************************************************************************/
public static void register(QInstance instance, DbPerfQBitConfig config)
{
Map<String, String> labels = Map.of(COLLECTION_FAILED, "Database performance collection failed", QUERY_SPIKE, "Database query time spike",
CONNECTION_SATURATION, "Database connections saturated", BLOCKED_CHAIN, "Database blocking chain");
Map<String, String> descriptions = Map.of(COLLECTION_FAILED, "Collection failed; resolves after the next OK snapshot.",
QUERY_SPIKE, "Interval query time exceeds its seven-day median and configured floor.",
CONNECTION_SATURATION, "Observed connections reached the configured percentage of capacity.",
BLOCKED_CHAIN, "A blocked session exceeded the configured duration.");
for(String type : List.of(COLLECTION_FAILED, QUERY_SPIKE, CONNECTION_SATURATION, BLOCKED_CHAIN))
{
AlertsInstanceMetaData.ofOrWithNew(instance).withAlertType(new QAlertTypeMetaData().withName(type).withLabel(labels.get(type))
.withDescription(descriptions.get(type)).withSourceQBitName("db-performance")
.withDefaultSeverity(type.equals(COLLECTION_FAILED) ? AlertSeverity.ERROR : AlertSeverity.WARNING)
.withSubjectTableName(type.equals(QUERY_SPIKE) ? "db_perf_statement" : "db_perf_snapshot")
.withVisibilityPermission(config.getPermissionBaseName() + ".view"));
}
}
/*******************************************************************************
** A failed evaluation is logged without SQL, credentials or exception text.
*******************************************************************************/
public void evaluate(QRecord snapshot)
{
try
{
runtime.asSystem(() ->
{
evaluateStored(snapshot);
return (null);
});
}
catch(Exception e)
{
LOG.warn("Database performance alert evaluation could not read stored observations");
}
}
/*******************************************************************************
** Read the current settings every pass. Failed/partial collection cannot
** clear connection, blocking or query signals without fresh measurements.
*******************************************************************************/
private void evaluateStored(QRecord snapshot) throws Exception
{
String backend = snapshot.getValueString("backendName");
QRecord settings = DbPerfStore.first("db_perf_settings", DbPerfStore.eq("backendName", backend));
if(settings == null || !Boolean.TRUE.equals(settings.getValueBoolean("alertsEnabled")))
{
return;
}
String status = snapshot.getValueString("status");
if("FAILED".equals(status))
{
signal(COLLECTION_FAILED, backend, null, true, "Database performance collection failed", snapshot.getValueString("errorMessage"), snapshot, "db_perf_snapshot");
return;
}
if("OK".equals(status))
{
signal(COLLECTION_FAILED, backend, null, false, "Collection recovered", "An OK snapshot was collected", snapshot, "db_perf_snapshot");
}
else if(!"PARTIAL".equals(status) || snapshot.getValue("connectionsTotal") == null)
{
return;
}
double maximum = RuleContext.number(snapshot, "maxConnections");
if(maximum > 0)
{
double used = 100 * RuleContext.number(snapshot, "connectionsTotal") / maximum;
signal(CONNECTION_SATURATION, backend, null, used >= RuleContext.number(settings, "connectionAlertPercent"),
"Database connection capacity", used + "% of " + maximum + " connections", snapshot, "db_perf_snapshot");
}
Instant now = snapshot.getValueInstant("capturedAt");
RuleContext ctx = new RuleContext(runtime, backend, new QRecord(), settings, now);
List<QRecord> activity = DbPerfStore.query("db_perf_activity_sample", DbPerfStore.eq("snapshotId", snapshot.getValueInteger("id")));
long blocked = activity.stream().filter(r -> "BLOCKED".equals(r.getValueString("reason")))
.filter(r -> Math.max(RuleContext.number(r, "querySeconds"), RuleContext.number(r, "xactSeconds")) >= RuleContext.number(settings, "blockedAlertSeconds")).count();
signal(BLOCKED_CHAIN, backend, null, blocked > 0, "Database blocking chain", blocked + " blocked sessions exceed the duration threshold", snapshot, "db_perf_snapshot");
if("OK".equals(status) || "NO_READ_ALL_STATS".equals(settings.getValueString("statsAccess")))
{
spikes(ctx, snapshot);
}
}
/*******************************************************************************
** A baseline excludes the current observation and needs twelve prior samples.
** Missing top-N entries clear after one hour, including after a restart.
*******************************************************************************/
private void spikes(RuleContext ctx, QRecord snapshot) throws Exception
{
Map<Integer, StatementHistory> history = history(ctx, snapshot.getValueInteger("id"));
for(QRecord statement : ctx.identities("statement"))
{
StatementHistory samples = history.getOrDefault(statement.getValueInteger("id"), new StatementHistory());
if(samples.current == null)
{
Instant last = samples.last == null ? Instant.EPOCH : samples.last;
if(!last.plusSeconds(3600).isAfter(ctx.now))
{
signal(QUERY_SPIKE, ctx.backend, statement.getValueString("id"), false, "Query absent", "Statement absent from snapshots for at least one hour", statement, "db_perf_statement");
}
continue;
}
double[] previous = samples.sortedPrevious();
if(previous.length < 12)
{
if(samples.current < RuleContext.number(ctx.settings, "spikeMinTotalMs"))
{
signal(QUERY_SPIKE, ctx.backend, statement.getValueString("id"), false, "Query recovered", "Interval time is below the configured floor", statement, "db_perf_statement");
}
continue;
}
int middle = previous.length / 2;
double median = previous.length % 2 == 0 ? (previous[middle - 1] + previous[middle]) / 2 : previous[middle];
double time = samples.current;
boolean spike = time >= RuleContext.number(ctx.settings, "spikeMinTotalMs") && time >= median * RuleContext.number(ctx.settings, "spikeFactor");
String normalized = SqlNormalizer.normalize(statement.getValueString("queryText"));
String text = normalized.substring(0, Math.min(200, normalized.length()));
signal(QUERY_SPIKE, ctx.backend, statement.getValueString("id"), spike, "Database query time spike",
"Statement " + statement.getValueInteger("id") + ": " + time + " ms; prior median " + median + " ms. " + text, statement, "db_perf_statement");
}
}
/*******************************************************************************
** Seven days of per-statement interval times, read in keyset pages and kept
** as primitive doubles. kof22 writes ~33,000 query snapshots a day, so the
** window is ~230,000 rows; holding them as QRecords on every snapshot
** evaluation cost hundreds of MB of worker heap.
*******************************************************************************/
static Map<Integer, StatementHistory> history(RuleContext ctx, Integer currentSnapshotId) throws Exception
{
Set<Integer> valid = ctx.window("snapshot", 7).stream().filter(r -> List.of("OK", "PARTIAL").contains(r.getValueString("status")))
.map(r -> r.getValueInteger("id")).collect(Collectors.toSet());
QQueryFilter filter = new QQueryFilter(new QFilterCriteria("capturedAt", QCriteriaOperator.GREATER_THAN, ctx.now.minusSeconds(7 * 86400L)),
new QFilterCriteria("capturedAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, ctx.now));
Map<Integer, StatementHistory> history = new HashMap<>();
DbPerfStore.forEachPage("db_perf_query_snapshot", filter, DbPerfStore.BATCH_SIZE, page ->
{
for(QRecord row : page)
{
Integer snapshotId = row.getValueInteger("snapshotId");
if(!valid.contains(snapshotId))
{
continue;
}
StatementHistory statement = history.computeIfAbsent(row.getValueInteger("statementId"), ignored -> new StatementHistory());
Instant at = row.getValueInstant("capturedAt");
double totalMs = RuleContext.number(row, "totalMs");
if(statement.last == null || at.isAfter(statement.last))
{
statement.last = at;
}
if(snapshotId.equals(currentSnapshotId) && statement.current == null)
{
statement.current = totalMs;
}
if(at.isBefore(ctx.now))
{
statement.add(totalMs);
}
}
});
return (history);
}
/*******************************************************************************
** One statement's samples: the current interval, the latest capture time
** and earlier interval times.
*******************************************************************************/
static final class StatementHistory
{
private Double current;
private Instant last;
private double[] previous = new double[16];
private int size;
/***************************************************************************
** Append an earlier interval time.
***************************************************************************/
void add(double value)
{
if(size == previous.length)
{
previous = Arrays.copyOf(previous, size * 2);
}
previous[size++] = value;
}
/***************************************************************************
** Earlier interval times, ascending.
***************************************************************************/
double[] sortedPrevious()
{
double[] sorted = Arrays.copyOf(previous, size);
Arrays.sort(sorted);
return (sorted);
}
}
/*******************************************************************************
** Stable backend/subject keys make raises and resolutions restart-safe.
*******************************************************************************/
private static void signal(String type, String backend, String subject, boolean raised, String title, String summary, QRecord row, String table)
{
String key = type.replace("dbperf.", "db_perf.").replace('-', '_') + ":" + backend + (subject == null ? "" : ":" + subject);
if(raised)
{
QAlerts.raise(new AlertSignal(type).withDedupKey(key).withTitle(title).withSummary(summary)
.withRecordRef(AlertRecordRef.of(table, row.getValueInteger("id"))));
}
else
{
QAlerts.resolve(type, key, "Condition no longer detected");
}
}
}
@@ -0,0 +1,102 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.time.Clock;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfSettingsService;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Each rule has its own failure boundary and atomic finding reconciliation.
*******************************************************************************/
public class AnalyzeService
{
private final DbPerfRuntime runtime;
private final Clock clock;
private final List<DbPerfRule> rules;
/*******************************************************************************
** All seeded rules, using the production clock.
*******************************************************************************/
public AnalyzeService(DbPerfRuntime runtime)
{
this(runtime, Clock.systemUTC(), Arrays.asList(StoredRule.values()));
}
/*******************************************************************************
** Explicit rules permit isolation tests and future provider-independent rules.
*******************************************************************************/
public AnalyzeService(DbPerfRuntime runtime, Clock clock, List<DbPerfRule> rules)
{
this.runtime = runtime;
this.clock = clock;
this.rules = List.copyOf(rules);
}
/*******************************************************************************
** Error text is sanitized so no database exception leaks SQL or literals.
*******************************************************************************/
public record Result(String ruleCode, FindingService.Counts counts, String error)
{
}
/*******************************************************************************
** Disabled rules are neither evaluated nor reconciled.
*******************************************************************************/
public List<Result> run(String backend) throws Exception
{
if(!runtime.getConfig().getBackendName().equals(backend))
{
throw (new IllegalArgumentException("Backend is not configured for this runtime"));
}
QRecord settings = new DbPerfSettingsService(runtime).ensure(backend);
List<Result> results = new ArrayList<>();
for(DbPerfRule rule : rules)
{
try
{
QRecord row = runtime.asSystem(() -> DbPerfStore.first("db_perf_rule", DbPerfStore.eq("code", rule.code())));
if(row == null || !Boolean.TRUE.equals(row.getValueBoolean("enabled")))
{
continue;
}
List<FindingDraft> drafts = runtime.asSystem(() -> rule.evaluate(new RuleContext(runtime, backend, row, settings, clock.instant())));
results.add(new Result(rule.code(), new FindingService(runtime, clock).apply(backend, rule.code(), drafts), null));
}
catch(Exception e)
{
results.add(new Result(rule.code(), new FindingService.Counts(0, 0, 0), "Evaluation failed (" + e.getClass().getSimpleName() + ")"));
}
}
return (results);
}
}
@@ -0,0 +1,42 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.util.List;
/*******************************************************************************
** A suggestion reads collected evidence only; it never probes the monitored DB.
*******************************************************************************/
public interface DbPerfRule
{
/*******************************************************************************
** Seeded rule identity.
*******************************************************************************/
String code();
/*******************************************************************************
** Evaluate the configured thresholds against stored observations.
*******************************************************************************/
List<FindingDraft> evaluate(RuleContext ctx) throws Exception;
}
@@ -0,0 +1,68 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.math.BigDecimal;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Immutable proposal. The lifecycle service owns identity, timestamps and state.
*******************************************************************************/
public record FindingDraft(String ruleCode, String dedupeKey, String severity, double impact,
String title, String summary, String evidence, String recommendation,
String migrationSnippet, String subjectTable, Integer statementId,
Integer tableStatId, Integer indexStatId, String unitName)
{
/*******************************************************************************
** Rank even small findings, with impact clamped to a finite fraction.
*******************************************************************************/
public BigDecimal score()
{
int weight = switch(severity)
{
case "INFO" -> 1;
case "WARNING" -> 2;
case "ERROR" -> 3;
case "CRITICAL" -> 4;
default -> throw new IllegalArgumentException("Unknown finding severity");
};
if(!Double.isFinite(impact))
{
throw (new IllegalArgumentException("Impact must be finite"));
}
return (BigDecimal.valueOf(weight * (0.1 + Math.max(0, Math.min(1, impact)))));
}
/*******************************************************************************
** Persist only the generated evidence; do not overwrite human status fields.
*******************************************************************************/
public QRecord record()
{
return (new QRecord().withValue("ruleCode", ruleCode).withValue("dedupeKey", dedupeKey)
.withValue("severity", severity).withValue("score", score()).withValue("title", title)
.withValue("summary", summary).withValue("evidence", evidence).withValue("recommendation", recommendation)
.withValue("migrationSnippet", migrationSnippet).withValue("subjectTable", subjectTable)
.withValue("statementId", statementId).withValue("tableStatId", tableStatId).withValue("indexStatId", indexStatId).withValue("unitName", unitName));
}
}
@@ -0,0 +1,176 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.time.Clock;
import java.time.Instant;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryLockMode;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
/*******************************************************************************
** Atomic reconciliation of one rule. Human acknowledgement and ignore survive
** recurrence; only resolved findings reopen. Disabled rules are entirely inert.
*******************************************************************************/
public class FindingService
{
private final DbPerfRuntime runtime;
private final Clock clock;
/*******************************************************************************
** Production clock.
*******************************************************************************/
public FindingService(DbPerfRuntime runtime)
{
this(runtime, Clock.systemUTC());
}
/*******************************************************************************
** Testable observation time.
*******************************************************************************/
public FindingService(DbPerfRuntime runtime, Clock clock)
{
this.runtime = runtime;
this.clock = clock;
}
/*******************************************************************************
** Counts describe actual writes, not the size of the input list.
*******************************************************************************/
public record Counts(int created, int updated, int resolved)
{
}
/*******************************************************************************
** Validate scope before any mutations; a failed rule never clears findings.
*******************************************************************************/
public Counts apply(String backend, String ruleCode, List<FindingDraft> drafts) throws Exception
{
if(!runtime.getConfig().getBackendName().equals(backend))
{
throw (new IllegalArgumentException("Backend is not configured for this runtime"));
}
synchronized(runtime)
{
return (runtime.transaction(() -> reconcile(backend, ruleCode, drafts)));
}
}
/*******************************************************************************
** Serialize analyzer and human status writes across host instances. A busy
** rule reports a retryable error instead of overwriting a concurrent Ignore.
** Call inside the runtime transaction and synchronized(runtime) boundary.
*******************************************************************************/
public static QRecord lockRule(DbPerfRuntime runtime, String code) throws Exception
{
RDBMSBackendMetaData backend = (RDBMSBackendMetaData) runtime.getInstance().getBackend(runtime.getConfig().getBackendName());
if(backend.getActionStrategy().supportsForUpdateSkipLocked())
{
List<QRecord> rows = new QueryAction().execute(new QueryInput("db_perf_rule").withFilter(DbPerfStore.eq("code", code))
.withTransaction(QContext.getQBackendTransaction()).withLockMode(QueryLockMode.FOR_UPDATE_SKIP_LOCKED)).getRecords();
if(rows.isEmpty() && DbPerfStore.first("db_perf_rule", DbPerfStore.eq("code", code)) != null)
{
throw (new IllegalStateException("Rule is being updated; retry analysis or the finding action"));
}
return (rows.isEmpty() ? null : rows.getFirst());
}
return (DbPerfStore.first("db_perf_rule", DbPerfStore.eq("code", code)));
}
/*******************************************************************************
** Re-read the enabled flag in the transaction, preserving administrator edits.
*******************************************************************************/
private Counts reconcile(String backend, String ruleCode, List<FindingDraft> drafts) throws Exception
{
QRecord rule = lockRule(runtime, ruleCode);
if(rule == null || !Boolean.TRUE.equals(rule.getValueBoolean("enabled")))
{
return (new Counts(0, 0, 0));
}
Map<String, QRecord> existing = new HashMap<>();
for(QRecord row : RuleContext.read("db_perf_finding", DbPerfStore.eq("backendName", backend)
.withCriteria(new QFilterCriteria("ruleCode", QCriteriaOperator.EQUALS, ruleCode))))
{
existing.put(row.getValueString("dedupeKey"), row);
}
Instant now = clock.instant();
Set<String> seen = new HashSet<>();
int created = 0;
int updated = 0;
int resolved = 0;
for(FindingDraft draft : drafts)
{
if(!ruleCode.equals(draft.ruleCode()) || !draft.dedupeKey().startsWith(ruleCode + ":" + backend + ":") || !seen.add(draft.dedupeKey()))
{
throw (new IllegalArgumentException("Invalid or duplicate finding identity"));
}
QRecord old = existing.get(draft.dedupeKey());
QRecord row = draft.record().withValue("backendName", backend).withValue("ruleId", rule.getValueInteger("id"))
.withValue("lastSeenAt", now);
if(old == null)
{
row.withValue("status", "OPEN").withValue("firstSeenAt", now).withValue("occurrences", 1);
DbPerfStore.insert("db_perf_finding", List.of(row));
created++;
}
else
{
row.withValue("id", old.getValueInteger("id")).withValue("occurrences", (int) RuleContext.number(old, "occurrences") + 1);
if("RESOLVED".equals(old.getValueString("status")))
{
row.withValue("status", "OPEN").withValue("resolvedAt", null).withValue("statusNote", "Detected again").withValue("statusChangedBy", null);
}
DbPerfStore.update("db_perf_finding", List.of(row));
updated++;
}
}
for(QRecord old : existing.values())
{
if(!seen.contains(old.getValueString("dedupeKey")) && List.of("OPEN", "ACKNOWLEDGED").contains(old.getValueString("status")))
{
DbPerfStore.update("db_perf_finding", List.of(new QRecord().withValue("id", old.getValueInteger("id"))
.withValue("status", "RESOLVED").withValue("statusNote", "No longer detected").withValue("resolvedAt", now).withValue("statusChangedBy", null)));
resolved++;
}
}
return (new Counts(created, updated, resolved));
}
}
@@ -0,0 +1,125 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.time.Instant;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Reviewable Liquibase text only. These suggestions are never executed.
*******************************************************************************/
public final class MigrationSnippets
{
private static final Set<String> RESERVED = Set.of("user", "order", "group", "select", "table", "index", "where", "from", "to", "primary", "constraint", "limit", "offset", "references", "column", "default", "check", "unique", "all", "and", "or", "not", "null", "true", "false", "case", "when", "end", "join", "on", "in", "as", "by", "values", "grant", "current_user", "authorization", "returning",
"analyse", "analyze", "array", "asc", "asymmetric", "both", "cast", "collate", "collation", "concurrently", "current_catalog", "current_date", "current_role", "current_schema", "current_time", "current_timestamp",
"desc", "distinct", "do", "else", "except", "fetch", "for", "foreign", "freeze", "ilike", "initially", "intersect", "into", "is", "isnull", "lateral", "leading", "like", "localtime", "localtimestamp", "natural", "notnull",
"only", "placing", "session_user", "similar", "some", "symmetric", "system_user", "tablesample", "then", "trailing", "union", "using", "variadic", "verbose", "window", "with");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private MigrationSnippets()
{
}
/*******************************************************************************
** Quote identifiers with punctuation, mixed case, or reserved spelling.
*******************************************************************************/
public static String quote(String name)
{
return (name.matches("[a-z_][a-z0-9_]*") && !RESERVED.contains(name) ? name : "\"" + name.replace("\"", "\"\"") + "\"");
}
/*******************************************************************************
** ASCII names are byte-bounded as well as character-bounded for PostgreSQL.
*******************************************************************************/
public static String indexName(String table, List<String> columns)
{
String raw = "idx_" + table + "_" + String.join("_", columns);
String safe = raw.toLowerCase(Locale.ROOT).replaceAll("[^a-z0-9_]", "_");
return (safe.length() <= 63 && safe.equals(raw) ? safe : safe.substring(0, Math.min(54, safe.length())) + "_" + String.format("%08x", raw.hashCode()));
}
/*******************************************************************************
** Create concurrently, with a matching schema-qualified rollback.
*******************************************************************************/
public static String create(Instant now, QRecord table, List<String> columns, String reason)
{
String name = indexName(table.getValueString("tableName"), columns);
String schema = quote(table.getValueString("schemaName"));
return (header(now, name, "MISSING_INDEX", reason)
+ "CREATE INDEX CONCURRENTLY IF NOT EXISTS " + quote(name) + " ON " + schema + "." + quote(table.getValueString("tableName"))
+ " (" + columns.stream().map(MigrationSnippets::quote).collect(Collectors.joining(", ")) + ");\n"
+ "-- rollback DROP INDEX CONCURRENTLY IF EXISTS " + schema + "." + quote(name) + ";\n");
}
/*******************************************************************************
** Preserve the collected definition in rollback; no guessed index structure.
*******************************************************************************/
public static String drop(Instant now, QRecord index, String rule, String reason)
{
String definition = index.getValueString("definition");
if(definition == null || !definition.matches("(?is)^CREATE\\s+(UNIQUE\\s+)?INDEX\\s+.*"))
{
return (null);
}
String rollback = definition.replaceFirst("(?i)^(CREATE\\s+(?:UNIQUE\\s+)?INDEX)\\s+(?:CONCURRENTLY\\s+)?", "$1 CONCURRENTLY ").replaceAll("[\\r\\n]+", " ").replaceAll(";+$", "");
return (header(now, index.getValueString("indexName"), rule, reason)
+ "DROP INDEX CONCURRENTLY IF EXISTS " + quote(index.getValueString("schemaName")) + "." + quote(index.getValueString("indexName")) + ";\n"
+ "-- rollback " + rollback + ";\n");
}
/*******************************************************************************
** Optional tuning suggestion; rollback resets the override to server defaults.
*******************************************************************************/
public static String vacuum(Instant now, QRecord table, String reason)
{
String target = quote(table.getValueString("schemaName")) + "." + quote(table.getValueString("tableName"));
return (header(now, table.getValueString("tableName"), "BLOAT", reason)
+ "ALTER TABLE " + target + " SET (autovacuum_vacuum_scale_factor = 0.05);\n"
+ "-- rollback ALTER TABLE " + target + " RESET (autovacuum_vacuum_scale_factor);\n");
}
/*******************************************************************************
** A stable daily changeset identity and a single-line evidence comment.
*******************************************************************************/
private static String header(Instant now, String subject, String rule, String reason)
{
String date = DateTimeFormatter.ofPattern("yyyyMMdd").withZone(ZoneOffset.UTC).format(now);
return ("-- liquibase formatted sql\n-- changeset db-performance:" + date + "-" + rule.toLowerCase(Locale.ROOT) + "-" + subject.replaceAll("[^a-zA-Z0-9_-]", "_")
+ " runInTransaction:false\n-- comment: Suggested by Database Performance (" + rule + "): " + reason.replaceAll("[\\r\\n]+", " ") + "\n");
}
}
@@ -0,0 +1,189 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
/*******************************************************************************
** Conservative tokenizer for simple WHERE equality/IN and equijoins. Complex
** subqueries and expressions are deliberately left for the Query plan action.
*******************************************************************************/
public final class PredicateParser
{
private static final Pattern TOKEN = Pattern.compile("\"(?:\"\"|[^\"])*\"|[a-zA-Z_][a-zA-Z_0-9$]*|\\?|[^\\s]");
private static final Set<String> KEYWORDS = Set.of("where", "join", "inner", "outer", "left", "right", "full", "cross", "on", "group", "order", "limit", "having", "set", "returning", "union");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private PredicateParser()
{
}
/*******************************************************************************
** Decode SQL identifiers; unquoted names fold to lower case.
*******************************************************************************/
public static String identifier(String token)
{
return (token.startsWith("\"") ? token.substring(1, token.length() - 1).replace("\"\"", "\"") : token.toLowerCase(Locale.ROOT));
}
/*******************************************************************************
** Only simple column indexes can be compared as prefixes.
*******************************************************************************/
public static List<String> indexColumns(String columns)
{
if(columns == null || columns.isBlank())
{
return (List.of());
}
List<String> result = new ArrayList<>();
Matcher matcher = TOKEN.matcher(columns);
boolean wantColumn = true;
while(matcher.find())
{
String token = matcher.group();
if(wantColumn && isIdentifier(token))
{
result.add(identifier(token));
}
else if(wantColumn || !token.equals(","))
{
return (List.of());
}
wantColumn = !wantColumn;
}
return (wantColumn ? List.of() : result);
}
/*******************************************************************************
** Resolve aliases before collecting predicates so joined tables do not lend
** their columns to the main table's suggested index.
*******************************************************************************/
public static List<String> candidates(String sql, String schema, String table)
{
List<String> tokens = new ArrayList<>();
Matcher matcher = TOKEN.matcher(SqlNormalizer.normalize(sql));
while(matcher.find())
{
tokens.add(matcher.group());
}
if(tokens.stream().filter(t -> t.equals("select")).count() > 1 || tokens.contains("union"))
{
return (List.of());
}
Map<String, Boolean> aliases = new LinkedHashMap<>();
for(int i = 0; i + 1 < tokens.size(); i++)
{
if(Set.of("from", "join", "update").contains(tokens.get(i)) && isIdentifier(tokens.get(i + 1)))
{
int end = i + 2;
String relation = identifier(tokens.get(i + 1));
boolean schemaMatches = true;
if(end + 1 < tokens.size() && tokens.get(end).equals("."))
{
schemaMatches = relation.equals(schema);
relation = identifier(tokens.get(end + 1));
end += 2;
}
boolean target = schemaMatches && relation.equals(table);
aliases.put(relation, target);
if(end < tokens.size() && tokens.get(end).equals("as"))
{
end++;
}
if(end < tokens.size() && isIdentifier(tokens.get(end)) && !KEYWORDS.contains(tokens.get(end)))
{
aliases.put(identifier(tokens.get(end)), target);
}
}
}
Set<String> result = new LinkedHashSet<>();
boolean predicate = false;
for(int i = 0; i < tokens.size(); i++)
{
String token = tokens.get(i);
if(token.equals("where") || token.equals("on"))
{
predicate = true;
continue;
}
if(KEYWORDS.contains(token) && !token.equals("on"))
{
predicate = false;
}
if(!predicate || !isIdentifier(token) || i > 0 && tokens.get(i - 1).equals("."))
{
continue;
}
int end = i + 1;
String column = identifier(token);
boolean belongs = !aliases.isEmpty() && aliases.values().stream().allMatch(Boolean::booleanValue);
if(end + 1 < tokens.size() && tokens.get(end).equals("."))
{
belongs = Boolean.TRUE.equals(aliases.get(column));
column = identifier(tokens.get(end + 1));
end += 2;
}
if(end + 1 >= tokens.size())
{
continue;
}
String operator = tokens.get(end);
String right = tokens.get(end + 1);
boolean joinedColumn = end + 3 < tokens.size() && tokens.get(end + 2).equals(".") && aliases.containsKey(identifier(right));
if(belongs && (operator.equals("in") && right.equals("(") || operator.equals("=") && (right.equals("?") || joinedColumn)))
{
result.add(column);
}
if(operator.equals("=") && end + 3 < tokens.size() && tokens.get(end + 2).equals(".")
&& Boolean.TRUE.equals(aliases.get(identifier(right))))
{
result.add(identifier(tokens.get(end + 3)));
}
}
return (List.copyOf(result));
}
/*******************************************************************************
** Tokens are already balanced quoted identifiers or simple words.
*******************************************************************************/
private static boolean isIdentifier(String token)
{
return (token.startsWith("\"") || token.matches("[a-zA-Z_][a-zA-Z_0-9$]*"));
}
}
@@ -0,0 +1,258 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
/*******************************************************************************
** Backend-scoped, time-bounded reads of the qbit's own tables. Ratios in storage
** are percentages. Attribution belongs to the single configured backend.
*******************************************************************************/
public class RuleContext
{
public final DbPerfRuntime runtime;
public final String backend;
public final QRecord rule;
public final QRecord settings;
public final Instant now;
private final Map<String, List<QRecord>> cache = new HashMap<>();
/*******************************************************************************
** An explicit observation time makes windows repeatable in tests and runs.
*******************************************************************************/
public RuleContext(DbPerfRuntime runtime, String backend, QRecord rule, QRecord settings, Instant now)
{
if(!runtime.getConfig().getBackendName().equals(backend))
{
throw (new IllegalArgumentException("Backend is not configured for this runtime"));
}
this.runtime = runtime;
this.backend = backend;
this.rule = rule;
this.settings = settings;
this.now = now;
}
/*******************************************************************************
** Seeded thresholds remain editable by the administrator.
*******************************************************************************/
public double threshold(int ordinal)
{
return (number(rule, "threshold" + ordinal));
}
/*******************************************************************************
** Page by primary key rather than silently accepting backend row limits.
*******************************************************************************/
public static List<QRecord> read(String table, QQueryFilter filter) throws Exception
{
List<QRecord> rows = new ArrayList<>();
int last = 0;
while(true)
{
List<QRecord> page = DbPerfStore.query(table, new QQueryFilter().withSubFilter(filter)
.withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, last))
.withOrderBy(new QFilterOrderBy("id", true)).withLimit(DbPerfStore.BATCH_SIZE));
rows.addAll(page);
if(page.size() < DbPerfStore.BATCH_SIZE)
{
return (rows);
}
last = page.getLast().getValueInteger("id");
}
}
/*******************************************************************************
** Current identities, filtered by backend. Cached only for this evaluation.
*******************************************************************************/
public List<QRecord> identities(String suffix) throws Exception
{
String key = "identity:" + suffix;
if(!cache.containsKey(key))
{
cache.put(key, read("db_perf_" + suffix, DbPerfStore.eq("backendName", backend)));
}
return (cache.get(key));
}
/*******************************************************************************
** Read observations inside (now-days, now], excluding other backends through
** their snapshot foreign key. Partial snapshots still contain useful data.
*******************************************************************************/
public List<QRecord> window(String suffix, int days) throws Exception
{
String key = suffix + ":" + days;
if(!cache.containsKey(key))
{
String time = Set.of("attribution", "repeat_pattern").contains(suffix) ? "windowEnd" : "capturedAt";
QQueryFilter filter = new QQueryFilter(new QFilterCriteria(time, QCriteriaOperator.GREATER_THAN, now.minusSeconds(days * 86400L)),
new QFilterCriteria(time, QCriteriaOperator.LESS_THAN_OR_EQUALS, now));
if(suffix.equals("snapshot"))
{
filter.withCriteria(new QFilterCriteria("backendName", QCriteriaOperator.EQUALS, backend));
}
List<QRecord> rows = suffix.equals("attribution") ? attributionTotals(filter) : read("db_perf_" + suffix, filter);
if(!Set.of("snapshot", "attribution", "repeat_pattern").contains(suffix))
{
Set<Integer> ids = window("snapshot", days).stream()
.filter(r -> List.of("OK", "PARTIAL").contains(r.getValueString("status")))
.map(r -> r.getValueInteger("id")).collect(Collectors.toSet());
rows = rows.stream().filter(r -> ids.contains(r.getValueInteger("snapshotId"))).toList();
}
cache.put(key, rows);
}
return (cache.get(key));
}
/*******************************************************************************
** Most attribution groups the rules receive (prod: ~2,000 per day).
*******************************************************************************/
public static final int MAX_ATTRIBUTION_GROUPS = 50_000;
/*******************************************************************************
** Attribution summed in the database per unit, table and operation: the
** fields the rules read (statements, totalMs and the other counters), never
** the day's detail rows. A day is ~570,000 rows in kof22 and loading it made
** every hourly dbPerfAnalyze fill the worker heap (2026-10-16).
*******************************************************************************/
private static List<QRecord> attributionTotals(QQueryFilter filter) throws QException
{
List<String> dimensions = List.of("unitKind", "unitName", "tableName", "operation");
AggregateInput input = new AggregateInput("db_perf_attribution").withFilter(filter).withLimit(MAX_ATTRIBUTION_GROUPS)
.withTransaction(QContext.getQBackendTransaction());
Map<String, GroupBy> groupBys = new LinkedHashMap<>();
for(String dimension : dimensions)
{
GroupBy groupBy = new GroupBy(QFieldType.STRING, dimension);
groupBys.put(dimension, groupBy);
input.withGroupBy(groupBy);
}
Map<String, Aggregate> aggregates = new LinkedHashMap<>();
for(String counter : List.of("units", "statements", "rowsCount", "errors", "totalMs"))
{
aggregates.put(counter, new Aggregate(counter, AggregateOperator.SUM));
}
aggregates.put("maxMs", new Aggregate("maxMs", AggregateOperator.MAX));
aggregates.values().forEach(input::withAggregate);
List<QRecord> rows = new ArrayList<>();
for(AggregateResult result : new AggregateAction().execute(input).getResults())
{
QRecord row = new QRecord();
groupBys.forEach((field, groupBy) -> row.setValue(field, result.getGroupByValue(groupBy)));
aggregates.forEach((field, aggregate) -> row.setValue(field, result.getAggregateValue(aggregate)));
rows.add(row);
}
return (rows);
}
/*******************************************************************************
** Latest inventory only: retired tables and dropped indexes cannot recur.
*******************************************************************************/
public List<QRecord> inventory(String suffix) throws Exception
{
Instant latest = settings.getValueInstant("lastTableSnapshotAt");
return (identities(suffix).stream().filter(r -> latest == null || latest.equals(r.getValueInstant("lastSeenAt"))).toList());
}
/*******************************************************************************
** Scale observed table deltas to a day when the observation window is short.
** Table sampling is independent of the faster database snapshot interval.
*******************************************************************************/
public double perDay(QRecord table, String field) throws Exception
{
List<QRecord> rows = window("table_snapshot", 1).stream()
.filter(r -> table.getValueInteger("id").equals(r.getValueInteger("tableStatId"))).toList();
if(rows.isEmpty())
{
return (0);
}
Instant first = rows.stream().map(r -> r.getValueInstant("capturedAt")).min(Instant::compareTo).orElseThrow();
Instant last = rows.stream().map(r -> r.getValueInstant("capturedAt")).max(Instant::compareTo).orElseThrow();
double seconds = Math.min(86400, Duration.between(first, last).getSeconds() + Math.max(60, number(settings, "tableSnapshotMinutes") * 60));
return (rows.stream().mapToDouble(r -> number(r, field)).sum() * 86400 / seconds);
}
/*******************************************************************************
** Minimum age of the current database statistics, including retained history.
*******************************************************************************/
public double statsAgeDays() throws Exception
{
QRecord latest = DbPerfStore.first("db_perf_snapshot", DbPerfStore.eq("backendName", backend)
.withCriteria(new QFilterCriteria("rawStatsReset", QCriteriaOperator.IS_NOT_BLANK))
.withOrderBy(new QFilterOrderBy("capturedAt", false)));
Instant start = latest == null ? null : latest.getValueInstant("rawStatsReset");
if(start == null)
{
QRecord first = DbPerfStore.first("db_perf_snapshot", DbPerfStore.eq("backendName", backend)
.withOrderBy(new QFilterOrderBy("capturedAt", true)));
start = first == null ? now : first.getValueInstant("capturedAt");
}
return (Math.max(0, Duration.between(start, now).toSeconds() / 86400.0));
}
/*******************************************************************************
** Null counters mean no observed activity, never an invented measurement.
*******************************************************************************/
public static double number(QRecord row, String field)
{
return (row == null || row.getValue(field) == null ? 0 : row.getValueBigDecimal(field).doubleValue());
}
}
@@ -0,0 +1,516 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.analyze;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** The fourteen seeded rules. Every decision is based on stored collector rows;
** administrators control severities and thresholds through db_perf_rule.
*******************************************************************************/
public enum StoredRule implements DbPerfRule
{
SLOW_QUERY, MISSING_INDEX, UNUSED_INDEX, DUPLICATE_INDEX, N_PLUS_ONE, HOT_JOB,
OVER_AUDITED, BLOAT, VACUUM_STALE, POOL_SIZING, IDLE_IN_TRANSACTION,
CACHE_HIT_LOW, TEMP_SPILL, DEADLOCKS;
/*******************************************************************************
** Seed identity.
*******************************************************************************/
@Override
public String code()
{
return (name());
}
/*******************************************************************************
** Disabled evaluation is inert even when called outside AnalyzeService.
*******************************************************************************/
@Override
public List<FindingDraft> evaluate(RuleContext ctx) throws Exception
{
if(!Boolean.TRUE.equals(ctx.rule.getValueBoolean("enabled")))
{
return (List.of());
}
return (switch(this)
{
case SLOW_QUERY -> slow(ctx);
case MISSING_INDEX, OVER_AUDITED, BLOAT, VACUUM_STALE -> tables(ctx);
case UNUSED_INDEX, DUPLICATE_INDEX -> indexes(ctx);
case N_PLUS_ONE -> repeats(ctx);
case HOT_JOB -> jobs(ctx);
case POOL_SIZING -> pools(ctx);
case IDLE_IN_TRANSACTION -> idle(ctx);
case CACHE_HIT_LOW, DEADLOCKS, TEMP_SPILL -> database(ctx);
});
}
/*******************************************************************************
** Mean and share use the collector's 24-hour aggregates, excluding stale rows.
*******************************************************************************/
private List<FindingDraft> slow(RuleContext ctx) throws Exception
{
List<FindingDraft> findings = new ArrayList<>();
for(QRecord row : ctx.identities("statement"))
{
if(row.getValueInstant("lastSeenAt") == null || row.getValueInstant("lastSeenAt").isBefore(ctx.now.minusSeconds(86400)))
{
continue;
}
double share = n(row, "shareOfDbTime24h");
if(n(row, "meanMs24h") >= ctx.threshold(1) || share >= ctx.threshold(2))
{
String sql = URLEncoder.encode("EXPLAIN " + row.getValueString("queryText"), StandardCharsets.UTF_8).replace("+", "%20");
String recommendation = "Open the [Query plan action](/record/db_perf_statement/" + row.getValueInteger("id")
+ ") to inspect this query, or [Explain in SQL Workbench](" + ctx.runtime.getConfig().getSqlWorkbenchRunnerPath() + "?sql=" + sql + "). Review scan and join costs before changing indexes.";
findings.add(draft(ctx, row.getValueString("id"), share / 100, "High query execution cost over 24 hours",
evidence("Calls", n(row, "calls24h"), "Mean ms", n(row, "meanMs24h"), "Total ms", n(row, "totalMs24h"), "DB time %", share), recommendation, null, row, "statement", null));
}
}
return (findings);
}
/*******************************************************************************
** Inventory-based rules share the same current table list and daily deltas.
*******************************************************************************/
private List<FindingDraft> tables(RuleContext ctx) throws Exception
{
List<FindingDraft> findings = new ArrayList<>();
for(QRecord table : ctx.inventory("table_stat"))
{
String name = table.getValueString("tableName");
String subject = table.getValueString("schemaName") + "." + name;
if(this == MISSING_INDEX)
{
double reads = ctx.perDay(table, "seqTupRead");
if(n(table, "seqScanShare") < ctx.threshold(1) || reads < ctx.threshold(2))
{
continue;
}
Set<String> columns = new LinkedHashSet<>();
for(QRecord statement : ctx.identities("statement").stream()
.filter(s -> name.equals(s.getValueString("mainTable")) || subject.equals(s.getValueString("mainTable")))
.filter(s -> n(s, "totalMs24h") > 0 && s.getValueInstant("lastSeenAt") != null && !s.getValueInstant("lastSeenAt").isBefore(ctx.now.minusSeconds(86400)))
.sorted(Comparator.comparingDouble((QRecord s) -> n(s, "totalMs24h")).reversed()).limit(20).toList())
{
columns.addAll(PredicateParser.candidates(statement.getValueString("queryText"), table.getValueString("schemaName"), name));
}
for(QRecord index : ctx.inventory("index_stat"))
{
List<String> indexed = PredicateParser.indexColumns(index.getValueString("columns"));
if(sameTable(table, index) && Boolean.TRUE.equals(index.getValueBoolean("isValid")) && !indexed.isEmpty() && plainIndex(index))
{
columns.remove(indexed.getFirst());
}
}
if(!columns.isEmpty())
{
String reason = reads + " sequential tuples/day; " + n(table, "seqScanShare") + "% sequential scans";
findings.add(draft(ctx, subject, n(table, "seqScanShare") / 100, reason,
evidence("Sequential scan %", n(table, "seqScanShare"), "Tuples/day", reads, "Candidate columns", String.join(", ", columns)),
"Check Query plan and selectivity for " + String.join(", ", columns) + ". Review the suggested column order and migration before committing it.",
MigrationSnippets.create(ctx.now, table, List.copyOf(columns), reason), table, "table", null));
}
}
else if(this == BLOAT && n(table, "deadRatio") >= ctx.threshold(1) && n(table, "nDeadTup") >= ctx.threshold(2))
{
String reason = n(table, "nDeadTup") + " dead tuples (" + n(table, "deadRatio") + "%)";
findings.add(draft(ctx, subject, n(table, "deadRatio") / 100, reason,
evidence("Dead tuples", n(table, "nDeadTup"), "Live tuples", n(table, "nLiveTup"), "Dead %", n(table, "deadRatio")),
"Review long transactions and autovacuum capacity. Consider a lower per-table vacuum scale factor; the optional rollback resets it to the server default.",
MigrationSnippets.vacuum(ctx.now, table, reason), table, "table", null));
}
else if(this == VACUUM_STALE)
{
Instant vacuum = table.getValueInstant("lastVacuum");
Instant auto = table.getValueInstant("lastAutovacuum");
if(vacuum == null || auto != null && auto.isAfter(vacuum))
{
vacuum = auto;
}
double age = vacuum == null ? ctx.statsAgeDays() : Duration.between(vacuum, ctx.now).toSeconds() / 86400.0;
double writes = n(table, "nTupIns") + n(table, "nTupUpd") + n(table, "nTupDel");
if(writes > 0 && age > ctx.threshold(1) || n(table, "freezeAge") >= ctx.threshold(2))
{
findings.add(draft(ctx, subject, Math.min(1, n(table, "freezeAge") / Math.max(1, ctx.threshold(2))), "Vacuum maintenance needs attention",
evidence("Days since vacuum (or observation age)", age, "Writes", writes, "Freeze age", n(table, "freezeAge")),
"Review autovacuum settings and long transactions preventing vacuum. Schedule maintenance and investigate transaction freeze age.", null, table, "table", null));
}
}
else if(this == OVER_AUDITED && name.toLowerCase(Locale.ROOT).matches("(?:audit.*|.*_audit.*)"))
{
double inserts = ctx.perDay(table, "nTupIns");
if(inserts >= ctx.threshold(1))
{
Map<String, Double> units = new LinkedHashMap<>();
for(QRecord row : ctx.window("attribution", 1))
{
if((name.equals(row.getValueString("tableName")) || subject.equals(row.getValueString("tableName"))) && "INSERT".equalsIgnoreCase(row.getValueString("operation")))
{
units.merge(String.valueOf(row.getValueString("unitName")), n(row, "statements"), Double::sum);
}
}
String contributors = units.entrySet().stream().sorted(Map.Entry.<String, Double>comparingByValue().reversed()).limit(5)
.map(e -> e.getKey() + " (" + e.getValue() + " inserts)").collect(Collectors.joining(", "));
Map<String, Double> sources = new LinkedHashMap<>();
for(QRecord row : ctx.window("attribution", 1))
{
String source = row.getValueString("tableName");
if(source != null && !source.toLowerCase(Locale.ROOT).matches("(?:audit.*|.*_audit.*)")
&& units.containsKey(row.getValueString("unitName")) && Set.of("INSERT", "UPDATE", "DELETE").contains(String.valueOf(row.getValueString("operation")).toUpperCase(Locale.ROOT)))
{
sources.merge(source, n(row, "statements"), Double::sum);
}
}
String sourceTables = sources.entrySet().stream().sorted(Map.Entry.<String, Double>comparingByValue().reversed()).limit(5)
.map(e -> e.getKey() + " (" + e.getValue() + " writes)").collect(Collectors.joining(", "));
findings.add(draft(ctx, subject, Math.min(1, inserts / Math.max(1, ctx.threshold(1) * 10)), "High audit insert volume",
evidence("Audit inserts/day", inserts, "Attributed units", contributors.isEmpty() ? "Unavailable" : contributors),
"Lower AuditLevel on noisy source tables after reviewing audit requirements. " + (contributors.isEmpty() ? "Source-table attribution is unavailable." : "Largest audit-producing units: " + contributors + ".")
+ (sourceTables.isEmpty() ? "" : " Source tables with the most writes in those units: " + sourceTables + ". These are correlated writes, not measured per-source audit row counts."),
null, table, "table", null));
}
}
}
return (findings);
}
/*******************************************************************************
** Never propose dropping unique, primary, invalid or structurally incomparable
** indexes. Equivalent peers choose one survivor deterministically.
*******************************************************************************/
private List<FindingDraft> indexes(RuleContext ctx) throws Exception
{
List<QRecord> indexes = ctx.inventory("index_stat");
double bytes = indexes.stream().mapToDouble(r -> n(r, "sizeBytes")).sum();
List<FindingDraft> findings = new ArrayList<>();
for(QRecord index : indexes)
{
if(Boolean.TRUE.equals(index.getValueBoolean("isUnique")) || Boolean.TRUE.equals(index.getValueBoolean("isPrimary")) || !Boolean.TRUE.equals(index.getValueBoolean("isValid")))
{
continue;
}
String reason = null;
if(this == UNUSED_INDEX && n(index, "idxScan") == 0 && ctx.statsAgeDays() >= ctx.threshold(1) && n(index, "sizeBytes") / 1048576 >= ctx.threshold(2))
{
reason = "Zero scans over " + ctx.statsAgeDays() + " days; " + n(index, "sizeBytes") / 1048576 + " MB";
}
if(this == DUPLICATE_INDEX && plainIndex(index))
{
List<String> columns = PredicateParser.indexColumns(index.getValueString("columns"));
for(QRecord other : indexes)
{
List<String> covering = PredicateParser.indexColumns(other.getValueString("columns"));
if(!index.getValueInteger("id").equals(other.getValueInteger("id")) && Objects.equals(index.getValueInteger("tableStatId"), other.getValueInteger("tableStatId"))
&& Boolean.TRUE.equals(other.getValueBoolean("isValid")) && plainIndex(other) && !columns.isEmpty() && covering.size() >= columns.size()
&& covering.subList(0, columns.size()).equals(columns))
{
boolean exact = columns.equals(covering);
boolean unique = Boolean.TRUE.equals(other.getValueBoolean("isUnique")) || Boolean.TRUE.equals(other.getValueBoolean("isPrimary"));
if(exact && (unique || other.getValueInteger("id") < index.getValueInteger("id")) || !exact && !unique)
{
reason = "Redundant with " + other.getValueString("indexName") + ": " + other.getValueString("definition");
break;
}
}
}
}
if(reason != null)
{
findings.add(draft(ctx, index.getValueString("schemaName") + "." + index.getValueString("indexName"), n(index, "sizeBytes") / Math.max(1, bytes), reason,
evidence("Index bytes", n(index, "sizeBytes"), "Scans", n(index, "idxScan"), "Write cost", n(index, "writeCost"), "Definition", index.getValueString("definition")),
"Review rare workloads and dependencies before dropping this index. Commit the migration only after verifying the replacement and rollback.",
MigrationSnippets.drop(ctx.now, index, code(), reason), index, "index", null));
}
}
return (findings);
}
/*******************************************************************************
** Column prefixes alone cannot prove equivalence for partial/expression or
** non-btree indexes. Such structures need human review, not a drop snippet.
*******************************************************************************/
private static boolean plainIndex(QRecord index)
{
String definition = index.getValueString("definition");
if(definition == null)
{
return (false);
}
String lower = definition.toLowerCase(Locale.ROOT);
return (!PredicateParser.indexColumns(index.getValueString("columns")).isEmpty() && !lower.contains(" where ") && !lower.contains(" include ")
&& (!lower.contains(" using ") || lower.contains(" using btree ")));
}
/*******************************************************************************
** Aggregate pods/windows before applying the minimum affected-unit threshold.
*******************************************************************************/
private List<FindingDraft> repeats(RuleContext ctx) throws Exception
{
Map<String, List<QRecord>> groups = group(ctx.window("repeat_pattern", 1), "unitKind", "unitName", "fingerprint");
List<FindingDraft> findings = new ArrayList<>();
for(var entry : groups.entrySet())
{
List<QRecord> rows = entry.getValue();
double max = rows.stream().mapToDouble(r -> n(r, "maxRepeats")).max().orElse(0);
double units = rows.stream().mapToDouble(r -> n(r, "unitsAffected")).sum();
if(max >= ctx.threshold(1) && units >= ctx.threshold(2))
{
QRecord row = rows.getFirst();
findings.add(draft(ctx, entry.getKey(), Math.min(1, max / Math.max(1, ctx.threshold(1) * 10)), "Repeated queries in " + row.getValueString("unitKind") + " " + row.getValueString("unitName"),
evidence("Max repeats", max, "Units affected", units, "Total repeats", rows.stream().mapToDouble(r -> n(r, "totalRepeats")).sum()),
"Batch lookups on " + row.getValueString("tableName") + " in " + row.getValueString("unitName") + " into one query with IN (...), a QQQ QueryJoin, or a record pre-fetch.", null, row, "unit", row.getValueString("unitName")));
}
}
return (findings);
}
/*******************************************************************************
** Share is relative to all attributed work, including HTTP routes.
*******************************************************************************/
private List<FindingDraft> jobs(RuleContext ctx) throws Exception
{
List<QRecord> rows = ctx.window("attribution", 1);
double total = rows.stream().mapToDouble(r -> n(r, "totalMs")).sum();
List<FindingDraft> findings = new ArrayList<>();
for(var entry : group(rows, "unitKind", "unitName").entrySet())
{
QRecord row = entry.getValue().getFirst();
double time = entry.getValue().stream().mapToDouble(r -> n(r, "totalMs")).sum();
double share = total == 0 ? 0 : time / total;
if(List.of("SCHEDULED_JOB", "PROCESS").contains(row.getValueString("unitKind")) && total > 0 && share * 100 >= ctx.threshold(1))
{
findings.add(draft(ctx, entry.getKey(), share, "Expensive " + row.getValueString("unitKind") + " " + row.getValueString("unitName"),
evidence("Total ms", time, "Attributed time %", share * 100), "Review the schedule and batch database operations in " + row.getValueString("unitName") + ".", null, row, "unit", row.getValueString("unitName")));
}
}
return (findings);
}
/*******************************************************************************
** Per-application peaks sum users/states within each snapshot first.
*******************************************************************************/
private List<FindingDraft> pools(RuleContext ctx) throws Exception
{
double peak = 0;
double idle = 0;
double maxConnections = 0;
for(QRecord row : ctx.window("snapshot", 1))
{
if(n(row, "maxConnections") > 0)
{
peak = Math.max(peak, n(row, "connectionsTotal") / n(row, "maxConnections") * 100);
maxConnections = Math.max(maxConnections, n(row, "maxConnections"));
}
if(n(row, "connectionsTotal") > 0)
{
idle = Math.max(idle, n(row, "connectionsIdle") / n(row, "connectionsTotal") * 100);
}
}
if(maxConnections == 0 || peak < ctx.threshold(1) && idle < ctx.threshold(2))
{
return (List.of());
}
Map<String, Double> peaks = new LinkedHashMap<>();
Map<String, Double> activePeaks = new LinkedHashMap<>();
for(List<QRecord> rows : group(ctx.window("connection_sample", 1), "snapshotId", "applicationName").values())
{
String app = String.valueOf(rows.getFirst().getValueString("applicationName"));
peaks.merge(app, rows.stream().mapToDouble(r -> n(r, "connectionCount")).sum(), Math::max);
activePeaks.merge(app, rows.stream().filter(r -> "active".equals(r.getValueString("state"))).mapToDouble(r -> n(r, "connectionCount")).sum(), Math::max);
}
double desired = activePeaks.values().stream().mapToDouble(v -> Math.max(1, Math.ceil(v * 1.2))).sum();
double scale = Math.min(1, maxConnections * 0.8 / Math.max(1, desired));
String sizes = activePeaks.entrySet().stream().map(e -> e.getKey() + ": combined pool cap " + Math.max(1, (int) Math.floor(Math.max(1, Math.ceil(e.getValue() * 1.2)) * scale))
+ " across replicas (peak active " + e.getValue().intValue() + ")").collect(Collectors.joining("; "));
String applications = peaks.entrySet().stream().map(e -> e.getKey() + ": peak " + e.getValue().intValue()).collect(Collectors.joining("; "));
return (List.of(draft(ctx, "database", Math.max(peak, idle) / 100, "Review connection pool sizes",
evidence("Peak used %", peak, "Peak idle %", idle, "Max connections", maxConnections, "Application peaks", applications),
"Starting pool sizes from observed active demand: " + sizes + ". Validate under representative load and divide combined caps across replicas; reserve capacity for administration and other applications. Observed application peaks: " + applications,
null, null, "database", null)));
}
/*******************************************************************************
** One finding per application, retaining the worst observed transaction age.
*******************************************************************************/
private List<FindingDraft> idle(RuleContext ctx) throws Exception
{
List<QRecord> rows = ctx.window("activity_sample", 1).stream().filter(r -> "IDLE_IN_TRANSACTION".equals(r.getValueString("reason"))).toList();
List<FindingDraft> findings = new ArrayList<>();
for(var entry : group(rows, "applicationName").entrySet())
{
double seconds = entry.getValue().stream().mapToDouble(r -> n(r, "xactSeconds")).max().orElse(0);
if(seconds >= ctx.threshold(1))
{
String application = String.valueOf(entry.getValue().getFirst().getValueString("applicationName"));
findings.add(draft(ctx, entry.getKey(), Math.min(1, seconds / Math.max(1, ctx.threshold(1) * 10)), "Idle transaction in " + application,
evidence("Maximum transaction seconds", seconds, "Samples", entry.getValue().size()),
"Fix transaction scope in " + application + "; consider idle_in_transaction_session_timeout.", null, null, "database", application));
}
}
return (findings);
}
/*******************************************************************************
** Database counters are already deltas; temporary blocks are 8 KiB each.
*******************************************************************************/
private List<FindingDraft> database(RuleContext ctx) throws Exception
{
List<QRecord> snapshots = ctx.window("snapshot", 1).stream().filter(r -> "OK".equals(r.getValueString("status")) || this != CACHE_HIT_LOW && "PARTIAL".equals(r.getValueString("status"))).toList();
List<FindingDraft> findings = new ArrayList<>();
if(this == CACHE_HIT_LOW)
{
var average = snapshots.stream().filter(r -> r.getValue("cacheHitRatio") != null).mapToDouble(r -> n(r, "cacheHitRatio")).average();
if(average.isPresent() && average.getAsDouble() < ctx.threshold(1))
{
findings.add(draft(ctx, "database", 1 - average.getAsDouble() / 100, "Low average database cache hit ratio", evidence("Average hit %", average.getAsDouble(), "OK snapshots", snapshots.size()),
"Review query scans and the working set before increasing database memory.", null, null, "database", null));
}
}
else if(this == DEADLOCKS)
{
double count = snapshots.stream().mapToDouble(r -> n(r, "deadlocks")).sum();
if(!snapshots.isEmpty() && count >= ctx.threshold(1))
{
findings.add(draft(ctx, "database", Math.min(1, count / Math.max(1, ctx.threshold(1) * 10)), "Database deadlocks detected", evidence("Deadlocks in 24h", count),
"Use consistent lock ordering and shorter transactions. Review affected application operations.", null, null, "database", null));
}
}
else
{
double databaseMb = snapshots.stream().mapToDouble(r -> n(r, "tempBytes")).sum() / 1048576;
Map<Integer, QRecord> statements = ctx.identities("statement").stream().collect(Collectors.toMap(r -> r.getValueInteger("id"), r -> r));
for(var entry : group(ctx.window("query_snapshot", 1), "statementId").entrySet())
{
double mb = entry.getValue().stream().mapToDouble(r -> n(r, "tempBlksWritten")).sum() / 128;
if(mb >= ctx.threshold(1))
{
QRecord statement = statements.get(entry.getValue().getFirst().getValueInteger("statementId"));
findings.add(draft(ctx, entry.getKey(), mb / Math.max(1, databaseMb), "Query spilled temporary data to disk",
evidence("Query temp MB", mb, "Database temp MB", databaseMb), "Inspect sorts and hash joins in Query plan. Tune the query and review work_mem per operation, accounting for concurrent queries.", null, statement, "statement", null));
}
}
if(databaseMb >= ctx.threshold(1))
{
findings.add(draft(ctx, "database", Math.min(1, databaseMb / Math.max(1, ctx.threshold(1) * 10)), "Database temporary disk spill",
evidence("Database temp MB in 24h", databaseMb), "Review spilling queries, including those outside the stored top-N. Size work_mem with concurrency in mind.", null, null, "database", null));
}
}
return (findings);
}
/*******************************************************************************
** Delimit parts with lengths to avoid collisions between unit names.
*******************************************************************************/
private static Map<String, List<QRecord>> group(List<QRecord> rows, String... fields)
{
Map<String, List<QRecord>> groups = new LinkedHashMap<>();
for(QRecord row : rows)
{
String key = java.util.Arrays.stream(fields).map(f -> String.valueOf(row.getValue(f))).map(s -> s.length() + ":" + s).collect(Collectors.joining("|"));
groups.computeIfAbsent(key, ignored -> new ArrayList<>()).add(row);
}
return (groups);
}
/*******************************************************************************
** Shared foreign-key comparison.
*******************************************************************************/
private static boolean sameTable(QRecord table, QRecord index)
{
return (table.getValueInteger("id").equals(index.getValueInteger("tableStatId")));
}
/*******************************************************************************
** Numeric convenience keeps units visible at the comparison sites.
*******************************************************************************/
private static double n(QRecord row, String field)
{
return (RuleContext.number(row, field));
}
/*******************************************************************************
** Small Markdown table with escaped cell delimiters.
*******************************************************************************/
private static String evidence(Object... cells)
{
StringBuilder text = new StringBuilder("| Measurement | Value |\n| --- | --- |\n");
for(int i = 0; i < cells.length; i += 2)
{
text.append("| ").append(cells[i]).append(" | ").append(String.valueOf(cells[i + 1]).replace("|", "\\|").replace("\n", " ")).append(" |\n");
}
return (text.toString());
}
/*******************************************************************************
** Centralize persisted links and severity; no rule invents a fixed severity.
*******************************************************************************/
private FindingDraft draft(RuleContext ctx, String subject, double impact, String summary, String evidence, String recommendation,
String snippet, QRecord row, String kind, String unit)
{
String table = row == null ? null : row.getValueString(kind.equals("statement") ? "mainTable" : "tableName");
Integer id = row == null ? null : row.getValueInteger("id");
String key = code() + ":" + ctx.backend + ":" + subject;
if(key.length() > 300)
{
key = code() + ":" + ctx.backend + ":" + SqlNormalizer.fingerprint(subject);
}
String title = ctx.rule.getValueString("label") + ": " + (unit == null ? subject : unit);
return (new FindingDraft(code(), key, ctx.rule.getValueString("severity"), impact,
title.substring(0, Math.min(300, title.length())), summary.substring(0, Math.min(2000, summary.length())), evidence, recommendation, snippet, table,
kind.equals("statement") ? id : null, kind.equals("table") ? id : kind.equals("index") ? row.getValueInteger("tableStatId") : null,
kind.equals("index") ? id : null, unit));
}
}
@@ -0,0 +1,67 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
/*******************************************************************************
** Database Performance activity under the authenticated UI API.
*******************************************************************************/
public class ActivitySpec extends DbPerformanceJsonSpec
{
/*******************************************************************************
** Method and path relative to /qqq/v1.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withHttpMethod(HttpMethod.GET).withPath("/qbit/db-performance/activity")
.withShortSummary("Database Performance activity").withLongDescription("Requires db_performance.view and uses the caller's session."));
}
/*******************************************************************************
** Versioned response shape.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("DbPerformanceActivityV1");
}
/*******************************************************************************
** Execute in the caller's context; also directly testable without transport.
*******************************************************************************/
@Override
public Object handle(JsonInput input) throws Exception
{
return (data().activity(input.getQueryParam("backend"), true));
}
}
@@ -0,0 +1,349 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import java.math.BigDecimal;
import java.sql.Connection;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qbits.dbperformance.provider.DbStatsProvider;
import com.kingsrook.qbits.dbperformance.provider.DbStatsProviders;
import com.kingsrook.qbits.dbperformance.provider.SessionInfo;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
/*******************************************************************************
** Read models shared by the API and widgets, under the caller's QQQ permissions.
*******************************************************************************/
public class DbPerformanceData
{
private final DbPerfRuntime runtime;
/*******************************************************************************
** Resolve one configured monitored backend, never arbitrary host credentials.
*******************************************************************************/
public DbPerformanceData(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Explicit permission check even for hosts that disable core action enforcement.
*******************************************************************************/
public void requireView(String backend) throws QException
{
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".view"))
{
throw (new QPermissionDeniedException("Database Performance view permission is required."));
}
if(backend != null && !runtime.getConfig().getBackendName().equals(backend))
{
throw (new QBadRequestException("Choose the configured monitored backend."));
}
}
/*******************************************************************************
** Backend equality filter, reused by every backend-owned table.
*******************************************************************************/
public QQueryFilter backendFilter()
{
return (DbPerfStore.eq("backendName", runtime.getConfig().getBackendName()));
}
/*******************************************************************************
** Read settings without creating or changing them during a dashboard request.
*******************************************************************************/
public QRecord settings() throws QException
{
requireView(null);
return (DbPerfStore.first("db_perf_settings", backendFilter()));
}
/*******************************************************************************
** Latest collected numbers, including degraded and failed collection status.
*******************************************************************************/
public QRecord latest() throws QException
{
requireView(null);
return (DbPerfStore.first("db_perf_snapshot", backendFilter().withOrderBy(new QFilterOrderBy("capturedAt", false))));
}
/*******************************************************************************
** Snapshot deltas over the trailing day, ordered for charting.
*******************************************************************************/
public List<QRecord> snapshots() throws QException
{
requireView(null);
return (DbPerfStore.query("db_perf_snapshot", backendFilter().withCriteria(new QFilterCriteria("capturedAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, Instant.now().minusSeconds(86400)))
.withOrderBy(new QFilterOrderBy("capturedAt", true))));
}
/*******************************************************************************
** Ranked statements with literal-free text and canonical history links.
*******************************************************************************/
public List<Map<String, Object>> statements(int limit, String fingerprint) throws QException
{
requireView(null);
QQueryFilter filter = backendFilter().withOrderBy(new QFilterOrderBy("totalMs24h", false)).withLimit(limit);
if(fingerprint != null)
{
filter.withCriteria(new QFilterCriteria("fingerprint", QCriteriaOperator.EQUALS, fingerprint));
}
List<Map<String, Object>> results = new ArrayList<>();
for(QRecord record : DbPerfStore.query("db_perf_statement", filter))
{
Map<String, Object> item = fields(record, "id", "calls24h", "meanMs24h", "totalMs24h", "shareOfDbTime24h");
String text = SqlNormalizer.normalize(record.getValueString("queryText"));
item.put("queryText", fingerprint == null ? text.substring(0, Math.min(300, text.length())) : text);
item.put("historyPath", recordPath("db_perf_statement", record.getValueInteger("id")));
item.put("hitRatio", hitRatio(record));
results.add(item);
}
return (results);
}
/*******************************************************************************
** 24-hour cache hit ratio uses deltas rather than cumulative statement counters.
*******************************************************************************/
private BigDecimal hitRatio(QRecord record) throws QException
{
List<QRecord> rows = DbPerfStore.query("db_perf_query_snapshot", DbPerfStore.eq("statementId", record.getValueInteger("id"))
.withCriteria(new QFilterCriteria("capturedAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, Instant.now().minusSeconds(86400))));
double hits = rows.stream().mapToDouble(row -> number(row, "sharedBlksHit")).sum();
double reads = rows.stream().mapToDouble(row -> number(row, "sharedBlksRead")).sum();
return (hits + reads == 0 ? null : BigDecimal.valueOf(hits / (hits + reads)));
}
/*******************************************************************************
** Open and acknowledged findings stay actionable and are ordered by score.
*******************************************************************************/
public List<QRecord> findings(int limit) throws QException
{
requireView(null);
return (DbPerfStore.query("db_perf_finding", backendFilter()
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.IN, List.of("OPEN", "ACKNOWLEDGED")))
.withOrderBy(new QFilterOrderBy("score", false)).withLimit(limit)));
}
/*******************************************************************************
** Small overview response; last snapshot metrics are interval deltas.
*******************************************************************************/
public Map<String, Object> overview(String backend) throws QException
{
requireView(backend);
Map<String, Integer> counts = new LinkedHashMap<>();
List<Map<String, Object>> top = new ArrayList<>();
for(QRecord record : findings(Integer.MAX_VALUE))
{
counts.merge(record.getValueString("severity"), 1, Integer::sum);
if(top.size() < 5)
{
top.add(fields(record, "id", "title", "severity", "score"));
}
}
return (map("settings", fields(settings(), "provider", "statsAccess", "statsAccessMessage", "lastSnapshotAt"),
"snapshot", fields(latest(), "id", "capturedAt", "status", "connectionsTotal", "maxConnections", "cacheHitRatio", "tempBytes", "deadlocks", "statementTotalMs", "statementCalls"),
"topStatements", statements(10, null), "findingsBySeverity", counts, "topFindings", top));
}
/*******************************************************************************
** Link arbitrary SQL shapes back to their normalized statement history.
*******************************************************************************/
public Map<String, Object> lookup(String backend, String sql) throws QException
{
requireView(backend);
if(sql == null || sql.isBlank())
{
throw (new QBadRequestException("Supply SQL to look up."));
}
String normalized = SqlNormalizer.normalize(sql);
String fingerprint = SqlNormalizer.fingerprint(normalized);
return (map("normalizedSql", normalized, "fingerprint", fingerprint, "matches", statements(100, fingerprint)));
}
/*******************************************************************************
** Live activity is read now, not from saved samples, and is never persisted.
*******************************************************************************/
public Map<String, Object> activity(String backend, boolean allowRawText) throws Exception
{
requireView(backend);
try(Connection connection = ConnectionManager.getConnection((RDBMSBackendMetaData) runtime.getInstance().getBackend(runtime.getConfig().getBackendName())))
{
DbStatsProvider provider = DbStatsProviders.forConnection(connection);
return (activity(provider.readSessions(connection), allowRawText));
}
catch(java.sql.SQLException e)
{
throw (new QException("Live activity is unavailable. Check the backend and statistics access."));
}
}
/*******************************************************************************
** Pure projection also used by provider fixture tests; widgets always pass false.
*******************************************************************************/
public Map<String, Object> activity(List<SessionInfo> sessions, boolean allowRawText) throws QException
{
requireView(null);
boolean redacted = !allowRawText || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".view_live_text");
Map<Integer, Set<Integer>> blocked = new LinkedHashMap<>();
for(SessionInfo session : sessions)
{
for(Integer blocker : session.blockedBy())
{
blocked.computeIfAbsent(blocker, key -> new HashSet<>()).add(session.pid());
}
}
List<Map<String, Object>> rows = new ArrayList<>();
List<Map<String, Object>> chains = new ArrayList<>();
Set<Integer> allBlocked = new HashSet<>();
blocked.values().forEach(allBlocked::addAll);
for(SessionInfo session : sessions)
{
String normalized = SqlNormalizer.normalize(session.queryText());
rows.add(map("pid", session.pid(), "user", session.dbUser(), "application", session.applicationName(), "state", session.state(),
"waitEvent", session.waitEvent(), "querySeconds", session.querySeconds(), "xactSeconds", session.xactSeconds(),
"blockedBy", session.blockedBy(), "blockingCount", blocked.getOrDefault(session.pid(), Set.of()).size(),
"fingerprint", SqlNormalizer.fingerprint(normalized), "normalizedText", normalized, "query", redacted ? null : session.queryText(), "liveTextRedacted", redacted));
}
for(Integer root : blocked.keySet())
{
if(!allBlocked.contains(root))
{
Set<Integer> descendants = new java.util.TreeSet<>();
descendants(root, blocked, descendants);
descendants.remove(root);
chains.add(map("rootPid", root, "blockedPids", new ArrayList<>(descendants)));
}
}
return (map("sessions", rows, "blockingChains", chains, "liveTextRedacted", redacted));
}
/*******************************************************************************
** Traverse transitive blocking with cycle protection.
*******************************************************************************/
private static void descendants(Integer pid, Map<Integer, Set<Integer>> blocked, Set<Integer> result)
{
for(Integer child : blocked.getOrDefault(pid, Set.of()))
{
if(result.add(child))
{
descendants(child, blocked, result);
}
}
}
/*******************************************************************************
** One record path builder for API and widgets, including custom app names.
*******************************************************************************/
public String recordPath(String table, Integer id)
{
return ("/app/" + runtime.getConfig().getAppName() + "/" + table + "/" + id);
}
/*******************************************************************************
** Explicit allowlist prevents returning raw counters, configuration or future fields.
*******************************************************************************/
public static Map<String, Object> fields(QRecord record, String... names)
{
Map<String, Object> result = new LinkedHashMap<>();
for(String name : names)
{
result.put(name, record == null ? null : record.getValue(name));
}
return (result);
}
/*******************************************************************************
** Nullable JSON object with stable property order.
*******************************************************************************/
public static Map<String, Object> map(Object... pairs)
{
Map<String, Object> result = new LinkedHashMap<>();
for(int i = 0; i < pairs.length; i += 2)
{
result.put((String) pairs[i], pairs[i + 1]);
}
return (result);
}
/*******************************************************************************
** Nullable statistics arithmetic.
*******************************************************************************/
public static double number(QRecord row, String field)
{
return (row == null || row.getValue(field) == null ? 0 : ((Number) row.getValue(field)).doubleValue());
}
}
@@ -0,0 +1,100 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.middleware.javalin.specs.BasicResponse;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec;
import com.kingsrook.qqq.openapi.model.In;
import com.kingsrook.qqq.openapi.model.Method;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
import com.kingsrook.qqq.openapi.model.Type;
import io.javalin.http.HttpStatus;
/*******************************************************************************
** Common schemas and error contract, with no alternate route or permission pipeline.
*******************************************************************************/
public abstract class DbPerformanceJsonSpec extends AbstractJsonEndpointSpec
{
/*******************************************************************************
** Stable operation identifiers for generated clients.
*******************************************************************************/
@Override
public Method defineMethod()
{
return (super.defineMethod().withOperationId("dbPerformance" + getClass().getSimpleName().replace("Spec", "")));
}
/*******************************************************************************
** Publish all nested response and request component schemas.
*******************************************************************************/
@Override
public Map<String, Schema> defineComponentSchemas()
{
return (DbPerformanceSchemas.all());
}
/*******************************************************************************
** Backend defaults to the qbit's single configured monitored backend for reads.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (List.of(new Parameter().withName("backend").withIn(In.QUERY).withRequired(false)
.withSchema(new Schema().withType(Type.STRING)).withDescription("Configured monitored backend; defaults to the qbit backend.")));
}
/*******************************************************************************
** Framework exceptions map to the same status in every UI client.
*******************************************************************************/
@Override
public List<BasicResponse> defineAdditionalBasicResponses()
{
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid request", "DbPerformanceErrorV1"),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", "DbPerformanceErrorV1"),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", "DbPerformanceErrorV1")));
}
/*******************************************************************************
** Resolve per-request runtime from the authenticated QContext.
*******************************************************************************/
protected DbPerformanceData data()
{
return (new DbPerformanceData(DbPerfRuntime.of(QContext.getQInstance())));
}
}
@@ -0,0 +1,151 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import com.kingsrook.qqq.openapi.model.Schema;
import com.kingsrook.qqq.openapi.model.Type;
/*******************************************************************************
** Explicit versioned schemas; every public property is described for generated clients.
*******************************************************************************/
public final class DbPerformanceSchemas
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerformanceSchemas()
{
}
/*******************************************************************************
** Shared schema components referenced by the four independently versioned operations.
*******************************************************************************/
public static Map<String, Schema> all()
{
Map<String, Schema> schemas = new TreeMap<>();
schemas.put("DbPerformanceErrorV1", object("error", text()));
schemas.put("DbPerformanceSettingsV1", object("provider", text(), "statsAccess", text(), "statsAccessMessage", text(), "lastSnapshotAt", text()));
schemas.put("DbPerformanceSnapshotV1", object("id", integer(), "capturedAt", text(), "status", text(), "connectionsTotal", integer(), "maxConnections", integer(),
"cacheHitRatio", number(), "tempBytes", integer(), "deadlocks", integer(), "statementTotalMs", number(), "statementCalls", integer()));
schemas.put("DbPerformanceStatementV1", object("id", integer(), "queryText", text().withDescription("Normalized SQL, truncated to 300 characters."),
"calls24h", integer(), "meanMs24h", number(), "totalMs24h", number(), "shareOfDbTime24h", number(), "hitRatio", number(), "historyPath", text()));
schemas.put("DbPerformanceFindingV1", object("id", integer(), "title", text(), "severity", text(), "score", number()));
schemas.put("DbPerformanceOverviewV1", object("settings", ref("Settings"), "snapshot", ref("Snapshot"), "topStatements", array(ref("Statement")),
"findingsBySeverity", new Schema().withType(Type.OBJECT).withAdditionalProperties(integer()), "topFindings", array(ref("Finding"))));
schemas.put("DbPerformanceSessionV1", object("pid", integer(), "user", text(), "application", text(), "state", text(), "waitEvent", text(),
"querySeconds", number(), "xactSeconds", number(), "blockedBy", array(integer()), "blockingCount", integer(), "fingerprint", text(), "normalizedText", text(),
"query", text().withDescription("Raw live SQL only with view_live_text; never persisted."), "liveTextRedacted", bool()));
schemas.put("DbPerformanceBlockingChainV1", object("rootPid", integer(), "blockedPids", array(integer())));
schemas.put("DbPerformanceActivityV1", object("sessions", array(ref("Session")), "blockingChains", array(ref("BlockingChain")), "liveTextRedacted", bool()));
schemas.put("DbPerformanceLookupV1", object("normalizedSql", text(), "fingerprint", text(), "matches", array(ref("Statement"))));
schemas.put("DbPerformanceExplainRequestV1", object("backend", text(), "statementId", integer(), "sql", text(),
"mode", new Schema().withType(Type.STRING).withEnumValues(List.of("PLAN", "ANALYZE")), "parameters", array(text()))
.withDescription("Supply exactly one of statementId or sql. Parameters are strings, accepted only for ANALYZE and never stored."));
schemas.put("DbPerformanceExplainResultV1", object("mode", text(), "planJson", text(), "planText", text(), "totalCost", number(), "actualTotalMs", number(),
"warnings", array(text()), "explainId", integer(), "sqlWorkbenchUrl", text(), "assistantPrompt", text()));
return (schemas);
}
/*******************************************************************************
** Nullable scalar for metrics that have not yet been collected.
*******************************************************************************/
private static Schema text()
{
return (new Schema().withType(Type.STRING).withNullable(true));
}
/*******************************************************************************
** Integral count or identifier.
*******************************************************************************/
private static Schema integer()
{
return (new Schema().withType(Type.INTEGER).withNullable(true));
}
/*******************************************************************************
** Fractional duration, cost or ratio.
*******************************************************************************/
private static Schema number()
{
return (new Schema().withType(Type.NUMBER).withNullable(true));
}
/*******************************************************************************
** Required boolean flag.
*******************************************************************************/
private static Schema bool()
{
return (new Schema().withType(Type.BOOLEAN));
}
/*******************************************************************************
** Stable component reference.
*******************************************************************************/
public static Schema ref(String suffix)
{
return (new Schema().withRefToSchema("DbPerformance" + suffix + "V1"));
}
/*******************************************************************************
** A list with an explicitly typed item.
*******************************************************************************/
private static Schema array(Schema item)
{
return (new Schema().withType(Type.ARRAY).withItems(item));
}
/*******************************************************************************
** Deterministically ordered properties for stable OpenAPI generation.
*******************************************************************************/
private static Schema object(Object... pairs)
{
Map<String, Schema> properties = new TreeMap<>();
for(int i = 0; i < pairs.length; i += 2)
{
properties.put((String) pairs[i], (Schema) pairs[i + 1]);
}
return (new Schema().withType(Type.OBJECT).withProperties(properties));
}
}
@@ -0,0 +1,65 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData;
/*******************************************************************************
** Discoverable UI API extension, never raw Javalin routes.
*******************************************************************************/
public final class DbPerformanceUiApi
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerformanceUiApi()
{
}
/*******************************************************************************
** Add the versioned extension once per host instance.
*******************************************************************************/
public static void register(QInstance instance)
{
MiddlewareSpecExtensionsMetaData.ofOrWithNew(instance).withExtension(extension());
}
/*******************************************************************************
** Public contract used for routing, discovery and committed OpenAPI generation.
*******************************************************************************/
public static MiddlewareSpecExtension extension()
{
return (new MiddlewareSpecExtension().withName("db-performance").withSemanticVersion("1.0.0")
.withDescription("Database performance, live activity and audited query plans")
.withSpec(OverviewSpec.class).withSpec(ActivitySpec.class).withSpec(ExplainSpec.class).withSpec(LookupSpec.class));
}
}
@@ -0,0 +1,152 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.explain.ExplainRequest;
import com.kingsrook.qbits.dbperformance.explain.ExplainService;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
/*******************************************************************************
** Audited plan execution. Service denial is a core permission exception (HTTP 403).
*******************************************************************************/
public class ExplainSpec extends DbPerformanceJsonSpec
{
/*******************************************************************************
** JSON-only POST, with stronger permission for ANALYZE.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withHttpMethod(HttpMethod.POST).withPath("/qbit/db-performance/explain")
.withShortSummary("Explain a PostgreSQL query").withLongDescription("PLAN requires db_performance.explain; ANALYZE requires db_performance.explain_analyze. Values and ANALYZE plans are never persisted."));
}
/*******************************************************************************
** All inputs are in the body.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (List.of());
}
/*******************************************************************************
** Versioned response shape.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("DbPerformanceExplainResultV1");
}
/*******************************************************************************
** Versioned request shape.
*******************************************************************************/
@Override
protected Schema defineRequestBodySchema()
{
return (DbPerformanceSchemas.ref("ExplainRequest"));
}
/*******************************************************************************
** Validate JSON types without copying arbitrary values to errors or logs.
*******************************************************************************/
@Override
public Object handle(JsonInput input) throws Exception
{
Map<String, Object> body = input.getBody();
ExplainRequest.Mode mode = null;
try
{
mode = ExplainRequest.Mode.valueOf(string(body, "mode"));
}
catch(IllegalArgumentException | NullPointerException e)
{
// The service audits invalid modes before returning a user-facing error.
}
Integer id = null;
if(body.get("statementId") != null)
{
try
{
id = new java.math.BigDecimal(body.get("statementId").toString()).intValueExact();
}
catch(ArithmeticException | NumberFormatException e)
{
throw (new QBadRequestException("statementId must be an integer."));
}
}
List<String> parameters = new ArrayList<>();
if(body.get("parameters") != null)
{
if(!(body.get("parameters") instanceof List<?> values))
{
throw (new QBadRequestException("parameters must be an array of strings."));
}
for(Object value : values)
{
if(!(value instanceof String))
{
throw (new QBadRequestException("parameters must be an array of strings."));
}
parameters.add((String) value);
}
}
return (new ExplainService(DbPerfRuntime.of(QContext.getQInstance())).explain(new ExplainRequest(string(body, "backend"), id, string(body, "sql"), mode, parameters)));
}
/*******************************************************************************
** Accept strings only, rather than coercing objects into executable SQL.
*******************************************************************************/
private static String string(Map<String, Object> body, String key) throws QBadRequestException
{
Object value = body.get(key);
if(value != null && !(value instanceof String))
{
throw (new QBadRequestException(key + " must be a string."));
}
return ((String) value);
}
}
@@ -0,0 +1,86 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.In;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
import com.kingsrook.qqq.openapi.model.Type;
/*******************************************************************************
** Database Performance lookup under the authenticated UI API.
*******************************************************************************/
public class LookupSpec extends DbPerformanceJsonSpec
{
/*******************************************************************************
** Method and path relative to /qqq/v1.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withHttpMethod(HttpMethod.GET).withPath("/qbit/db-performance/statements/lookup")
.withShortSummary("Database Performance lookup").withLongDescription("Requires db_performance.view and uses the caller's session."));
}
/*******************************************************************************
** Versioned response shape.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("DbPerformanceLookupV1");
}
/*******************************************************************************
** Execute in the caller's context; also directly testable without transport.
*******************************************************************************/
@Override
public Object handle(JsonInput input) throws Exception
{
return (data().lookup(input.getQueryParam("backend"), input.getQueryParam("sql")));
}
/*******************************************************************************
** Lookup requires a SQL shape in addition to the optional backend.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
List<Parameter> parameters = new ArrayList<>(super.defineRequestParameters());
parameters.add(new Parameter().withName("sql").withIn(In.QUERY).withRequired(true).withSchema(new Schema().withType(Type.STRING)));
return (parameters);
}
}
@@ -0,0 +1,67 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.api;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
/*******************************************************************************
** Database Performance overview under the authenticated UI API.
*******************************************************************************/
public class OverviewSpec extends DbPerformanceJsonSpec
{
/*******************************************************************************
** Method and path relative to /qqq/v1.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withHttpMethod(HttpMethod.GET).withPath("/qbit/db-performance/overview")
.withShortSummary("Database Performance overview").withLongDescription("Requires db_performance.view and uses the caller's session."));
}
/*******************************************************************************
** Versioned response shape.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("DbPerformanceOverviewV1");
}
/*******************************************************************************
** Execute in the caller's context; also directly testable without transport.
*******************************************************************************/
@Override
public Object handle(JsonInput input) throws Exception
{
return (data().overview(input.getQueryParam("backend")));
}
}
@@ -0,0 +1,270 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.attribution;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfSettingsService;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.instrumentation.StatementExecutionListeners;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** One daemon per runtime. Windows are detached before any database operation;
** failed windows are dropped and later windows continue without retry queues.
*******************************************************************************/
public class AttributionFlusher implements AutoCloseable
{
private static final QLogger LOG = QLogger.getLogger(AttributionFlusher.class);
private final DbPerfRuntime runtime;
private final AttributionListener listener;
private ScheduledExecutorService executor;
private Thread shutdownHook;
private boolean closed;
private long failures;
/*******************************************************************************
** Construct without starting threads, for hosts and deterministic tests.
*******************************************************************************/
public AttributionFlusher(DbPerfRuntime runtime)
{
this.runtime = runtime;
this.listener = new AttributionListener(runtime);
}
/*******************************************************************************
** Register once after the instance and schema are ready.
*******************************************************************************/
public synchronized void start()
{
if(executor != null || closed || !Boolean.TRUE.equals(runtime.getConfig().getAttributionEnabled()))
{
return;
}
refreshSettings();
StatementExecutionListeners.register(listener);
executor = Executors.newSingleThreadScheduledExecutor(task ->
{
Thread thread = new Thread(task, "db-perf-attribution");
thread.setDaemon(true);
return (thread);
});
shutdownHook = new Thread(this::close, "db-perf-attribution-shutdown");
Runtime.getRuntime().addShutdownHook(shutdownHook);
schedule();
}
/*******************************************************************************
** Reschedule after each pass so administrative interval changes take effect.
*******************************************************************************/
private synchronized void schedule()
{
if(!closed)
{
executor.schedule(() ->
{
flushNow();
schedule();
}, listener.settings().flushSeconds(), TimeUnit.SECONDS);
}
}
/*******************************************************************************
** Exactly one InsertAction per nonempty table, with all rows in the window.
** Serialize flushers only; statement threads never acquire this monitor.
*******************************************************************************/
public synchronized void flushNow()
{
boolean previous = AttributionListener.suppress(true);
try
{
Instant end = Instant.now();
AttributionListener.Window window = listener.drain(end);
List<QRecord> attribution = records(window.attribution(), window.start(), end, false);
List<QRecord> repeats = records(window.repeats(), window.start(), end, true);
if(!attribution.isEmpty() || !repeats.isEmpty())
{
runtime.transaction(() ->
{
insert("db_perf_attribution", attribution);
insert("db_perf_repeat_pattern", repeats);
return (null);
});
}
}
catch(Exception e)
{
failed();
}
finally
{
refreshSettings();
AttributionListener.suppress(previous);
}
}
/*******************************************************************************
** Refresh even after failed writes, including while attribution is disabled.
*******************************************************************************/
public synchronized void refreshSettings()
{
boolean previous = AttributionListener.suppress(true);
try
{
listener.refresh(new DbPerfSettingsService(runtime).ensure(runtime.getConfig().getBackendName()));
}
catch(Exception e)
{
failed();
}
finally
{
AttributionListener.suppress(previous);
}
}
/*******************************************************************************
** Rate-limited fixed text; database errors may contain sensitive SQL.
*******************************************************************************/
private void failed()
{
failures++;
if(failures == 1 || failures % 1000 == 0)
{
LOG.warn("Database Performance attribution write or settings refresh failed; the failed window is dropped");
}
}
/*******************************************************************************
** Convert nanoseconds to persisted milliseconds without floating point loss.
*******************************************************************************/
private List<QRecord> records(Map<AttributionListener.Key, AttributionListener.Counters> values, Instant start, Instant end, boolean repeat)
{
List<QRecord> records = new ArrayList<>(values.size());
for(Map.Entry<AttributionListener.Key, AttributionListener.Counters> entry : values.entrySet())
{
AttributionListener.Key key = entry.getKey();
AttributionListener.Counters counts = entry.getValue();
QRecord row = new QRecord().withValue("windowStart", start).withValue("windowEnd", end)
.withValue("podName", truncate(runtime.getConfig().getPodName(), 200)).withValue("unitKind", key.kind())
.withValue("unitName", truncate(key.name(), 300)).withValue("processName", truncate(key.process(), 200))
.withValue("tableName", truncate(key.table(), 200)).withValue("fingerprint", key.fingerprint())
.withValue("sqlText", truncate(counts.sql, 4000)).withValue("sampleRate", counts.sampleRate < 0 ? null : BigDecimal.valueOf(counts.sampleRate));
if(repeat)
{
row.withValue("unitsAffected", counts.units.sum()).withValue("totalRepeats", counts.statements.sum())
.withValue("maxRepeats", (int) Math.min(Integer.MAX_VALUE, counts.max.get()));
}
else
{
row.withValue("operation", key.operation()).withValue("units", counts.units.sum()).withValue("statements", counts.statements.sum())
.withValue("totalMs", BigDecimal.valueOf(counts.nanos.sum(), 6)).withValue("maxMs", BigDecimal.valueOf(counts.max.get(), 6))
.withValue("rowsCount", counts.rows.sum()).withValue("errors", counts.errors.sum());
}
records.add(row);
}
return (records);
}
/*******************************************************************************
** Bound text at the persistence boundary after normalization.
*******************************************************************************/
private static String truncate(String value, int length)
{
return (value == null || value.length() <= length ? value : value.substring(0, length));
}
/*******************************************************************************
** Reject record-level failures so the whole window transaction rolls back.
*******************************************************************************/
private static void insert(String table, List<QRecord> rows) throws Exception
{
if(!rows.isEmpty())
{
List<QRecord> inserted = InsertAction.executeForRecords(new InsertInput(table).withRecords(rows).withTransaction(QContext.getQBackendTransaction()));
if(inserted.size() != rows.size() || inserted.stream().anyMatch(row -> row.getErrors() != null && !row.getErrors().isEmpty()))
{
throw (new IllegalStateException("Attribution insert failed"));
}
}
}
/*******************************************************************************
** Listener exposed for deterministic delivery and registration tests.
*******************************************************************************/
public AttributionListener getListener()
{
return (listener);
}
/*******************************************************************************
** Unregister first, stop scheduling, and flush one final detached window.
*******************************************************************************/
@Override
public synchronized void close()
{
if(closed)
{
return;
}
closed = true;
StatementExecutionListeners.unregister(listener);
if(executor != null)
{
executor.shutdownNow();
}
if(shutdownHook != null && Thread.currentThread() != shutdownHook)
{
try
{
Runtime.getRuntime().removeShutdownHook(shutdownHook);
}
catch(IllegalStateException expected)
{
// JVM shutdown is already running the hook.
}
}
flushNow();
}
}
@@ -0,0 +1,425 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.attribution;
import java.time.Instant;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.LongAdder;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.instrumentation.StatementExecution;
import com.kingsrook.qqq.backend.core.instrumentation.StatementExecutionListener;
import com.kingsrook.qqq.backend.core.instrumentation.UnitOfWork;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Bounded, in-memory attribution. No table actions run on statement threads.
** Completed units belong to the window in which they end, including long units.
*******************************************************************************/
public class AttributionListener implements StatementExecutionListener
{
private static final String OTHER = "(other)";
private static final Key OVERFLOW = new Key("OTHER", OTHER, OTHER, OTHER, "OTHER", SqlNormalizer.fingerprint(OTHER), -1);
private static final ThreadLocal<Boolean> SUPPRESSED = ThreadLocal.withInitial(() -> false);
private final DbPerfRuntime runtime;
private final ConcurrentHashMap<String, Shape> shapes = new ConcurrentHashMap<>();
private final ThreadLocal<Buffer> current = new ThreadLocal<>();
private volatile Settings settings = new Settings(false, 1.0, 60, 10000, 10);
private Window window = new Window(Instant.now(), new HashMap<>(), new HashMap<>());
/*******************************************************************************
** Bind to one instance and monitored backend.
*******************************************************************************/
public AttributionListener(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Ignore self-observation, then sample and count without database access.
*******************************************************************************/
@Override
public void onStatement(StatementExecution execution)
{
Settings copy = settings;
if(!copy.enabled() || !Boolean.TRUE.equals(runtime.getConfig().getAttributionEnabled()) || SUPPRESSED.get()
|| !runtime.getConfig().getBackendName().equals(execution.backendName())
|| (execution.tableName() != null && execution.tableName().startsWith(DbPerfQBitConfig.TABLE_NAME_PREFIX))
|| (QContext.getQInstance() != null && QContext.getQInstance() != runtime.getInstance()))
{
return;
}
UnitOfWork unit = execution.unitOfWork();
Buffer buffer = current.get();
if(buffer != null && (unit == null || buffer.unit.getId() != unit.getId()))
{
merge(buffer);
current.remove();
buffer = null;
}
if(unit != null && buffer == null)
{
buffer = new Buffer(unit, copy, sampled(unit.getId(), copy.sampleRate()));
current.set(buffer);
}
if(buffer == null ? !sampled(ThreadLocalRandom.current().nextLong(), copy.sampleRate()) : !buffer.sampled)
{
return;
}
Shape shape = shape(execution.sql());
Key key = new Key(unit == null ? "OTHER" : unit.getKind().name(), unit == null ? name(execution.processName()) : "",
execution.processName(), execution.tableName(), execution.operation() == null ? "OTHER" : execution.operation().name(), shape.fingerprint(), buffer == null ? copy.sampleRate() : buffer.settings.sampleRate());
if(buffer == null)
{
synchronized(this)
{
Key target = bounded(window.attribution(), key, copy.maxKeys());
Counters counts = window.attribution().computeIfAbsent(target, ignored -> new Counters(target == OVERFLOW ? OTHER : shape.sql(), key.sampleRate()));
counts.sampleRate(key.sampleRate());
counts.add(execution);
counts.units.increment();
}
}
else
{
Key target = bounded(buffer.counts, key, Math.max(2, buffer.settings.maxKeys()));
buffer.counts.computeIfAbsent(target, ignored -> new Counters(target == OVERFLOW ? OTHER : shape.sql(), key.sampleRate())).add(execution);
}
}
/*******************************************************************************
** Read the final route name, then release all per-thread counters.
*******************************************************************************/
@Override
public void onUnitOfWorkEnd(UnitOfWork unit)
{
Buffer buffer = current.get();
if(buffer != null && buffer.unit.getId() == unit.getId())
{
current.remove();
if(settings.enabled() && Boolean.TRUE.equals(runtime.getConfig().getAttributionEnabled()))
{
merge(buffer);
}
}
}
/*******************************************************************************
** Merge each unit once per destination, even when several shapes overflow.
** Repeat counts are combined across operations before threshold evaluation.
*******************************************************************************/
private synchronized void merge(Buffer buffer)
{
Set<Key> touched = new HashSet<>();
Map<Key, Counters> repeats = new HashMap<>();
for(Map.Entry<Key, Counters> entry : buffer.counts.entrySet())
{
Key local = entry.getKey();
Key key = local == OVERFLOW ? OVERFLOW : new Key(local.kind(), name(buffer.unit.getName()), local.process(), local.table(), local.operation(), local.fingerprint(), buffer.settings.sampleRate());
Key target = bounded(window.attribution(), key, settings.maxKeys());
Counters counts = window.attribution().computeIfAbsent(target, ignored -> new Counters(target == OVERFLOW ? OTHER : entry.getValue().sql, entry.getValue().sampleRate));
counts.merge(entry.getValue());
if(touched.add(target))
{
counts.units.increment();
}
// Collapsed shapes cannot prove repetition of any one fingerprint.
if(local != OVERFLOW)
{
Key repeat = new Key(key.kind(), key.name(), key.process(), key.table(), null, key.fingerprint(), key.sampleRate());
repeats.computeIfAbsent(repeat, ignored -> new Counters(entry.getValue().sql, entry.getValue().sampleRate)).merge(entry.getValue());
}
}
Map<Key, Long> totals = new HashMap<>();
Map<Key, Long> maxima = new HashMap<>();
for(Map.Entry<Key, Counters> entry : repeats.entrySet())
{
long count = entry.getValue().statements.sum();
if(count >= buffer.settings.threshold())
{
Key target = bounded(window.repeats(), entry.getKey(), settings.maxKeys());
window.repeats().computeIfAbsent(target, ignored -> new Counters(target == OVERFLOW ? OTHER : entry.getValue().sql, entry.getValue().sampleRate));
window.repeats().get(target).sampleRate(entry.getValue().sampleRate);
totals.merge(target, count, Long::sum);
maxima.merge(target, count, Math::max);
}
}
for(Map.Entry<Key, Long> entry : totals.entrySet())
{
Counters counts = window.repeats().get(entry.getKey());
counts.units.increment();
counts.statements.add(entry.getValue());
counts.max.accumulateAndGet(maxima.get(entry.getKey()), Math::max);
}
}
/*******************************************************************************
** Reserve one slot for a universal overflow key, preserving the strict cap.
*******************************************************************************/
private static Key bounded(Map<Key, Counters> counts, Key key, int limit)
{
return (counts.containsKey(key) || (key != OVERFLOW && counts.size() < limit - (counts.containsKey(OVERFLOW) ? 0 : 1)) ? key : OVERFLOW);
}
/*******************************************************************************
** Normalize once per raw SQL shape; cold-cache admission alone is serialized.
*******************************************************************************/
private Shape shape(String sql)
{
String raw = sql == null ? "" : sql;
Shape cached = shapes.get(raw);
if(cached != null)
{
return (cached);
}
String normalized = SqlNormalizer.normalize(raw);
Shape created = new Shape(normalized, SqlNormalizer.fingerprint(normalized));
synchronized(shapes)
{
if(shapes.size() >= 5000)
{
shapes.clear();
}
shapes.put(raw, created);
}
return (created);
}
/*******************************************************************************
** SplitMix64 finalizer distributes sequential unit IDs without allocations.
*******************************************************************************/
private static boolean sampled(long id, double rate)
{
long hash = (id ^ (id >>> 30)) * 0xbf58476d1ce4e5b9L;
hash = (hash ^ (hash >>> 27)) * 0x94d049bb133111ebL;
hash ^= hash >>> 31;
return ((hash >>> 11) * 0x1.0p-53 < rate);
}
/*******************************************************************************
** Use a stable label for units without a name.
*******************************************************************************/
private static String name(String value)
{
return (value == null || value.isBlank() ? "(none)" : value);
}
/*******************************************************************************
** Detach an immutable-in-practice window; no writers retain its counters.
*******************************************************************************/
synchronized Window drain(Instant end)
{
Window previous = window;
window = new Window(end, new HashMap<>(), new HashMap<>());
return (previous);
}
/*******************************************************************************
** Publish a validated immutable settings copy only from the flusher.
*******************************************************************************/
void refresh(QRecord row)
{
double rate = row.getValueBigDecimal("attributionSampleRate") == null ? 1.0 : row.getValueBigDecimal("attributionSampleRate").doubleValue();
settings = new Settings(!Boolean.FALSE.equals(row.getValueBoolean("attributionEnabled")), Math.max(0, Math.min(1, rate)),
positive(row, "attributionFlushSeconds", 60), positive(row, "attributionMaxKeys", 10000), positive(row, "nPlusOneThreshold", 10));
}
/*******************************************************************************
** Invalid administrative values use defaults.
*******************************************************************************/
private static int positive(QRecord row, String field, int fallback)
{
Integer value = row.getValueInteger(field);
return (value == null || value <= 0 ? fallback : value);
}
/*******************************************************************************
** Current settings for scheduling and window annotations.
*******************************************************************************/
Settings settings()
{
return (settings);
}
/*******************************************************************************
** Set thread suppression, returning the previous state for nested callers.
*******************************************************************************/
static boolean suppress(boolean value)
{
boolean previous = SUPPRESSED.get();
if(value)
{
SUPPRESSED.set(true);
}
else
{
SUPPRESSED.remove();
}
return (previous);
}
/*******************************************************************************
** Immutable administrative settings.
*******************************************************************************/
record Settings(boolean enabled, double sampleRate, int flushSeconds, int maxKeys, int threshold)
{
}
/*******************************************************************************
** Attribution dimensions; sampling transitions remain separately interpretable.
** Null operation denotes a repeat pattern.
*******************************************************************************/
record Key(String kind, String name, String process, String table, String operation, String fingerprint, double sampleRate)
{
}
/*******************************************************************************
** Cached literal-free text and its digest.
*******************************************************************************/
private record Shape(String sql, String fingerprint)
{
}
/*******************************************************************************
** Detached window transferred to the flusher.
*******************************************************************************/
record Window(Instant start, Map<Key, Counters> attribution, Map<Key, Counters> repeats)
{
}
/*******************************************************************************
** Thread-confined unit, including its once-only sampling decision.
*******************************************************************************/
private static class Buffer
{
private final UnitOfWork unit;
private final Settings settings;
private final boolean sampled;
private final Map<Key, Counters> counts = new HashMap<>();
/*******************************************************************************
** Capture unit settings so refresh cannot partially sample a unit.
*******************************************************************************/
private Buffer(UnitOfWork unit, Settings settings, boolean sampled)
{
this.unit = unit;
this.settings = settings;
this.sampled = sampled;
}
}
/*******************************************************************************
** Additive counters plus an atomic maximum, shared by both window tables.
*******************************************************************************/
static class Counters
{
final String sql;
double sampleRate;
final LongAdder units = new LongAdder();
final LongAdder statements = new LongAdder();
final LongAdder nanos = new LongAdder();
final AtomicLong max = new AtomicLong();
final LongAdder rows = new LongAdder();
final LongAdder errors = new LongAdder();
/*******************************************************************************
** Retain only normalized SQL.
*******************************************************************************/
Counters(String sql, double sampleRate)
{
this.sql = sql;
this.sampleRate = sampleRate;
}
/*******************************************************************************
** Count one completed statement, including failures and unknown row counts.
*******************************************************************************/
void add(StatementExecution execution)
{
statements.increment();
nanos.add(execution.elapsedNanos());
max.accumulateAndGet(execution.elapsedNanos(), Math::max);
rows.add(execution.rowCount() == null ? 0 : execution.rowCount());
if(!execution.succeeded())
{
errors.increment();
}
}
/*******************************************************************************
** An overflow bucket spanning sampling rates has no single valid rate.
*******************************************************************************/
void sampleRate(double rate)
{
if(sampleRate != rate)
{
sampleRate = -1;
}
}
/*******************************************************************************
** Fold statement totals; distinct units are managed by the caller.
*******************************************************************************/
void merge(Counters other)
{
sampleRate(other.sampleRate);
statements.add(other.statements.sum());
nanos.add(other.nanos.sum());
max.accumulateAndGet(other.max.get(), Math::max);
rows.add(other.rows.sum());
errors.add(other.errors.sum());
}
}
}
@@ -0,0 +1,69 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.attribution;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qqq.backend.core.instances.QRuntimeServiceInterface;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
** QApplicationLauncher lifecycle, after instance initialization and migrations.
*******************************************************************************/
public class AttributionRuntimeService implements QRuntimeServiceInterface
{
private DbPerfRuntime runtime;
/*******************************************************************************
** Name used by the host launcher.
*******************************************************************************/
@Override
public String getName()
{
return ("dbPerformanceAttribution");
}
/*******************************************************************************
** Start on each application pod, independently of the job scheduler.
*******************************************************************************/
@Override
public void start(QInstance instance)
{
runtime = DbPerfRuntime.of(instance);
runtime.startAttribution();
}
/*******************************************************************************
** Stop listener delivery and persist the final completed units.
*******************************************************************************/
@Override
public void stop()
{
if(runtime != null)
{
runtime.stopAttribution();
}
}
}
@@ -0,0 +1,74 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Insert-only per-backend defaults; user edits are never overwritten.
*******************************************************************************/
public class DbPerfSettingsService
{
private final DbPerfRuntime runtime;
/*******************************************************************************
** Runtime dependency.
*******************************************************************************/
public DbPerfSettingsService(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Load or insert the settings row. A concurrent winner is re-read.
*******************************************************************************/
public QRecord ensure(String backendName) throws Exception
{
return (runtime.asSystem(() ->
{
QRecord existing = DbPerfStore.first("db_perf_settings", DbPerfStore.eq("backendName", backendName));
if(existing != null)
{
return (existing);
}
try
{
return (DbPerfStore.insert("db_perf_settings", List.of(new QRecord().withValue("backendName", backendName)
.withValue("attributionEnabled", runtime.getConfig().getAttributionEnabled()))).getFirst());
}
catch(Exception e)
{
QRecord winner = DbPerfStore.first("db_perf_settings", DbPerfStore.eq("backendName", backendName));
if(winner != null)
{
return (winner);
}
throw (e);
}
}));
}
}
@@ -0,0 +1,204 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Small QQQ action adapter. All mutations preserve audit/customizer semantics.
*******************************************************************************/
public final class DbPerfStore
{
public static final int BATCH_SIZE = 1000;
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfStore()
{
}
/*******************************************************************************
** Read rows using the ambient system transaction.
*******************************************************************************/
public static List<QRecord> query(String table, QQueryFilter filter) throws QException
{
return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withTransaction(QContext.getQBackendTransaction())).getRecords());
}
/*******************************************************************************
** Keyset-paged read for tables that can hold more rows than fit in the heap
** (attribution writes hundreds of thousands per day). At most pageSize
** records are held at once; the consumer must not keep the page.
*******************************************************************************/
public static void forEachPage(String table, QQueryFilter filter, int pageSize, PageConsumer consumer) throws Exception
{
Serializable lastId = 0;
while(true)
{
QQueryFilter page = new QQueryFilter().withSubFilter(filter).withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId))
.withOrderBy(new QFilterOrderBy("id", true)).withLimit(pageSize);
List<QRecord> rows = query(table, page);
if(rows.isEmpty())
{
return;
}
consumer.accept(rows);
if(rows.size() < pageSize)
{
return;
}
lastId = rows.getLast().getValue("id");
}
}
/*******************************************************************************
** Receives one bounded page from forEachPage.
*******************************************************************************/
@FunctionalInterface
public interface PageConsumer
{
/***************************************************************************
** Handle one page.
***************************************************************************/
void accept(List<QRecord> page) throws Exception;
}
/*******************************************************************************
** First matching record.
*******************************************************************************/
public static QRecord first(String table, QQueryFilter filter) throws QException
{
List<QRecord> rows = query(table, filter.withLimit(1));
return (rows.isEmpty() ? null : rows.getFirst());
}
/*******************************************************************************
** A field equality predicate.
*******************************************************************************/
public static QQueryFilter eq(String field, Serializable value)
{
return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
}
/*******************************************************************************
** Bounded batch insertion, returning generated IDs and rejecting row errors.
*******************************************************************************/
public static List<QRecord> insert(String table, List<QRecord> rows) throws QException
{
List<QRecord> result = new ArrayList<>();
for(int i = 0; i < rows.size(); i += BATCH_SIZE)
{
List<QRecord> inserted = InsertAction.executeForRecords(new InsertInput(table).withRecords(new ArrayList<>(rows.subList(i, Math.min(rows.size(), i + BATCH_SIZE)))).withTransaction(QContext.getQBackendTransaction()));
check(inserted);
result.addAll(inserted);
}
return (result);
}
/*******************************************************************************
** Batch updates, throwing on row errors rather than silently losing data.
*******************************************************************************/
public static void update(String table, List<QRecord> rows) throws QException
{
for(int i = 0; i < rows.size(); i += BATCH_SIZE)
{
check(UpdateAction.executeForRecords(new UpdateInput(table).withRecords(new ArrayList<>(rows.subList(i, Math.min(rows.size(), i + BATCH_SIZE)))).withTransaction(QContext.getQBackendTransaction())));
}
}
/*******************************************************************************
** Keyset deletion: select at most 1000 matching IDs, delete through QQQ,
** then advance the key. No OFFSET skips and no unbounded ID collection.
*******************************************************************************/
public static int delete(String table, QQueryFilter filter) throws QException
{
int count = 0;
int lastId = 0;
while(true)
{
QQueryFilter page = new QQueryFilter().withSubFilter(filter).withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId))
.withOrderBy(new QFilterOrderBy("id", true)).withLimit(BATCH_SIZE);
List<QRecord> rows = query(table, page);
if(rows.isEmpty())
{
return (count);
}
List<Serializable> ids = rows.stream().map(row -> (Serializable) row.getValueInteger("id")).toList();
var deleted = new DeleteAction().execute(new DeleteInput(table).withQueryFilter(new QQueryFilter().withSubFilter(filter).withCriteria(new QFilterCriteria("id", QCriteriaOperator.IN, ids))).withTransaction(QContext.getQBackendTransaction()));
if(deleted.getRecordsWithErrors() != null)
{
check(deleted.getRecordsWithErrors());
}
if(deleted.getDeletedRecordCount() != ids.size())
{
throw (new QException("Database Performance deletion did not remove all selected records"));
}
lastId = rows.getLast().getValueInteger("id");
count += rows.size();
}
}
/*******************************************************************************
** Reject action-level record errors without persisting potentially raw SQL.
*******************************************************************************/
private static void check(List<QRecord> rows) throws QException
{
for(QRecord row : rows)
{
if(row.getErrors() != null && !row.getErrors().isEmpty())
{
throw (new QException("Database Performance record write failed: " + row.getErrors().getFirst().getClass().getSimpleName()));
}
}
}
}
@@ -0,0 +1,83 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.Objects;
import com.kingsrook.qbits.dbperformance.provider.StatementStats;
/*******************************************************************************
** Pure cumulative-counter arithmetic with explicit baselines and reset detection.
*******************************************************************************/
public final class DeltaCalculator
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DeltaCalculator()
{
}
/*******************************************************************************
** A first observation is not historical activity. Only an entry created
** within the previous capture interval can count from zero immediately.
*******************************************************************************/
public static Delta calculate(StatementStats previous, StatementStats current, Instant previousCapture, Instant globalReset)
{
boolean fresh = previous == null;
boolean baseline = fresh && (previousCapture == null || current.statsSince() == null || !current.statsSince().isAfter(previousCapture));
boolean reset = !fresh && (current.calls() < previous.calls()
|| current.statsSince() != null && !Objects.equals(current.statsSince(), previous.statsSince())
|| globalReset != null && previousCapture != null && globalReset.isAfter(previousCapture));
if(baseline)
{
return (new Delta(0, BigDecimal.ZERO, 0, 0, 0, 0, false, true));
}
boolean zero = fresh || reset;
return (new Delta(counter(zero ? 0L : previous.calls(), current.calls(), false),
current.totalExecMs().subtract(zero ? BigDecimal.ZERO : previous.totalExecMs()).max(BigDecimal.ZERO),
counter(zero ? 0L : previous.rows(), current.rows(), false),
counter(zero ? 0L : previous.sharedBlksHit(), current.sharedBlksHit(), false),
counter(zero ? 0L : previous.sharedBlksRead(), current.sharedBlksRead(), false),
counter(zero ? 0L : previous.tempBlksWritten(), current.tempBlksWritten(), false), reset, false));
}
/*******************************************************************************
** Null means no baseline; a decrease means reset for this counter.
*******************************************************************************/
public static long counter(Long previous, long current, boolean reset)
{
return (previous == null ? 0 : reset || current < previous ? current : current - previous);
}
/*******************************************************************************
** Interval statement activity; maximum duration remains a since-reset gauge.
*******************************************************************************/
public record Delta(long calls, BigDecimal totalMs, long rows, long sharedBlksHit, long sharedBlksRead, long tempBlksWritten, boolean reset, boolean baseline)
{
}
}
@@ -0,0 +1,119 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.time.Clock;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Configurable retention through bounded QQQ keyset deletes.
*******************************************************************************/
public class RetentionService
{
private final DbPerfRuntime runtime;
private final Clock clock;
/*******************************************************************************
** Production clock.
*******************************************************************************/
public RetentionService(DbPerfRuntime runtime)
{
this(runtime, Clock.systemUTC());
}
/*******************************************************************************
** Clock injection for exact boundary tests.
*******************************************************************************/
public RetentionService(DbPerfRuntime runtime, Clock clock)
{
this.runtime = runtime;
this.clock = clock;
}
/*******************************************************************************
** Only closed old findings are removed; FK actions preserve open findings.
*******************************************************************************/
public int purge() throws Exception
{
QRecord settings = new DbPerfSettingsService(runtime).ensure(runtime.getConfig().getBackendName());
Instant detailCutoff = clock.instant().minusSeconds(86400L * SnapshotCollector.positive(settings, "detailRetentionDays", 30));
int rollupDays = SnapshotCollector.positive(settings, "rollupRetentionDays", 365);
Instant staleCutoff = clock.instant().minusSeconds(86400L * rollupDays);
LocalDate dayCutoff = LocalDate.now(clock.withZone(ZoneId.of(runtime.getConfig().getTimeZone()))).minusDays(rollupDays);
return (runtime.asSystem(() ->
{
int count = 0;
for(String table : List.of("db_perf_query_snapshot", "db_perf_table_snapshot", "db_perf_activity_sample", "db_perf_connection_sample"))
{
count += DbPerfStore.delete(table, before("capturedAt", detailCutoff));
}
for(String table : List.of("db_perf_attribution", "db_perf_repeat_pattern"))
{
count += DbPerfStore.delete(table, before("windowStart", detailCutoff));
}
count += DbPerfStore.delete("db_perf_explain", before("requestedAt", detailCutoff));
count += DbPerfStore.delete("db_perf_snapshot", backend(before("capturedAt", detailCutoff)));
for(String table : List.of("db_perf_query_daily", "db_perf_table_daily", "db_perf_attribution_daily"))
{
count += DbPerfStore.delete(table, new QQueryFilter(new QFilterCriteria("day", QCriteriaOperator.LESS_THAN, dayCutoff)));
}
for(String table : List.of("db_perf_index_stat", "db_perf_statement", "db_perf_table_stat"))
{
count += DbPerfStore.delete(table, backend(before("lastSeenAt", staleCutoff)));
}
count += DbPerfStore.delete("db_perf_finding", backend(before("lastSeenAt", staleCutoff))
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.IN, List.of("RESOLVED", "IGNORED"))));
return (count);
}));
}
/*******************************************************************************
** Scope backend-bearing rows to the configured monitored backend.
*******************************************************************************/
private QQueryFilter backend(QQueryFilter filter)
{
return (filter.withCriteria(new QFilterCriteria("backendName", QCriteriaOperator.EQUALS, runtime.getConfig().getBackendName())));
}
/*******************************************************************************
** Strict age boundary preserves records at the cutoff.
*******************************************************************************/
private static QQueryFilter before(String field, Instant cutoff)
{
return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.LESS_THAN, cutoff)));
}
}
@@ -0,0 +1,243 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Daily aggregates replaced atomically; reruns include late arriving detail rows.
*******************************************************************************/
public class RollupService
{
private final DbPerfRuntime runtime;
/*******************************************************************************
** Runtime dependency.
*******************************************************************************/
public RollupService(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Replace all three rollups for a local calendar day in one transaction.
*******************************************************************************/
public void rollupDay(LocalDate day) throws Exception
{
synchronized(runtime)
{
runtime.transaction(() ->
{
ZoneId zone = ZoneId.of(runtime.getConfig().getTimeZone());
Instant start = day.atStartOfDay(zone).toInstant();
Instant end = day.plusDays(1).atStartOfDay(zone).toInstant();
rollupQueries(day, start, end);
rollupTables(day, start, end);
rollupAttribution(day, start, end);
return (null);
});
}
}
/*******************************************************************************
** A day needs backfill only if retained detail exists without its daily rows.
** Existing history must survive after a short detail-retention window expires.
*******************************************************************************/
public boolean needsBackfill(LocalDate day) throws Exception
{
return (runtime.asSystem(() ->
{
ZoneId zone = ZoneId.of(runtime.getConfig().getTimeZone());
Instant start = day.atStartOfDay(zone).toInstant();
Instant end = day.plusDays(1).atStartOfDay(zone).toInstant();
for(String kind : List.of("query", "table", "attribution"))
{
String detail = "db_perf_" + kind + (kind.equals("attribution") ? "" : "_snapshot");
String field = kind.equals("attribution") ? "windowStart" : "capturedAt";
if(DbPerfStore.first("db_perf_" + kind + "_daily", DbPerfStore.eq("day", day)) == null
&& DbPerfStore.first(detail, range(field, start, end)) != null)
{
return (true);
}
}
return (false);
}));
}
/*******************************************************************************
** Weighted statement means/hit ratios, since-reset maximum, exact day share.
*******************************************************************************/
private void rollupQueries(LocalDate day, Instant start, Instant end) throws Exception
{
Map<Integer, QRecord> groups = new LinkedHashMap<>();
Map<Integer, long[]> hits = new LinkedHashMap<>();
BigDecimal denominator = BigDecimal.ZERO;
for(QRecord snapshot : DbPerfStore.query("db_perf_snapshot", range("capturedAt", start, end)
.withCriteria(new QFilterCriteria("backendName", QCriteriaOperator.EQUALS, runtime.getConfig().getBackendName()))
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.IN, List.of("OK", "PARTIAL")))))
{
denominator = denominator.add(SnapshotCollector.decimal(snapshot, "statementTotalMs"));
}
for(QRecord row : DbPerfStore.query("db_perf_query_snapshot", range("capturedAt", start, end)))
{
int id = row.getValueInteger("statementId");
QRecord aggregate = groups.computeIfAbsent(id, key -> new QRecord().withValue("statementId", key).withValue("day", day));
sum(aggregate, row, "calls", "rowsCount");
sumDecimal(aggregate, row, "totalMs");
aggregate.setValue("maxMs", SnapshotCollector.decimal(aggregate, "maxMs").max(SnapshotCollector.decimal(row, "maxMs")));
aggregate.setValue("snapshots", (int) SnapshotCollector.number(aggregate, "snapshots") + 1);
long[] counts = hits.computeIfAbsent(id, key -> new long[2]);
counts[0] += SnapshotCollector.number(row, "sharedBlksHit");
counts[1] += SnapshotCollector.number(row, "sharedBlksRead");
}
for(Map.Entry<Integer, QRecord> entry : groups.entrySet())
{
QRecord row = entry.getValue();
long[] counts = hits.get(entry.getKey());
row.withValue("meanMs", SnapshotCollector.divide(SnapshotCollector.decimal(row, "totalMs"), SnapshotCollector.number(row, "calls")))
.withValue("hitRatio", SnapshotCollector.ratio(counts[0], counts[0] + counts[1]))
.withValue("shareOfDbTime", denominator.signum() == 0 ? BigDecimal.ZERO : SnapshotCollector.decimal(row, "totalMs").multiply(BigDecimal.valueOf(100)).divide(denominator, 4, RoundingMode.HALF_UP));
}
replace("db_perf_query_daily", day, new ArrayList<>(groups.values()));
}
/*******************************************************************************
** Sum scan/write deltas, keeping the last size and tuple gauges of the day.
*******************************************************************************/
private void rollupTables(LocalDate day, Instant start, Instant end) throws Exception
{
Map<Integer, QRecord> groups = new LinkedHashMap<>();
for(QRecord row : DbPerfStore.query("db_perf_table_snapshot", range("capturedAt", start, end).withOrderBy(new QFilterOrderBy("capturedAt", true)).withOrderBy(new QFilterOrderBy("id", true))))
{
QRecord aggregate = groups.computeIfAbsent(row.getValueInteger("tableStatId"), key -> new QRecord().withValue("tableStatId", key).withValue("day", day));
sum(aggregate, row, "seqScan", "idxScan", "seqTupRead", "nTupIns", "nTupUpd", "nTupDel");
for(String gauge : List.of("nLiveTup", "nDeadTup", "totalBytes"))
{
aggregate.setValue(gauge, row.getValue(gauge));
}
}
replace("db_perf_table_daily", day, new ArrayList<>(groups.values()));
}
/*******************************************************************************
** Merge pods by all attribution dimensions, using an unambiguous length-
** prefixed digest for null-safe uniqueness and bounded index size.
*******************************************************************************/
private void rollupAttribution(LocalDate day, Instant start, Instant end) throws Exception
{
Map<String, QRecord> groups = new LinkedHashMap<>();
List<String> dimensions = List.of("unitKind", "unitName", "processName", "tableName", "operation", "fingerprint");
DbPerfStore.forEachPage("db_perf_attribution", range("windowStart", start, end), DbPerfStore.BATCH_SIZE, page -> mergeAttribution(day, page, dimensions, groups));
replace("db_perf_attribution_daily", day, new ArrayList<>(groups.values()));
}
/*******************************************************************************
** Fold one page of attribution detail into the day's groups. Memory is
** bounded by distinct dimension keys, never by the day's detail row count.
*******************************************************************************/
private static void mergeAttribution(LocalDate day, List<QRecord> page, List<String> dimensions, Map<String, QRecord> groups)
{
for(QRecord row : page)
{
StringBuilder key = new StringBuilder();
for(String field : dimensions)
{
String value = row.getValueString(field);
key.append(value == null ? "-1:" : value.length() + ":" + value);
}
String hash = SqlNormalizer.fingerprint(key.toString());
QRecord aggregate = groups.computeIfAbsent(hash, ignored -> new QRecord().withValue("day", day).withValue("rollupKey", hash));
for(String field : dimensions)
{
aggregate.setValue(field, row.getValue(field));
}
aggregate.setValue("sqlText", row.getValueString("sqlText"));
sum(aggregate, row, "units", "statements", "rowsCount", "errors");
sumDecimal(aggregate, row, "totalMs");
aggregate.setValue("maxMs", SnapshotCollector.decimal(aggregate, "maxMs").max(SnapshotCollector.decimal(row, "maxMs")));
}
}
/*******************************************************************************
** Half-open day window works on 23/25-hour local days too.
*******************************************************************************/
private static QQueryFilter range(String field, Instant start, Instant end)
{
return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, start), new QFilterCriteria(field, QCriteriaOperator.LESS_THAN, end)));
}
/*******************************************************************************
** Replace inside the caller's transaction.
*******************************************************************************/
private static void replace(String table, LocalDate day, List<QRecord> rows) throws Exception
{
DbPerfStore.delete(table, DbPerfStore.eq("day", day));
DbPerfStore.insert(table, rows);
}
/*******************************************************************************
** Sum integral counters.
*******************************************************************************/
private static void sum(QRecord target, QRecord source, String... fields)
{
for(String field : fields)
{
target.setValue(field, SnapshotCollector.number(target, field) + SnapshotCollector.number(source, field));
}
}
/*******************************************************************************
** Sum elapsed time without floating-point precision loss.
*******************************************************************************/
private static void sumDecimal(QRecord target, QRecord source, String field)
{
target.setValue(field, SnapshotCollector.decimal(target, field).add(SnapshotCollector.decimal(source, field)));
}
}
@@ -0,0 +1,659 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.collect;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.sql.Connection;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qbits.dbperformance.provider.ConnectionCount;
import com.kingsrook.qbits.dbperformance.provider.DatabaseStats;
import com.kingsrook.qbits.dbperformance.provider.DbStatsProvider;
import com.kingsrook.qbits.dbperformance.provider.DbStatsProviders;
import com.kingsrook.qbits.dbperformance.provider.IndexStats;
import com.kingsrook.qbits.dbperformance.provider.ProviderCapabilities;
import com.kingsrook.qbits.dbperformance.provider.SessionInfo;
import com.kingsrook.qbits.dbperformance.provider.StatementStats;
import com.kingsrook.qbits.dbperformance.provider.TableStats;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** Transactional delta collection; raw JDBC is used only for statistics reads.
*******************************************************************************/
public class SnapshotCollector
{
private static final QLogger LOG = QLogger.getLogger(SnapshotCollector.class);
private final DbPerfRuntime runtime;
private final Clock clock;
private final DbStatsProvider providerOverride;
/*******************************************************************************
** Production provider is chosen from the backend connection.
*******************************************************************************/
public SnapshotCollector(DbPerfRuntime runtime)
{
this(runtime, Clock.systemUTC(), null);
}
/*******************************************************************************
** Explicit clock and provider allow deterministic reset/failure tests.
*******************************************************************************/
public SnapshotCollector(DbPerfRuntime runtime, Clock clock, DbStatsProvider providerOverride)
{
this.runtime = runtime;
this.clock = clock;
this.providerOverride = providerOverride;
}
/*******************************************************************************
** Serialize within an instance and commit counters with their detail rows.
** A failure leaves a FAILED run record and never escapes to the scheduler.
*******************************************************************************/
public QRecord collect(String backendName, boolean forceTables)
{
synchronized(runtime)
{
long started = System.nanoTime();
QRecord snapshot = new QRecord().withValue("backendName", backendName).withValue("capturedAt", clock.instant()).withValue("status", "RUNNING");
try
{
if(!Objects.equals(backendName, runtime.getConfig().getBackendName()))
{
throw (new IllegalArgumentException("Backend is not configured for this runtime"));
}
QRecord settings = new DbPerfSettingsService(runtime).ensure(backendName);
failAbandonedRuns(backendName, settings);
QRecord inserted = runtime.asSystem(() -> DbPerfStore.insert("db_perf_snapshot", List.of(snapshot)).getFirst());
snapshot.setValue("id", inserted.getValueInteger("id"));
runtime.transaction(() ->
{
if(!Boolean.TRUE.equals(settings.getValueBoolean("enabled")))
{
snapshot.withValue("status", "PARTIAL").withValue("errorMessage", "Collection is disabled in Settings.");
}
else
{
collectInTransaction(backendName, forceTables, settings, snapshot);
}
snapshot.setValue("durationMs", (int) Math.min(Integer.MAX_VALUE, (System.nanoTime() - started) / 1000000));
DbPerfStore.update("db_perf_snapshot", List.of(snapshot));
return (null);
});
}
catch(Throwable e)
{
//////////////////////////////////////////////////////////////////////////
// Errors too (a StackOverflowError once left runs RUNNING with no log). //
// Never persist or log a JDBC exception message: it may contain SQL //
// literals. The class and the throwing frame identify the cause. //
//////////////////////////////////////////////////////////////////////////
if(e instanceof OutOfMemoryError outOfMemory)
{
throw (outOfMemory);
}
LOG.error("Database performance snapshot failed", logPair("backend", backendName), logPair("snapshotId", snapshot.getValueInteger("id")),
logPair("exception", e.getClass().getName()), logPair("at", e.getStackTrace().length == 0 ? null : e.getStackTrace()[0].toString()));
snapshot.withValue("status", "FAILED").withValue("errorMessage", "Collection failed (" + e.getClass().getSimpleName() + "). Check database connectivity, schema and statistics access.")
.withValue("durationMs", (int) Math.min(Integer.MAX_VALUE, (System.nanoTime() - started) / 1000000));
if(snapshot.getValueInteger("id") != null)
{
try
{
runtime.asSystem(() ->
{
DbPerfStore.update("db_perf_snapshot", List.of(new QRecord().withValue("id", snapshot.getValueInteger("id"))
.withValue("status", "FAILED").withValue("errorMessage", snapshot.getValueString("errorMessage")).withValue("durationMs", snapshot.getValueInteger("durationMs"))));
return (null);
});
}
catch(Exception persistenceFailure)
{
snapshot.setValue("errorMessage", snapshot.getValueString("errorMessage") + " Failure status could not be saved.");
}
}
}
new com.kingsrook.qbits.dbperformance.alerts.SnapshotEvaluator(runtime).evaluate(snapshot);
return (snapshot);
}
}
/*******************************************************************************
** Runs left RUNNING by a crashed pod or an earlier failure are closed as
** FAILED once they are older than three collection intervals.
*******************************************************************************/
private void failAbandonedRuns(String backendName, QRecord settings) throws Exception
{
int interval = positive(settings, "snapshotIntervalMinutes", 5);
Instant cutoff = clock.instant().minus(Duration.ofMinutes(3L * interval));
List<QRecord> abandoned = runtime.asSystem(() -> DbPerfStore.query("db_perf_snapshot", DbPerfStore.eq("backendName", backendName)
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "RUNNING"))
.withCriteria(new QFilterCriteria("capturedAt", QCriteriaOperator.LESS_THAN, cutoff))));
if(abandoned.isEmpty())
{
return;
}
List<QRecord> updates = abandoned.stream().map(row -> new QRecord().withValue("id", row.getValueInteger("id")).withValue("status", "FAILED")
.withValue("errorMessage", "Collection did not finish (the run was interrupted or its pod stopped).")).toList();
runtime.asSystem(() ->
{
DbPerfStore.update("db_perf_snapshot", updates);
return (null);
});
LOG.warn("Closed abandoned database performance snapshots", logPair("backend", backendName), logPair("count", updates.size()));
}
/*******************************************************************************
** Statistics read and writes under the ambient system transaction.
*******************************************************************************/
private void collectInTransaction(String backendName, boolean forceTables, QRecord settings, QRecord snapshot) throws Exception
{
RDBMSBackendMetaData storageBackend = (RDBMSBackendMetaData) runtime.getInstance().getBackend(backendName);
if(storageBackend.getActionStrategy().supportsForUpdateSkipLocked())
{
var locked = new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput("db_perf_settings")
.withFilter(DbPerfStore.eq("backendName", backendName)).withTransaction(com.kingsrook.qqq.backend.core.context.QContext.getQBackendTransaction())
.withLockMode(com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryLockMode.FOR_UPDATE_SKIP_LOCKED)).getRecords();
if(locked.isEmpty())
{
snapshot.withValue("status", "PARTIAL").withValue("errorMessage", "Another collector is already running for this backend.");
return;
}
settings = locked.getFirst();
if(!Boolean.TRUE.equals(settings.getValueBoolean("enabled")))
{
snapshot.withValue("status", "PARTIAL").withValue("errorMessage", "Collection is disabled in Settings.");
return;
}
}
Instant now = snapshot.getValueInstant("capturedAt");
QQueryFilter filter = DbPerfStore.eq("backendName", backendName)
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.IN, List.of("OK", "PARTIAL")))
.withCriteria(new QFilterCriteria("rawXactCommit", QCriteriaOperator.IS_NOT_BLANK))
.withOrderBy(new QFilterOrderBy("capturedAt", false));
QRecord previous = DbPerfStore.first("db_perf_snapshot", filter);
Instant previousCapture = previous == null ? null : previous.getValueInstant("capturedAt");
snapshot.withValue("intervalSeconds", previousCapture == null ? 0 : Math.max(0, (int) Duration.between(previousCapture, now).toSeconds()))
.withValue("statsResetDetected", false).withValue("includesTables", false);
RDBMSBackendMetaData backend = (RDBMSBackendMetaData) runtime.getInstance().getBackend(backendName);
try(Connection connection = ConnectionManager.getConnection(backend))
{
connection.setReadOnly(true);
DbStatsProvider provider = providerOverride == null ? DbStatsProviders.forConnection(connection) : providerOverride;
ProviderCapabilities capability = provider.capabilities(connection);
settings.withValue("provider", provider.getName()).withValue("statsAccess", capability.statsAccess()).withValue("statsAccessMessage", capability.message());
if(capability.supported())
{
DatabaseStats database = provider.readDatabase(connection);
boolean databaseReset = previous != null && database.statsReset() != null && !Objects.equals(database.statsReset(), previous.getValueInstant("rawStatsReset"));
if(previous != null)
{
for(Map.Entry<String, Long> entry : database.counters().entrySet())
{
String raw = "raw" + Character.toUpperCase(entry.getKey().charAt(0)) + entry.getKey().substring(1);
if(previous.getValueLong(raw) != null && entry.getValue() < previous.getValueLong(raw))
{
databaseReset = true;
}
}
}
for(Map.Entry<String, Long> entry : database.counters().entrySet())
{
String raw = "raw" + Character.toUpperCase(entry.getKey().charAt(0)) + entry.getKey().substring(1);
Long before = previous == null ? null : previous.getValueLong(raw);
snapshot.withValue(entry.getKey(), DeltaCalculator.counter(before, entry.getValue(), databaseReset)).withValue(raw, entry.getValue());
if(before != null && entry.getValue() < before)
{
databaseReset = true;
}
}
snapshot.withValue("rawStatsReset", database.statsReset()).withValue("statsResetDetected", databaseReset)
.withValue("databaseSizeBytes", database.sizeBytes()).withValue("maxConnections", database.maxConnections())
.withValue("cacheHitRatio", ratio(number(snapshot, "blksHit"), number(snapshot, "blksHit") + number(snapshot, "blksRead")));
if(capability.statements())
{
Instant reset = provider.readStatementResetInfo(connection).statsReset();
snapshot.setValue("rawStatementStatsReset", reset);
if(reset != null && previousCapture != null && reset.isAfter(previousCapture))
{
snapshot.setValue("statsResetDetected", true);
}
collectStatements(backendName, provider.readStatements(connection), previousCapture, reset, snapshot, settings);
}
List<ConnectionCount> connections = provider.readConnections(connection);
collectConnections(connections, snapshot);
collectSessions(provider.readSessions(connection), snapshot, settings, backendName);
Instant lastTables = settings.getValueInstant("lastTableSnapshotAt");
if(forceTables || lastTables == null || !lastTables.plusSeconds(60L * positive(settings, "tableSnapshotMinutes", 60)).isAfter(now))
{
collectTables(backendName, provider.readTables(connection), provider.readIndexes(connection), snapshot, databaseReset);
settings.setValue("lastTableSnapshotAt", now);
snapshot.setValue("includesTables", true);
}
}
snapshot.withValue("status", "FULL".equals(capability.statsAccess()) ? "OK" : "PARTIAL")
.withValue("errorMessage", "FULL".equals(capability.statsAccess()) ? null : capability.message());
QRecord runtimeSettings = new QRecord().withValue("id", settings.getValueInteger("id")).withValue("lastSnapshotAt", now)
.withValue("provider", settings.getValueString("provider")).withValue("statsAccess", settings.getValueString("statsAccess"))
.withValue("statsAccessMessage", settings.getValueString("statsAccessMessage"));
if(Boolean.TRUE.equals(snapshot.getValueBoolean("includesTables")))
{
runtimeSettings.setValue("lastTableSnapshotAt", now);
}
DbPerfStore.update("db_perf_settings", List.of(runtimeSettings));
}
}
/*******************************************************************************
** Upsert all statement baselines, but retain only the top N interval rows.
** The denominator and total calls include every observed statement delta.
*******************************************************************************/
private void collectStatements(String backend, List<StatementStats> current, Instant previousCapture, Instant reset, QRecord snapshot, QRecord settings) throws Exception
{
Map<String, QRecord> previous = new HashMap<>();
for(QRecord row : DbPerfStore.query("db_perf_statement", DbPerfStore.eq("backendName", backend)))
{
previous.put(statementKey(row.getValueLong("queryId"), row.getValueString("dbUser"), Boolean.TRUE.equals(row.getValueBoolean("toplevel"))), row);
}
List<QRecord> inserts = new ArrayList<>();
List<QRecord> updates = new ArrayList<>();
List<QRecord> deltas = new ArrayList<>();
Map<QRecord, String> keys = new LinkedHashMap<>();
BigDecimal total = BigDecimal.ZERO;
long calls = 0;
Instant now = snapshot.getValueInstant("capturedAt");
for(StatementStats entry : current)
{
String key = statementKey(entry.queryId(), entry.dbUser(), entry.toplevel());
QRecord old = previous.get(key);
DeltaCalculator.Delta delta = DeltaCalculator.calculate(old == null ? null : storedStatement(old), entry, previousCapture, reset);
String text = SqlNormalizer.normalize(entry.queryText());
QRecord row = new QRecord().withValue("backendName", backend).withValue("queryId", entry.queryId()).withValue("dbUser", entry.dbUser()).withValue("toplevel", entry.toplevel())
.withValue("queryText", text).withValue("fingerprint", SqlNormalizer.fingerprint(text)).withValue("operation", SqlNormalizer.operation(text)).withValue("mainTable", SqlNormalizer.mainTable(text))
.withValue("lastSeenAt", now).withValue("lastCalls", entry.calls()).withValue("lastTotalExecMs", entry.totalExecMs()).withValue("lastRows", entry.rows())
.withValue("lastSharedBlksHit", entry.sharedBlksHit()).withValue("lastSharedBlksRead", entry.sharedBlksRead()).withValue("lastTempBlksWritten", entry.tempBlksWritten())
.withValue("lastMaxExecMs", entry.maxExecMs()).withValue("lastStatsSince", entry.statsSince());
if(old == null)
{
row.setValue("firstSeenAt", now);
inserts.add(row);
}
else
{
row.setValue("id", old.getValueInteger("id"));
updates.add(row);
}
if(delta.reset())
{
snapshot.setValue("statsResetDetected", true);
}
total = total.add(delta.totalMs());
calls += delta.calls();
if(!delta.baseline() && (delta.calls() > 0 || delta.totalMs().signum() > 0))
{
QRecord detail = new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", now)
.withValue("calls", delta.calls()).withValue("totalMs", delta.totalMs()).withValue("meanMs", divide(delta.totalMs(), delta.calls()))
.withValue("maxMs", entry.maxExecMs()).withValue("rowsCount", delta.rows()).withValue("sharedBlksHit", delta.sharedBlksHit())
.withValue("sharedBlksRead", delta.sharedBlksRead()).withValue("hitRatio", ratio(delta.sharedBlksHit(), delta.sharedBlksHit() + delta.sharedBlksRead()))
.withValue("tempBlksWritten", delta.tempBlksWritten());
deltas.add(detail);
keys.put(detail, key);
}
}
for(QRecord inserted : DbPerfStore.insert("db_perf_statement", inserts))
{
previous.put(statementKey(inserted.getValueLong("queryId"), inserted.getValueString("dbUser"), Boolean.TRUE.equals(inserted.getValueBoolean("toplevel"))), inserted);
}
DbPerfStore.update("db_perf_statement", updates);
deltas.sort(Comparator.comparing((QRecord row) -> row.getValueBigDecimal("totalMs")).reversed());
List<QRecord> top = deltas.subList(0, Math.min(deltas.size(), positive(settings, "topStatementsPerSnapshot", 200)));
for(QRecord row : top)
{
row.withValue("statementId", previous.get(keys.get(row)).getValueInteger("id"))
.withValue("shareOfDbTime", total.signum() == 0 ? BigDecimal.ZERO : row.getValueBigDecimal("totalMs").multiply(BigDecimal.valueOf(100)).divide(total, 4, RoundingMode.HALF_UP));
}
DbPerfStore.insert("db_perf_query_snapshot", top);
snapshot.withValue("statementCount", current.size()).withValue("statementCalls", calls).withValue("statementTotalMs", total).withValue("storedStatementCount", top.size());
refresh24hours(backend, previous, current, snapshot);
}
/*******************************************************************************
** Stored counters have the same shape as provider counters.
*******************************************************************************/
private StatementStats storedStatement(QRecord row)
{
return (new StatementStats(number(row, "queryId"), row.getValueString("dbUser"), Boolean.TRUE.equals(row.getValueBoolean("toplevel")), row.getValueString("queryText"), number(row, "lastCalls"), decimal(row, "lastTotalExecMs"), BigDecimal.ZERO, decimal(row, "lastMaxExecMs"), number(row, "lastRows"), number(row, "lastSharedBlksHit"), number(row, "lastSharedBlksRead"), number(row, "lastTempBlksWritten"), row.getValueInstant("lastStatsSince"), null));
}
/*******************************************************************************
** Recompute touched statements against the exact all-statement denominator.
*******************************************************************************/
private void refresh24hours(String backend, Map<String, QRecord> known, List<StatementStats> touched, QRecord snapshot) throws Exception
{
Instant since = snapshot.getValueInstant("capturedAt").minusSeconds(86400);
BigDecimal denominator = decimal(snapshot, "statementTotalMs");
for(QRecord row : DbPerfStore.query("db_perf_snapshot", DbPerfStore.eq("backendName", backend).withCriteria(new QFilterCriteria("capturedAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since))
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.IN, List.of("OK", "PARTIAL")))))
{
denominator = denominator.add(decimal(row, "statementTotalMs"));
}
Map<Integer, List<QRecord>> details = DbPerfStore.query("db_perf_query_snapshot", new QQueryFilter(new QFilterCriteria("capturedAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)))
.stream().collect(Collectors.groupingBy(row -> row.getValueInteger("statementId")));
//////////////////////////////////////////////////////////////////////////
// refresh every statement with activity in the window, plus any whose //
// stored 24h numbers are non-zero (so idle statements decay to zero) //
//////////////////////////////////////////////////////////////////////////
Set<Integer> ids = new LinkedHashSet<>(details.keySet());
for(StatementStats entry : touched)
{
ids.add(known.get(statementKey(entry.queryId(), entry.dbUser(), entry.toplevel())).getValueInteger("id"));
}
for(QRecord row : known.values())
{
if(row.getValueInteger("id") != null && number(row, "calls24h") != 0)
{
ids.add(row.getValueInteger("id"));
}
}
List<QRecord> updates = new ArrayList<>();
for(Integer id : ids)
{
List<QRecord> rows = details.getOrDefault(id, List.of());
long calls = rows.stream().mapToLong(row -> number(row, "calls")).sum();
BigDecimal total = rows.stream().map(row -> decimal(row, "totalMs")).reduce(BigDecimal.ZERO, BigDecimal::add);
updates.add(new QRecord().withValue("id", id).withValue("calls24h", calls).withValue("totalMs24h", total).withValue("meanMs24h", divide(total, calls))
.withValue("shareOfDbTime24h", denominator.signum() == 0 ? BigDecimal.ZERO : total.multiply(BigDecimal.valueOf(100)).divide(denominator, 4, RoundingMode.HALF_UP)));
}
DbPerfStore.update("db_perf_statement", updates);
}
/*******************************************************************************
** Upsert table/index gauges; snapshot scan/write/maintenance counters as deltas.
*******************************************************************************/
private void collectTables(String backend, List<TableStats> tables, List<IndexStats> indexes, QRecord snapshot, boolean reset) throws Exception
{
Map<String, QRecord> known = new HashMap<>();
for(QRecord row : DbPerfStore.query("db_perf_table_stat", DbPerfStore.eq("backendName", backend)))
{
known.put(row.getValueString("schemaName") + "." + row.getValueString("tableName"), row);
}
List<QRecord> inserts = new ArrayList<>();
List<QRecord> updates = new ArrayList<>();
List<QRecord> details = new ArrayList<>();
Map<QRecord, String> detailKeys = new LinkedHashMap<>();
Instant now = snapshot.getValueInstant("capturedAt");
for(TableStats entry : tables)
{
String key = entry.schemaName() + "." + entry.tableName();
QRecord old = known.get(key);
QRecord row = new QRecord().withValue("backendName", backend).withValue("schemaName", entry.schemaName()).withValue("tableName", entry.tableName()).withValue("relid", entry.relid()).withValue("lastSeenAt", now);
entry.counters().forEach(row::setValue);
entry.maintenance().forEach(row::setValue);
row.withValue("deadRatio", ratio(number(row, "nDeadTup"), number(row, "nLiveTup") + number(row, "nDeadTup")))
.withValue("seqScanShare", ratio(number(row, "seqScan"), number(row, "seqScan") + number(row, "idxScan")));
if(old == null)
{
inserts.add(row);
}
else
{
row.setValue("id", old.getValueInteger("id"));
updates.add(row);
}
if(old != null)
{
QRecord weekAgo = DbPerfStore.first("db_perf_table_snapshot", DbPerfStore.eq("tableStatId", old.getValueInteger("id"))
.withCriteria(new QFilterCriteria("capturedAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, now.minusSeconds(7 * 86400L)))
.withOrderBy(new QFilterOrderBy("capturedAt", false)));
row.setValue("growthBytes7d", weekAgo == null ? null : number(row, "totalBytes") - number(weekAgo, "totalBytes"));
}
QRecord detail = new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", now);
boolean tableReset = reset || old != null && number(old, "relid") != entry.relid();
for(String counter : List.of("seqScan", "seqTupRead", "idxScan", "idxTupFetch", "nTupIns", "nTupUpd", "nTupDel", "nTupHotUpd", "autovacuumCount", "autoanalyzeCount"))
{
detail.setValue(counter, DeltaCalculator.counter(old == null ? null : old.getValueLong(counter), number(row, counter), tableReset));
}
for(String gauge : List.of("nLiveTup", "nDeadTup", "totalBytes", "tableBytes", "indexBytes", "freezeAge"))
{
detail.setValue(gauge, number(row, gauge));
}
details.add(detail);
detailKeys.put(detail, key);
}
for(QRecord inserted : DbPerfStore.insert("db_perf_table_stat", inserts))
{
known.put(inserted.getValueString("schemaName") + "." + inserted.getValueString("tableName"), inserted);
}
DbPerfStore.update("db_perf_table_stat", updates);
for(QRecord detail : details)
{
detail.setValue("tableStatId", known.get(detailKeys.get(detail)).getValueInteger("id"));
}
DbPerfStore.insert("db_perf_table_snapshot", details);
collectIndexes(backend, indexes, known, now);
}
/*******************************************************************************
** Indexes reference collected tables and are written in batches.
*******************************************************************************/
private void collectIndexes(String backend, List<IndexStats> entries, Map<String, QRecord> tables, Instant now) throws Exception
{
Map<String, QRecord> old = new HashMap<>();
for(QRecord row : DbPerfStore.query("db_perf_index_stat", DbPerfStore.eq("backendName", backend)))
{
old.put(row.getValueString("schemaName") + "." + row.getValueString("indexName"), row);
}
List<QRecord> inserts = new ArrayList<>();
List<QRecord> updates = new ArrayList<>();
for(IndexStats entry : entries)
{
QRecord table = tables.get(entry.schemaName() + "." + entry.tableName());
if(table == null)
{
continue;
}
QRecord row = new QRecord().withValue("backendName", backend).withValue("tableStatId", table.getValueInteger("id"))
.withValue("schemaName", entry.schemaName()).withValue("tableName", entry.tableName()).withValue("indexName", entry.indexName()).withValue("relid", entry.relid())
.withValue("definition", truncate(entry.definition(), 2000)).withValue("columns", truncate(entry.columns(), 1000)).withValue("isUnique", entry.unique()).withValue("isPrimary", entry.primary()).withValue("isValid", entry.valid())
.withValue("idxScan", entry.idxScan()).withValue("idxTupRead", entry.idxTupRead()).withValue("idxTupFetch", entry.idxTupFetch()).withValue("sizeBytes", entry.sizeBytes()).withValue("writeCost", entry.writeCost())
.withValue("scansSinceReset", entry.idxScan()).withValue("lastSeenAt", now);
QRecord previous = old.get(entry.schemaName() + "." + entry.indexName());
if(previous == null)
{
inserts.add(row);
}
else
{
row.setValue("id", previous.getValueInteger("id"));
updates.add(row);
}
}
DbPerfStore.insert("db_perf_index_stat", inserts);
DbPerfStore.update("db_perf_index_stat", updates);
}
/*******************************************************************************
** All-database connection counts for capacity comparison.
*******************************************************************************/
private void collectConnections(List<ConnectionCount> connections, QRecord snapshot) throws Exception
{
List<QRecord> rows = new ArrayList<>();
int total = 0;
int active = 0;
int idle = 0;
int idleTx = 0;
for(ConnectionCount entry : connections)
{
total += entry.connectionCount();
active += "active".equals(entry.state()) ? entry.connectionCount() : 0;
idle += "idle".equals(entry.state()) ? entry.connectionCount() : 0;
idleTx += entry.state() != null && entry.state().startsWith("idle in transaction") ? entry.connectionCount() : 0;
rows.add(new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", snapshot.getValueInstant("capturedAt"))
.withValue("dbUser", entry.dbUser()).withValue("applicationName", truncate(entry.applicationName(), 200)).withValue("state", entry.state()).withValue("connectionCount", entry.connectionCount()));
}
DbPerfStore.insert("db_perf_connection_sample", rows);
snapshot.withValue("connectionsTotal", total).withValue("connectionsActive", active).withValue("connectionsIdle", idle).withValue("connectionsIdleInTx", idleTx);
}
/*******************************************************************************
** Persist only notable sessions; normalize before truncating or hashing.
*******************************************************************************/
private void collectSessions(List<SessionInfo> sessions, QRecord snapshot, QRecord settings, String backend) throws Exception
{
Map<Integer, Integer> blocking = new HashMap<>();
for(SessionInfo session : sessions)
{
session.blockedBy().forEach(pid -> blocking.merge(pid, 1, Integer::sum));
}
Map<String, QRecord> statements = new HashMap<>();
for(QRecord row : DbPerfStore.query("db_perf_statement", DbPerfStore.eq("backendName", backend)))
{
if(Boolean.TRUE.equals(row.getValueBoolean("toplevel")))
{
statements.put(row.getValueString("queryId") + "/" + row.getValueString("dbUser"), row);
}
}
List<QRecord> rows = new ArrayList<>();
for(SessionInfo entry : sessions)
{
String reason = !entry.blockedBy().isEmpty() ? "BLOCKED" : blocking.containsKey(entry.pid()) ? "BLOCKING"
: entry.state() != null && entry.state().startsWith("idle in transaction") && entry.stateChange() != null && Duration.between(entry.stateChange(), snapshot.getValueInstant("capturedAt")).getSeconds() >= positive(settings, "idleInTransactionSeconds", 60) ? "IDLE_IN_TRANSACTION"
: "active".equals(entry.state()) && entry.querySeconds() != null && entry.querySeconds().doubleValue() >= positive(settings, "activityLongRunningSeconds", 30) ? "LONG_RUNNING" : null;
if(reason == null)
{
continue;
}
QRecord statement = statements.get(entry.queryId() + "/" + entry.dbUser());
String normalized = SqlNormalizer.normalize(statement == null ? entry.queryText() : statement.getValueString("queryText"));
rows.add(new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", snapshot.getValueInstant("capturedAt"))
.withValue("pid", entry.pid()).withValue("dbUser", entry.dbUser()).withValue("applicationName", truncate(entry.applicationName(), 200)).withValue("clientAddr", entry.clientAddr())
.withValue("state", entry.state()).withValue("waitEventType", entry.waitEventType()).withValue("waitEvent", entry.waitEvent()).withValue("backendType", entry.backendType())
.withValue("xactStart", entry.xactStart()).withValue("queryStart", entry.queryStart()).withValue("stateChange", entry.stateChange()).withValue("querySeconds", entry.querySeconds()).withValue("xactSeconds", entry.xactSeconds())
.withValue("blockedBy", truncate(entry.blockedBy().stream().map(String::valueOf).collect(Collectors.joining(",")), 500)).withValue("blockingCount", blocking.getOrDefault(entry.pid(), 0))
.withValue("reason", reason).withValue("queryId", entry.queryId()).withValue("statementId", statement == null ? null : statement.getValueInteger("id"))
.withValue("queryFingerprint", SqlNormalizer.fingerprint(normalized)).withValue("queryTextNormalized", truncate(normalized, 4000)));
}
DbPerfStore.insert("db_perf_activity_sample", rows);
}
/*******************************************************************************
** Stable statement identity including role and nesting level.
*******************************************************************************/
private static String statementKey(long queryId, String user, boolean top)
{
return (queryId + "/" + top + "/" + user);
}
/*******************************************************************************
** Null counters are zero.
*******************************************************************************/
public static long number(QRecord row, String field)
{
Long value = row.getValueLong(field);
return (value == null ? 0 : value);
}
/*******************************************************************************
** Null decimal counters are zero.
*******************************************************************************/
public static BigDecimal decimal(QRecord row, String field)
{
BigDecimal value = row.getValueBigDecimal(field);
return (value == null ? BigDecimal.ZERO : value);
}
/*******************************************************************************
** Percentage, not a fractional ratio; matches seeded rule thresholds.
*******************************************************************************/
public static BigDecimal ratio(long numerator, long denominator)
{
return (denominator == 0 ? BigDecimal.ZERO : BigDecimal.valueOf(numerator).multiply(BigDecimal.valueOf(100)).divide(BigDecimal.valueOf(denominator), 4, RoundingMode.HALF_UP));
}
/*******************************************************************************
** Weighted mean with a zero-call guard.
*******************************************************************************/
public static BigDecimal divide(BigDecimal total, long count)
{
return (count == 0 ? BigDecimal.ZERO : total.divide(BigDecimal.valueOf(count), 3, RoundingMode.HALF_UP));
}
/*******************************************************************************
** Invalid persisted intervals fall back to safe positive defaults.
*******************************************************************************/
public static int positive(QRecord settings, String field, int fallback)
{
Integer value = settings.getValueInteger(field);
return (value == null || value < 1 ? fallback : value);
}
/*******************************************************************************
** Bound display strings after normalization where applicable.
*******************************************************************************/
private static String truncate(String value, int length)
{
return (value == null ? null : value.substring(0, Math.min(length, value.length())));
}
}
@@ -0,0 +1,42 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.util.List;
/*******************************************************************************
** Transient explain input. Parameters must never be persisted or logged.
*******************************************************************************/
public record ExplainRequest(String backendName, Integer statementId, String sql, Mode mode, List<String> parameters)
{
/*******************************************************************************
** Requested execution mode; GENERIC is selected automatically for PLAN.
*******************************************************************************/
public enum Mode
{
PLAN, ANALYZE
}
}
@@ -0,0 +1,34 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.math.BigDecimal;
import java.util.List;
/*******************************************************************************
** Transient response; ANALYZE plans are returned only to the authorized caller.
*******************************************************************************/
public record ExplainResult(String mode, String planJson, String planText, BigDecimal totalCost, BigDecimal actualTotalMs,
List<String> warnings, Integer explainId, String sqlWorkbenchUrl, String assistantPrompt)
{
}
@@ -0,0 +1,312 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Types;
import java.time.Instant;
import java.util.Collections;
import java.util.List;
import java.util.UUID;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfSettingsService;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qbits.dbperformance.collect.SnapshotCollector;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import org.postgresql.core.QueryExecutor;
import org.postgresql.jdbc.PgStatement;
/*******************************************************************************
** Audited, read-only plan execution on a dedicated monitored-backend connection.
** Neither database error messages nor supplied values enter the persistent audit.
*******************************************************************************/
public class ExplainService
{
private final DbPerfRuntime runtime;
/*******************************************************************************
** Use the registered host runtime.
*******************************************************************************/
public ExplainService(DbPerfRuntime runtime)
{
this.runtime = runtime;
}
/*******************************************************************************
** Audit every request, including validation and authorization failures.
*******************************************************************************/
public ExplainResult explain(ExplainRequest request) throws Exception
{
long started = System.nanoTime();
QSession session = QContext.getQSession();
QRecord audit = new QRecord().withValue("requestedAt", Instant.now()).withValue("outcome", "ERROR")
.withValue("requestedBy", session == null || session.getUser() == null ? null : session.getUser().getIdReference())
.withValue("mode", request != null && request.mode() == ExplainRequest.Mode.ANALYZE ? "ANALYZE" : "PLAN")
.withValue("parameterCount", request == null || request.parameters() == null ? 0 : request.parameters().size());
String sql = request == null ? "" : request.sql();
String json = null;
PlanTextRenderer.Plan plan = null;
QException failure = null;
List<String> warnings = List.of();
try
{
String permission = runtime.getConfig().getPermissionBaseName() + ("ANALYZE".equals(audit.getValueString("mode")) ? ".explain_analyze" : ".explain");
if(session == null || !session.hasPermission(permission))
{
throw (new QPermissionDeniedException("Query plan permission required: " + permission));
}
if(request == null || request.mode() == null)
{
throw (new QBadRequestException("Choose PLAN or ANALYZE."));
}
if(!runtime.getConfig().getBackendName().equals(request.backendName()))
{
throw (new QBadRequestException("Choose the configured monitored backend."));
}
if((request.statementId() == null) == (sql == null || sql.isBlank()))
{
throw (new QBadRequestException("Supply either a statementId or SQL, not both."));
}
boolean storedStatement = request.statementId() != null;
if(storedStatement)
{
QRecord row = runtime.asSystem(() -> DbPerfStore.first("db_perf_statement", DbPerfStore.eq("id", request.statementId())));
if(row == null || !request.backendName().equals(row.getValueString("backendName")))
{
throw (new QBadRequestException("The statement was not found on the monitored backend."));
}
audit.setValue("statementId", request.statementId());
sql = row.getValueString("queryText");
}
ExplainSql.Parsed parsed = ExplainSql.parse(sql);
List<String> parameters = request.parameters() == null ? List.of() : request.parameters();
audit.setValue("parameterCount", parsed.parameterCount());
boolean analyze = request.mode() == ExplainRequest.Mode.ANALYZE;
if(analyze)
{
parsed.requireSelect();
if(parameters.size() != parsed.parameterCount())
{
throw (new QBadRequestException("Supply exactly " + parsed.parameterCount() + " parameter values for ANALYZE."));
}
}
else if(!parameters.isEmpty())
{
throw (new QBadRequestException("Parameter values are accepted only for ANALYZE."));
}
boolean generic = !analyze && parsed.parameterCount() > 0;
audit.setValue("mode", analyze ? "ANALYZE" : generic ? "GENERIC" : "PLAN");
QRecord settings = new DbPerfSettingsService(runtime).ensure(request.backendName());
json = execute(parsed.sql(), parameters, analyze, generic, SnapshotCollector.positive(settings, "explainTimeoutMs", 10000));
plan = PlanTextRenderer.render(json);
if(!analyze && (storedStatement || sql.equals(SqlNormalizer.normalize(sql))))
{
audit.setValue("planJson", json);
audit.setValue("planText", plan.text());
audit.setValue("totalCost", plan.totalCost());
}
else
{
warnings = List.of("This plan is transient; only a literal-free request audit is stored.");
}
audit.setValue("outcome", "OK");
}
catch(QPermissionDeniedException e)
{
audit.setValue("outcome", "DENIED");
failure = e;
}
catch(QBadRequestException e)
{
failure = e;
}
catch(Exception e)
{
boolean timeout = isTimeout(e);
audit.setValue("outcome", timeout ? "TIMEOUT" : "ERROR");
failure = new QException(timeout ? "Query plan timed out. Reduce the query scope or ask an administrator to review the explain timeout."
: "Query plan failed. Check SQL, parameter types and database access. Writes are not allowed.");
}
finally
{
String normalized = SqlNormalizer.normalize(sql);
audit.setValue("sqlText", normalized);
audit.setValue("sqlFingerprint", SqlNormalizer.fingerprint(normalized));
audit.setValue("durationMs", (System.nanoTime() - started) / 1000000);
audit.setValue("errorMessage", failure == null ? null : failure.getMessage());
QRecord inserted = runtime.asSystem(() -> DbPerfStore.insert("db_perf_explain", List.of(audit)).getFirst());
audit.setValue("id", inserted.getValueInteger("id"));
}
if(failure != null)
{
throw (failure);
}
return (new ExplainResult(audit.getValueString("mode"), json, plan.text(), plan.totalCost(), plan.actualTotalMs(), warnings,
audit.getValueInteger("id"), runtime.getConfig().getSqlWorkbenchRunnerPath() + "?sql=" + encode("EXPLAIN " + sql) + "&backend=" + encode(request.backendName()),
"Explain this PostgreSQL query plan and suggest improvements.\n\nQuery:\n" + SqlNormalizer.normalize(sql) + "\n\nPlan:\n" + plan.safeText()));
}
/*******************************************************************************
** Roll back even on parse, execution or timeout failure, deallocate after rollback,
** and restore pool state before releasing this dedicated connection.
*******************************************************************************/
private String execute(String sql, List<String> parameters, boolean analyze, boolean generic, int timeout) throws Exception
{
try(Connection connection = openConnection())
{
if(!"PostgreSQL".equals(connection.getMetaData().getDatabaseProductName()))
{
throw (new QBadRequestException("Query plans currently require PostgreSQL."));
}
if(generic && connection.getMetaData().getDatabaseMajorVersion() < 16)
{
throw (new QBadRequestException("Generic plans require PostgreSQL 16 or later. Use ANALYZE with parameter values if authorized."));
}
boolean autoCommit = connection.getAutoCommit();
boolean readOnly = connection.isReadOnly();
String preparedName = "dbperf_" + UUID.randomUUID().toString().replace("-", "");
boolean prepared = false;
try
{
connection.setAutoCommit(false);
try(Statement statement = connection.createStatement())
{
statement.execute("SET TRANSACTION READ ONLY");
statement.execute("SET LOCAL statement_timeout = " + timeout);
statement.execute("SET LOCAL standard_conforming_strings = on");
if(!analyze)
{
// GENERIC_PLAN must not be parsed as an external JDBC bind request.
statement.unwrap(PgStatement.class).executeWithFlags("EXPLAIN (" + (generic ? "GENERIC_PLAN, " : "") + "FORMAT JSON) " + sql, QueryExecutor.QUERY_EXECUTE_AS_SIMPLE);
try(ResultSet rows = statement.getResultSet())
{
rows.next();
return (rows.getString(1));
}
}
statement.execute("PREPARE " + preparedName + " AS " + sql);
prepared = true;
}
String arguments = parameters.isEmpty() ? "" : "(" + String.join(", ", Collections.nCopies(parameters.size(), "?")) + ")";
try(PreparedStatement statement = connection.prepareStatement("EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) EXECUTE " + preparedName + arguments))
{
for(int i = 0; i < parameters.size(); i++)
{
statement.setObject(i + 1, parameters.get(i), Types.OTHER);
}
// EXECUTE's arguments are utility expressions: unknown bind types cannot
// be inferred in extended Parse. Keep JDBC binding/escaping, but send this
// one command in simple mode so PostgreSQL coerces the quoted strings.
statement.unwrap(PgStatement.class).executeWithFlags(QueryExecutor.QUERY_EXECUTE_AS_SIMPLE);
try(ResultSet rows = statement.getResultSet())
{
rows.next();
return (rows.getString(1));
}
}
}
finally
{
try
{
connection.rollback();
if(prepared)
{
try(Statement cleanup = connection.createStatement())
{
cleanup.execute("DEALLOCATE " + preparedName);
}
finally
{
connection.rollback();
}
}
}
finally
{
connection.setReadOnly(readOnly);
connection.setAutoCommit(autoCommit);
}
}
}
}
/*******************************************************************************
** Always obtain a dedicated connection, independent of any caller QQQ transaction.
** Protected for contract tests that verify pool state restoration on driver errors.
*******************************************************************************/
protected Connection openConnection() throws SQLException
{
RDBMSBackendMetaData backend = (RDBMSBackendMetaData) runtime.getInstance().getBackend(runtime.getConfig().getBackendName());
return (ConnectionManager.getConnection(backend));
}
/*******************************************************************************
** SQLSTATE survives JDBC wrapper exceptions without exposing their messages.
*******************************************************************************/
private static boolean isTimeout(Throwable exception)
{
for(Throwable current = exception; current != null; current = current.getCause())
{
if(current instanceof SQLException sql && "57014".equals(sql.getSQLState()))
{
return (true);
}
}
return (false);
}
/*******************************************************************************
** UTF-8 query parameter encoding.
*******************************************************************************/
private static String encode(String value)
{
return (URLEncoder.encode(value, StandardCharsets.UTF_8));
}
}
@@ -0,0 +1,257 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
/*******************************************************************************
** Lex executable SQL without changing literals, identifiers or parameter positions.
** This is a conservative SELECT guard; PostgreSQL READ ONLY is the write boundary.
*******************************************************************************/
public final class ExplainSql
{
private static final Pattern DOLLAR_QUOTE = Pattern.compile("\\$(?:[A-Za-z_][A-Za-z_0-9]*)?\\$");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private ExplainSql()
{
}
/*******************************************************************************
** Executable SQL with contiguous PostgreSQL bind slots and unquoted tokens.
*******************************************************************************/
public record Parsed(String sql, List<String> tokens, int parameterCount)
{
/*******************************************************************************
** Reject writes, SELECT INTO and every locking SELECT, including nested CTEs.
*******************************************************************************/
public void requireSelect() throws QBadRequestException
{
if(tokens.isEmpty() || !Set.of("SELECT", "WITH").contains(tokens.getFirst())
|| tokens.stream().anyMatch(Set.of("INSERT", "UPDATE", "DELETE", "MERGE", "INTO", "SHARE")::contains))
{
throw (new QBadRequestException("ANALYZE requires a single SELECT without writes, INTO or row locks."));
}
}
}
/*******************************************************************************
** Skip SQL comments and quoted regions, reject multiple statements, and convert ?.
** Mixing JDBC and PostgreSQL parameter styles is rejected as ambiguous.
*******************************************************************************/
public static Parsed parse(String sql) throws QBadRequestException
{
if(sql == null || sql.isBlank() || sql.length() > 100000)
{
throw (new QBadRequestException("Supply one SQL statement of at most 100000 characters."));
}
StringBuilder result = new StringBuilder();
List<String> tokens = new ArrayList<>();
Set<Integer> positions = new HashSet<>();
int jdbc = 0;
int i = 0;
boolean ended = false;
while(i < sql.length())
{
int start = i;
char c = sql.charAt(i);
if(Character.isWhitespace(c))
{
result.append(c);
i++;
continue;
}
if(sql.startsWith("--", i))
{
int end = sql.indexOf('\n', i);
i = end < 0 ? sql.length() : end;
result.append(' ');
continue;
}
if(sql.startsWith("/*", i))
{
int depth = 1;
i += 2;
while(i < sql.length() && depth > 0)
{
if(sql.startsWith("/*", i))
{
depth++;
i += 2;
}
else if(sql.startsWith("*/", i))
{
depth--;
i += 2;
}
else
{
i++;
}
}
if(depth != 0)
{
throw (new QBadRequestException("Unterminated SQL comment."));
}
result.append(' ');
continue;
}
if(ended)
{
throw (new QBadRequestException("Only one SQL statement is allowed."));
}
if(c == ';')
{
ended = true;
i++;
continue;
}
boolean escaped = (c == 'e' || c == 'E') && i + 1 < sql.length() && sql.charAt(i + 1) == '\'';
if(c == '\'' || c == '"' || escaped)
{
char quote = escaped ? '\'' : c;
i = quotedEnd(sql, i + (escaped ? 1 : 0), quote, escaped);
result.append(sql, start, i);
tokens.add("<QUOTED>");
continue;
}
if(c == '$')
{
Matcher delimiter = DOLLAR_QUOTE.matcher(sql).region(i, sql.length());
if(delimiter.lookingAt())
{
int end = sql.indexOf(delimiter.group(), delimiter.end());
if(end < 0)
{
throw (new QBadRequestException("Unterminated dollar-quoted SQL string."));
}
i = end + delimiter.group().length();
result.append(sql, start, i);
tokens.add("<QUOTED>");
continue;
}
i++;
while(i < sql.length() && Character.isDigit(sql.charAt(i)))
{
i++;
}
if(i > start + 1)
{
try
{
int position = Integer.parseInt(sql.substring(start + 1, i));
if(position < 1 || position > 65535)
{
throw (new NumberFormatException());
}
positions.add(position);
}
catch(NumberFormatException e)
{
throw (new QBadRequestException("SQL parameter positions must be between 1 and 65535."));
}
}
result.append(sql, start, i);
continue;
}
if(c == '?')
{
// PostgreSQL's multi-character JSON operators are not JDBC bind slots.
if(i + 1 < sql.length() && "|&?".indexOf(sql.charAt(i + 1)) >= 0)
{
result.append(sql, i, i + 2);
i += 2;
}
else
{
result.append('$').append(++jdbc);
i++;
}
continue;
}
if(Character.isLetter(c) || c == '_')
{
i++;
while(i < sql.length() && (Character.isLetterOrDigit(sql.charAt(i)) || "_$".indexOf(sql.charAt(i)) >= 0))
{
i++;
}
tokens.add(sql.substring(start, i).toUpperCase(Locale.ROOT));
result.append(sql, start, i);
continue;
}
result.append(c);
i++;
}
int maximum = positions.stream().mapToInt(Integer::intValue).max().orElse(0);
if(jdbc > 0 && maximum > 0 || maximum != positions.size())
{
throw (new QBadRequestException("Use either ? parameters or contiguous $1 through $n parameters."));
}
return (new Parsed(result.toString().trim(), List.copyOf(tokens), Math.max(jdbc, maximum)));
}
/*******************************************************************************
** A quoted region cannot introduce executable keywords or statement delimiters.
*******************************************************************************/
private static int quotedEnd(String sql, int start, char quote, boolean escaped) throws QBadRequestException
{
int i = start + 1;
while(i < sql.length())
{
if(escaped && sql.charAt(i) == '\\')
{
i += 2;
}
else if(sql.charAt(i++) == quote)
{
if(i < sql.length() && sql.charAt(i) == quote)
{
i++;
}
else
{
return (i);
}
}
}
throw (new QBadRequestException("Unterminated SQL quote."));
}
}
@@ -0,0 +1,102 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.math.BigDecimal;
import java.util.List;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.dbperformance.normalize.SqlNormalizer;
/*******************************************************************************
** Readable PostgreSQL JSON plan tree, with a literal-free prompt variant.
*******************************************************************************/
public final class PlanTextRenderer
{
private static final ObjectMapper MAPPER = new ObjectMapper();
/*******************************************************************************
** Utility class.
*******************************************************************************/
private PlanTextRenderer()
{
}
/*******************************************************************************
** A parsed plan and its top-level measures.
*******************************************************************************/
public record Plan(String text, String safeText, BigDecimal totalCost, BigDecimal actualTotalMs)
{
}
/*******************************************************************************
** PostgreSQL emits an array containing the plan and optional execution timing.
*******************************************************************************/
public static Plan render(String json) throws Exception
{
JsonNode document = MAPPER.readTree(json);
JsonNode root = document.isArray() ? document.path(0) : document;
JsonNode plan = root.path("Plan");
if(!plan.isObject())
{
throw (new IllegalArgumentException("PostgreSQL did not return a plan tree."));
}
StringBuilder text = new StringBuilder();
StringBuilder safe = new StringBuilder();
append(plan, 0, text, false);
append(plan, 0, safe, true);
JsonNode actual = root.has("Execution Time") ? root.get("Execution Time") : plan.get("Actual Total Time");
return (new Plan(text.toString().stripTrailing(), safe.toString().stripTrailing(),
plan.has("Total Cost") ? plan.get("Total Cost").decimalValue() : null, actual == null ? null : actual.decimalValue()));
}
/*******************************************************************************
** Preserve tree structure and planner metrics; redact expression literals for AI.
*******************************************************************************/
private static void append(JsonNode node, int depth, StringBuilder out, boolean redact)
{
out.append(" ".repeat(depth)).append(node.path("Node Type").asText("Unknown"));
for(String field : List.of("Relation Name", "Index Name", "Startup Cost", "Total Cost", "Plan Rows", "Actual Total Time", "Actual Rows", "Filter", "Index Cond", "Hash Cond", "Join Filter", "Recheck Cond"))
{
if(node.has(field))
{
String value = node.get(field).asText();
boolean expression = field.endsWith("Cond") || field.endsWith("Filter");
out.append(" | ").append(field).append(": ").append(redact && expression ? SqlNormalizer.normalize(value) : value);
}
}
out.append('\n');
for(JsonNode child : node.path("Plans"))
{
append(child, depth + 1, out, redact);
}
}
}
@@ -0,0 +1,144 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.explain;
import java.util.Arrays;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfPermissions;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QRecordListMetaData;
/*******************************************************************************
** The one-record Query plan action; user-supplied values remain transient.
*******************************************************************************/
public class QueryPlanProcess implements BackendStep
{
public static final String NAME = "dbPerfQueryPlan";
/*******************************************************************************
** Register form, execution and copyable result fields, without a navigation item.
*******************************************************************************/
public static void register(QInstance instance, DbPerfQBitConfig config)
{
instance.addPossibleValueSource(new QPossibleValueSource().withName("db_perf_ExplainRequestMode").withType(QPossibleValueSourceType.ENUM)
.withEnumValues(List.of(new QPossibleValue<>("PLAN", "PLAN"), new QPossibleValue<>("ANALYZE", "ANALYZE"))));
instance.addProcess(new QProcessMetaData().withName(NAME).withLabel("Query plan").withTableName("db_perf_statement")
.withMinInputRecords(1).withMaxInputRecords(1)
.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".explain"))
.withStep(new QFrontendStepMetaData().withName("review").withLabel("Review")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM))
.withFormFields(List.of(new QFieldMetaData("mode", QFieldType.STRING).withLabel("Mode").withPossibleValueSourceName("db_perf_ExplainRequestMode").withDefaultValue("PLAN").withIsRequired(true),
new QFieldMetaData("parameterValues", QFieldType.TEXT).withLabel("Parameter values (ANALYZE only, one per line)").withIsSensitive(true))))
.withStep(new QBackendStepMetaData().withName("explain").withCode(new QCodeReference(QueryPlanProcess.class))
.withInputData(new QFunctionInputMetaData().withRecordListMetaData(new QRecordListMetaData().withTableName("db_perf_statement"))))
.withStep(new QFrontendStepMetaData().withName("result").withLabel("Result")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.VIEW_FORM))
.withViewField(code("planText", "Query plan"))
.withViewField(new QFieldMetaData("totalCost", QFieldType.DECIMAL).withLabel("Total cost"))
.withViewField(new QFieldMetaData("sqlWorkbenchUrl", QFieldType.STRING).withLabel("Open in SQL Workbench")
.withFieldAdornment(new FieldAdornment(AdornmentType.LINK)))
.withViewField(code("assistantPrompt", "Copy to assistant"))));
}
/*******************************************************************************
** Monospace text that remains selectable and copyable.
*******************************************************************************/
private static QFieldMetaData code(String name, String label)
{
return (new QFieldMetaData(name, QFieldType.TEXT).withLabel(label)
.withFieldAdornment(new FieldAdornment(AdornmentType.CODE_EDITOR).withValue(AdornmentType.CodeEditorValues.languageMode("text"))));
}
/*******************************************************************************
** Re-check permissions inside the service even when invoked without UI middleware.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
try
{
if(input.getRecords() == null || input.getRecords().size() != 1)
{
throw (new QException("Select exactly one statement."));
}
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
String values = input.getValueString("parameterValues");
List<String> parameters = values == null || values.isEmpty() ? List.of() : Arrays.asList(values.split("\\R", -1));
ExplainRequest.Mode mode;
try
{
mode = ExplainRequest.Mode.valueOf(input.getValueString("mode") == null ? "PLAN" : input.getValueString("mode"));
}
catch(IllegalArgumentException e)
{
mode = null;
}
ExplainResult result = new ExplainService(runtime).explain(new ExplainRequest(runtime.getConfig().getBackendName(),
input.getRecords().getFirst().getValueInteger("id"), null, mode, parameters));
output.addValue("planText", result.planText());
output.addValue("totalCost", result.totalCost());
output.addValue("sqlWorkbenchUrl", result.sqlWorkbenchUrl());
output.addValue("assistantPrompt", result.assistantPrompt());
}
catch(QException e)
{
throw (e);
}
catch(Exception e)
{
throw (new QException("Unable to complete the query plan request."));
}
finally
{
input.getValues().remove("parameterValues");
output.getValues().remove("parameterValues");
}
}
}
@@ -0,0 +1,153 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.health;
import java.time.Clock;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.analyze.RuleContext;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.health.HealthCategory;
import com.kingsrook.qqq.backend.core.health.HealthCheckKind;
import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData;
import com.kingsrook.qqq.backend.core.health.HealthCheckOutcome;
import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData;
import com.kingsrook.qqq.backend.core.health.HealthDetails;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
/*******************************************************************************
** Passive core health checks. No live database probes and no SQL in details.
*******************************************************************************/
public class DbPerfHealthChecks
{
private final DbPerfRuntime runtime;
private final Clock clock;
/*******************************************************************************
** Bind observations to this runtime and clock.
*******************************************************************************/
public DbPerfHealthChecks(DbPerfRuntime runtime, Clock clock)
{
this.runtime = runtime;
this.clock = clock;
}
/*******************************************************************************
** Register the three central database dependency checks.
*******************************************************************************/
public static void register(QInstance instance)
{
DbPerfHealthChecks checks = new DbPerfHealthChecks(DbPerfRuntime.of(instance), Clock.systemUTC());
for(String name : List.of("collector", "stats_access", "connections"))
{
HealthChecksMetaData.ofOrWithNew(instance).withCheck(new HealthCheckMetaData().withName("db_performance." + name)
.withLabel("Database performance " + name.replace('_', ' ')).withComponent("db_performance")
.withCategory(HealthCategory.DATABASE).withKind(HealthCheckKind.DEPENDENCY).withIntervalSeconds(60).withTimeoutMs(5000)
.withCheckInstance(input -> checks.check(name)));
}
}
/*******************************************************************************
** Errors expose only a remedy, never a JDBC exception or query text.
*******************************************************************************/
public HealthCheckOutcome check(String name)
{
try
{
return (runtime.asSystem(() -> evaluate(name)));
}
catch(Exception e)
{
return (HealthCheckOutcome.down("Unable to read Database Performance health; check connectivity and migrations"));
}
}
/*******************************************************************************
** Missing settings never cause a write during a health request.
*******************************************************************************/
private HealthCheckOutcome evaluate(String name) throws Exception
{
String backend = runtime.getConfig().getBackendName();
QRecord settings = DbPerfStore.first("db_perf_settings", DbPerfStore.eq("backendName", backend));
if(settings == null || !Boolean.TRUE.equals(settings.getValueBoolean("enabled")))
{
return (HealthCheckOutcome.unknown("Database performance collection is disabled or not configured"));
}
if(name.equals("stats_access"))
{
String access = settings.getValueString("statsAccess");
if(access == null)
{
return (HealthCheckOutcome.unknown("No statistics access observation yet"));
}
String user = ((RDBMSBackendMetaData) runtime.getInstance().getBackend(backend)).getUsername();
HealthCheckOutcome outcome = switch(access)
{
case "FULL" -> HealthCheckOutcome.up("Full statistics access");
case "NO_READ_ALL_STATS" -> HealthCheckOutcome.degraded("grant pg_read_all_stats to " + user);
case "NO_PG_STAT_STATEMENTS" -> HealthCheckOutcome.down("Enable pg_stat_statements in shared_preload_libraries and create the extension");
case "UNSUPPORTED" -> HealthCheckOutcome.down("Use a supported PostgreSQL version (13 or newer) and statistics provider");
default -> HealthCheckOutcome.unknown("Statistics access has not been classified");
};
String safe = List.of("FULL", "NO_READ_ALL_STATS", "NO_PG_STAT_STATEMENTS", "UNSUPPORTED").contains(access) ? access : "UNKNOWN";
return (outcome.withDetails(new HealthDetails().withValue("statsAccess", safe)));
}
var filter = DbPerfStore.eq("backendName", backend).withOrderBy(new QFilterOrderBy("capturedAt", false));
if(name.equals("collector"))
{
filter.withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "OK"));
}
QRecord snapshot = DbPerfStore.first("db_perf_snapshot", filter);
if(snapshot == null)
{
return (name.equals("collector") ? HealthCheckOutcome.down("No successful snapshot; run Collect now and check statistics access") : HealthCheckOutcome.unknown("No connection sample yet"));
}
if(name.equals("collector"))
{
long age = Math.max(0, Duration.between(snapshot.getValueInstant("capturedAt"), clock.instant()).getSeconds());
double interval = Math.max(1, RuleContext.number(settings, "snapshotIntervalMinutes")) * 60;
HealthCheckOutcome outcome = age <= 3 * interval ? HealthCheckOutcome.up("Collector is current")
: age <= 6 * interval ? HealthCheckOutcome.degraded("Collector is delayed") : HealthCheckOutcome.down("Collector is stale; check the collection schedule");
return (outcome.withDetails(new HealthDetails().withValue("ageSeconds", age).withValue("intervalSeconds", interval)));
}
double maximum = RuleContext.number(snapshot, "maxConnections");
if(maximum <= 0 || snapshot.getValue("connectionsTotal") == null)
{
return (HealthCheckOutcome.unknown("Latest snapshot has no connection capacity measurement"));
}
double used = 100 * RuleContext.number(snapshot, "connectionsTotal") / maximum;
HealthCheckOutcome outcome = used >= 95 ? HealthCheckOutcome.down("Connection usage is at least 95%")
: used >= RuleContext.number(settings, "connectionAlertPercent") ? HealthCheckOutcome.degraded("Connection usage reached the alert threshold") : HealthCheckOutcome.up("Connection capacity is available");
return (outcome.withDetails(new HealthDetails().withValue("usedPercent", used).withValue("connectionsTotal", snapshot.getValueInteger("connectionsTotal")).withValue("maxConnections", snapshot.getValueInteger("maxConnections"))));
}
}
@@ -0,0 +1,94 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.metadata;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundle;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundleEntry;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
/*******************************************************************************
** Data-driven app/table help, also visible when the host has no help-content tables.
*******************************************************************************/
public final class DbPerfHelp
{
public static final String RESOURCE = "help/qbit-db-performance.yaml";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfHelp()
{
}
/*******************************************************************************
** Register the Maven-versioned bundle and fill only unclaimed help slots.
*******************************************************************************/
public static void register(QInstance instance, DbPerfQBitConfig config)
{
HelpBundles.register(instance, RESOURCE);
HelpBundle bundle = HelpBundles.of(instance).getBundles().getLast();
for(HelpBundleEntry entry : bundle.getEntries())
{
String key = entry.getKey();
if(key.startsWith("app:dbPerformance;"))
{
entry.setKey(key.replace("app:dbPerformance;", "app:" + config.getAppName() + ";"));
}
if(key.startsWith("table:") && key.contains(";slot:"))
{
String name = key.substring(6, key.indexOf(';'));
String slot = key.substring(key.indexOf(";slot:") + 6);
var table = instance.getTable(name);
if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(slot)))
{
table.withHelpContent(slot, content(entry));
}
}
else if(key.equals("app:dbPerformance;slot:help"))
{
var app = instance.getApp(config.getAppName());
if(app.getHelpContent() == null || !app.getHelpContent().containsKey("help"))
{
app.withHelpContent("help", content(entry));
}
}
}
}
/*******************************************************************************
** Render a concise summary followed by the full data-driven markdown explanation.
*******************************************************************************/
private static QHelpContent content(HelpBundleEntry entry)
{
return (new QHelpContent().withContentAsMarkdown(entry.getSummary() + "\n\n" + (entry.getContent() == null ? "" : entry.getContent())));
}
}
@@ -0,0 +1,757 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.metadata;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfPermissions;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qbits.dbperformance.model.DbPerfActivitySample;
import com.kingsrook.qbits.dbperformance.model.DbPerfAttribution;
import com.kingsrook.qbits.dbperformance.model.DbPerfAttributionDaily;
import com.kingsrook.qbits.dbperformance.model.DbPerfConnectionSample;
import com.kingsrook.qbits.dbperformance.model.DbPerfExplain;
import com.kingsrook.qbits.dbperformance.model.DbPerfFinding;
import com.kingsrook.qbits.dbperformance.model.DbPerfIndexStat;
import com.kingsrook.qbits.dbperformance.model.DbPerfQueryDaily;
import com.kingsrook.qbits.dbperformance.model.DbPerfQuerySnapshot;
import com.kingsrook.qbits.dbperformance.model.DbPerfRepeatPattern;
import com.kingsrook.qbits.dbperformance.model.DbPerfRule;
import com.kingsrook.qbits.dbperformance.model.DbPerfSettings;
import com.kingsrook.qbits.dbperformance.model.DbPerfSnapshot;
import com.kingsrook.qbits.dbperformance.model.DbPerfStatement;
import com.kingsrook.qbits.dbperformance.model.DbPerfTableDaily;
import com.kingsrook.qbits.dbperformance.model.DbPerfTableSnapshot;
import com.kingsrook.qbits.dbperformance.model.DbPerfTableStat;
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
import com.kingsrook.qqq.backend.core.actions.ai.AiTableMetaData;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinType;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
/*******************************************************************************
** QQQ tables, exact permissions, sections and parent-child record lists.
*******************************************************************************/
public final class DbPerfMetaData
{
public static final String CHANGELOG = "db/liquibase/qbit-db-performance/db_performance.changelog.sql";
public static final List<String> TABLES = List.of("db_perf_settings", "db_perf_rule", "db_perf_snapshot", "db_perf_statement", "db_perf_query_snapshot", "db_perf_query_daily", "db_perf_table_stat", "db_perf_table_snapshot", "db_perf_table_daily", "db_perf_index_stat", "db_perf_activity_sample", "db_perf_connection_sample", "db_perf_attribution", "db_perf_attribution_daily", "db_perf_repeat_pattern", "db_perf_finding", "db_perf_explain");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfMetaData()
{
}
/*******************************************************************************
** Register the complete Phase A metadata.
*******************************************************************************/
public static void addTo(QInstance instance, DbPerfQBitConfig config) throws QException
{
QTableMetaData settings = base(config, DbPerfSettings.TABLE_NAME, "Settings", DbPerfSettings.class);
settings.withRecordLabelFormat("%s").withRecordLabelFields("backendName");
settings.withUniqueKey(new UniqueKey("backendName"));
settings.getField("id").setBackendName("id");
settings.getField("backendName").setBackendName("backend_name");
settings.getField("enabled").setBackendName("enabled");
settings.getField("enabled").setDefaultValue(true);
settings.getField("provider").setBackendName("provider");
settings.getField("provider").setPossibleValueSourceName("db_perf_Provider");
settings.getField("snapshotIntervalMinutes").setBackendName("snapshot_interval_minutes");
settings.getField("snapshotIntervalMinutes").setDefaultValue(5);
settings.getField("tableSnapshotMinutes").setBackendName("table_snapshot_minutes");
settings.getField("tableSnapshotMinutes").setDefaultValue(60);
settings.getField("detailRetentionDays").setBackendName("detail_retention_days");
settings.getField("detailRetentionDays").setDefaultValue(30);
settings.getField("rollupRetentionDays").setBackendName("rollup_retention_days");
settings.getField("rollupRetentionDays").setDefaultValue(365);
settings.getField("topStatementsPerSnapshot").setBackendName("top_statements_per_snapshot");
settings.getField("topStatementsPerSnapshot").setDefaultValue(200);
settings.getField("activityLongRunningSeconds").setBackendName("activity_long_running_seconds");
settings.getField("activityLongRunningSeconds").setDefaultValue(30);
settings.getField("idleInTransactionSeconds").setBackendName("idle_in_transaction_seconds");
settings.getField("idleInTransactionSeconds").setDefaultValue(60);
settings.getField("attributionEnabled").setBackendName("attribution_enabled");
settings.getField("attributionEnabled").setDefaultValue(true);
settings.getField("attributionSampleRate").setBackendName("attribution_sample_rate");
settings.getField("attributionSampleRate").setDefaultValue(new java.math.BigDecimal("1.0"));
settings.getField("attributionFlushSeconds").setBackendName("attribution_flush_seconds");
settings.getField("attributionFlushSeconds").setDefaultValue(60);
settings.getField("attributionMaxKeys").setBackendName("attribution_max_keys");
settings.getField("attributionMaxKeys").setDefaultValue(10000);
settings.getField("nPlusOneThreshold").setBackendName("n_plus_one_threshold");
settings.getField("nPlusOneThreshold").setDefaultValue(10);
settings.getField("explainTimeoutMs").setBackendName("explain_timeout_ms");
settings.getField("explainTimeoutMs").setDefaultValue(10000);
settings.getField("alertsEnabled").setBackendName("alerts_enabled");
settings.getField("alertsEnabled").setDefaultValue(true);
settings.getField("spikeFactor").setBackendName("spike_factor");
settings.getField("spikeFactor").setDefaultValue(new java.math.BigDecimal("3"));
settings.getField("spikeMinTotalMs").setBackendName("spike_min_total_ms");
settings.getField("spikeMinTotalMs").setDefaultValue(5000);
settings.getField("connectionAlertPercent").setBackendName("connection_alert_percent");
settings.getField("connectionAlertPercent").setDefaultValue(80);
settings.getField("blockedAlertSeconds").setBackendName("blocked_alert_seconds");
settings.getField("blockedAlertSeconds").setDefaultValue(30);
settings.getField("lastSnapshotAt").setBackendName("last_snapshot_at");
settings.getField("lastTableSnapshotAt").setBackendName("last_table_snapshot_at");
settings.getField("statsAccess").setBackendName("stats_access");
settings.getField("statsAccess").setPossibleValueSourceName("db_perf_StatsAccess");
settings.getField("statsAccessMessage").setBackendName("stats_access_message");
settings.getField("createDate").setBackendName("create_date");
settings.getField("modifyDate").setBackendName("modify_date");
settings.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
settings.withSupplementalMetaData(new AiTableMetaData().withExposure(AiExposure.NONE));
settings.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_DELETE);
settings.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "enabled", "provider", "snapshotIntervalMinutes", "tableSnapshotMinutes", "detailRetentionDays")));
settings.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("rollupRetentionDays", "topStatementsPerSnapshot", "activityLongRunningSeconds", "idleInTransactionSeconds", "attributionEnabled", "attributionSampleRate", "attributionFlushSeconds", "attributionMaxKeys", "nPlusOneThreshold", "explainTimeoutMs", "alertsEnabled", "spikeFactor", "spikeMinTotalMs", "connectionAlertPercent", "blockedAlertSeconds", "lastSnapshotAt", "lastTableSnapshotAt", "statsAccess", "statsAccessMessage")));
settings.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(settings);
QTableMetaData rule = base(config, DbPerfRule.TABLE_NAME, "Suggestion Rules", DbPerfRule.class);
rule.withRecordLabelFormat("%s").withRecordLabelFields("label");
rule.withUniqueKey(new UniqueKey("code"));
rule.getField("id").setBackendName("id");
rule.getField("code").setBackendName("code");
rule.getField("label").setBackendName("label");
rule.getField("description").setBackendName("description");
rule.getField("enabled").setBackendName("enabled");
rule.getField("enabled").setDefaultValue(true);
rule.getField("severity").setBackendName("severity");
rule.getField("severity").setPossibleValueSourceName("db_perf_Severity");
rule.getField("threshold1").setBackendName("threshold_1");
rule.getField("threshold2").setBackendName("threshold_2");
rule.getField("threshold1Label").setBackendName("threshold_1_label");
rule.getField("threshold2Label").setBackendName("threshold_2_label");
rule.getField("createDate").setBackendName("create_date");
rule.getField("modifyDate").setBackendName("modify_date");
rule.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
rule.withSupplementalMetaData(new AiTableMetaData().withExposure(AiExposure.NONE));
rule.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "code", "label", "description", "enabled", "severity", "threshold1")));
rule.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("threshold2", "threshold1Label", "threshold2Label")));
rule.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(rule);
QTableMetaData snapshot = base(config, DbPerfSnapshot.TABLE_NAME, "Snapshots", DbPerfSnapshot.class);
snapshot.withRecordLabelFormat("%s").withRecordLabelFields("capturedAt");
snapshot.withUniqueKey(new UniqueKey("backendName", "capturedAt"));
snapshot.getField("id").setBackendName("id");
snapshot.getField("backendName").setBackendName("backend_name");
snapshot.getField("capturedAt").setBackendName("captured_at");
snapshot.getField("intervalSeconds").setBackendName("interval_seconds");
snapshot.getField("status").setBackendName("status");
snapshot.getField("status").setPossibleValueSourceName("db_perf_SnapshotStatus");
snapshot.getField("durationMs").setBackendName("duration_ms");
snapshot.getField("errorMessage").setBackendName("error_message");
snapshot.getField("statsResetDetected").setBackendName("stats_reset_detected");
snapshot.getField("includesTables").setBackendName("includes_tables");
snapshot.getField("xactCommit").setBackendName("xact_commit");
snapshot.getField("xactRollback").setBackendName("xact_rollback");
snapshot.getField("blksHit").setBackendName("blks_hit");
snapshot.getField("blksRead").setBackendName("blks_read");
snapshot.getField("cacheHitRatio").setBackendName("cache_hit_ratio");
snapshot.getField("tempFiles").setBackendName("temp_files");
snapshot.getField("tempBytes").setBackendName("temp_bytes");
snapshot.getField("deadlocks").setBackendName("deadlocks");
snapshot.getField("conflicts").setBackendName("conflicts");
snapshot.getField("connectionsTotal").setBackendName("connections_total");
snapshot.getField("connectionsActive").setBackendName("connections_active");
snapshot.getField("connectionsIdle").setBackendName("connections_idle");
snapshot.getField("connectionsIdleInTx").setBackendName("connections_idle_in_tx");
snapshot.getField("maxConnections").setBackendName("max_connections");
snapshot.getField("databaseSizeBytes").setBackendName("database_size_bytes");
snapshot.getField("statementCount").setBackendName("statement_count");
snapshot.getField("statementCalls").setBackendName("statement_calls");
snapshot.getField("statementTotalMs").setBackendName("statement_total_ms");
snapshot.getField("storedStatementCount").setBackendName("stored_statement_count");
snapshot.getField("createDate").setBackendName("create_date");
snapshot.getField("rawXactCommit").setBackendName("raw_xact_commit");
snapshot.getField("rawXactRollback").setBackendName("raw_xact_rollback");
snapshot.getField("rawBlksHit").setBackendName("raw_blks_hit");
snapshot.getField("rawBlksRead").setBackendName("raw_blks_read");
snapshot.getField("rawTempFiles").setBackendName("raw_temp_files");
snapshot.getField("rawTempBytes").setBackendName("raw_temp_bytes");
snapshot.getField("rawDeadlocks").setBackendName("raw_deadlocks");
snapshot.getField("rawConflicts").setBackendName("raw_conflicts");
snapshot.getField("rawStatsReset").setBackendName("raw_stats_reset");
snapshot.getField("rawStatementStatsReset").setBackendName("raw_statement_stats_reset");
snapshot.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
snapshot.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "capturedAt", "intervalSeconds", "status", "durationMs", "errorMessage")));
snapshot.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("statsResetDetected", "includesTables", "xactCommit", "xactRollback", "blksHit", "blksRead", "cacheHitRatio", "tempFiles", "tempBytes", "deadlocks", "conflicts", "connectionsTotal", "connectionsActive", "connectionsIdle", "connectionsIdleInTx", "maxConnections", "databaseSizeBytes", "statementCount", "statementCalls", "statementTotalMs", "storedStatementCount")));
snapshot.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "rawXactCommit", "rawXactRollback", "rawBlksHit", "rawBlksRead", "rawTempFiles", "rawTempBytes", "rawDeadlocks", "rawConflicts", "rawStatsReset", "rawStatementStatsReset")));
instance.addTable(snapshot);
QTableMetaData statement = base(config, DbPerfStatement.TABLE_NAME, "Queries", DbPerfStatement.class);
statement.withRecordLabelFormat("%s").withRecordLabelFields("queryText");
statement.withUniqueKey(new UniqueKey("backendName", "queryId", "dbUser", "toplevel"));
statement.getField("id").setBackendName("id");
statement.getField("backendName").setBackendName("backend_name");
statement.getField("queryId").setBackendName("query_id");
statement.getField("dbUser").setBackendName("db_user");
statement.getField("toplevel").setBackendName("toplevel");
statement.getField("fingerprint").setBackendName("fingerprint");
statement.getField("queryText").setBackendName("query_text");
statement.getField("queryText").setType(QFieldType.TEXT);
statement.getField("operation").setBackendName("operation");
statement.getField("operation").setPossibleValueSourceName("db_perf_Operation");
statement.getField("mainTable").setBackendName("main_table");
statement.getField("firstSeenAt").setBackendName("first_seen_at");
statement.getField("lastSeenAt").setBackendName("last_seen_at");
statement.getField("lastCalls").setBackendName("last_calls");
statement.getField("lastTotalExecMs").setBackendName("last_total_exec_ms");
statement.getField("lastRows").setBackendName("last_rows");
statement.getField("lastSharedBlksHit").setBackendName("last_shared_blks_hit");
statement.getField("lastSharedBlksRead").setBackendName("last_shared_blks_read");
statement.getField("lastTempBlksWritten").setBackendName("last_temp_blks_written");
statement.getField("lastMaxExecMs").setBackendName("last_max_exec_ms");
statement.getField("lastStatsSince").setBackendName("last_stats_since");
statement.getField("calls24h").setBackendName("calls_24h");
statement.getField("totalMs24h").setBackendName("total_ms_24h");
statement.getField("meanMs24h").setBackendName("mean_ms_24h");
statement.getField("shareOfDbTime24h").setBackendName("share_of_db_time_24h");
statement.getField("createDate").setBackendName("create_date");
statement.getField("modifyDate").setBackendName("modify_date");
statement.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
statement.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "queryId", "dbUser", "toplevel", "fingerprint", "queryText")));
statement.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("operation", "mainTable", "firstSeenAt", "lastSeenAt", "lastCalls", "lastTotalExecMs", "lastRows", "lastSharedBlksHit", "lastSharedBlksRead", "lastTempBlksWritten", "lastMaxExecMs", "lastStatsSince", "calls24h", "totalMs24h", "meanMs24h", "shareOfDbTime24h")));
statement.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(statement);
QTableMetaData querySnapshot = base(config, DbPerfQuerySnapshot.TABLE_NAME, "Query Snapshots", DbPerfQuerySnapshot.class);
querySnapshot.withRecordLabelFormat("%s").withRecordLabelFields("id");
querySnapshot.getField("id").setBackendName("id");
querySnapshot.getField("snapshotId").setBackendName("snapshot_id");
querySnapshot.getField("snapshotId").setPossibleValueSourceName("db_perf_snapshot");
querySnapshot.getField("statementId").setBackendName("statement_id");
querySnapshot.getField("statementId").setPossibleValueSourceName("db_perf_statement");
querySnapshot.getField("capturedAt").setBackendName("captured_at");
querySnapshot.getField("calls").setBackendName("calls");
querySnapshot.getField("totalMs").setBackendName("total_ms");
querySnapshot.getField("meanMs").setBackendName("mean_ms");
querySnapshot.getField("maxMs").setBackendName("max_ms");
querySnapshot.getField("rowsCount").setBackendName("rows_count");
querySnapshot.getField("sharedBlksHit").setBackendName("shared_blks_hit");
querySnapshot.getField("sharedBlksRead").setBackendName("shared_blks_read");
querySnapshot.getField("hitRatio").setBackendName("hit_ratio");
querySnapshot.getField("tempBlksWritten").setBackendName("temp_blks_written");
querySnapshot.getField("shareOfDbTime").setBackendName("share_of_db_time");
querySnapshot.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
querySnapshot.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "snapshotId", "statementId", "capturedAt", "calls", "totalMs", "meanMs")));
querySnapshot.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("maxMs", "rowsCount", "sharedBlksHit", "sharedBlksRead", "hitRatio", "tempBlksWritten", "shareOfDbTime")));
instance.addTable(querySnapshot);
QTableMetaData queryDaily = base(config, DbPerfQueryDaily.TABLE_NAME, "Query Daily", DbPerfQueryDaily.class);
queryDaily.withRecordLabelFormat("%s").withRecordLabelFields("day");
queryDaily.withUniqueKey(new UniqueKey("statementId", "day"));
queryDaily.getField("id").setBackendName("id");
queryDaily.getField("statementId").setBackendName("statement_id");
queryDaily.getField("statementId").setPossibleValueSourceName("db_perf_statement");
queryDaily.getField("day").setBackendName("day");
queryDaily.getField("calls").setBackendName("calls");
queryDaily.getField("totalMs").setBackendName("total_ms");
queryDaily.getField("meanMs").setBackendName("mean_ms");
queryDaily.getField("maxMs").setBackendName("max_ms");
queryDaily.getField("rowsCount").setBackendName("rows_count");
queryDaily.getField("hitRatio").setBackendName("hit_ratio");
queryDaily.getField("shareOfDbTime").setBackendName("share_of_db_time");
queryDaily.getField("snapshots").setBackendName("snapshots");
queryDaily.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
queryDaily.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "statementId", "day", "calls", "totalMs", "meanMs", "maxMs")));
queryDaily.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("rowsCount", "hitRatio", "shareOfDbTime", "snapshots")));
instance.addTable(queryDaily);
QTableMetaData tableStat = base(config, DbPerfTableStat.TABLE_NAME, "Tables", DbPerfTableStat.class);
tableStat.withRecordLabelFormat("%s").withRecordLabelFields("tableName");
tableStat.withUniqueKey(new UniqueKey("backendName", "schemaName", "tableName"));
tableStat.getField("id").setBackendName("id");
tableStat.getField("backendName").setBackendName("backend_name");
tableStat.getField("schemaName").setBackendName("schema_name");
tableStat.getField("tableName").setBackendName("table_name");
tableStat.getField("relid").setBackendName("relid");
tableStat.getField("seqScan").setBackendName("seq_scan");
tableStat.getField("seqTupRead").setBackendName("seq_tup_read");
tableStat.getField("idxScan").setBackendName("idx_scan");
tableStat.getField("idxTupFetch").setBackendName("idx_tup_fetch");
tableStat.getField("nTupIns").setBackendName("n_tup_ins");
tableStat.getField("nTupUpd").setBackendName("n_tup_upd");
tableStat.getField("nTupDel").setBackendName("n_tup_del");
tableStat.getField("nTupHotUpd").setBackendName("n_tup_hot_upd");
tableStat.getField("nLiveTup").setBackendName("n_live_tup");
tableStat.getField("nDeadTup").setBackendName("n_dead_tup");
tableStat.getField("deadRatio").setBackendName("dead_ratio");
tableStat.getField("totalBytes").setBackendName("total_bytes");
tableStat.getField("tableBytes").setBackendName("table_bytes");
tableStat.getField("indexBytes").setBackendName("index_bytes");
tableStat.getField("toastBytes").setBackendName("toast_bytes");
tableStat.getField("growthBytes7d").setBackendName("growth_bytes_7d");
tableStat.getField("lastVacuum").setBackendName("last_vacuum");
tableStat.getField("lastAutovacuum").setBackendName("last_autovacuum");
tableStat.getField("lastAnalyze").setBackendName("last_analyze");
tableStat.getField("lastAutoanalyze").setBackendName("last_autoanalyze");
tableStat.getField("vacuumCount").setBackendName("vacuum_count");
tableStat.getField("autovacuumCount").setBackendName("autovacuum_count");
tableStat.getField("analyzeCount").setBackendName("analyze_count");
tableStat.getField("autoanalyzeCount").setBackendName("autoanalyze_count");
tableStat.getField("freezeAge").setBackendName("freeze_age");
tableStat.getField("seqScanShare").setBackendName("seq_scan_share");
tableStat.getField("lastSeenAt").setBackendName("last_seen_at");
tableStat.getField("createDate").setBackendName("create_date");
tableStat.getField("modifyDate").setBackendName("modify_date");
tableStat.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
tableStat.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "schemaName", "tableName", "relid", "seqScan", "seqTupRead")));
tableStat.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("idxScan", "idxTupFetch", "nTupIns", "nTupUpd", "nTupDel", "nTupHotUpd", "nLiveTup", "nDeadTup", "deadRatio", "totalBytes", "tableBytes", "indexBytes", "toastBytes", "growthBytes7d", "lastVacuum", "lastAutovacuum", "lastAnalyze", "lastAutoanalyze", "vacuumCount", "autovacuumCount", "analyzeCount", "autoanalyzeCount", "freezeAge", "seqScanShare", "lastSeenAt")));
tableStat.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(tableStat);
QTableMetaData tableSnapshot = base(config, DbPerfTableSnapshot.TABLE_NAME, "Table Snapshots", DbPerfTableSnapshot.class);
tableSnapshot.withRecordLabelFormat("%s").withRecordLabelFields("id");
tableSnapshot.getField("id").setBackendName("id");
tableSnapshot.getField("snapshotId").setBackendName("snapshot_id");
tableSnapshot.getField("snapshotId").setPossibleValueSourceName("db_perf_snapshot");
tableSnapshot.getField("tableStatId").setBackendName("table_stat_id");
tableSnapshot.getField("tableStatId").setPossibleValueSourceName("db_perf_table_stat");
tableSnapshot.getField("capturedAt").setBackendName("captured_at");
tableSnapshot.getField("seqScan").setBackendName("seq_scan");
tableSnapshot.getField("seqTupRead").setBackendName("seq_tup_read");
tableSnapshot.getField("idxScan").setBackendName("idx_scan");
tableSnapshot.getField("idxTupFetch").setBackendName("idx_tup_fetch");
tableSnapshot.getField("nTupIns").setBackendName("n_tup_ins");
tableSnapshot.getField("nTupUpd").setBackendName("n_tup_upd");
tableSnapshot.getField("nTupDel").setBackendName("n_tup_del");
tableSnapshot.getField("nTupHotUpd").setBackendName("n_tup_hot_upd");
tableSnapshot.getField("nLiveTup").setBackendName("n_live_tup");
tableSnapshot.getField("nDeadTup").setBackendName("n_dead_tup");
tableSnapshot.getField("totalBytes").setBackendName("total_bytes");
tableSnapshot.getField("tableBytes").setBackendName("table_bytes");
tableSnapshot.getField("indexBytes").setBackendName("index_bytes");
tableSnapshot.getField("autovacuumCount").setBackendName("autovacuum_count");
tableSnapshot.getField("autoanalyzeCount").setBackendName("autoanalyze_count");
tableSnapshot.getField("freezeAge").setBackendName("freeze_age");
tableSnapshot.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
tableSnapshot.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "snapshotId", "tableStatId", "capturedAt", "seqScan", "seqTupRead", "idxScan")));
tableSnapshot.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("idxTupFetch", "nTupIns", "nTupUpd", "nTupDel", "nTupHotUpd", "nLiveTup", "nDeadTup", "totalBytes", "tableBytes", "indexBytes", "autovacuumCount", "autoanalyzeCount", "freezeAge")));
instance.addTable(tableSnapshot);
QTableMetaData tableDaily = base(config, DbPerfTableDaily.TABLE_NAME, "Table Daily", DbPerfTableDaily.class);
tableDaily.withRecordLabelFormat("%s").withRecordLabelFields("day");
tableDaily.withUniqueKey(new UniqueKey("tableStatId", "day"));
tableDaily.getField("id").setBackendName("id");
tableDaily.getField("tableStatId").setBackendName("table_stat_id");
tableDaily.getField("tableStatId").setPossibleValueSourceName("db_perf_table_stat");
tableDaily.getField("day").setBackendName("day");
tableDaily.getField("seqScan").setBackendName("seq_scan");
tableDaily.getField("idxScan").setBackendName("idx_scan");
tableDaily.getField("seqTupRead").setBackendName("seq_tup_read");
tableDaily.getField("nTupIns").setBackendName("n_tup_ins");
tableDaily.getField("nTupUpd").setBackendName("n_tup_upd");
tableDaily.getField("nTupDel").setBackendName("n_tup_del");
tableDaily.getField("nLiveTup").setBackendName("n_live_tup");
tableDaily.getField("nDeadTup").setBackendName("n_dead_tup");
tableDaily.getField("totalBytes").setBackendName("total_bytes");
tableDaily.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
tableDaily.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "tableStatId", "day", "seqScan", "idxScan", "seqTupRead", "nTupIns")));
tableDaily.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("nTupUpd", "nTupDel", "nLiveTup", "nDeadTup", "totalBytes")));
instance.addTable(tableDaily);
QTableMetaData indexStat = base(config, DbPerfIndexStat.TABLE_NAME, "Indexes", DbPerfIndexStat.class);
indexStat.withRecordLabelFormat("%s").withRecordLabelFields("indexName");
indexStat.withUniqueKey(new UniqueKey("backendName", "schemaName", "indexName"));
indexStat.getField("id").setBackendName("id");
indexStat.getField("backendName").setBackendName("backend_name");
indexStat.getField("tableStatId").setBackendName("table_stat_id");
indexStat.getField("tableStatId").setPossibleValueSourceName("db_perf_table_stat");
indexStat.getField("schemaName").setBackendName("schema_name");
indexStat.getField("tableName").setBackendName("table_name");
indexStat.getField("indexName").setBackendName("index_name");
indexStat.getField("relid").setBackendName("relid");
indexStat.getField("definition").setBackendName("definition");
indexStat.getField("columns").setBackendName("columns");
indexStat.getField("isUnique").setBackendName("is_unique");
indexStat.getField("isPrimary").setBackendName("is_primary");
indexStat.getField("isValid").setBackendName("is_valid");
indexStat.getField("idxScan").setBackendName("idx_scan");
indexStat.getField("idxTupRead").setBackendName("idx_tup_read");
indexStat.getField("idxTupFetch").setBackendName("idx_tup_fetch");
indexStat.getField("sizeBytes").setBackendName("size_bytes");
indexStat.getField("writeCost").setBackendName("write_cost");
indexStat.getField("scansSinceReset").setBackendName("scans_since_reset");
indexStat.getField("lastSeenAt").setBackendName("last_seen_at");
indexStat.getField("createDate").setBackendName("create_date");
indexStat.getField("modifyDate").setBackendName("modify_date");
indexStat.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
indexStat.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "tableStatId", "schemaName", "tableName", "indexName", "relid")));
indexStat.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("definition", "columns", "isUnique", "isPrimary", "isValid", "idxScan", "idxTupRead", "idxTupFetch", "sizeBytes", "writeCost", "scansSinceReset", "lastSeenAt")));
indexStat.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(indexStat);
QTableMetaData activitySample = base(config, DbPerfActivitySample.TABLE_NAME, "Activity Samples", DbPerfActivitySample.class);
activitySample.withRecordLabelFormat("%s").withRecordLabelFields("id");
activitySample.getField("id").setBackendName("id");
activitySample.getField("snapshotId").setBackendName("snapshot_id");
activitySample.getField("snapshotId").setPossibleValueSourceName("db_perf_snapshot");
activitySample.getField("capturedAt").setBackendName("captured_at");
activitySample.getField("pid").setBackendName("pid");
activitySample.getField("dbUser").setBackendName("db_user");
activitySample.getField("applicationName").setBackendName("application_name");
activitySample.getField("clientAddr").setBackendName("client_addr");
activitySample.getField("state").setBackendName("state");
activitySample.getField("state").setPossibleValueSourceName("db_perf_State");
activitySample.getField("waitEventType").setBackendName("wait_event_type");
activitySample.getField("waitEvent").setBackendName("wait_event");
activitySample.getField("backendType").setBackendName("backend_type");
activitySample.getField("xactStart").setBackendName("xact_start");
activitySample.getField("queryStart").setBackendName("query_start");
activitySample.getField("stateChange").setBackendName("state_change");
activitySample.getField("querySeconds").setBackendName("query_seconds");
activitySample.getField("xactSeconds").setBackendName("xact_seconds");
activitySample.getField("blockedBy").setBackendName("blocked_by");
activitySample.getField("blockingCount").setBackendName("blocking_count");
activitySample.getField("reason").setBackendName("reason");
activitySample.getField("reason").setPossibleValueSourceName("db_perf_Reason");
activitySample.getField("queryId").setBackendName("query_id");
activitySample.getField("statementId").setBackendName("statement_id");
activitySample.getField("statementId").setPossibleValueSourceName("db_perf_statement");
activitySample.getField("queryFingerprint").setBackendName("query_fingerprint");
activitySample.getField("queryTextNormalized").setBackendName("query_text_normalized");
activitySample.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
activitySample.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "snapshotId", "capturedAt", "pid", "dbUser", "applicationName", "clientAddr")));
activitySample.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("state", "waitEventType", "waitEvent", "backendType", "xactStart", "queryStart", "stateChange", "querySeconds", "xactSeconds", "blockedBy", "blockingCount", "reason", "queryId", "statementId", "queryFingerprint", "queryTextNormalized")));
instance.addTable(activitySample);
QTableMetaData connectionSample = base(config, DbPerfConnectionSample.TABLE_NAME, "Connection Samples", DbPerfConnectionSample.class);
connectionSample.withRecordLabelFormat("%s").withRecordLabelFields("id");
connectionSample.getField("id").setBackendName("id");
connectionSample.getField("snapshotId").setBackendName("snapshot_id");
connectionSample.getField("snapshotId").setPossibleValueSourceName("db_perf_snapshot");
connectionSample.getField("capturedAt").setBackendName("captured_at");
connectionSample.getField("dbUser").setBackendName("db_user");
connectionSample.getField("applicationName").setBackendName("application_name");
connectionSample.getField("state").setBackendName("state");
connectionSample.getField("state").setPossibleValueSourceName("db_perf_State");
connectionSample.getField("connectionCount").setBackendName("connection_count");
connectionSample.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
connectionSample.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "snapshotId", "capturedAt", "dbUser", "applicationName", "state", "connectionCount")));
instance.addTable(connectionSample);
QTableMetaData attribution = base(config, DbPerfAttribution.TABLE_NAME, "QQQ Attribution", DbPerfAttribution.class);
attribution.withRecordLabelFormat("%s").withRecordLabelFields("id");
attribution.getField("id").setBackendName("id");
attribution.getField("windowStart").setBackendName("window_start");
attribution.getField("windowEnd").setBackendName("window_end");
attribution.getField("podName").setBackendName("pod_name");
attribution.getField("unitKind").setBackendName("unit_kind");
attribution.getField("unitKind").setPossibleValueSourceName("db_perf_UnitKind");
attribution.getField("unitName").setBackendName("unit_name");
attribution.getField("processName").setBackendName("process_name");
attribution.getField("tableName").setBackendName("table_name");
attribution.getField("operation").setBackendName("operation");
attribution.getField("operation").setPossibleValueSourceName("db_perf_Operation");
attribution.getField("fingerprint").setBackendName("fingerprint");
attribution.getField("sqlText").setBackendName("sql_text");
attribution.getField("units").setBackendName("units");
attribution.getField("statements").setBackendName("statements");
attribution.getField("totalMs").setBackendName("total_ms");
attribution.getField("maxMs").setBackendName("max_ms");
attribution.getField("rowsCount").setBackendName("rows_count");
attribution.getField("errors").setBackendName("errors");
attribution.getField("sampleRate").setBackendName("sample_rate");
attribution.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
attribution.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "windowStart", "windowEnd", "podName", "unitKind", "unitName", "processName")));
attribution.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("tableName", "operation", "fingerprint", "sqlText", "units", "statements", "totalMs", "maxMs", "rowsCount", "errors", "sampleRate")));
instance.addTable(attribution);
QTableMetaData attributionDaily = base(config, DbPerfAttributionDaily.TABLE_NAME, "QQQ Attribution Daily", DbPerfAttributionDaily.class);
attributionDaily.withRecordLabelFormat("%s").withRecordLabelFields("day");
attributionDaily.withUniqueKey(new UniqueKey("day", "rollupKey"));
attributionDaily.getField("id").setBackendName("id");
attributionDaily.getField("day").setBackendName("day");
attributionDaily.getField("rollupKey").setBackendName("rollup_key");
attributionDaily.getField("unitKind").setBackendName("unit_kind");
attributionDaily.getField("unitKind").setPossibleValueSourceName("db_perf_UnitKind");
attributionDaily.getField("unitName").setBackendName("unit_name");
attributionDaily.getField("processName").setBackendName("process_name");
attributionDaily.getField("tableName").setBackendName("table_name");
attributionDaily.getField("operation").setBackendName("operation");
attributionDaily.getField("operation").setPossibleValueSourceName("db_perf_Operation");
attributionDaily.getField("fingerprint").setBackendName("fingerprint");
attributionDaily.getField("sqlText").setBackendName("sql_text");
attributionDaily.getField("units").setBackendName("units");
attributionDaily.getField("statements").setBackendName("statements");
attributionDaily.getField("totalMs").setBackendName("total_ms");
attributionDaily.getField("maxMs").setBackendName("max_ms");
attributionDaily.getField("rowsCount").setBackendName("rows_count");
attributionDaily.getField("errors").setBackendName("errors");
attributionDaily.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
attributionDaily.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "day", "rollupKey", "unitKind", "unitName", "processName", "tableName")));
attributionDaily.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("operation", "fingerprint", "sqlText", "units", "statements", "totalMs", "maxMs", "rowsCount", "errors")));
instance.addTable(attributionDaily);
QTableMetaData repeatPattern = base(config, DbPerfRepeatPattern.TABLE_NAME, "Repeated Statements", DbPerfRepeatPattern.class);
repeatPattern.withRecordLabelFormat("%s").withRecordLabelFields("id");
repeatPattern.getField("id").setBackendName("id");
repeatPattern.getField("windowStart").setBackendName("window_start");
repeatPattern.getField("windowEnd").setBackendName("window_end");
repeatPattern.getField("podName").setBackendName("pod_name");
repeatPattern.getField("unitKind").setBackendName("unit_kind");
repeatPattern.getField("unitKind").setPossibleValueSourceName("db_perf_UnitKind");
repeatPattern.getField("unitName").setBackendName("unit_name");
repeatPattern.getField("processName").setBackendName("process_name");
repeatPattern.getField("tableName").setBackendName("table_name");
repeatPattern.getField("fingerprint").setBackendName("fingerprint");
repeatPattern.getField("sqlText").setBackendName("sql_text");
repeatPattern.getField("unitsAffected").setBackendName("units_affected");
repeatPattern.getField("maxRepeats").setBackendName("max_repeats");
repeatPattern.getField("totalRepeats").setBackendName("total_repeats");
repeatPattern.getField("sampleRate").setBackendName("sample_rate");
repeatPattern.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
repeatPattern.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "windowStart", "windowEnd", "podName", "unitKind", "unitName", "processName")));
repeatPattern.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("tableName", "fingerprint", "sqlText", "unitsAffected", "maxRepeats", "totalRepeats", "sampleRate")));
instance.addTable(repeatPattern);
QTableMetaData finding = base(config, DbPerfFinding.TABLE_NAME, "Findings", DbPerfFinding.class);
finding.withRecordLabelFormat("%s").withRecordLabelFields("title");
finding.withUniqueKey(new UniqueKey("dedupeKey"));
finding.getField("id").setBackendName("id");
finding.getField("backendName").setBackendName("backend_name");
finding.getField("ruleId").setBackendName("rule_id");
finding.getField("ruleId").setPossibleValueSourceName("db_perf_rule");
finding.getField("ruleCode").setBackendName("rule_code");
finding.getField("dedupeKey").setBackendName("dedupe_key");
finding.getField("status").setBackendName("status");
finding.getField("status").setDefaultValue("OPEN");
finding.getField("status").setPossibleValueSourceName("db_perf_FindingStatus");
finding.getField("severity").setBackendName("severity");
finding.getField("severity").setPossibleValueSourceName("db_perf_Severity");
finding.getField("score").setBackendName("score");
finding.getField("title").setBackendName("title");
finding.getField("summary").setBackendName("summary");
finding.getField("evidence").setBackendName("evidence");
finding.getField("evidence").setType(QFieldType.TEXT);
finding.getField("recommendation").setBackendName("recommendation");
finding.getField("recommendation").setType(QFieldType.TEXT);
finding.getField("migrationSnippet").setBackendName("migration_snippet");
finding.getField("migrationSnippet").setType(QFieldType.TEXT);
finding.getField("subjectTable").setBackendName("subject_table");
finding.getField("statementId").setBackendName("statement_id");
finding.getField("statementId").setPossibleValueSourceName("db_perf_statement");
finding.getField("tableStatId").setBackendName("table_stat_id");
finding.getField("tableStatId").setPossibleValueSourceName("db_perf_table_stat");
finding.getField("indexStatId").setBackendName("index_stat_id");
finding.getField("indexStatId").setPossibleValueSourceName("db_perf_index_stat");
finding.getField("unitName").setBackendName("unit_name");
finding.getField("firstSeenAt").setBackendName("first_seen_at");
finding.getField("lastSeenAt").setBackendName("last_seen_at");
finding.getField("occurrences").setBackendName("occurrences");
finding.getField("resolvedAt").setBackendName("resolved_at");
finding.getField("statusNote").setBackendName("status_note");
finding.getField("statusChangedBy").setBackendName("status_changed_by");
finding.getField("createDate").setBackendName("create_date");
finding.getField("modifyDate").setBackendName("modify_date");
finding.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
finding.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
finding.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "backendName", "ruleId", "ruleCode", "dedupeKey", "status", "severity")));
finding.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("score", "title", "summary", "evidence", "recommendation", "migrationSnippet", "subjectTable", "statementId", "tableStatId", "indexStatId", "unitName", "firstSeenAt", "lastSeenAt", "occurrences", "resolvedAt", "statusNote", "statusChangedBy")));
finding.withSection(new QFieldSection("system", new QIcon("table_rows"), Tier.T3, List.of("createDate", "modifyDate")));
instance.addTable(finding);
QTableMetaData explain = base(config, DbPerfExplain.TABLE_NAME, "Query Plans", DbPerfExplain.class);
explain.withRecordLabelFormat("%s").withRecordLabelFields("requestedAt");
explain.getField("id").setBackendName("id");
explain.getField("statementId").setBackendName("statement_id");
explain.getField("statementId").setPossibleValueSourceName("db_perf_statement");
explain.getField("requestedBy").setBackendName("requested_by");
explain.getField("requestedAt").setBackendName("requested_at");
explain.getField("mode").setBackendName("mode");
explain.getField("mode").setPossibleValueSourceName("db_perf_ExplainMode");
explain.getField("sqlFingerprint").setBackendName("sql_fingerprint");
explain.getField("sqlText").setBackendName("sql_text");
explain.getField("sqlText").setType(QFieldType.TEXT);
explain.getField("parameterCount").setBackendName("parameter_count");
explain.getField("durationMs").setBackendName("duration_ms");
explain.getField("outcome").setBackendName("outcome");
explain.getField("outcome").setPossibleValueSourceName("db_perf_Outcome");
explain.getField("errorMessage").setBackendName("error_message");
explain.getField("planJson").setBackendName("plan_json");
explain.getField("planJson").setType(QFieldType.TEXT);
explain.getField("planText").setBackendName("plan_text");
explain.getField("planText").setType(QFieldType.TEXT);
explain.getField("totalCost").setBackendName("total_cost");
explain.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
explain.withSection(new QFieldSection("identity", new QIcon("table_rows"), Tier.T1, List.of("id", "statementId", "requestedBy", "requestedAt", "mode", "sqlFingerprint", "sqlText")));
explain.withSection(new QFieldSection("details", new QIcon("table_rows"), Tier.T2, List.of("parameterCount", "durationMs", "outcome", "errorMessage", "planJson", "planText", "totalCost")));
instance.addTable(explain);
instance.addPossibleValueSource(source("db_perf_Provider", DbPerfPossibleValues.Provider.values()));
instance.addPossibleValueSource(source("db_perf_StatsAccess", DbPerfPossibleValues.StatsAccess.values()));
instance.addPossibleValueSource(source("db_perf_SnapshotStatus", DbPerfPossibleValues.SnapshotStatus.values()));
instance.addPossibleValueSource(source("db_perf_Severity", DbPerfPossibleValues.Severity.values()));
instance.addPossibleValueSource(source("db_perf_Operation", DbPerfPossibleValues.Operation.values()));
instance.addPossibleValueSource(source("db_perf_UnitKind", DbPerfPossibleValues.UnitKind.values()));
instance.addPossibleValueSource(source("db_perf_FindingStatus", DbPerfPossibleValues.FindingStatus.values()));
instance.addPossibleValueSource(source("db_perf_ExplainMode", DbPerfPossibleValues.ExplainMode.values()));
instance.addPossibleValueSource(source("db_perf_Outcome", DbPerfPossibleValues.Outcome.values()));
instance.addPossibleValueSource(source("db_perf_Reason", DbPerfPossibleValues.Reason.values()));
instance.addPossibleValueSource(source("db_perf_State", DbPerfPossibleValues.State.values()));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_settings"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_rule"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_snapshot"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_statement"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_query_snapshot"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_query_daily"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_table_stat"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_table_snapshot"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_table_daily"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_index_stat"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_activity_sample"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_connection_sample"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_attribution"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_attribution_daily"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_repeat_pattern"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_finding"));
instance.addPossibleValueSource(QPossibleValueSource.newForTable("db_perf_explain"));
child(instance, "db_perf_snapshot", "db_perf_query_snapshot", "snapshotId");
child(instance, "db_perf_statement", "db_perf_query_snapshot", "statementId");
child(instance, "db_perf_statement", "db_perf_query_daily", "statementId");
child(instance, "db_perf_snapshot", "db_perf_table_snapshot", "snapshotId");
child(instance, "db_perf_table_stat", "db_perf_table_snapshot", "tableStatId");
child(instance, "db_perf_table_stat", "db_perf_table_daily", "tableStatId");
child(instance, "db_perf_table_stat", "db_perf_index_stat", "tableStatId");
child(instance, "db_perf_snapshot", "db_perf_activity_sample", "snapshotId");
child(instance, "db_perf_snapshot", "db_perf_connection_sample", "snapshotId");
child(instance, "db_perf_rule", "db_perf_finding", "ruleId");
child(instance, "db_perf_statement", "db_perf_finding", "statementId");
child(instance, "db_perf_table_stat", "db_perf_finding", "tableStatId");
child(instance, "db_perf_statement", "db_perf_explain", "statementId");
QAppMetaData app = new QAppMetaData().withName(config.getAppName()).withLabel("Database Performance").withIcon(new QIcon("storage"))
.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".view"))
.withSection(new QAppSection().withName("data").withLabel("Data").withIcon(new QIcon("table_rows"))
.withTables(List.of("db_perf_finding", "db_perf_statement", "db_perf_table_stat", "db_perf_index_stat", "db_perf_snapshot", "db_perf_settings", "db_perf_rule")));
instance.addApp(app);
if(config.getParentAppName() != null)
{
instance.getApp(config.getParentAppName()).withChild(app);
}
}
/*******************************************************************************
** A distinct icon per table, so nav entries are told apart at a glance.
*******************************************************************************/
static final Map<String, String> TABLE_ICONS = Map.ofEntries(
Map.entry("db_perf_finding", "tips_and_updates"),
Map.entry("db_perf_statement", "query_stats"),
Map.entry("db_perf_table_stat", "grid_on"),
Map.entry("db_perf_index_stat", "bookmarks"),
Map.entry("db_perf_snapshot", "photo_camera"),
Map.entry("db_perf_settings", "settings_suggest"),
Map.entry("db_perf_rule", "playlist_add_check"),
Map.entry("db_perf_explain", "device_hub"),
Map.entry("db_perf_query_snapshot", "timeline"),
Map.entry("db_perf_query_daily", "calendar_month"),
Map.entry("db_perf_table_snapshot", "stacked_line_chart"),
Map.entry("db_perf_table_daily", "date_range"),
Map.entry("db_perf_activity_sample", "monitor_heart"),
Map.entry("db_perf_connection_sample", "lan"),
Map.entry("db_perf_attribution", "hub"),
Map.entry("db_perf_attribution_daily", "event_note"),
Map.entry("db_perf_repeat_pattern", "repeat"));
/*******************************************************************************
** Common storage, permissions and write guard.
*******************************************************************************/
private static QTableMetaData base(DbPerfQBitConfig config, String name, String label, Class<? extends QRecordEntity> entity) throws QException
{
QTableMetaData table = new QTableMetaData().withName(name).withLabel(label).withIcon(new QIcon(TABLE_ICONS.getOrDefault(name, "table_rows")))
.withBackendName(config.getBackendName()).withBackendDetails(new RDBMSTableBackendDetails().withTableName(name))
.withPrimaryKeyField("id").withFieldsFromEntity(entity)
.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".view"));
table.getPermissionRules().setCustomPermissionChecker(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties(DbPerfTablePermissionChecker.class,
java.util.Map.of("permissionName", config.getPermissionBaseName() + ".view")));
QInstanceEnricher.setInferredFieldBackendNames(table);
for(TableCustomizers role : List.of(TableCustomizers.PRE_INSERT_RECORD, TableCustomizers.PRE_UPDATE_RECORD, TableCustomizers.PRE_DELETE_RECORD))
{
table.withCustomizer(role, new QCodeReference(DbPerfWriteGuard.class));
}
return (table);
}
/*******************************************************************************
** Enum-backed stored values.
*******************************************************************************/
private static QPossibleValueSource source(String name, PossibleValueEnum<String>[] values)
{
List<QPossibleValue<?>> choices = new ArrayList<>();
for(PossibleValueEnum<String> value : values)
{
choices.add(new QPossibleValue<>(value.getPossibleValueId(), value.getPossibleValueLabel()));
}
return (new QPossibleValueSource().withName(name).withType(QPossibleValueSourceType.ENUM).withEnumValues(choices));
}
/*******************************************************************************
** A join and its record-screen child list.
*******************************************************************************/
private static void child(QInstance instance, String parent, String child, String field)
{
String name = parent + "_" + child;
QJoinMetaData join = new QJoinMetaData().withName(name).withLeftTable(parent).withRightTable(child)
.withType(JoinType.ONE_TO_MANY).withJoinOn(new JoinOn("id", field));
instance.addJoin(join);
instance.addWidget(ChildRecordListRenderer.widgetMetaDataBuilder(join).withLabel(instance.getTable(child).getLabel())
.withMaxRows(25).withOrderBys(List.of(new QFilterOrderBy("id", false))).getWidgetMetaData()
.withPermissionRules(instance.getTable(parent).getPermissionRules()));
instance.getTable(parent).withSection(new QFieldSection(name, instance.getTable(child).getLabel(), new QIcon("list"), Tier.T2, null).withWidgetName(name));
}
}
@@ -0,0 +1,369 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.metadata;
import java.util.Locale;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
/*******************************************************************************
** Enum-backed possible values with readable labels.
*******************************************************************************/
public final class DbPerfPossibleValues
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfPossibleValues()
{
}
/*******************************************************************************
** Values for Provider.
*******************************************************************************/
public enum Provider implements PossibleValueEnum<String>
{
POSTGRES, MYSQL, UNSUPPORTED;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for StatsAccess.
*******************************************************************************/
public enum StatsAccess implements PossibleValueEnum<String>
{
FULL, NO_READ_ALL_STATS, NO_PG_STAT_STATEMENTS, UNSUPPORTED;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for SnapshotStatus.
*******************************************************************************/
public enum SnapshotStatus implements PossibleValueEnum<String>
{
RUNNING, OK, PARTIAL, FAILED;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for Severity.
*******************************************************************************/
public enum Severity implements PossibleValueEnum<String>
{
INFO, WARNING, ERROR, CRITICAL;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for Operation.
*******************************************************************************/
public enum Operation implements PossibleValueEnum<String>
{
SELECT, INSERT, UPDATE, DELETE, WITH, OTHER;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for UnitKind.
*******************************************************************************/
public enum UnitKind implements PossibleValueEnum<String>
{
HTTP_REQUEST, PROCESS, SCHEDULED_JOB, OTHER;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for FindingStatus.
*******************************************************************************/
public enum FindingStatus implements PossibleValueEnum<String>
{
OPEN, ACKNOWLEDGED, RESOLVED, IGNORED;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for ExplainMode.
*******************************************************************************/
public enum ExplainMode implements PossibleValueEnum<String>
{
PLAN, GENERIC, ANALYZE;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for Outcome.
*******************************************************************************/
public enum Outcome implements PossibleValueEnum<String>
{
OK, ERROR, DENIED, TIMEOUT;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for Reason.
*******************************************************************************/
public enum Reason implements PossibleValueEnum<String>
{
LONG_RUNNING, IDLE_IN_TRANSACTION, BLOCKED, BLOCKING;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name());
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
/*******************************************************************************
** Values for State.
*******************************************************************************/
public enum State implements PossibleValueEnum<String>
{
active, idle, idle_in_transaction, idle_in_transaction_aborted, fastpath_function_call, disabled;
/*******************************************************************************
** Stored value.
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (name().replace('_', ' '));
}
/*******************************************************************************
** Human-readable label.
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
String text = name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(text.charAt(0)) + text.substring(1));
}
}
}
@@ -0,0 +1,93 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.metadata;
import com.kingsrook.qqq.backend.core.actions.permissions.AvailablePermission;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionSubType;
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
import com.kingsrook.qqq.backend.core.actions.permissions.UseOtherPermissionNameCustomPermissionChecker;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithName;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
/*******************************************************************************
** Exact view and settings-write permissions without READ_WRITE permission names.
*******************************************************************************/
public class DbPerfTablePermissionChecker extends UseOtherPermissionNameCustomPermissionChecker
{
/*******************************************************************************
** Reads require view; only settings/rules have human write permissions.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
if(QContext.getQSession() instanceof QSystemUserSession)
{
return;
}
if(input instanceof InsertInput || input instanceof UpdateInput || input instanceof DeleteInput)
{
boolean settings = "db_perf_settings".equals(metadata.getName());
boolean rule = "db_perf_rule".equals(metadata.getName());
if((!settings && !rule) || settings && !(input instanceof UpdateInput)
|| QContext.getQSession() == null || !QContext.getQSession().hasPermission(managePermission()))
{
throw (new QPermissionDeniedException("Database Performance write permission denied"));
}
return;
}
super.checkPermissionsThrowing(input, metadata);
}
/*******************************************************************************
** Publish the exact write permission in host role editors.
*******************************************************************************/
@Override
public AvailablePermission buildAvailablePermission(QPermissionRules rules, PermissionSubType subtype, String base, MetaDataWithName metadata, String objectType)
{
AvailablePermission permission = super.buildAvailablePermission(rules, subtype, base, metadata, objectType);
if(permission != null && (subtype == TablePermissionSubType.INSERT || subtype == TablePermissionSubType.EDIT || subtype == TablePermissionSubType.DELETE))
{
permission.setName(managePermission());
}
return (permission);
}
/*******************************************************************************
** The configurable base is shared with the view permission.
*******************************************************************************/
private String managePermission()
{
String view = getPermissionName();
return (view.substring(0, view.length() - "view".length()) + "manage_settings");
}
}
@@ -0,0 +1,114 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.metadata;
import java.util.List;
import java.util.Optional;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.model.statusmessages.PermissionDeniedMessage;
/*******************************************************************************
** Enforces, in the core actions, what a table's capabilities hide in the UI
** and API: an insert, update or delete the table does not offer is rejected
** unless the qbit itself (the system user) makes it. So performance records only change through the engine and record actions,
** whoever calls the table actions (admins included).
*******************************************************************************/
public class DbPerfWriteGuard implements TableCustomizerInterface
{
/*******************************************************************************
**
*******************************************************************************/
@Override
public List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
{
return (deny(insertInput.getTableName(), Capability.TABLE_INSERT, records));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
{
return (deny(updateInput.getTableName(), Capability.TABLE_UPDATE, records));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
{
return (deny(deleteInput.getTableName(), Capability.TABLE_DELETE, records));
}
/*******************************************************************************
** Add an error to every record when the table lacks the capability and the
** caller is not the system user.
*******************************************************************************/
static List<QRecord> deny(String tableName, Capability capability, List<QRecord> records)
{
if(QContext.getQSession() instanceof QSystemUserSession)
{
return (records);
}
QTableMetaData table = QContext.getQInstance().getTable(tableName);
if(table != null && table.getDisabledCapabilities() != null && table.getDisabledCapabilities().contains(capability))
{
for(QRecord record : records)
{
record.addError(new PermissionDeniedMessage("Only the Database Performance runtime changes this table."));
}
}
if(table != null && (tableName.equals("db_perf_settings") || tableName.equals("db_perf_rule")))
{
String view = table.getPermissionRules().getPermissionBaseName();
String manage = view.substring(0, view.length() - "view".length()) + "manage_settings";
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(manage))
{
for(QRecord record : records)
{
record.addError(new PermissionDeniedMessage("Managing settings requires " + manage));
}
}
}
return (records);
}
}
@@ -0,0 +1,767 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Activity Samples persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfActivitySample extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_activity_sample";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer snapshotId;
@QField
private Instant capturedAt;
@QField
private Integer pid;
@QField(maxLength = 100)
private String dbUser;
@QField(maxLength = 200)
private String applicationName;
@QField(maxLength = 100)
private String clientAddr;
@QField(maxLength = 40)
private String state;
@QField(maxLength = 60)
private String waitEventType;
@QField(maxLength = 100)
private String waitEvent;
@QField(maxLength = 60)
private String backendType;
@QField
private Instant xactStart;
@QField
private Instant queryStart;
@QField
private Instant stateChange;
@QField
private BigDecimal querySeconds;
@QField
private BigDecimal xactSeconds;
@QField(maxLength = 500)
private String blockedBy;
@QField
private Integer blockingCount;
@QField(maxLength = 30)
private String reason;
@QField
private Long queryId;
@QField
private Integer statementId;
@QField(maxLength = 64)
private String queryFingerprint;
@QField(maxLength = 4000)
private String queryTextNormalized;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfActivitySample()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfActivitySample(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfActivitySample withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for snapshotId.
*******************************************************************************/
public Integer getSnapshotId()
{
return (this.snapshotId);
}
/*******************************************************************************
** Setter for snapshotId.
*******************************************************************************/
public void setSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
}
/*******************************************************************************
** Fluent setter for snapshotId.
*******************************************************************************/
public DbPerfActivitySample withSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
return (this);
}
/*******************************************************************************
** Getter for capturedAt.
*******************************************************************************/
public Instant getCapturedAt()
{
return (this.capturedAt);
}
/*******************************************************************************
** Setter for capturedAt.
*******************************************************************************/
public void setCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
}
/*******************************************************************************
** Fluent setter for capturedAt.
*******************************************************************************/
public DbPerfActivitySample withCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
return (this);
}
/*******************************************************************************
** Getter for pid.
*******************************************************************************/
public Integer getPid()
{
return (this.pid);
}
/*******************************************************************************
** Setter for pid.
*******************************************************************************/
public void setPid(Integer pid)
{
this.pid = pid;
}
/*******************************************************************************
** Fluent setter for pid.
*******************************************************************************/
public DbPerfActivitySample withPid(Integer pid)
{
this.pid = pid;
return (this);
}
/*******************************************************************************
** Getter for dbUser.
*******************************************************************************/
public String getDbUser()
{
return (this.dbUser);
}
/*******************************************************************************
** Setter for dbUser.
*******************************************************************************/
public void setDbUser(String dbUser)
{
this.dbUser = dbUser;
}
/*******************************************************************************
** Fluent setter for dbUser.
*******************************************************************************/
public DbPerfActivitySample withDbUser(String dbUser)
{
this.dbUser = dbUser;
return (this);
}
/*******************************************************************************
** Getter for applicationName.
*******************************************************************************/
public String getApplicationName()
{
return (this.applicationName);
}
/*******************************************************************************
** Setter for applicationName.
*******************************************************************************/
public void setApplicationName(String applicationName)
{
this.applicationName = applicationName;
}
/*******************************************************************************
** Fluent setter for applicationName.
*******************************************************************************/
public DbPerfActivitySample withApplicationName(String applicationName)
{
this.applicationName = applicationName;
return (this);
}
/*******************************************************************************
** Getter for clientAddr.
*******************************************************************************/
public String getClientAddr()
{
return (this.clientAddr);
}
/*******************************************************************************
** Setter for clientAddr.
*******************************************************************************/
public void setClientAddr(String clientAddr)
{
this.clientAddr = clientAddr;
}
/*******************************************************************************
** Fluent setter for clientAddr.
*******************************************************************************/
public DbPerfActivitySample withClientAddr(String clientAddr)
{
this.clientAddr = clientAddr;
return (this);
}
/*******************************************************************************
** Getter for state.
*******************************************************************************/
public String getState()
{
return (this.state);
}
/*******************************************************************************
** Setter for state.
*******************************************************************************/
public void setState(String state)
{
this.state = state;
}
/*******************************************************************************
** Fluent setter for state.
*******************************************************************************/
public DbPerfActivitySample withState(String state)
{
this.state = state;
return (this);
}
/*******************************************************************************
** Getter for waitEventType.
*******************************************************************************/
public String getWaitEventType()
{
return (this.waitEventType);
}
/*******************************************************************************
** Setter for waitEventType.
*******************************************************************************/
public void setWaitEventType(String waitEventType)
{
this.waitEventType = waitEventType;
}
/*******************************************************************************
** Fluent setter for waitEventType.
*******************************************************************************/
public DbPerfActivitySample withWaitEventType(String waitEventType)
{
this.waitEventType = waitEventType;
return (this);
}
/*******************************************************************************
** Getter for waitEvent.
*******************************************************************************/
public String getWaitEvent()
{
return (this.waitEvent);
}
/*******************************************************************************
** Setter for waitEvent.
*******************************************************************************/
public void setWaitEvent(String waitEvent)
{
this.waitEvent = waitEvent;
}
/*******************************************************************************
** Fluent setter for waitEvent.
*******************************************************************************/
public DbPerfActivitySample withWaitEvent(String waitEvent)
{
this.waitEvent = waitEvent;
return (this);
}
/*******************************************************************************
** Getter for backendType.
*******************************************************************************/
public String getBackendType()
{
return (this.backendType);
}
/*******************************************************************************
** Setter for backendType.
*******************************************************************************/
public void setBackendType(String backendType)
{
this.backendType = backendType;
}
/*******************************************************************************
** Fluent setter for backendType.
*******************************************************************************/
public DbPerfActivitySample withBackendType(String backendType)
{
this.backendType = backendType;
return (this);
}
/*******************************************************************************
** Getter for xactStart.
*******************************************************************************/
public Instant getXactStart()
{
return (this.xactStart);
}
/*******************************************************************************
** Setter for xactStart.
*******************************************************************************/
public void setXactStart(Instant xactStart)
{
this.xactStart = xactStart;
}
/*******************************************************************************
** Fluent setter for xactStart.
*******************************************************************************/
public DbPerfActivitySample withXactStart(Instant xactStart)
{
this.xactStart = xactStart;
return (this);
}
/*******************************************************************************
** Getter for queryStart.
*******************************************************************************/
public Instant getQueryStart()
{
return (this.queryStart);
}
/*******************************************************************************
** Setter for queryStart.
*******************************************************************************/
public void setQueryStart(Instant queryStart)
{
this.queryStart = queryStart;
}
/*******************************************************************************
** Fluent setter for queryStart.
*******************************************************************************/
public DbPerfActivitySample withQueryStart(Instant queryStart)
{
this.queryStart = queryStart;
return (this);
}
/*******************************************************************************
** Getter for stateChange.
*******************************************************************************/
public Instant getStateChange()
{
return (this.stateChange);
}
/*******************************************************************************
** Setter for stateChange.
*******************************************************************************/
public void setStateChange(Instant stateChange)
{
this.stateChange = stateChange;
}
/*******************************************************************************
** Fluent setter for stateChange.
*******************************************************************************/
public DbPerfActivitySample withStateChange(Instant stateChange)
{
this.stateChange = stateChange;
return (this);
}
/*******************************************************************************
** Getter for querySeconds.
*******************************************************************************/
public BigDecimal getQuerySeconds()
{
return (this.querySeconds);
}
/*******************************************************************************
** Setter for querySeconds.
*******************************************************************************/
public void setQuerySeconds(BigDecimal querySeconds)
{
this.querySeconds = querySeconds;
}
/*******************************************************************************
** Fluent setter for querySeconds.
*******************************************************************************/
public DbPerfActivitySample withQuerySeconds(BigDecimal querySeconds)
{
this.querySeconds = querySeconds;
return (this);
}
/*******************************************************************************
** Getter for xactSeconds.
*******************************************************************************/
public BigDecimal getXactSeconds()
{
return (this.xactSeconds);
}
/*******************************************************************************
** Setter for xactSeconds.
*******************************************************************************/
public void setXactSeconds(BigDecimal xactSeconds)
{
this.xactSeconds = xactSeconds;
}
/*******************************************************************************
** Fluent setter for xactSeconds.
*******************************************************************************/
public DbPerfActivitySample withXactSeconds(BigDecimal xactSeconds)
{
this.xactSeconds = xactSeconds;
return (this);
}
/*******************************************************************************
** Getter for blockedBy.
*******************************************************************************/
public String getBlockedBy()
{
return (this.blockedBy);
}
/*******************************************************************************
** Setter for blockedBy.
*******************************************************************************/
public void setBlockedBy(String blockedBy)
{
this.blockedBy = blockedBy;
}
/*******************************************************************************
** Fluent setter for blockedBy.
*******************************************************************************/
public DbPerfActivitySample withBlockedBy(String blockedBy)
{
this.blockedBy = blockedBy;
return (this);
}
/*******************************************************************************
** Getter for blockingCount.
*******************************************************************************/
public Integer getBlockingCount()
{
return (this.blockingCount);
}
/*******************************************************************************
** Setter for blockingCount.
*******************************************************************************/
public void setBlockingCount(Integer blockingCount)
{
this.blockingCount = blockingCount;
}
/*******************************************************************************
** Fluent setter for blockingCount.
*******************************************************************************/
public DbPerfActivitySample withBlockingCount(Integer blockingCount)
{
this.blockingCount = blockingCount;
return (this);
}
/*******************************************************************************
** Getter for reason.
*******************************************************************************/
public String getReason()
{
return (this.reason);
}
/*******************************************************************************
** Setter for reason.
*******************************************************************************/
public void setReason(String reason)
{
this.reason = reason;
}
/*******************************************************************************
** Fluent setter for reason.
*******************************************************************************/
public DbPerfActivitySample withReason(String reason)
{
this.reason = reason;
return (this);
}
/*******************************************************************************
** Getter for queryId.
*******************************************************************************/
public Long getQueryId()
{
return (this.queryId);
}
/*******************************************************************************
** Setter for queryId.
*******************************************************************************/
public void setQueryId(Long queryId)
{
this.queryId = queryId;
}
/*******************************************************************************
** Fluent setter for queryId.
*******************************************************************************/
public DbPerfActivitySample withQueryId(Long queryId)
{
this.queryId = queryId;
return (this);
}
/*******************************************************************************
** Getter for statementId.
*******************************************************************************/
public Integer getStatementId()
{
return (this.statementId);
}
/*******************************************************************************
** Setter for statementId.
*******************************************************************************/
public void setStatementId(Integer statementId)
{
this.statementId = statementId;
}
/*******************************************************************************
** Fluent setter for statementId.
*******************************************************************************/
public DbPerfActivitySample withStatementId(Integer statementId)
{
this.statementId = statementId;
return (this);
}
/*******************************************************************************
** Getter for queryFingerprint.
*******************************************************************************/
public String getQueryFingerprint()
{
return (this.queryFingerprint);
}
/*******************************************************************************
** Setter for queryFingerprint.
*******************************************************************************/
public void setQueryFingerprint(String queryFingerprint)
{
this.queryFingerprint = queryFingerprint;
}
/*******************************************************************************
** Fluent setter for queryFingerprint.
*******************************************************************************/
public DbPerfActivitySample withQueryFingerprint(String queryFingerprint)
{
this.queryFingerprint = queryFingerprint;
return (this);
}
/*******************************************************************************
** Getter for queryTextNormalized.
*******************************************************************************/
public String getQueryTextNormalized()
{
return (this.queryTextNormalized);
}
/*******************************************************************************
** Setter for queryTextNormalized.
*******************************************************************************/
public void setQueryTextNormalized(String queryTextNormalized)
{
this.queryTextNormalized = queryTextNormalized;
}
/*******************************************************************************
** Fluent setter for queryTextNormalized.
*******************************************************************************/
public DbPerfActivitySample withQueryTextNormalized(String queryTextNormalized)
{
this.queryTextNormalized = queryTextNormalized;
return (this);
}
}
@@ -0,0 +1,612 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** QQQ Attribution persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfAttribution extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_attribution";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Instant windowStart;
@QField
private Instant windowEnd;
@QField(maxLength = 200)
private String podName;
@QField(maxLength = 20)
private String unitKind;
@QField(maxLength = 300)
private String unitName;
@QField(maxLength = 200)
private String processName;
@QField(maxLength = 200)
private String tableName;
@QField(maxLength = 20)
private String operation;
@QField(maxLength = 64)
private String fingerprint;
@QField(maxLength = 4000)
private String sqlText;
@QField
private Long units;
@QField
private Long statements;
@QField
private BigDecimal totalMs;
@QField
private BigDecimal maxMs;
@QField
private Long rowsCount;
@QField
private Long errors;
@QField
private BigDecimal sampleRate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfAttribution()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfAttribution(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfAttribution withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for windowStart.
*******************************************************************************/
public Instant getWindowStart()
{
return (this.windowStart);
}
/*******************************************************************************
** Setter for windowStart.
*******************************************************************************/
public void setWindowStart(Instant windowStart)
{
this.windowStart = windowStart;
}
/*******************************************************************************
** Fluent setter for windowStart.
*******************************************************************************/
public DbPerfAttribution withWindowStart(Instant windowStart)
{
this.windowStart = windowStart;
return (this);
}
/*******************************************************************************
** Getter for windowEnd.
*******************************************************************************/
public Instant getWindowEnd()
{
return (this.windowEnd);
}
/*******************************************************************************
** Setter for windowEnd.
*******************************************************************************/
public void setWindowEnd(Instant windowEnd)
{
this.windowEnd = windowEnd;
}
/*******************************************************************************
** Fluent setter for windowEnd.
*******************************************************************************/
public DbPerfAttribution withWindowEnd(Instant windowEnd)
{
this.windowEnd = windowEnd;
return (this);
}
/*******************************************************************************
** Getter for podName.
*******************************************************************************/
public String getPodName()
{
return (this.podName);
}
/*******************************************************************************
** Setter for podName.
*******************************************************************************/
public void setPodName(String podName)
{
this.podName = podName;
}
/*******************************************************************************
** Fluent setter for podName.
*******************************************************************************/
public DbPerfAttribution withPodName(String podName)
{
this.podName = podName;
return (this);
}
/*******************************************************************************
** Getter for unitKind.
*******************************************************************************/
public String getUnitKind()
{
return (this.unitKind);
}
/*******************************************************************************
** Setter for unitKind.
*******************************************************************************/
public void setUnitKind(String unitKind)
{
this.unitKind = unitKind;
}
/*******************************************************************************
** Fluent setter for unitKind.
*******************************************************************************/
public DbPerfAttribution withUnitKind(String unitKind)
{
this.unitKind = unitKind;
return (this);
}
/*******************************************************************************
** Getter for unitName.
*******************************************************************************/
public String getUnitName()
{
return (this.unitName);
}
/*******************************************************************************
** Setter for unitName.
*******************************************************************************/
public void setUnitName(String unitName)
{
this.unitName = unitName;
}
/*******************************************************************************
** Fluent setter for unitName.
*******************************************************************************/
public DbPerfAttribution withUnitName(String unitName)
{
this.unitName = unitName;
return (this);
}
/*******************************************************************************
** Getter for processName.
*******************************************************************************/
public String getProcessName()
{
return (this.processName);
}
/*******************************************************************************
** Setter for processName.
*******************************************************************************/
public void setProcessName(String processName)
{
this.processName = processName;
}
/*******************************************************************************
** Fluent setter for processName.
*******************************************************************************/
public DbPerfAttribution withProcessName(String processName)
{
this.processName = processName;
return (this);
}
/*******************************************************************************
** Getter for tableName.
*******************************************************************************/
public String getTableName()
{
return (this.tableName);
}
/*******************************************************************************
** Setter for tableName.
*******************************************************************************/
public void setTableName(String tableName)
{
this.tableName = tableName;
}
/*******************************************************************************
** Fluent setter for tableName.
*******************************************************************************/
public DbPerfAttribution withTableName(String tableName)
{
this.tableName = tableName;
return (this);
}
/*******************************************************************************
** Getter for operation.
*******************************************************************************/
public String getOperation()
{
return (this.operation);
}
/*******************************************************************************
** Setter for operation.
*******************************************************************************/
public void setOperation(String operation)
{
this.operation = operation;
}
/*******************************************************************************
** Fluent setter for operation.
*******************************************************************************/
public DbPerfAttribution withOperation(String operation)
{
this.operation = operation;
return (this);
}
/*******************************************************************************
** Getter for fingerprint.
*******************************************************************************/
public String getFingerprint()
{
return (this.fingerprint);
}
/*******************************************************************************
** Setter for fingerprint.
*******************************************************************************/
public void setFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
}
/*******************************************************************************
** Fluent setter for fingerprint.
*******************************************************************************/
public DbPerfAttribution withFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
return (this);
}
/*******************************************************************************
** Getter for sqlText.
*******************************************************************************/
public String getSqlText()
{
return (this.sqlText);
}
/*******************************************************************************
** Setter for sqlText.
*******************************************************************************/
public void setSqlText(String sqlText)
{
this.sqlText = sqlText;
}
/*******************************************************************************
** Fluent setter for sqlText.
*******************************************************************************/
public DbPerfAttribution withSqlText(String sqlText)
{
this.sqlText = sqlText;
return (this);
}
/*******************************************************************************
** Getter for units.
*******************************************************************************/
public Long getUnits()
{
return (this.units);
}
/*******************************************************************************
** Setter for units.
*******************************************************************************/
public void setUnits(Long units)
{
this.units = units;
}
/*******************************************************************************
** Fluent setter for units.
*******************************************************************************/
public DbPerfAttribution withUnits(Long units)
{
this.units = units;
return (this);
}
/*******************************************************************************
** Getter for statements.
*******************************************************************************/
public Long getStatements()
{
return (this.statements);
}
/*******************************************************************************
** Setter for statements.
*******************************************************************************/
public void setStatements(Long statements)
{
this.statements = statements;
}
/*******************************************************************************
** Fluent setter for statements.
*******************************************************************************/
public DbPerfAttribution withStatements(Long statements)
{
this.statements = statements;
return (this);
}
/*******************************************************************************
** Getter for totalMs.
*******************************************************************************/
public BigDecimal getTotalMs()
{
return (this.totalMs);
}
/*******************************************************************************
** Setter for totalMs.
*******************************************************************************/
public void setTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
}
/*******************************************************************************
** Fluent setter for totalMs.
*******************************************************************************/
public DbPerfAttribution withTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
return (this);
}
/*******************************************************************************
** Getter for maxMs.
*******************************************************************************/
public BigDecimal getMaxMs()
{
return (this.maxMs);
}
/*******************************************************************************
** Setter for maxMs.
*******************************************************************************/
public void setMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
}
/*******************************************************************************
** Fluent setter for maxMs.
*******************************************************************************/
public DbPerfAttribution withMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
return (this);
}
/*******************************************************************************
** Getter for rowsCount.
*******************************************************************************/
public Long getRowsCount()
{
return (this.rowsCount);
}
/*******************************************************************************
** Setter for rowsCount.
*******************************************************************************/
public void setRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
}
/*******************************************************************************
** Fluent setter for rowsCount.
*******************************************************************************/
public DbPerfAttribution withRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
return (this);
}
/*******************************************************************************
** Getter for errors.
*******************************************************************************/
public Long getErrors()
{
return (this.errors);
}
/*******************************************************************************
** Setter for errors.
*******************************************************************************/
public void setErrors(Long errors)
{
this.errors = errors;
}
/*******************************************************************************
** Fluent setter for errors.
*******************************************************************************/
public DbPerfAttribution withErrors(Long errors)
{
this.errors = errors;
return (this);
}
/*******************************************************************************
** Getter for sampleRate.
*******************************************************************************/
public BigDecimal getSampleRate()
{
return (this.sampleRate);
}
/*******************************************************************************
** Setter for sampleRate.
*******************************************************************************/
public void setSampleRate(BigDecimal sampleRate)
{
this.sampleRate = sampleRate;
}
/*******************************************************************************
** Fluent setter for sampleRate.
*******************************************************************************/
public DbPerfAttribution withSampleRate(BigDecimal sampleRate)
{
this.sampleRate = sampleRate;
return (this);
}
}
@@ -0,0 +1,550 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.LocalDate;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** QQQ Attribution Daily persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfAttributionDaily extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_attribution_daily";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private LocalDate day;
@QField(maxLength = 64)
private String rollupKey;
@QField(maxLength = 20)
private String unitKind;
@QField(maxLength = 300)
private String unitName;
@QField(maxLength = 200)
private String processName;
@QField(maxLength = 200)
private String tableName;
@QField(maxLength = 20)
private String operation;
@QField(maxLength = 64)
private String fingerprint;
@QField(maxLength = 4000)
private String sqlText;
@QField
private Long units;
@QField
private Long statements;
@QField
private BigDecimal totalMs;
@QField
private BigDecimal maxMs;
@QField
private Long rowsCount;
@QField
private Long errors;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfAttributionDaily()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfAttributionDaily(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfAttributionDaily withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for day.
*******************************************************************************/
public LocalDate getDay()
{
return (this.day);
}
/*******************************************************************************
** Setter for day.
*******************************************************************************/
public void setDay(LocalDate day)
{
this.day = day;
}
/*******************************************************************************
** Fluent setter for day.
*******************************************************************************/
public DbPerfAttributionDaily withDay(LocalDate day)
{
this.day = day;
return (this);
}
/*******************************************************************************
** Getter for rollupKey.
*******************************************************************************/
public String getRollupKey()
{
return (this.rollupKey);
}
/*******************************************************************************
** Setter for rollupKey.
*******************************************************************************/
public void setRollupKey(String rollupKey)
{
this.rollupKey = rollupKey;
}
/*******************************************************************************
** Fluent setter for rollupKey.
*******************************************************************************/
public DbPerfAttributionDaily withRollupKey(String rollupKey)
{
this.rollupKey = rollupKey;
return (this);
}
/*******************************************************************************
** Getter for unitKind.
*******************************************************************************/
public String getUnitKind()
{
return (this.unitKind);
}
/*******************************************************************************
** Setter for unitKind.
*******************************************************************************/
public void setUnitKind(String unitKind)
{
this.unitKind = unitKind;
}
/*******************************************************************************
** Fluent setter for unitKind.
*******************************************************************************/
public DbPerfAttributionDaily withUnitKind(String unitKind)
{
this.unitKind = unitKind;
return (this);
}
/*******************************************************************************
** Getter for unitName.
*******************************************************************************/
public String getUnitName()
{
return (this.unitName);
}
/*******************************************************************************
** Setter for unitName.
*******************************************************************************/
public void setUnitName(String unitName)
{
this.unitName = unitName;
}
/*******************************************************************************
** Fluent setter for unitName.
*******************************************************************************/
public DbPerfAttributionDaily withUnitName(String unitName)
{
this.unitName = unitName;
return (this);
}
/*******************************************************************************
** Getter for processName.
*******************************************************************************/
public String getProcessName()
{
return (this.processName);
}
/*******************************************************************************
** Setter for processName.
*******************************************************************************/
public void setProcessName(String processName)
{
this.processName = processName;
}
/*******************************************************************************
** Fluent setter for processName.
*******************************************************************************/
public DbPerfAttributionDaily withProcessName(String processName)
{
this.processName = processName;
return (this);
}
/*******************************************************************************
** Getter for tableName.
*******************************************************************************/
public String getTableName()
{
return (this.tableName);
}
/*******************************************************************************
** Setter for tableName.
*******************************************************************************/
public void setTableName(String tableName)
{
this.tableName = tableName;
}
/*******************************************************************************
** Fluent setter for tableName.
*******************************************************************************/
public DbPerfAttributionDaily withTableName(String tableName)
{
this.tableName = tableName;
return (this);
}
/*******************************************************************************
** Getter for operation.
*******************************************************************************/
public String getOperation()
{
return (this.operation);
}
/*******************************************************************************
** Setter for operation.
*******************************************************************************/
public void setOperation(String operation)
{
this.operation = operation;
}
/*******************************************************************************
** Fluent setter for operation.
*******************************************************************************/
public DbPerfAttributionDaily withOperation(String operation)
{
this.operation = operation;
return (this);
}
/*******************************************************************************
** Getter for fingerprint.
*******************************************************************************/
public String getFingerprint()
{
return (this.fingerprint);
}
/*******************************************************************************
** Setter for fingerprint.
*******************************************************************************/
public void setFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
}
/*******************************************************************************
** Fluent setter for fingerprint.
*******************************************************************************/
public DbPerfAttributionDaily withFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
return (this);
}
/*******************************************************************************
** Getter for sqlText.
*******************************************************************************/
public String getSqlText()
{
return (this.sqlText);
}
/*******************************************************************************
** Setter for sqlText.
*******************************************************************************/
public void setSqlText(String sqlText)
{
this.sqlText = sqlText;
}
/*******************************************************************************
** Fluent setter for sqlText.
*******************************************************************************/
public DbPerfAttributionDaily withSqlText(String sqlText)
{
this.sqlText = sqlText;
return (this);
}
/*******************************************************************************
** Getter for units.
*******************************************************************************/
public Long getUnits()
{
return (this.units);
}
/*******************************************************************************
** Setter for units.
*******************************************************************************/
public void setUnits(Long units)
{
this.units = units;
}
/*******************************************************************************
** Fluent setter for units.
*******************************************************************************/
public DbPerfAttributionDaily withUnits(Long units)
{
this.units = units;
return (this);
}
/*******************************************************************************
** Getter for statements.
*******************************************************************************/
public Long getStatements()
{
return (this.statements);
}
/*******************************************************************************
** Setter for statements.
*******************************************************************************/
public void setStatements(Long statements)
{
this.statements = statements;
}
/*******************************************************************************
** Fluent setter for statements.
*******************************************************************************/
public DbPerfAttributionDaily withStatements(Long statements)
{
this.statements = statements;
return (this);
}
/*******************************************************************************
** Getter for totalMs.
*******************************************************************************/
public BigDecimal getTotalMs()
{
return (this.totalMs);
}
/*******************************************************************************
** Setter for totalMs.
*******************************************************************************/
public void setTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
}
/*******************************************************************************
** Fluent setter for totalMs.
*******************************************************************************/
public DbPerfAttributionDaily withTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
return (this);
}
/*******************************************************************************
** Getter for maxMs.
*******************************************************************************/
public BigDecimal getMaxMs()
{
return (this.maxMs);
}
/*******************************************************************************
** Setter for maxMs.
*******************************************************************************/
public void setMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
}
/*******************************************************************************
** Fluent setter for maxMs.
*******************************************************************************/
public DbPerfAttributionDaily withMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
return (this);
}
/*******************************************************************************
** Getter for rowsCount.
*******************************************************************************/
public Long getRowsCount()
{
return (this.rowsCount);
}
/*******************************************************************************
** Setter for rowsCount.
*******************************************************************************/
public void setRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
}
/*******************************************************************************
** Fluent setter for rowsCount.
*******************************************************************************/
public DbPerfAttributionDaily withRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
return (this);
}
/*******************************************************************************
** Getter for errors.
*******************************************************************************/
public Long getErrors()
{
return (this.errors);
}
/*******************************************************************************
** Setter for errors.
*******************************************************************************/
public void setErrors(Long errors)
{
this.errors = errors;
}
/*******************************************************************************
** Fluent setter for errors.
*******************************************************************************/
public DbPerfAttributionDaily withErrors(Long errors)
{
this.errors = errors;
return (this);
}
}
@@ -0,0 +1,270 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Connection Samples persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfConnectionSample extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_connection_sample";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer snapshotId;
@QField
private Instant capturedAt;
@QField(maxLength = 100)
private String dbUser;
@QField(maxLength = 200)
private String applicationName;
@QField(maxLength = 40)
private String state;
@QField
private Integer connectionCount;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfConnectionSample()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfConnectionSample(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfConnectionSample withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for snapshotId.
*******************************************************************************/
public Integer getSnapshotId()
{
return (this.snapshotId);
}
/*******************************************************************************
** Setter for snapshotId.
*******************************************************************************/
public void setSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
}
/*******************************************************************************
** Fluent setter for snapshotId.
*******************************************************************************/
public DbPerfConnectionSample withSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
return (this);
}
/*******************************************************************************
** Getter for capturedAt.
*******************************************************************************/
public Instant getCapturedAt()
{
return (this.capturedAt);
}
/*******************************************************************************
** Setter for capturedAt.
*******************************************************************************/
public void setCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
}
/*******************************************************************************
** Fluent setter for capturedAt.
*******************************************************************************/
public DbPerfConnectionSample withCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
return (this);
}
/*******************************************************************************
** Getter for dbUser.
*******************************************************************************/
public String getDbUser()
{
return (this.dbUser);
}
/*******************************************************************************
** Setter for dbUser.
*******************************************************************************/
public void setDbUser(String dbUser)
{
this.dbUser = dbUser;
}
/*******************************************************************************
** Fluent setter for dbUser.
*******************************************************************************/
public DbPerfConnectionSample withDbUser(String dbUser)
{
this.dbUser = dbUser;
return (this);
}
/*******************************************************************************
** Getter for applicationName.
*******************************************************************************/
public String getApplicationName()
{
return (this.applicationName);
}
/*******************************************************************************
** Setter for applicationName.
*******************************************************************************/
public void setApplicationName(String applicationName)
{
this.applicationName = applicationName;
}
/*******************************************************************************
** Fluent setter for applicationName.
*******************************************************************************/
public DbPerfConnectionSample withApplicationName(String applicationName)
{
this.applicationName = applicationName;
return (this);
}
/*******************************************************************************
** Getter for state.
*******************************************************************************/
public String getState()
{
return (this.state);
}
/*******************************************************************************
** Setter for state.
*******************************************************************************/
public void setState(String state)
{
this.state = state;
}
/*******************************************************************************
** Fluent setter for state.
*******************************************************************************/
public DbPerfConnectionSample withState(String state)
{
this.state = state;
return (this);
}
/*******************************************************************************
** Getter for connectionCount.
*******************************************************************************/
public Integer getConnectionCount()
{
return (this.connectionCount);
}
/*******************************************************************************
** Setter for connectionCount.
*******************************************************************************/
public void setConnectionCount(Integer connectionCount)
{
this.connectionCount = connectionCount;
}
/*******************************************************************************
** Fluent setter for connectionCount.
*******************************************************************************/
public DbPerfConnectionSample withConnectionCount(Integer connectionCount)
{
this.connectionCount = connectionCount;
return (this);
}
}
@@ -0,0 +1,488 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Query Plans persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfExplain extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_explain";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer statementId;
@QField(maxLength = 100)
private String requestedBy;
@QField
private Instant requestedAt;
@QField(maxLength = 20)
private String mode;
@QField(maxLength = 64)
private String sqlFingerprint;
@QField
private String sqlText;
@QField
private Integer parameterCount;
@QField
private Integer durationMs;
@QField(maxLength = 20)
private String outcome;
@QField(maxLength = 2000)
private String errorMessage;
@QField
private String planJson;
@QField
private String planText;
@QField
private BigDecimal totalCost;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfExplain()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfExplain(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfExplain withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for statementId.
*******************************************************************************/
public Integer getStatementId()
{
return (this.statementId);
}
/*******************************************************************************
** Setter for statementId.
*******************************************************************************/
public void setStatementId(Integer statementId)
{
this.statementId = statementId;
}
/*******************************************************************************
** Fluent setter for statementId.
*******************************************************************************/
public DbPerfExplain withStatementId(Integer statementId)
{
this.statementId = statementId;
return (this);
}
/*******************************************************************************
** Getter for requestedBy.
*******************************************************************************/
public String getRequestedBy()
{
return (this.requestedBy);
}
/*******************************************************************************
** Setter for requestedBy.
*******************************************************************************/
public void setRequestedBy(String requestedBy)
{
this.requestedBy = requestedBy;
}
/*******************************************************************************
** Fluent setter for requestedBy.
*******************************************************************************/
public DbPerfExplain withRequestedBy(String requestedBy)
{
this.requestedBy = requestedBy;
return (this);
}
/*******************************************************************************
** Getter for requestedAt.
*******************************************************************************/
public Instant getRequestedAt()
{
return (this.requestedAt);
}
/*******************************************************************************
** Setter for requestedAt.
*******************************************************************************/
public void setRequestedAt(Instant requestedAt)
{
this.requestedAt = requestedAt;
}
/*******************************************************************************
** Fluent setter for requestedAt.
*******************************************************************************/
public DbPerfExplain withRequestedAt(Instant requestedAt)
{
this.requestedAt = requestedAt;
return (this);
}
/*******************************************************************************
** Getter for mode.
*******************************************************************************/
public String getMode()
{
return (this.mode);
}
/*******************************************************************************
** Setter for mode.
*******************************************************************************/
public void setMode(String mode)
{
this.mode = mode;
}
/*******************************************************************************
** Fluent setter for mode.
*******************************************************************************/
public DbPerfExplain withMode(String mode)
{
this.mode = mode;
return (this);
}
/*******************************************************************************
** Getter for sqlFingerprint.
*******************************************************************************/
public String getSqlFingerprint()
{
return (this.sqlFingerprint);
}
/*******************************************************************************
** Setter for sqlFingerprint.
*******************************************************************************/
public void setSqlFingerprint(String sqlFingerprint)
{
this.sqlFingerprint = sqlFingerprint;
}
/*******************************************************************************
** Fluent setter for sqlFingerprint.
*******************************************************************************/
public DbPerfExplain withSqlFingerprint(String sqlFingerprint)
{
this.sqlFingerprint = sqlFingerprint;
return (this);
}
/*******************************************************************************
** Getter for sqlText.
*******************************************************************************/
public String getSqlText()
{
return (this.sqlText);
}
/*******************************************************************************
** Setter for sqlText.
*******************************************************************************/
public void setSqlText(String sqlText)
{
this.sqlText = sqlText;
}
/*******************************************************************************
** Fluent setter for sqlText.
*******************************************************************************/
public DbPerfExplain withSqlText(String sqlText)
{
this.sqlText = sqlText;
return (this);
}
/*******************************************************************************
** Getter for parameterCount.
*******************************************************************************/
public Integer getParameterCount()
{
return (this.parameterCount);
}
/*******************************************************************************
** Setter for parameterCount.
*******************************************************************************/
public void setParameterCount(Integer parameterCount)
{
this.parameterCount = parameterCount;
}
/*******************************************************************************
** Fluent setter for parameterCount.
*******************************************************************************/
public DbPerfExplain withParameterCount(Integer parameterCount)
{
this.parameterCount = parameterCount;
return (this);
}
/*******************************************************************************
** Getter for durationMs.
*******************************************************************************/
public Integer getDurationMs()
{
return (this.durationMs);
}
/*******************************************************************************
** Setter for durationMs.
*******************************************************************************/
public void setDurationMs(Integer durationMs)
{
this.durationMs = durationMs;
}
/*******************************************************************************
** Fluent setter for durationMs.
*******************************************************************************/
public DbPerfExplain withDurationMs(Integer durationMs)
{
this.durationMs = durationMs;
return (this);
}
/*******************************************************************************
** Getter for outcome.
*******************************************************************************/
public String getOutcome()
{
return (this.outcome);
}
/*******************************************************************************
** Setter for outcome.
*******************************************************************************/
public void setOutcome(String outcome)
{
this.outcome = outcome;
}
/*******************************************************************************
** Fluent setter for outcome.
*******************************************************************************/
public DbPerfExplain withOutcome(String outcome)
{
this.outcome = outcome;
return (this);
}
/*******************************************************************************
** Getter for errorMessage.
*******************************************************************************/
public String getErrorMessage()
{
return (this.errorMessage);
}
/*******************************************************************************
** Setter for errorMessage.
*******************************************************************************/
public void setErrorMessage(String errorMessage)
{
this.errorMessage = errorMessage;
}
/*******************************************************************************
** Fluent setter for errorMessage.
*******************************************************************************/
public DbPerfExplain withErrorMessage(String errorMessage)
{
this.errorMessage = errorMessage;
return (this);
}
/*******************************************************************************
** Getter for planJson.
*******************************************************************************/
public String getPlanJson()
{
return (this.planJson);
}
/*******************************************************************************
** Setter for planJson.
*******************************************************************************/
public void setPlanJson(String planJson)
{
this.planJson = planJson;
}
/*******************************************************************************
** Fluent setter for planJson.
*******************************************************************************/
public DbPerfExplain withPlanJson(String planJson)
{
this.planJson = planJson;
return (this);
}
/*******************************************************************************
** Getter for planText.
*******************************************************************************/
public String getPlanText()
{
return (this.planText);
}
/*******************************************************************************
** Setter for planText.
*******************************************************************************/
public void setPlanText(String planText)
{
this.planText = planText;
}
/*******************************************************************************
** Fluent setter for planText.
*******************************************************************************/
public DbPerfExplain withPlanText(String planText)
{
this.planText = planText;
return (this);
}
/*******************************************************************************
** Getter for totalCost.
*******************************************************************************/
public BigDecimal getTotalCost()
{
return (this.totalCost);
}
/*******************************************************************************
** Setter for totalCost.
*******************************************************************************/
public void setTotalCost(BigDecimal totalCost)
{
this.totalCost = totalCost;
}
/*******************************************************************************
** Fluent setter for totalCost.
*******************************************************************************/
public DbPerfExplain withTotalCost(BigDecimal totalCost)
{
this.totalCost = totalCost;
return (this);
}
}
@@ -0,0 +1,860 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Findings persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfFinding extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_finding";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(maxLength = 100)
private String backendName;
@QField
private Integer ruleId;
@QField(maxLength = 40)
private String ruleCode;
@QField(maxLength = 300)
private String dedupeKey;
@QField(maxLength = 20)
private String status;
@QField(maxLength = 20)
private String severity;
@QField
private BigDecimal score;
@QField(maxLength = 300)
private String title;
@QField(maxLength = 2000)
private String summary;
@QField
private String evidence;
@QField
private String recommendation;
@QField
private String migrationSnippet;
@QField(maxLength = 200)
private String subjectTable;
@QField
private Integer statementId;
@QField
private Integer tableStatId;
@QField
private Integer indexStatId;
@QField(maxLength = 300)
private String unitName;
@QField
private Instant firstSeenAt;
@QField
private Instant lastSeenAt;
@QField
private Integer occurrences;
@QField
private Instant resolvedAt;
@QField(maxLength = 2000)
private String statusNote;
@QField(maxLength = 100)
private String statusChangedBy;
@QField
private Instant createDate;
@QField
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfFinding()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfFinding(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfFinding withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for backendName.
*******************************************************************************/
public String getBackendName()
{
return (this.backendName);
}
/*******************************************************************************
** Setter for backendName.
*******************************************************************************/
public void setBackendName(String backendName)
{
this.backendName = backendName;
}
/*******************************************************************************
** Fluent setter for backendName.
*******************************************************************************/
public DbPerfFinding withBackendName(String backendName)
{
this.backendName = backendName;
return (this);
}
/*******************************************************************************
** Getter for ruleId.
*******************************************************************************/
public Integer getRuleId()
{
return (this.ruleId);
}
/*******************************************************************************
** Setter for ruleId.
*******************************************************************************/
public void setRuleId(Integer ruleId)
{
this.ruleId = ruleId;
}
/*******************************************************************************
** Fluent setter for ruleId.
*******************************************************************************/
public DbPerfFinding withRuleId(Integer ruleId)
{
this.ruleId = ruleId;
return (this);
}
/*******************************************************************************
** Getter for ruleCode.
*******************************************************************************/
public String getRuleCode()
{
return (this.ruleCode);
}
/*******************************************************************************
** Setter for ruleCode.
*******************************************************************************/
public void setRuleCode(String ruleCode)
{
this.ruleCode = ruleCode;
}
/*******************************************************************************
** Fluent setter for ruleCode.
*******************************************************************************/
public DbPerfFinding withRuleCode(String ruleCode)
{
this.ruleCode = ruleCode;
return (this);
}
/*******************************************************************************
** Getter for dedupeKey.
*******************************************************************************/
public String getDedupeKey()
{
return (this.dedupeKey);
}
/*******************************************************************************
** Setter for dedupeKey.
*******************************************************************************/
public void setDedupeKey(String dedupeKey)
{
this.dedupeKey = dedupeKey;
}
/*******************************************************************************
** Fluent setter for dedupeKey.
*******************************************************************************/
public DbPerfFinding withDedupeKey(String dedupeKey)
{
this.dedupeKey = dedupeKey;
return (this);
}
/*******************************************************************************
** Getter for status.
*******************************************************************************/
public String getStatus()
{
return (this.status);
}
/*******************************************************************************
** Setter for status.
*******************************************************************************/
public void setStatus(String status)
{
this.status = status;
}
/*******************************************************************************
** Fluent setter for status.
*******************************************************************************/
public DbPerfFinding withStatus(String status)
{
this.status = status;
return (this);
}
/*******************************************************************************
** Getter for severity.
*******************************************************************************/
public String getSeverity()
{
return (this.severity);
}
/*******************************************************************************
** Setter for severity.
*******************************************************************************/
public void setSeverity(String severity)
{
this.severity = severity;
}
/*******************************************************************************
** Fluent setter for severity.
*******************************************************************************/
public DbPerfFinding withSeverity(String severity)
{
this.severity = severity;
return (this);
}
/*******************************************************************************
** Getter for score.
*******************************************************************************/
public BigDecimal getScore()
{
return (this.score);
}
/*******************************************************************************
** Setter for score.
*******************************************************************************/
public void setScore(BigDecimal score)
{
this.score = score;
}
/*******************************************************************************
** Fluent setter for score.
*******************************************************************************/
public DbPerfFinding withScore(BigDecimal score)
{
this.score = score;
return (this);
}
/*******************************************************************************
** Getter for title.
*******************************************************************************/
public String getTitle()
{
return (this.title);
}
/*******************************************************************************
** Setter for title.
*******************************************************************************/
public void setTitle(String title)
{
this.title = title;
}
/*******************************************************************************
** Fluent setter for title.
*******************************************************************************/
public DbPerfFinding withTitle(String title)
{
this.title = title;
return (this);
}
/*******************************************************************************
** Getter for summary.
*******************************************************************************/
public String getSummary()
{
return (this.summary);
}
/*******************************************************************************
** Setter for summary.
*******************************************************************************/
public void setSummary(String summary)
{
this.summary = summary;
}
/*******************************************************************************
** Fluent setter for summary.
*******************************************************************************/
public DbPerfFinding withSummary(String summary)
{
this.summary = summary;
return (this);
}
/*******************************************************************************
** Getter for evidence.
*******************************************************************************/
public String getEvidence()
{
return (this.evidence);
}
/*******************************************************************************
** Setter for evidence.
*******************************************************************************/
public void setEvidence(String evidence)
{
this.evidence = evidence;
}
/*******************************************************************************
** Fluent setter for evidence.
*******************************************************************************/
public DbPerfFinding withEvidence(String evidence)
{
this.evidence = evidence;
return (this);
}
/*******************************************************************************
** Getter for recommendation.
*******************************************************************************/
public String getRecommendation()
{
return (this.recommendation);
}
/*******************************************************************************
** Setter for recommendation.
*******************************************************************************/
public void setRecommendation(String recommendation)
{
this.recommendation = recommendation;
}
/*******************************************************************************
** Fluent setter for recommendation.
*******************************************************************************/
public DbPerfFinding withRecommendation(String recommendation)
{
this.recommendation = recommendation;
return (this);
}
/*******************************************************************************
** Getter for migrationSnippet.
*******************************************************************************/
public String getMigrationSnippet()
{
return (this.migrationSnippet);
}
/*******************************************************************************
** Setter for migrationSnippet.
*******************************************************************************/
public void setMigrationSnippet(String migrationSnippet)
{
this.migrationSnippet = migrationSnippet;
}
/*******************************************************************************
** Fluent setter for migrationSnippet.
*******************************************************************************/
public DbPerfFinding withMigrationSnippet(String migrationSnippet)
{
this.migrationSnippet = migrationSnippet;
return (this);
}
/*******************************************************************************
** Getter for subjectTable.
*******************************************************************************/
public String getSubjectTable()
{
return (this.subjectTable);
}
/*******************************************************************************
** Setter for subjectTable.
*******************************************************************************/
public void setSubjectTable(String subjectTable)
{
this.subjectTable = subjectTable;
}
/*******************************************************************************
** Fluent setter for subjectTable.
*******************************************************************************/
public DbPerfFinding withSubjectTable(String subjectTable)
{
this.subjectTable = subjectTable;
return (this);
}
/*******************************************************************************
** Getter for statementId.
*******************************************************************************/
public Integer getStatementId()
{
return (this.statementId);
}
/*******************************************************************************
** Setter for statementId.
*******************************************************************************/
public void setStatementId(Integer statementId)
{
this.statementId = statementId;
}
/*******************************************************************************
** Fluent setter for statementId.
*******************************************************************************/
public DbPerfFinding withStatementId(Integer statementId)
{
this.statementId = statementId;
return (this);
}
/*******************************************************************************
** Getter for tableStatId.
*******************************************************************************/
public Integer getTableStatId()
{
return (this.tableStatId);
}
/*******************************************************************************
** Setter for tableStatId.
*******************************************************************************/
public void setTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
}
/*******************************************************************************
** Fluent setter for tableStatId.
*******************************************************************************/
public DbPerfFinding withTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
return (this);
}
/*******************************************************************************
** Getter for indexStatId.
*******************************************************************************/
public Integer getIndexStatId()
{
return (this.indexStatId);
}
/*******************************************************************************
** Setter for indexStatId.
*******************************************************************************/
public void setIndexStatId(Integer indexStatId)
{
this.indexStatId = indexStatId;
}
/*******************************************************************************
** Fluent setter for indexStatId.
*******************************************************************************/
public DbPerfFinding withIndexStatId(Integer indexStatId)
{
this.indexStatId = indexStatId;
return (this);
}
/*******************************************************************************
** Getter for unitName.
*******************************************************************************/
public String getUnitName()
{
return (this.unitName);
}
/*******************************************************************************
** Setter for unitName.
*******************************************************************************/
public void setUnitName(String unitName)
{
this.unitName = unitName;
}
/*******************************************************************************
** Fluent setter for unitName.
*******************************************************************************/
public DbPerfFinding withUnitName(String unitName)
{
this.unitName = unitName;
return (this);
}
/*******************************************************************************
** Getter for firstSeenAt.
*******************************************************************************/
public Instant getFirstSeenAt()
{
return (this.firstSeenAt);
}
/*******************************************************************************
** Setter for firstSeenAt.
*******************************************************************************/
public void setFirstSeenAt(Instant firstSeenAt)
{
this.firstSeenAt = firstSeenAt;
}
/*******************************************************************************
** Fluent setter for firstSeenAt.
*******************************************************************************/
public DbPerfFinding withFirstSeenAt(Instant firstSeenAt)
{
this.firstSeenAt = firstSeenAt;
return (this);
}
/*******************************************************************************
** Getter for lastSeenAt.
*******************************************************************************/
public Instant getLastSeenAt()
{
return (this.lastSeenAt);
}
/*******************************************************************************
** Setter for lastSeenAt.
*******************************************************************************/
public void setLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
}
/*******************************************************************************
** Fluent setter for lastSeenAt.
*******************************************************************************/
public DbPerfFinding withLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
return (this);
}
/*******************************************************************************
** Getter for occurrences.
*******************************************************************************/
public Integer getOccurrences()
{
return (this.occurrences);
}
/*******************************************************************************
** Setter for occurrences.
*******************************************************************************/
public void setOccurrences(Integer occurrences)
{
this.occurrences = occurrences;
}
/*******************************************************************************
** Fluent setter for occurrences.
*******************************************************************************/
public DbPerfFinding withOccurrences(Integer occurrences)
{
this.occurrences = occurrences;
return (this);
}
/*******************************************************************************
** Getter for resolvedAt.
*******************************************************************************/
public Instant getResolvedAt()
{
return (this.resolvedAt);
}
/*******************************************************************************
** Setter for resolvedAt.
*******************************************************************************/
public void setResolvedAt(Instant resolvedAt)
{
this.resolvedAt = resolvedAt;
}
/*******************************************************************************
** Fluent setter for resolvedAt.
*******************************************************************************/
public DbPerfFinding withResolvedAt(Instant resolvedAt)
{
this.resolvedAt = resolvedAt;
return (this);
}
/*******************************************************************************
** Getter for statusNote.
*******************************************************************************/
public String getStatusNote()
{
return (this.statusNote);
}
/*******************************************************************************
** Setter for statusNote.
*******************************************************************************/
public void setStatusNote(String statusNote)
{
this.statusNote = statusNote;
}
/*******************************************************************************
** Fluent setter for statusNote.
*******************************************************************************/
public DbPerfFinding withStatusNote(String statusNote)
{
this.statusNote = statusNote;
return (this);
}
/*******************************************************************************
** Getter for statusChangedBy.
*******************************************************************************/
public String getStatusChangedBy()
{
return (this.statusChangedBy);
}
/*******************************************************************************
** Setter for statusChangedBy.
*******************************************************************************/
public void setStatusChangedBy(String statusChangedBy)
{
this.statusChangedBy = statusChangedBy;
}
/*******************************************************************************
** Fluent setter for statusChangedBy.
*******************************************************************************/
public DbPerfFinding withStatusChangedBy(String statusChangedBy)
{
this.statusChangedBy = statusChangedBy;
return (this);
}
/*******************************************************************************
** Getter for createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate.
*******************************************************************************/
public DbPerfFinding withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate.
*******************************************************************************/
public DbPerfFinding withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,704 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Indexes persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfIndexStat extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_index_stat";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(maxLength = 100)
private String backendName;
@QField
private Integer tableStatId;
@QField(maxLength = 100)
private String schemaName;
@QField(maxLength = 200)
private String tableName;
@QField(maxLength = 200)
private String indexName;
@QField
private Long relid;
@QField(maxLength = 2000)
private String definition;
@QField(maxLength = 1000)
private String columns;
@QField
private Boolean isUnique;
@QField
private Boolean isPrimary;
@QField
private Boolean isValid;
@QField
private Long idxScan;
@QField
private Long idxTupRead;
@QField
private Long idxTupFetch;
@QField
private Long sizeBytes;
@QField
private Long writeCost;
@QField
private Long scansSinceReset;
@QField
private Instant lastSeenAt;
@QField
private Instant createDate;
@QField
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfIndexStat()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfIndexStat(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfIndexStat withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for backendName.
*******************************************************************************/
public String getBackendName()
{
return (this.backendName);
}
/*******************************************************************************
** Setter for backendName.
*******************************************************************************/
public void setBackendName(String backendName)
{
this.backendName = backendName;
}
/*******************************************************************************
** Fluent setter for backendName.
*******************************************************************************/
public DbPerfIndexStat withBackendName(String backendName)
{
this.backendName = backendName;
return (this);
}
/*******************************************************************************
** Getter for tableStatId.
*******************************************************************************/
public Integer getTableStatId()
{
return (this.tableStatId);
}
/*******************************************************************************
** Setter for tableStatId.
*******************************************************************************/
public void setTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
}
/*******************************************************************************
** Fluent setter for tableStatId.
*******************************************************************************/
public DbPerfIndexStat withTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
return (this);
}
/*******************************************************************************
** Getter for schemaName.
*******************************************************************************/
public String getSchemaName()
{
return (this.schemaName);
}
/*******************************************************************************
** Setter for schemaName.
*******************************************************************************/
public void setSchemaName(String schemaName)
{
this.schemaName = schemaName;
}
/*******************************************************************************
** Fluent setter for schemaName.
*******************************************************************************/
public DbPerfIndexStat withSchemaName(String schemaName)
{
this.schemaName = schemaName;
return (this);
}
/*******************************************************************************
** Getter for tableName.
*******************************************************************************/
public String getTableName()
{
return (this.tableName);
}
/*******************************************************************************
** Setter for tableName.
*******************************************************************************/
public void setTableName(String tableName)
{
this.tableName = tableName;
}
/*******************************************************************************
** Fluent setter for tableName.
*******************************************************************************/
public DbPerfIndexStat withTableName(String tableName)
{
this.tableName = tableName;
return (this);
}
/*******************************************************************************
** Getter for indexName.
*******************************************************************************/
public String getIndexName()
{
return (this.indexName);
}
/*******************************************************************************
** Setter for indexName.
*******************************************************************************/
public void setIndexName(String indexName)
{
this.indexName = indexName;
}
/*******************************************************************************
** Fluent setter for indexName.
*******************************************************************************/
public DbPerfIndexStat withIndexName(String indexName)
{
this.indexName = indexName;
return (this);
}
/*******************************************************************************
** Getter for relid.
*******************************************************************************/
public Long getRelid()
{
return (this.relid);
}
/*******************************************************************************
** Setter for relid.
*******************************************************************************/
public void setRelid(Long relid)
{
this.relid = relid;
}
/*******************************************************************************
** Fluent setter for relid.
*******************************************************************************/
public DbPerfIndexStat withRelid(Long relid)
{
this.relid = relid;
return (this);
}
/*******************************************************************************
** Getter for definition.
*******************************************************************************/
public String getDefinition()
{
return (this.definition);
}
/*******************************************************************************
** Setter for definition.
*******************************************************************************/
public void setDefinition(String definition)
{
this.definition = definition;
}
/*******************************************************************************
** Fluent setter for definition.
*******************************************************************************/
public DbPerfIndexStat withDefinition(String definition)
{
this.definition = definition;
return (this);
}
/*******************************************************************************
** Getter for columns.
*******************************************************************************/
public String getColumns()
{
return (this.columns);
}
/*******************************************************************************
** Setter for columns.
*******************************************************************************/
public void setColumns(String columns)
{
this.columns = columns;
}
/*******************************************************************************
** Fluent setter for columns.
*******************************************************************************/
public DbPerfIndexStat withColumns(String columns)
{
this.columns = columns;
return (this);
}
/*******************************************************************************
** Getter for isUnique.
*******************************************************************************/
public Boolean getIsUnique()
{
return (this.isUnique);
}
/*******************************************************************************
** Setter for isUnique.
*******************************************************************************/
public void setIsUnique(Boolean isUnique)
{
this.isUnique = isUnique;
}
/*******************************************************************************
** Fluent setter for isUnique.
*******************************************************************************/
public DbPerfIndexStat withIsUnique(Boolean isUnique)
{
this.isUnique = isUnique;
return (this);
}
/*******************************************************************************
** Getter for isPrimary.
*******************************************************************************/
public Boolean getIsPrimary()
{
return (this.isPrimary);
}
/*******************************************************************************
** Setter for isPrimary.
*******************************************************************************/
public void setIsPrimary(Boolean isPrimary)
{
this.isPrimary = isPrimary;
}
/*******************************************************************************
** Fluent setter for isPrimary.
*******************************************************************************/
public DbPerfIndexStat withIsPrimary(Boolean isPrimary)
{
this.isPrimary = isPrimary;
return (this);
}
/*******************************************************************************
** Getter for isValid.
*******************************************************************************/
public Boolean getIsValid()
{
return (this.isValid);
}
/*******************************************************************************
** Setter for isValid.
*******************************************************************************/
public void setIsValid(Boolean isValid)
{
this.isValid = isValid;
}
/*******************************************************************************
** Fluent setter for isValid.
*******************************************************************************/
public DbPerfIndexStat withIsValid(Boolean isValid)
{
this.isValid = isValid;
return (this);
}
/*******************************************************************************
** Getter for idxScan.
*******************************************************************************/
public Long getIdxScan()
{
return (this.idxScan);
}
/*******************************************************************************
** Setter for idxScan.
*******************************************************************************/
public void setIdxScan(Long idxScan)
{
this.idxScan = idxScan;
}
/*******************************************************************************
** Fluent setter for idxScan.
*******************************************************************************/
public DbPerfIndexStat withIdxScan(Long idxScan)
{
this.idxScan = idxScan;
return (this);
}
/*******************************************************************************
** Getter for idxTupRead.
*******************************************************************************/
public Long getIdxTupRead()
{
return (this.idxTupRead);
}
/*******************************************************************************
** Setter for idxTupRead.
*******************************************************************************/
public void setIdxTupRead(Long idxTupRead)
{
this.idxTupRead = idxTupRead;
}
/*******************************************************************************
** Fluent setter for idxTupRead.
*******************************************************************************/
public DbPerfIndexStat withIdxTupRead(Long idxTupRead)
{
this.idxTupRead = idxTupRead;
return (this);
}
/*******************************************************************************
** Getter for idxTupFetch.
*******************************************************************************/
public Long getIdxTupFetch()
{
return (this.idxTupFetch);
}
/*******************************************************************************
** Setter for idxTupFetch.
*******************************************************************************/
public void setIdxTupFetch(Long idxTupFetch)
{
this.idxTupFetch = idxTupFetch;
}
/*******************************************************************************
** Fluent setter for idxTupFetch.
*******************************************************************************/
public DbPerfIndexStat withIdxTupFetch(Long idxTupFetch)
{
this.idxTupFetch = idxTupFetch;
return (this);
}
/*******************************************************************************
** Getter for sizeBytes.
*******************************************************************************/
public Long getSizeBytes()
{
return (this.sizeBytes);
}
/*******************************************************************************
** Setter for sizeBytes.
*******************************************************************************/
public void setSizeBytes(Long sizeBytes)
{
this.sizeBytes = sizeBytes;
}
/*******************************************************************************
** Fluent setter for sizeBytes.
*******************************************************************************/
public DbPerfIndexStat withSizeBytes(Long sizeBytes)
{
this.sizeBytes = sizeBytes;
return (this);
}
/*******************************************************************************
** Getter for writeCost.
*******************************************************************************/
public Long getWriteCost()
{
return (this.writeCost);
}
/*******************************************************************************
** Setter for writeCost.
*******************************************************************************/
public void setWriteCost(Long writeCost)
{
this.writeCost = writeCost;
}
/*******************************************************************************
** Fluent setter for writeCost.
*******************************************************************************/
public DbPerfIndexStat withWriteCost(Long writeCost)
{
this.writeCost = writeCost;
return (this);
}
/*******************************************************************************
** Getter for scansSinceReset.
*******************************************************************************/
public Long getScansSinceReset()
{
return (this.scansSinceReset);
}
/*******************************************************************************
** Setter for scansSinceReset.
*******************************************************************************/
public void setScansSinceReset(Long scansSinceReset)
{
this.scansSinceReset = scansSinceReset;
}
/*******************************************************************************
** Fluent setter for scansSinceReset.
*******************************************************************************/
public DbPerfIndexStat withScansSinceReset(Long scansSinceReset)
{
this.scansSinceReset = scansSinceReset;
return (this);
}
/*******************************************************************************
** Getter for lastSeenAt.
*******************************************************************************/
public Instant getLastSeenAt()
{
return (this.lastSeenAt);
}
/*******************************************************************************
** Setter for lastSeenAt.
*******************************************************************************/
public void setLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
}
/*******************************************************************************
** Fluent setter for lastSeenAt.
*******************************************************************************/
public DbPerfIndexStat withLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
return (this);
}
/*******************************************************************************
** Getter for createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate.
*******************************************************************************/
public DbPerfIndexStat withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate.
*******************************************************************************/
public DbPerfIndexStat withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,395 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.LocalDate;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Query Daily persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfQueryDaily extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_query_daily";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer statementId;
@QField
private LocalDate day;
@QField
private Long calls;
@QField
private BigDecimal totalMs;
@QField
private BigDecimal meanMs;
@QField
private BigDecimal maxMs;
@QField
private Long rowsCount;
@QField
private BigDecimal hitRatio;
@QField
private BigDecimal shareOfDbTime;
@QField
private Integer snapshots;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfQueryDaily()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfQueryDaily(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfQueryDaily withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for statementId.
*******************************************************************************/
public Integer getStatementId()
{
return (this.statementId);
}
/*******************************************************************************
** Setter for statementId.
*******************************************************************************/
public void setStatementId(Integer statementId)
{
this.statementId = statementId;
}
/*******************************************************************************
** Fluent setter for statementId.
*******************************************************************************/
public DbPerfQueryDaily withStatementId(Integer statementId)
{
this.statementId = statementId;
return (this);
}
/*******************************************************************************
** Getter for day.
*******************************************************************************/
public LocalDate getDay()
{
return (this.day);
}
/*******************************************************************************
** Setter for day.
*******************************************************************************/
public void setDay(LocalDate day)
{
this.day = day;
}
/*******************************************************************************
** Fluent setter for day.
*******************************************************************************/
public DbPerfQueryDaily withDay(LocalDate day)
{
this.day = day;
return (this);
}
/*******************************************************************************
** Getter for calls.
*******************************************************************************/
public Long getCalls()
{
return (this.calls);
}
/*******************************************************************************
** Setter for calls.
*******************************************************************************/
public void setCalls(Long calls)
{
this.calls = calls;
}
/*******************************************************************************
** Fluent setter for calls.
*******************************************************************************/
public DbPerfQueryDaily withCalls(Long calls)
{
this.calls = calls;
return (this);
}
/*******************************************************************************
** Getter for totalMs.
*******************************************************************************/
public BigDecimal getTotalMs()
{
return (this.totalMs);
}
/*******************************************************************************
** Setter for totalMs.
*******************************************************************************/
public void setTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
}
/*******************************************************************************
** Fluent setter for totalMs.
*******************************************************************************/
public DbPerfQueryDaily withTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
return (this);
}
/*******************************************************************************
** Getter for meanMs.
*******************************************************************************/
public BigDecimal getMeanMs()
{
return (this.meanMs);
}
/*******************************************************************************
** Setter for meanMs.
*******************************************************************************/
public void setMeanMs(BigDecimal meanMs)
{
this.meanMs = meanMs;
}
/*******************************************************************************
** Fluent setter for meanMs.
*******************************************************************************/
public DbPerfQueryDaily withMeanMs(BigDecimal meanMs)
{
this.meanMs = meanMs;
return (this);
}
/*******************************************************************************
** Getter for maxMs.
*******************************************************************************/
public BigDecimal getMaxMs()
{
return (this.maxMs);
}
/*******************************************************************************
** Setter for maxMs.
*******************************************************************************/
public void setMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
}
/*******************************************************************************
** Fluent setter for maxMs.
*******************************************************************************/
public DbPerfQueryDaily withMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
return (this);
}
/*******************************************************************************
** Getter for rowsCount.
*******************************************************************************/
public Long getRowsCount()
{
return (this.rowsCount);
}
/*******************************************************************************
** Setter for rowsCount.
*******************************************************************************/
public void setRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
}
/*******************************************************************************
** Fluent setter for rowsCount.
*******************************************************************************/
public DbPerfQueryDaily withRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
return (this);
}
/*******************************************************************************
** Getter for hitRatio.
*******************************************************************************/
public BigDecimal getHitRatio()
{
return (this.hitRatio);
}
/*******************************************************************************
** Setter for hitRatio.
*******************************************************************************/
public void setHitRatio(BigDecimal hitRatio)
{
this.hitRatio = hitRatio;
}
/*******************************************************************************
** Fluent setter for hitRatio.
*******************************************************************************/
public DbPerfQueryDaily withHitRatio(BigDecimal hitRatio)
{
this.hitRatio = hitRatio;
return (this);
}
/*******************************************************************************
** Getter for shareOfDbTime.
*******************************************************************************/
public BigDecimal getShareOfDbTime()
{
return (this.shareOfDbTime);
}
/*******************************************************************************
** Setter for shareOfDbTime.
*******************************************************************************/
public void setShareOfDbTime(BigDecimal shareOfDbTime)
{
this.shareOfDbTime = shareOfDbTime;
}
/*******************************************************************************
** Fluent setter for shareOfDbTime.
*******************************************************************************/
public DbPerfQueryDaily withShareOfDbTime(BigDecimal shareOfDbTime)
{
this.shareOfDbTime = shareOfDbTime;
return (this);
}
/*******************************************************************************
** Getter for snapshots.
*******************************************************************************/
public Integer getSnapshots()
{
return (this.snapshots);
}
/*******************************************************************************
** Setter for snapshots.
*******************************************************************************/
public void setSnapshots(Integer snapshots)
{
this.snapshots = snapshots;
}
/*******************************************************************************
** Fluent setter for snapshots.
*******************************************************************************/
public DbPerfQueryDaily withSnapshots(Integer snapshots)
{
this.snapshots = snapshots;
return (this);
}
}
@@ -0,0 +1,488 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Query Snapshots persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfQuerySnapshot extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_query_snapshot";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer snapshotId;
@QField
private Integer statementId;
@QField
private Instant capturedAt;
@QField
private Long calls;
@QField
private BigDecimal totalMs;
@QField
private BigDecimal meanMs;
@QField
private BigDecimal maxMs;
@QField
private Long rowsCount;
@QField
private Long sharedBlksHit;
@QField
private Long sharedBlksRead;
@QField
private BigDecimal hitRatio;
@QField
private Long tempBlksWritten;
@QField
private BigDecimal shareOfDbTime;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfQuerySnapshot()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfQuerySnapshot(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfQuerySnapshot withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for snapshotId.
*******************************************************************************/
public Integer getSnapshotId()
{
return (this.snapshotId);
}
/*******************************************************************************
** Setter for snapshotId.
*******************************************************************************/
public void setSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
}
/*******************************************************************************
** Fluent setter for snapshotId.
*******************************************************************************/
public DbPerfQuerySnapshot withSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
return (this);
}
/*******************************************************************************
** Getter for statementId.
*******************************************************************************/
public Integer getStatementId()
{
return (this.statementId);
}
/*******************************************************************************
** Setter for statementId.
*******************************************************************************/
public void setStatementId(Integer statementId)
{
this.statementId = statementId;
}
/*******************************************************************************
** Fluent setter for statementId.
*******************************************************************************/
public DbPerfQuerySnapshot withStatementId(Integer statementId)
{
this.statementId = statementId;
return (this);
}
/*******************************************************************************
** Getter for capturedAt.
*******************************************************************************/
public Instant getCapturedAt()
{
return (this.capturedAt);
}
/*******************************************************************************
** Setter for capturedAt.
*******************************************************************************/
public void setCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
}
/*******************************************************************************
** Fluent setter for capturedAt.
*******************************************************************************/
public DbPerfQuerySnapshot withCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
return (this);
}
/*******************************************************************************
** Getter for calls.
*******************************************************************************/
public Long getCalls()
{
return (this.calls);
}
/*******************************************************************************
** Setter for calls.
*******************************************************************************/
public void setCalls(Long calls)
{
this.calls = calls;
}
/*******************************************************************************
** Fluent setter for calls.
*******************************************************************************/
public DbPerfQuerySnapshot withCalls(Long calls)
{
this.calls = calls;
return (this);
}
/*******************************************************************************
** Getter for totalMs.
*******************************************************************************/
public BigDecimal getTotalMs()
{
return (this.totalMs);
}
/*******************************************************************************
** Setter for totalMs.
*******************************************************************************/
public void setTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
}
/*******************************************************************************
** Fluent setter for totalMs.
*******************************************************************************/
public DbPerfQuerySnapshot withTotalMs(BigDecimal totalMs)
{
this.totalMs = totalMs;
return (this);
}
/*******************************************************************************
** Getter for meanMs.
*******************************************************************************/
public BigDecimal getMeanMs()
{
return (this.meanMs);
}
/*******************************************************************************
** Setter for meanMs.
*******************************************************************************/
public void setMeanMs(BigDecimal meanMs)
{
this.meanMs = meanMs;
}
/*******************************************************************************
** Fluent setter for meanMs.
*******************************************************************************/
public DbPerfQuerySnapshot withMeanMs(BigDecimal meanMs)
{
this.meanMs = meanMs;
return (this);
}
/*******************************************************************************
** Getter for maxMs.
*******************************************************************************/
public BigDecimal getMaxMs()
{
return (this.maxMs);
}
/*******************************************************************************
** Setter for maxMs.
*******************************************************************************/
public void setMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
}
/*******************************************************************************
** Fluent setter for maxMs.
*******************************************************************************/
public DbPerfQuerySnapshot withMaxMs(BigDecimal maxMs)
{
this.maxMs = maxMs;
return (this);
}
/*******************************************************************************
** Getter for rowsCount.
*******************************************************************************/
public Long getRowsCount()
{
return (this.rowsCount);
}
/*******************************************************************************
** Setter for rowsCount.
*******************************************************************************/
public void setRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
}
/*******************************************************************************
** Fluent setter for rowsCount.
*******************************************************************************/
public DbPerfQuerySnapshot withRowsCount(Long rowsCount)
{
this.rowsCount = rowsCount;
return (this);
}
/*******************************************************************************
** Getter for sharedBlksHit.
*******************************************************************************/
public Long getSharedBlksHit()
{
return (this.sharedBlksHit);
}
/*******************************************************************************
** Setter for sharedBlksHit.
*******************************************************************************/
public void setSharedBlksHit(Long sharedBlksHit)
{
this.sharedBlksHit = sharedBlksHit;
}
/*******************************************************************************
** Fluent setter for sharedBlksHit.
*******************************************************************************/
public DbPerfQuerySnapshot withSharedBlksHit(Long sharedBlksHit)
{
this.sharedBlksHit = sharedBlksHit;
return (this);
}
/*******************************************************************************
** Getter for sharedBlksRead.
*******************************************************************************/
public Long getSharedBlksRead()
{
return (this.sharedBlksRead);
}
/*******************************************************************************
** Setter for sharedBlksRead.
*******************************************************************************/
public void setSharedBlksRead(Long sharedBlksRead)
{
this.sharedBlksRead = sharedBlksRead;
}
/*******************************************************************************
** Fluent setter for sharedBlksRead.
*******************************************************************************/
public DbPerfQuerySnapshot withSharedBlksRead(Long sharedBlksRead)
{
this.sharedBlksRead = sharedBlksRead;
return (this);
}
/*******************************************************************************
** Getter for hitRatio.
*******************************************************************************/
public BigDecimal getHitRatio()
{
return (this.hitRatio);
}
/*******************************************************************************
** Setter for hitRatio.
*******************************************************************************/
public void setHitRatio(BigDecimal hitRatio)
{
this.hitRatio = hitRatio;
}
/*******************************************************************************
** Fluent setter for hitRatio.
*******************************************************************************/
public DbPerfQuerySnapshot withHitRatio(BigDecimal hitRatio)
{
this.hitRatio = hitRatio;
return (this);
}
/*******************************************************************************
** Getter for tempBlksWritten.
*******************************************************************************/
public Long getTempBlksWritten()
{
return (this.tempBlksWritten);
}
/*******************************************************************************
** Setter for tempBlksWritten.
*******************************************************************************/
public void setTempBlksWritten(Long tempBlksWritten)
{
this.tempBlksWritten = tempBlksWritten;
}
/*******************************************************************************
** Fluent setter for tempBlksWritten.
*******************************************************************************/
public DbPerfQuerySnapshot withTempBlksWritten(Long tempBlksWritten)
{
this.tempBlksWritten = tempBlksWritten;
return (this);
}
/*******************************************************************************
** Getter for shareOfDbTime.
*******************************************************************************/
public BigDecimal getShareOfDbTime()
{
return (this.shareOfDbTime);
}
/*******************************************************************************
** Setter for shareOfDbTime.
*******************************************************************************/
public void setShareOfDbTime(BigDecimal shareOfDbTime)
{
this.shareOfDbTime = shareOfDbTime;
}
/*******************************************************************************
** Fluent setter for shareOfDbTime.
*******************************************************************************/
public DbPerfQuerySnapshot withShareOfDbTime(BigDecimal shareOfDbTime)
{
this.shareOfDbTime = shareOfDbTime;
return (this);
}
}
@@ -0,0 +1,487 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Repeated Statements persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfRepeatPattern extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_repeat_pattern";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Instant windowStart;
@QField
private Instant windowEnd;
@QField(maxLength = 200)
private String podName;
@QField(maxLength = 20)
private String unitKind;
@QField(maxLength = 300)
private String unitName;
@QField(maxLength = 200)
private String processName;
@QField(maxLength = 200)
private String tableName;
@QField(maxLength = 64)
private String fingerprint;
@QField(maxLength = 4000)
private String sqlText;
@QField
private Long unitsAffected;
@QField
private Integer maxRepeats;
@QField
private Long totalRepeats;
@QField
private BigDecimal sampleRate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfRepeatPattern()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfRepeatPattern(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfRepeatPattern withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for windowStart.
*******************************************************************************/
public Instant getWindowStart()
{
return (this.windowStart);
}
/*******************************************************************************
** Setter for windowStart.
*******************************************************************************/
public void setWindowStart(Instant windowStart)
{
this.windowStart = windowStart;
}
/*******************************************************************************
** Fluent setter for windowStart.
*******************************************************************************/
public DbPerfRepeatPattern withWindowStart(Instant windowStart)
{
this.windowStart = windowStart;
return (this);
}
/*******************************************************************************
** Getter for windowEnd.
*******************************************************************************/
public Instant getWindowEnd()
{
return (this.windowEnd);
}
/*******************************************************************************
** Setter for windowEnd.
*******************************************************************************/
public void setWindowEnd(Instant windowEnd)
{
this.windowEnd = windowEnd;
}
/*******************************************************************************
** Fluent setter for windowEnd.
*******************************************************************************/
public DbPerfRepeatPattern withWindowEnd(Instant windowEnd)
{
this.windowEnd = windowEnd;
return (this);
}
/*******************************************************************************
** Getter for podName.
*******************************************************************************/
public String getPodName()
{
return (this.podName);
}
/*******************************************************************************
** Setter for podName.
*******************************************************************************/
public void setPodName(String podName)
{
this.podName = podName;
}
/*******************************************************************************
** Fluent setter for podName.
*******************************************************************************/
public DbPerfRepeatPattern withPodName(String podName)
{
this.podName = podName;
return (this);
}
/*******************************************************************************
** Getter for unitKind.
*******************************************************************************/
public String getUnitKind()
{
return (this.unitKind);
}
/*******************************************************************************
** Setter for unitKind.
*******************************************************************************/
public void setUnitKind(String unitKind)
{
this.unitKind = unitKind;
}
/*******************************************************************************
** Fluent setter for unitKind.
*******************************************************************************/
public DbPerfRepeatPattern withUnitKind(String unitKind)
{
this.unitKind = unitKind;
return (this);
}
/*******************************************************************************
** Getter for unitName.
*******************************************************************************/
public String getUnitName()
{
return (this.unitName);
}
/*******************************************************************************
** Setter for unitName.
*******************************************************************************/
public void setUnitName(String unitName)
{
this.unitName = unitName;
}
/*******************************************************************************
** Fluent setter for unitName.
*******************************************************************************/
public DbPerfRepeatPattern withUnitName(String unitName)
{
this.unitName = unitName;
return (this);
}
/*******************************************************************************
** Getter for processName.
*******************************************************************************/
public String getProcessName()
{
return (this.processName);
}
/*******************************************************************************
** Setter for processName.
*******************************************************************************/
public void setProcessName(String processName)
{
this.processName = processName;
}
/*******************************************************************************
** Fluent setter for processName.
*******************************************************************************/
public DbPerfRepeatPattern withProcessName(String processName)
{
this.processName = processName;
return (this);
}
/*******************************************************************************
** Getter for tableName.
*******************************************************************************/
public String getTableName()
{
return (this.tableName);
}
/*******************************************************************************
** Setter for tableName.
*******************************************************************************/
public void setTableName(String tableName)
{
this.tableName = tableName;
}
/*******************************************************************************
** Fluent setter for tableName.
*******************************************************************************/
public DbPerfRepeatPattern withTableName(String tableName)
{
this.tableName = tableName;
return (this);
}
/*******************************************************************************
** Getter for fingerprint.
*******************************************************************************/
public String getFingerprint()
{
return (this.fingerprint);
}
/*******************************************************************************
** Setter for fingerprint.
*******************************************************************************/
public void setFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
}
/*******************************************************************************
** Fluent setter for fingerprint.
*******************************************************************************/
public DbPerfRepeatPattern withFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
return (this);
}
/*******************************************************************************
** Getter for sqlText.
*******************************************************************************/
public String getSqlText()
{
return (this.sqlText);
}
/*******************************************************************************
** Setter for sqlText.
*******************************************************************************/
public void setSqlText(String sqlText)
{
this.sqlText = sqlText;
}
/*******************************************************************************
** Fluent setter for sqlText.
*******************************************************************************/
public DbPerfRepeatPattern withSqlText(String sqlText)
{
this.sqlText = sqlText;
return (this);
}
/*******************************************************************************
** Getter for unitsAffected.
*******************************************************************************/
public Long getUnitsAffected()
{
return (this.unitsAffected);
}
/*******************************************************************************
** Setter for unitsAffected.
*******************************************************************************/
public void setUnitsAffected(Long unitsAffected)
{
this.unitsAffected = unitsAffected;
}
/*******************************************************************************
** Fluent setter for unitsAffected.
*******************************************************************************/
public DbPerfRepeatPattern withUnitsAffected(Long unitsAffected)
{
this.unitsAffected = unitsAffected;
return (this);
}
/*******************************************************************************
** Getter for maxRepeats.
*******************************************************************************/
public Integer getMaxRepeats()
{
return (this.maxRepeats);
}
/*******************************************************************************
** Setter for maxRepeats.
*******************************************************************************/
public void setMaxRepeats(Integer maxRepeats)
{
this.maxRepeats = maxRepeats;
}
/*******************************************************************************
** Fluent setter for maxRepeats.
*******************************************************************************/
public DbPerfRepeatPattern withMaxRepeats(Integer maxRepeats)
{
this.maxRepeats = maxRepeats;
return (this);
}
/*******************************************************************************
** Getter for totalRepeats.
*******************************************************************************/
public Long getTotalRepeats()
{
return (this.totalRepeats);
}
/*******************************************************************************
** Setter for totalRepeats.
*******************************************************************************/
public void setTotalRepeats(Long totalRepeats)
{
this.totalRepeats = totalRepeats;
}
/*******************************************************************************
** Fluent setter for totalRepeats.
*******************************************************************************/
public DbPerfRepeatPattern withTotalRepeats(Long totalRepeats)
{
this.totalRepeats = totalRepeats;
return (this);
}
/*******************************************************************************
** Getter for sampleRate.
*******************************************************************************/
public BigDecimal getSampleRate()
{
return (sampleRate);
}
/*******************************************************************************
** Setter for sampleRate.
*******************************************************************************/
public void setSampleRate(BigDecimal sampleRate)
{
this.sampleRate = sampleRate;
}
/*******************************************************************************
** Fluent setter for sampleRate.
*******************************************************************************/
public DbPerfRepeatPattern withSampleRate(BigDecimal sampleRate)
{
this.sampleRate = sampleRate;
return (this);
}
}
@@ -0,0 +1,426 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Suggestion Rules persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfRule extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_rule";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(isRequired = true, maxLength = 40)
private String code;
@QField(maxLength = 100)
private String label;
@QField(maxLength = 1000)
private String description;
@QField
private Boolean enabled;
@QField(maxLength = 20)
private String severity;
@QField
private BigDecimal threshold1;
@QField
private BigDecimal threshold2;
@QField(maxLength = 100)
private String threshold1Label;
@QField(maxLength = 100)
private String threshold2Label;
@QField
private Instant createDate;
@QField
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfRule()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfRule(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfRule withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for code.
*******************************************************************************/
public String getCode()
{
return (this.code);
}
/*******************************************************************************
** Setter for code.
*******************************************************************************/
public void setCode(String code)
{
this.code = code;
}
/*******************************************************************************
** Fluent setter for code.
*******************************************************************************/
public DbPerfRule withCode(String code)
{
this.code = code;
return (this);
}
/*******************************************************************************
** Getter for label.
*******************************************************************************/
public String getLabel()
{
return (this.label);
}
/*******************************************************************************
** Setter for label.
*******************************************************************************/
public void setLabel(String label)
{
this.label = label;
}
/*******************************************************************************
** Fluent setter for label.
*******************************************************************************/
public DbPerfRule withLabel(String label)
{
this.label = label;
return (this);
}
/*******************************************************************************
** Getter for description.
*******************************************************************************/
public String getDescription()
{
return (this.description);
}
/*******************************************************************************
** Setter for description.
*******************************************************************************/
public void setDescription(String description)
{
this.description = description;
}
/*******************************************************************************
** Fluent setter for description.
*******************************************************************************/
public DbPerfRule withDescription(String description)
{
this.description = description;
return (this);
}
/*******************************************************************************
** Getter for enabled.
*******************************************************************************/
public Boolean getEnabled()
{
return (this.enabled);
}
/*******************************************************************************
** Setter for enabled.
*******************************************************************************/
public void setEnabled(Boolean enabled)
{
this.enabled = enabled;
}
/*******************************************************************************
** Fluent setter for enabled.
*******************************************************************************/
public DbPerfRule withEnabled(Boolean enabled)
{
this.enabled = enabled;
return (this);
}
/*******************************************************************************
** Getter for severity.
*******************************************************************************/
public String getSeverity()
{
return (this.severity);
}
/*******************************************************************************
** Setter for severity.
*******************************************************************************/
public void setSeverity(String severity)
{
this.severity = severity;
}
/*******************************************************************************
** Fluent setter for severity.
*******************************************************************************/
public DbPerfRule withSeverity(String severity)
{
this.severity = severity;
return (this);
}
/*******************************************************************************
** Getter for threshold1.
*******************************************************************************/
public BigDecimal getThreshold1()
{
return (this.threshold1);
}
/*******************************************************************************
** Setter for threshold1.
*******************************************************************************/
public void setThreshold1(BigDecimal threshold1)
{
this.threshold1 = threshold1;
}
/*******************************************************************************
** Fluent setter for threshold1.
*******************************************************************************/
public DbPerfRule withThreshold1(BigDecimal threshold1)
{
this.threshold1 = threshold1;
return (this);
}
/*******************************************************************************
** Getter for threshold2.
*******************************************************************************/
public BigDecimal getThreshold2()
{
return (this.threshold2);
}
/*******************************************************************************
** Setter for threshold2.
*******************************************************************************/
public void setThreshold2(BigDecimal threshold2)
{
this.threshold2 = threshold2;
}
/*******************************************************************************
** Fluent setter for threshold2.
*******************************************************************************/
public DbPerfRule withThreshold2(BigDecimal threshold2)
{
this.threshold2 = threshold2;
return (this);
}
/*******************************************************************************
** Getter for threshold1Label.
*******************************************************************************/
public String getThreshold1Label()
{
return (this.threshold1Label);
}
/*******************************************************************************
** Setter for threshold1Label.
*******************************************************************************/
public void setThreshold1Label(String threshold1Label)
{
this.threshold1Label = threshold1Label;
}
/*******************************************************************************
** Fluent setter for threshold1Label.
*******************************************************************************/
public DbPerfRule withThreshold1Label(String threshold1Label)
{
this.threshold1Label = threshold1Label;
return (this);
}
/*******************************************************************************
** Getter for threshold2Label.
*******************************************************************************/
public String getThreshold2Label()
{
return (this.threshold2Label);
}
/*******************************************************************************
** Setter for threshold2Label.
*******************************************************************************/
public void setThreshold2Label(String threshold2Label)
{
this.threshold2Label = threshold2Label;
}
/*******************************************************************************
** Fluent setter for threshold2Label.
*******************************************************************************/
public DbPerfRule withThreshold2Label(String threshold2Label)
{
this.threshold2Label = threshold2Label;
return (this);
}
/*******************************************************************************
** Getter for createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate.
*******************************************************************************/
public DbPerfRule withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate.
*******************************************************************************/
public DbPerfRule withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,922 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Settings persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfSettings extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_settings";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(isRequired = true, maxLength = 100)
private String backendName;
@QField
private Boolean enabled;
@QField(maxLength = 20)
private String provider;
@QField
private Integer snapshotIntervalMinutes;
@QField
private Integer tableSnapshotMinutes;
@QField
private Integer detailRetentionDays;
@QField
private Integer rollupRetentionDays;
@QField
private Integer topStatementsPerSnapshot;
@QField
private Integer activityLongRunningSeconds;
@QField
private Integer idleInTransactionSeconds;
@QField
private Boolean attributionEnabled;
@QField
private BigDecimal attributionSampleRate;
@QField
private Integer attributionFlushSeconds;
@QField
private Integer attributionMaxKeys;
@QField
private Integer nPlusOneThreshold;
@QField
private Integer explainTimeoutMs;
@QField
private Boolean alertsEnabled;
@QField
private BigDecimal spikeFactor;
@QField
private Integer spikeMinTotalMs;
@QField
private Integer connectionAlertPercent;
@QField
private Integer blockedAlertSeconds;
@QField
private Instant lastSnapshotAt;
@QField
private Instant lastTableSnapshotAt;
@QField(maxLength = 30)
private String statsAccess;
@QField(maxLength = 500)
private String statsAccessMessage;
@QField
private Instant createDate;
@QField
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfSettings()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfSettings(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfSettings withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for backendName.
*******************************************************************************/
public String getBackendName()
{
return (this.backendName);
}
/*******************************************************************************
** Setter for backendName.
*******************************************************************************/
public void setBackendName(String backendName)
{
this.backendName = backendName;
}
/*******************************************************************************
** Fluent setter for backendName.
*******************************************************************************/
public DbPerfSettings withBackendName(String backendName)
{
this.backendName = backendName;
return (this);
}
/*******************************************************************************
** Getter for enabled.
*******************************************************************************/
public Boolean getEnabled()
{
return (this.enabled);
}
/*******************************************************************************
** Setter for enabled.
*******************************************************************************/
public void setEnabled(Boolean enabled)
{
this.enabled = enabled;
}
/*******************************************************************************
** Fluent setter for enabled.
*******************************************************************************/
public DbPerfSettings withEnabled(Boolean enabled)
{
this.enabled = enabled;
return (this);
}
/*******************************************************************************
** Getter for provider.
*******************************************************************************/
public String getProvider()
{
return (this.provider);
}
/*******************************************************************************
** Setter for provider.
*******************************************************************************/
public void setProvider(String provider)
{
this.provider = provider;
}
/*******************************************************************************
** Fluent setter for provider.
*******************************************************************************/
public DbPerfSettings withProvider(String provider)
{
this.provider = provider;
return (this);
}
/*******************************************************************************
** Getter for snapshotIntervalMinutes.
*******************************************************************************/
public Integer getSnapshotIntervalMinutes()
{
return (this.snapshotIntervalMinutes);
}
/*******************************************************************************
** Setter for snapshotIntervalMinutes.
*******************************************************************************/
public void setSnapshotIntervalMinutes(Integer snapshotIntervalMinutes)
{
this.snapshotIntervalMinutes = snapshotIntervalMinutes;
}
/*******************************************************************************
** Fluent setter for snapshotIntervalMinutes.
*******************************************************************************/
public DbPerfSettings withSnapshotIntervalMinutes(Integer snapshotIntervalMinutes)
{
this.snapshotIntervalMinutes = snapshotIntervalMinutes;
return (this);
}
/*******************************************************************************
** Getter for tableSnapshotMinutes.
*******************************************************************************/
public Integer getTableSnapshotMinutes()
{
return (this.tableSnapshotMinutes);
}
/*******************************************************************************
** Setter for tableSnapshotMinutes.
*******************************************************************************/
public void setTableSnapshotMinutes(Integer tableSnapshotMinutes)
{
this.tableSnapshotMinutes = tableSnapshotMinutes;
}
/*******************************************************************************
** Fluent setter for tableSnapshotMinutes.
*******************************************************************************/
public DbPerfSettings withTableSnapshotMinutes(Integer tableSnapshotMinutes)
{
this.tableSnapshotMinutes = tableSnapshotMinutes;
return (this);
}
/*******************************************************************************
** Getter for detailRetentionDays.
*******************************************************************************/
public Integer getDetailRetentionDays()
{
return (this.detailRetentionDays);
}
/*******************************************************************************
** Setter for detailRetentionDays.
*******************************************************************************/
public void setDetailRetentionDays(Integer detailRetentionDays)
{
this.detailRetentionDays = detailRetentionDays;
}
/*******************************************************************************
** Fluent setter for detailRetentionDays.
*******************************************************************************/
public DbPerfSettings withDetailRetentionDays(Integer detailRetentionDays)
{
this.detailRetentionDays = detailRetentionDays;
return (this);
}
/*******************************************************************************
** Getter for rollupRetentionDays.
*******************************************************************************/
public Integer getRollupRetentionDays()
{
return (this.rollupRetentionDays);
}
/*******************************************************************************
** Setter for rollupRetentionDays.
*******************************************************************************/
public void setRollupRetentionDays(Integer rollupRetentionDays)
{
this.rollupRetentionDays = rollupRetentionDays;
}
/*******************************************************************************
** Fluent setter for rollupRetentionDays.
*******************************************************************************/
public DbPerfSettings withRollupRetentionDays(Integer rollupRetentionDays)
{
this.rollupRetentionDays = rollupRetentionDays;
return (this);
}
/*******************************************************************************
** Getter for topStatementsPerSnapshot.
*******************************************************************************/
public Integer getTopStatementsPerSnapshot()
{
return (this.topStatementsPerSnapshot);
}
/*******************************************************************************
** Setter for topStatementsPerSnapshot.
*******************************************************************************/
public void setTopStatementsPerSnapshot(Integer topStatementsPerSnapshot)
{
this.topStatementsPerSnapshot = topStatementsPerSnapshot;
}
/*******************************************************************************
** Fluent setter for topStatementsPerSnapshot.
*******************************************************************************/
public DbPerfSettings withTopStatementsPerSnapshot(Integer topStatementsPerSnapshot)
{
this.topStatementsPerSnapshot = topStatementsPerSnapshot;
return (this);
}
/*******************************************************************************
** Getter for activityLongRunningSeconds.
*******************************************************************************/
public Integer getActivityLongRunningSeconds()
{
return (this.activityLongRunningSeconds);
}
/*******************************************************************************
** Setter for activityLongRunningSeconds.
*******************************************************************************/
public void setActivityLongRunningSeconds(Integer activityLongRunningSeconds)
{
this.activityLongRunningSeconds = activityLongRunningSeconds;
}
/*******************************************************************************
** Fluent setter for activityLongRunningSeconds.
*******************************************************************************/
public DbPerfSettings withActivityLongRunningSeconds(Integer activityLongRunningSeconds)
{
this.activityLongRunningSeconds = activityLongRunningSeconds;
return (this);
}
/*******************************************************************************
** Getter for idleInTransactionSeconds.
*******************************************************************************/
public Integer getIdleInTransactionSeconds()
{
return (this.idleInTransactionSeconds);
}
/*******************************************************************************
** Setter for idleInTransactionSeconds.
*******************************************************************************/
public void setIdleInTransactionSeconds(Integer idleInTransactionSeconds)
{
this.idleInTransactionSeconds = idleInTransactionSeconds;
}
/*******************************************************************************
** Fluent setter for idleInTransactionSeconds.
*******************************************************************************/
public DbPerfSettings withIdleInTransactionSeconds(Integer idleInTransactionSeconds)
{
this.idleInTransactionSeconds = idleInTransactionSeconds;
return (this);
}
/*******************************************************************************
** Getter for attributionEnabled.
*******************************************************************************/
public Boolean getAttributionEnabled()
{
return (this.attributionEnabled);
}
/*******************************************************************************
** Setter for attributionEnabled.
*******************************************************************************/
public void setAttributionEnabled(Boolean attributionEnabled)
{
this.attributionEnabled = attributionEnabled;
}
/*******************************************************************************
** Fluent setter for attributionEnabled.
*******************************************************************************/
public DbPerfSettings withAttributionEnabled(Boolean attributionEnabled)
{
this.attributionEnabled = attributionEnabled;
return (this);
}
/*******************************************************************************
** Getter for attributionSampleRate.
*******************************************************************************/
public BigDecimal getAttributionSampleRate()
{
return (this.attributionSampleRate);
}
/*******************************************************************************
** Setter for attributionSampleRate.
*******************************************************************************/
public void setAttributionSampleRate(BigDecimal attributionSampleRate)
{
this.attributionSampleRate = attributionSampleRate;
}
/*******************************************************************************
** Fluent setter for attributionSampleRate.
*******************************************************************************/
public DbPerfSettings withAttributionSampleRate(BigDecimal attributionSampleRate)
{
this.attributionSampleRate = attributionSampleRate;
return (this);
}
/*******************************************************************************
** Getter for attributionFlushSeconds.
*******************************************************************************/
public Integer getAttributionFlushSeconds()
{
return (this.attributionFlushSeconds);
}
/*******************************************************************************
** Setter for attributionFlushSeconds.
*******************************************************************************/
public void setAttributionFlushSeconds(Integer attributionFlushSeconds)
{
this.attributionFlushSeconds = attributionFlushSeconds;
}
/*******************************************************************************
** Fluent setter for attributionFlushSeconds.
*******************************************************************************/
public DbPerfSettings withAttributionFlushSeconds(Integer attributionFlushSeconds)
{
this.attributionFlushSeconds = attributionFlushSeconds;
return (this);
}
/*******************************************************************************
** Getter for attributionMaxKeys.
*******************************************************************************/
public Integer getAttributionMaxKeys()
{
return (this.attributionMaxKeys);
}
/*******************************************************************************
** Setter for attributionMaxKeys.
*******************************************************************************/
public void setAttributionMaxKeys(Integer attributionMaxKeys)
{
this.attributionMaxKeys = attributionMaxKeys;
}
/*******************************************************************************
** Fluent setter for attributionMaxKeys.
*******************************************************************************/
public DbPerfSettings withAttributionMaxKeys(Integer attributionMaxKeys)
{
this.attributionMaxKeys = attributionMaxKeys;
return (this);
}
/*******************************************************************************
** Getter for nPlusOneThreshold.
*******************************************************************************/
public Integer getNPlusOneThreshold()
{
return (this.nPlusOneThreshold);
}
/*******************************************************************************
** Setter for nPlusOneThreshold.
*******************************************************************************/
public void setNPlusOneThreshold(Integer nPlusOneThreshold)
{
this.nPlusOneThreshold = nPlusOneThreshold;
}
/*******************************************************************************
** Fluent setter for nPlusOneThreshold.
*******************************************************************************/
public DbPerfSettings withNPlusOneThreshold(Integer nPlusOneThreshold)
{
this.nPlusOneThreshold = nPlusOneThreshold;
return (this);
}
/*******************************************************************************
** Getter for explainTimeoutMs.
*******************************************************************************/
public Integer getExplainTimeoutMs()
{
return (this.explainTimeoutMs);
}
/*******************************************************************************
** Setter for explainTimeoutMs.
*******************************************************************************/
public void setExplainTimeoutMs(Integer explainTimeoutMs)
{
this.explainTimeoutMs = explainTimeoutMs;
}
/*******************************************************************************
** Fluent setter for explainTimeoutMs.
*******************************************************************************/
public DbPerfSettings withExplainTimeoutMs(Integer explainTimeoutMs)
{
this.explainTimeoutMs = explainTimeoutMs;
return (this);
}
/*******************************************************************************
** Getter for alertsEnabled.
*******************************************************************************/
public Boolean getAlertsEnabled()
{
return (this.alertsEnabled);
}
/*******************************************************************************
** Setter for alertsEnabled.
*******************************************************************************/
public void setAlertsEnabled(Boolean alertsEnabled)
{
this.alertsEnabled = alertsEnabled;
}
/*******************************************************************************
** Fluent setter for alertsEnabled.
*******************************************************************************/
public DbPerfSettings withAlertsEnabled(Boolean alertsEnabled)
{
this.alertsEnabled = alertsEnabled;
return (this);
}
/*******************************************************************************
** Getter for spikeFactor.
*******************************************************************************/
public BigDecimal getSpikeFactor()
{
return (this.spikeFactor);
}
/*******************************************************************************
** Setter for spikeFactor.
*******************************************************************************/
public void setSpikeFactor(BigDecimal spikeFactor)
{
this.spikeFactor = spikeFactor;
}
/*******************************************************************************
** Fluent setter for spikeFactor.
*******************************************************************************/
public DbPerfSettings withSpikeFactor(BigDecimal spikeFactor)
{
this.spikeFactor = spikeFactor;
return (this);
}
/*******************************************************************************
** Getter for spikeMinTotalMs.
*******************************************************************************/
public Integer getSpikeMinTotalMs()
{
return (this.spikeMinTotalMs);
}
/*******************************************************************************
** Setter for spikeMinTotalMs.
*******************************************************************************/
public void setSpikeMinTotalMs(Integer spikeMinTotalMs)
{
this.spikeMinTotalMs = spikeMinTotalMs;
}
/*******************************************************************************
** Fluent setter for spikeMinTotalMs.
*******************************************************************************/
public DbPerfSettings withSpikeMinTotalMs(Integer spikeMinTotalMs)
{
this.spikeMinTotalMs = spikeMinTotalMs;
return (this);
}
/*******************************************************************************
** Getter for connectionAlertPercent.
*******************************************************************************/
public Integer getConnectionAlertPercent()
{
return (this.connectionAlertPercent);
}
/*******************************************************************************
** Setter for connectionAlertPercent.
*******************************************************************************/
public void setConnectionAlertPercent(Integer connectionAlertPercent)
{
this.connectionAlertPercent = connectionAlertPercent;
}
/*******************************************************************************
** Fluent setter for connectionAlertPercent.
*******************************************************************************/
public DbPerfSettings withConnectionAlertPercent(Integer connectionAlertPercent)
{
this.connectionAlertPercent = connectionAlertPercent;
return (this);
}
/*******************************************************************************
** Getter for blockedAlertSeconds.
*******************************************************************************/
public Integer getBlockedAlertSeconds()
{
return (this.blockedAlertSeconds);
}
/*******************************************************************************
** Setter for blockedAlertSeconds.
*******************************************************************************/
public void setBlockedAlertSeconds(Integer blockedAlertSeconds)
{
this.blockedAlertSeconds = blockedAlertSeconds;
}
/*******************************************************************************
** Fluent setter for blockedAlertSeconds.
*******************************************************************************/
public DbPerfSettings withBlockedAlertSeconds(Integer blockedAlertSeconds)
{
this.blockedAlertSeconds = blockedAlertSeconds;
return (this);
}
/*******************************************************************************
** Getter for lastSnapshotAt.
*******************************************************************************/
public Instant getLastSnapshotAt()
{
return (this.lastSnapshotAt);
}
/*******************************************************************************
** Setter for lastSnapshotAt.
*******************************************************************************/
public void setLastSnapshotAt(Instant lastSnapshotAt)
{
this.lastSnapshotAt = lastSnapshotAt;
}
/*******************************************************************************
** Fluent setter for lastSnapshotAt.
*******************************************************************************/
public DbPerfSettings withLastSnapshotAt(Instant lastSnapshotAt)
{
this.lastSnapshotAt = lastSnapshotAt;
return (this);
}
/*******************************************************************************
** Getter for lastTableSnapshotAt.
*******************************************************************************/
public Instant getLastTableSnapshotAt()
{
return (this.lastTableSnapshotAt);
}
/*******************************************************************************
** Setter for lastTableSnapshotAt.
*******************************************************************************/
public void setLastTableSnapshotAt(Instant lastTableSnapshotAt)
{
this.lastTableSnapshotAt = lastTableSnapshotAt;
}
/*******************************************************************************
** Fluent setter for lastTableSnapshotAt.
*******************************************************************************/
public DbPerfSettings withLastTableSnapshotAt(Instant lastTableSnapshotAt)
{
this.lastTableSnapshotAt = lastTableSnapshotAt;
return (this);
}
/*******************************************************************************
** Getter for statsAccess.
*******************************************************************************/
public String getStatsAccess()
{
return (this.statsAccess);
}
/*******************************************************************************
** Setter for statsAccess.
*******************************************************************************/
public void setStatsAccess(String statsAccess)
{
this.statsAccess = statsAccess;
}
/*******************************************************************************
** Fluent setter for statsAccess.
*******************************************************************************/
public DbPerfSettings withStatsAccess(String statsAccess)
{
this.statsAccess = statsAccess;
return (this);
}
/*******************************************************************************
** Getter for statsAccessMessage.
*******************************************************************************/
public String getStatsAccessMessage()
{
return (this.statsAccessMessage);
}
/*******************************************************************************
** Setter for statsAccessMessage.
*******************************************************************************/
public void setStatsAccessMessage(String statsAccessMessage)
{
this.statsAccessMessage = statsAccessMessage;
}
/*******************************************************************************
** Fluent setter for statsAccessMessage.
*******************************************************************************/
public DbPerfSettings withStatsAccessMessage(String statsAccessMessage)
{
this.statsAccessMessage = statsAccessMessage;
return (this);
}
/*******************************************************************************
** Getter for createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate.
*******************************************************************************/
public DbPerfSettings withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate.
*******************************************************************************/
public DbPerfSettings withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,829 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Queries persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfStatement extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_statement";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(maxLength = 100)
private String backendName;
@QField
private Long queryId;
@QField(maxLength = 100)
private String dbUser;
@QField
private Boolean toplevel;
@QField(maxLength = 64)
private String fingerprint;
@QField
private String queryText;
@QField(maxLength = 20)
private String operation;
@QField(maxLength = 200)
private String mainTable;
@QField
private Instant firstSeenAt;
@QField
private Instant lastSeenAt;
@QField
private Long lastCalls;
@QField
private BigDecimal lastTotalExecMs;
@QField
private Long lastRows;
@QField
private Long lastSharedBlksHit;
@QField
private Long lastSharedBlksRead;
@QField
private Long lastTempBlksWritten;
@QField
private BigDecimal lastMaxExecMs;
@QField
private Instant lastStatsSince;
@QField
private Long calls24h;
@QField
private BigDecimal totalMs24h;
@QField
private BigDecimal meanMs24h;
@QField
private BigDecimal shareOfDbTime24h;
@QField
private Instant createDate;
@QField
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfStatement()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfStatement(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfStatement withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for backendName.
*******************************************************************************/
public String getBackendName()
{
return (this.backendName);
}
/*******************************************************************************
** Setter for backendName.
*******************************************************************************/
public void setBackendName(String backendName)
{
this.backendName = backendName;
}
/*******************************************************************************
** Fluent setter for backendName.
*******************************************************************************/
public DbPerfStatement withBackendName(String backendName)
{
this.backendName = backendName;
return (this);
}
/*******************************************************************************
** Getter for queryId.
*******************************************************************************/
public Long getQueryId()
{
return (this.queryId);
}
/*******************************************************************************
** Setter for queryId.
*******************************************************************************/
public void setQueryId(Long queryId)
{
this.queryId = queryId;
}
/*******************************************************************************
** Fluent setter for queryId.
*******************************************************************************/
public DbPerfStatement withQueryId(Long queryId)
{
this.queryId = queryId;
return (this);
}
/*******************************************************************************
** Getter for dbUser.
*******************************************************************************/
public String getDbUser()
{
return (this.dbUser);
}
/*******************************************************************************
** Setter for dbUser.
*******************************************************************************/
public void setDbUser(String dbUser)
{
this.dbUser = dbUser;
}
/*******************************************************************************
** Fluent setter for dbUser.
*******************************************************************************/
public DbPerfStatement withDbUser(String dbUser)
{
this.dbUser = dbUser;
return (this);
}
/*******************************************************************************
** Getter for toplevel.
*******************************************************************************/
public Boolean getToplevel()
{
return (this.toplevel);
}
/*******************************************************************************
** Setter for toplevel.
*******************************************************************************/
public void setToplevel(Boolean toplevel)
{
this.toplevel = toplevel;
}
/*******************************************************************************
** Fluent setter for toplevel.
*******************************************************************************/
public DbPerfStatement withToplevel(Boolean toplevel)
{
this.toplevel = toplevel;
return (this);
}
/*******************************************************************************
** Getter for fingerprint.
*******************************************************************************/
public String getFingerprint()
{
return (this.fingerprint);
}
/*******************************************************************************
** Setter for fingerprint.
*******************************************************************************/
public void setFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
}
/*******************************************************************************
** Fluent setter for fingerprint.
*******************************************************************************/
public DbPerfStatement withFingerprint(String fingerprint)
{
this.fingerprint = fingerprint;
return (this);
}
/*******************************************************************************
** Getter for queryText.
*******************************************************************************/
public String getQueryText()
{
return (this.queryText);
}
/*******************************************************************************
** Setter for queryText.
*******************************************************************************/
public void setQueryText(String queryText)
{
this.queryText = queryText;
}
/*******************************************************************************
** Fluent setter for queryText.
*******************************************************************************/
public DbPerfStatement withQueryText(String queryText)
{
this.queryText = queryText;
return (this);
}
/*******************************************************************************
** Getter for operation.
*******************************************************************************/
public String getOperation()
{
return (this.operation);
}
/*******************************************************************************
** Setter for operation.
*******************************************************************************/
public void setOperation(String operation)
{
this.operation = operation;
}
/*******************************************************************************
** Fluent setter for operation.
*******************************************************************************/
public DbPerfStatement withOperation(String operation)
{
this.operation = operation;
return (this);
}
/*******************************************************************************
** Getter for mainTable.
*******************************************************************************/
public String getMainTable()
{
return (this.mainTable);
}
/*******************************************************************************
** Setter for mainTable.
*******************************************************************************/
public void setMainTable(String mainTable)
{
this.mainTable = mainTable;
}
/*******************************************************************************
** Fluent setter for mainTable.
*******************************************************************************/
public DbPerfStatement withMainTable(String mainTable)
{
this.mainTable = mainTable;
return (this);
}
/*******************************************************************************
** Getter for firstSeenAt.
*******************************************************************************/
public Instant getFirstSeenAt()
{
return (this.firstSeenAt);
}
/*******************************************************************************
** Setter for firstSeenAt.
*******************************************************************************/
public void setFirstSeenAt(Instant firstSeenAt)
{
this.firstSeenAt = firstSeenAt;
}
/*******************************************************************************
** Fluent setter for firstSeenAt.
*******************************************************************************/
public DbPerfStatement withFirstSeenAt(Instant firstSeenAt)
{
this.firstSeenAt = firstSeenAt;
return (this);
}
/*******************************************************************************
** Getter for lastSeenAt.
*******************************************************************************/
public Instant getLastSeenAt()
{
return (this.lastSeenAt);
}
/*******************************************************************************
** Setter for lastSeenAt.
*******************************************************************************/
public void setLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
}
/*******************************************************************************
** Fluent setter for lastSeenAt.
*******************************************************************************/
public DbPerfStatement withLastSeenAt(Instant lastSeenAt)
{
this.lastSeenAt = lastSeenAt;
return (this);
}
/*******************************************************************************
** Getter for lastCalls.
*******************************************************************************/
public Long getLastCalls()
{
return (this.lastCalls);
}
/*******************************************************************************
** Setter for lastCalls.
*******************************************************************************/
public void setLastCalls(Long lastCalls)
{
this.lastCalls = lastCalls;
}
/*******************************************************************************
** Fluent setter for lastCalls.
*******************************************************************************/
public DbPerfStatement withLastCalls(Long lastCalls)
{
this.lastCalls = lastCalls;
return (this);
}
/*******************************************************************************
** Getter for lastTotalExecMs.
*******************************************************************************/
public BigDecimal getLastTotalExecMs()
{
return (this.lastTotalExecMs);
}
/*******************************************************************************
** Setter for lastTotalExecMs.
*******************************************************************************/
public void setLastTotalExecMs(BigDecimal lastTotalExecMs)
{
this.lastTotalExecMs = lastTotalExecMs;
}
/*******************************************************************************
** Fluent setter for lastTotalExecMs.
*******************************************************************************/
public DbPerfStatement withLastTotalExecMs(BigDecimal lastTotalExecMs)
{
this.lastTotalExecMs = lastTotalExecMs;
return (this);
}
/*******************************************************************************
** Getter for lastRows.
*******************************************************************************/
public Long getLastRows()
{
return (this.lastRows);
}
/*******************************************************************************
** Setter for lastRows.
*******************************************************************************/
public void setLastRows(Long lastRows)
{
this.lastRows = lastRows;
}
/*******************************************************************************
** Fluent setter for lastRows.
*******************************************************************************/
public DbPerfStatement withLastRows(Long lastRows)
{
this.lastRows = lastRows;
return (this);
}
/*******************************************************************************
** Getter for lastSharedBlksHit.
*******************************************************************************/
public Long getLastSharedBlksHit()
{
return (this.lastSharedBlksHit);
}
/*******************************************************************************
** Setter for lastSharedBlksHit.
*******************************************************************************/
public void setLastSharedBlksHit(Long lastSharedBlksHit)
{
this.lastSharedBlksHit = lastSharedBlksHit;
}
/*******************************************************************************
** Fluent setter for lastSharedBlksHit.
*******************************************************************************/
public DbPerfStatement withLastSharedBlksHit(Long lastSharedBlksHit)
{
this.lastSharedBlksHit = lastSharedBlksHit;
return (this);
}
/*******************************************************************************
** Getter for lastSharedBlksRead.
*******************************************************************************/
public Long getLastSharedBlksRead()
{
return (this.lastSharedBlksRead);
}
/*******************************************************************************
** Setter for lastSharedBlksRead.
*******************************************************************************/
public void setLastSharedBlksRead(Long lastSharedBlksRead)
{
this.lastSharedBlksRead = lastSharedBlksRead;
}
/*******************************************************************************
** Fluent setter for lastSharedBlksRead.
*******************************************************************************/
public DbPerfStatement withLastSharedBlksRead(Long lastSharedBlksRead)
{
this.lastSharedBlksRead = lastSharedBlksRead;
return (this);
}
/*******************************************************************************
** Getter for lastTempBlksWritten.
*******************************************************************************/
public Long getLastTempBlksWritten()
{
return (this.lastTempBlksWritten);
}
/*******************************************************************************
** Setter for lastTempBlksWritten.
*******************************************************************************/
public void setLastTempBlksWritten(Long lastTempBlksWritten)
{
this.lastTempBlksWritten = lastTempBlksWritten;
}
/*******************************************************************************
** Fluent setter for lastTempBlksWritten.
*******************************************************************************/
public DbPerfStatement withLastTempBlksWritten(Long lastTempBlksWritten)
{
this.lastTempBlksWritten = lastTempBlksWritten;
return (this);
}
/*******************************************************************************
** Getter for lastMaxExecMs.
*******************************************************************************/
public BigDecimal getLastMaxExecMs()
{
return (this.lastMaxExecMs);
}
/*******************************************************************************
** Setter for lastMaxExecMs.
*******************************************************************************/
public void setLastMaxExecMs(BigDecimal lastMaxExecMs)
{
this.lastMaxExecMs = lastMaxExecMs;
}
/*******************************************************************************
** Fluent setter for lastMaxExecMs.
*******************************************************************************/
public DbPerfStatement withLastMaxExecMs(BigDecimal lastMaxExecMs)
{
this.lastMaxExecMs = lastMaxExecMs;
return (this);
}
/*******************************************************************************
** Getter for lastStatsSince.
*******************************************************************************/
public Instant getLastStatsSince()
{
return (this.lastStatsSince);
}
/*******************************************************************************
** Setter for lastStatsSince.
*******************************************************************************/
public void setLastStatsSince(Instant lastStatsSince)
{
this.lastStatsSince = lastStatsSince;
}
/*******************************************************************************
** Fluent setter for lastStatsSince.
*******************************************************************************/
public DbPerfStatement withLastStatsSince(Instant lastStatsSince)
{
this.lastStatsSince = lastStatsSince;
return (this);
}
/*******************************************************************************
** Getter for calls24h.
*******************************************************************************/
public Long getCalls24h()
{
return (this.calls24h);
}
/*******************************************************************************
** Setter for calls24h.
*******************************************************************************/
public void setCalls24h(Long calls24h)
{
this.calls24h = calls24h;
}
/*******************************************************************************
** Fluent setter for calls24h.
*******************************************************************************/
public DbPerfStatement withCalls24h(Long calls24h)
{
this.calls24h = calls24h;
return (this);
}
/*******************************************************************************
** Getter for totalMs24h.
*******************************************************************************/
public BigDecimal getTotalMs24h()
{
return (this.totalMs24h);
}
/*******************************************************************************
** Setter for totalMs24h.
*******************************************************************************/
public void setTotalMs24h(BigDecimal totalMs24h)
{
this.totalMs24h = totalMs24h;
}
/*******************************************************************************
** Fluent setter for totalMs24h.
*******************************************************************************/
public DbPerfStatement withTotalMs24h(BigDecimal totalMs24h)
{
this.totalMs24h = totalMs24h;
return (this);
}
/*******************************************************************************
** Getter for meanMs24h.
*******************************************************************************/
public BigDecimal getMeanMs24h()
{
return (this.meanMs24h);
}
/*******************************************************************************
** Setter for meanMs24h.
*******************************************************************************/
public void setMeanMs24h(BigDecimal meanMs24h)
{
this.meanMs24h = meanMs24h;
}
/*******************************************************************************
** Fluent setter for meanMs24h.
*******************************************************************************/
public DbPerfStatement withMeanMs24h(BigDecimal meanMs24h)
{
this.meanMs24h = meanMs24h;
return (this);
}
/*******************************************************************************
** Getter for shareOfDbTime24h.
*******************************************************************************/
public BigDecimal getShareOfDbTime24h()
{
return (this.shareOfDbTime24h);
}
/*******************************************************************************
** Setter for shareOfDbTime24h.
*******************************************************************************/
public void setShareOfDbTime24h(BigDecimal shareOfDbTime24h)
{
this.shareOfDbTime24h = shareOfDbTime24h;
}
/*******************************************************************************
** Fluent setter for shareOfDbTime24h.
*******************************************************************************/
public DbPerfStatement withShareOfDbTime24h(BigDecimal shareOfDbTime24h)
{
this.shareOfDbTime24h = shareOfDbTime24h;
return (this);
}
/*******************************************************************************
** Getter for createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate.
*******************************************************************************/
public DbPerfStatement withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate.
*******************************************************************************/
public DbPerfStatement withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,425 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.time.LocalDate;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Table Daily persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfTableDaily extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_table_daily";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer tableStatId;
@QField
private LocalDate day;
@QField
private Long seqScan;
@QField
private Long idxScan;
@QField
private Long seqTupRead;
@QField
private Long nTupIns;
@QField
private Long nTupUpd;
@QField
private Long nTupDel;
@QField
private Long nLiveTup;
@QField
private Long nDeadTup;
@QField
private Long totalBytes;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfTableDaily()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfTableDaily(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfTableDaily withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for tableStatId.
*******************************************************************************/
public Integer getTableStatId()
{
return (this.tableStatId);
}
/*******************************************************************************
** Setter for tableStatId.
*******************************************************************************/
public void setTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
}
/*******************************************************************************
** Fluent setter for tableStatId.
*******************************************************************************/
public DbPerfTableDaily withTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
return (this);
}
/*******************************************************************************
** Getter for day.
*******************************************************************************/
public LocalDate getDay()
{
return (this.day);
}
/*******************************************************************************
** Setter for day.
*******************************************************************************/
public void setDay(LocalDate day)
{
this.day = day;
}
/*******************************************************************************
** Fluent setter for day.
*******************************************************************************/
public DbPerfTableDaily withDay(LocalDate day)
{
this.day = day;
return (this);
}
/*******************************************************************************
** Getter for seqScan.
*******************************************************************************/
public Long getSeqScan()
{
return (this.seqScan);
}
/*******************************************************************************
** Setter for seqScan.
*******************************************************************************/
public void setSeqScan(Long seqScan)
{
this.seqScan = seqScan;
}
/*******************************************************************************
** Fluent setter for seqScan.
*******************************************************************************/
public DbPerfTableDaily withSeqScan(Long seqScan)
{
this.seqScan = seqScan;
return (this);
}
/*******************************************************************************
** Getter for idxScan.
*******************************************************************************/
public Long getIdxScan()
{
return (this.idxScan);
}
/*******************************************************************************
** Setter for idxScan.
*******************************************************************************/
public void setIdxScan(Long idxScan)
{
this.idxScan = idxScan;
}
/*******************************************************************************
** Fluent setter for idxScan.
*******************************************************************************/
public DbPerfTableDaily withIdxScan(Long idxScan)
{
this.idxScan = idxScan;
return (this);
}
/*******************************************************************************
** Getter for seqTupRead.
*******************************************************************************/
public Long getSeqTupRead()
{
return (this.seqTupRead);
}
/*******************************************************************************
** Setter for seqTupRead.
*******************************************************************************/
public void setSeqTupRead(Long seqTupRead)
{
this.seqTupRead = seqTupRead;
}
/*******************************************************************************
** Fluent setter for seqTupRead.
*******************************************************************************/
public DbPerfTableDaily withSeqTupRead(Long seqTupRead)
{
this.seqTupRead = seqTupRead;
return (this);
}
/*******************************************************************************
** Getter for nTupIns.
*******************************************************************************/
public Long getNTupIns()
{
return (this.nTupIns);
}
/*******************************************************************************
** Setter for nTupIns.
*******************************************************************************/
public void setNTupIns(Long nTupIns)
{
this.nTupIns = nTupIns;
}
/*******************************************************************************
** Fluent setter for nTupIns.
*******************************************************************************/
public DbPerfTableDaily withNTupIns(Long nTupIns)
{
this.nTupIns = nTupIns;
return (this);
}
/*******************************************************************************
** Getter for nTupUpd.
*******************************************************************************/
public Long getNTupUpd()
{
return (this.nTupUpd);
}
/*******************************************************************************
** Setter for nTupUpd.
*******************************************************************************/
public void setNTupUpd(Long nTupUpd)
{
this.nTupUpd = nTupUpd;
}
/*******************************************************************************
** Fluent setter for nTupUpd.
*******************************************************************************/
public DbPerfTableDaily withNTupUpd(Long nTupUpd)
{
this.nTupUpd = nTupUpd;
return (this);
}
/*******************************************************************************
** Getter for nTupDel.
*******************************************************************************/
public Long getNTupDel()
{
return (this.nTupDel);
}
/*******************************************************************************
** Setter for nTupDel.
*******************************************************************************/
public void setNTupDel(Long nTupDel)
{
this.nTupDel = nTupDel;
}
/*******************************************************************************
** Fluent setter for nTupDel.
*******************************************************************************/
public DbPerfTableDaily withNTupDel(Long nTupDel)
{
this.nTupDel = nTupDel;
return (this);
}
/*******************************************************************************
** Getter for nLiveTup.
*******************************************************************************/
public Long getNLiveTup()
{
return (this.nLiveTup);
}
/*******************************************************************************
** Setter for nLiveTup.
*******************************************************************************/
public void setNLiveTup(Long nLiveTup)
{
this.nLiveTup = nLiveTup;
}
/*******************************************************************************
** Fluent setter for nLiveTup.
*******************************************************************************/
public DbPerfTableDaily withNLiveTup(Long nLiveTup)
{
this.nLiveTup = nLiveTup;
return (this);
}
/*******************************************************************************
** Getter for nDeadTup.
*******************************************************************************/
public Long getNDeadTup()
{
return (this.nDeadTup);
}
/*******************************************************************************
** Setter for nDeadTup.
*******************************************************************************/
public void setNDeadTup(Long nDeadTup)
{
this.nDeadTup = nDeadTup;
}
/*******************************************************************************
** Fluent setter for nDeadTup.
*******************************************************************************/
public DbPerfTableDaily withNDeadTup(Long nDeadTup)
{
this.nDeadTup = nDeadTup;
return (this);
}
/*******************************************************************************
** Getter for totalBytes.
*******************************************************************************/
public Long getTotalBytes()
{
return (this.totalBytes);
}
/*******************************************************************************
** Setter for totalBytes.
*******************************************************************************/
public void setTotalBytes(Long totalBytes)
{
this.totalBytes = totalBytes;
}
/*******************************************************************************
** Fluent setter for totalBytes.
*******************************************************************************/
public DbPerfTableDaily withTotalBytes(Long totalBytes)
{
this.totalBytes = totalBytes;
return (this);
}
}
@@ -0,0 +1,673 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.model;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Table Snapshots persisted through QQQ table actions.
*******************************************************************************/
public class DbPerfTableSnapshot extends QRecordEntity
{
public static final String TABLE_NAME = "db_perf_table_snapshot";
@QField(isPrimaryKey = true, isEditable = false)
private Integer id;
@QField
private Integer snapshotId;
@QField
private Integer tableStatId;
@QField
private Instant capturedAt;
@QField
private Long seqScan;
@QField
private Long seqTupRead;
@QField
private Long idxScan;
@QField
private Long idxTupFetch;
@QField
private Long nTupIns;
@QField
private Long nTupUpd;
@QField
private Long nTupDel;
@QField
private Long nTupHotUpd;
@QField
private Long nLiveTup;
@QField
private Long nDeadTup;
@QField
private Long totalBytes;
@QField
private Long tableBytes;
@QField
private Long indexBytes;
@QField
private Long autovacuumCount;
@QField
private Long autoanalyzeCount;
@QField
private Long freezeAge;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public DbPerfTableSnapshot()
{
}
/*******************************************************************************
** Populate from a QQQ record.
*******************************************************************************/
public DbPerfTableSnapshot(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id.
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id.
*******************************************************************************/
public DbPerfTableSnapshot withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for snapshotId.
*******************************************************************************/
public Integer getSnapshotId()
{
return (this.snapshotId);
}
/*******************************************************************************
** Setter for snapshotId.
*******************************************************************************/
public void setSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
}
/*******************************************************************************
** Fluent setter for snapshotId.
*******************************************************************************/
public DbPerfTableSnapshot withSnapshotId(Integer snapshotId)
{
this.snapshotId = snapshotId;
return (this);
}
/*******************************************************************************
** Getter for tableStatId.
*******************************************************************************/
public Integer getTableStatId()
{
return (this.tableStatId);
}
/*******************************************************************************
** Setter for tableStatId.
*******************************************************************************/
public void setTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
}
/*******************************************************************************
** Fluent setter for tableStatId.
*******************************************************************************/
public DbPerfTableSnapshot withTableStatId(Integer tableStatId)
{
this.tableStatId = tableStatId;
return (this);
}
/*******************************************************************************
** Getter for capturedAt.
*******************************************************************************/
public Instant getCapturedAt()
{
return (this.capturedAt);
}
/*******************************************************************************
** Setter for capturedAt.
*******************************************************************************/
public void setCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
}
/*******************************************************************************
** Fluent setter for capturedAt.
*******************************************************************************/
public DbPerfTableSnapshot withCapturedAt(Instant capturedAt)
{
this.capturedAt = capturedAt;
return (this);
}
/*******************************************************************************
** Getter for seqScan.
*******************************************************************************/
public Long getSeqScan()
{
return (this.seqScan);
}
/*******************************************************************************
** Setter for seqScan.
*******************************************************************************/
public void setSeqScan(Long seqScan)
{
this.seqScan = seqScan;
}
/*******************************************************************************
** Fluent setter for seqScan.
*******************************************************************************/
public DbPerfTableSnapshot withSeqScan(Long seqScan)
{
this.seqScan = seqScan;
return (this);
}
/*******************************************************************************
** Getter for seqTupRead.
*******************************************************************************/
public Long getSeqTupRead()
{
return (this.seqTupRead);
}
/*******************************************************************************
** Setter for seqTupRead.
*******************************************************************************/
public void setSeqTupRead(Long seqTupRead)
{
this.seqTupRead = seqTupRead;
}
/*******************************************************************************
** Fluent setter for seqTupRead.
*******************************************************************************/
public DbPerfTableSnapshot withSeqTupRead(Long seqTupRead)
{
this.seqTupRead = seqTupRead;
return (this);
}
/*******************************************************************************
** Getter for idxScan.
*******************************************************************************/
public Long getIdxScan()
{
return (this.idxScan);
}
/*******************************************************************************
** Setter for idxScan.
*******************************************************************************/
public void setIdxScan(Long idxScan)
{
this.idxScan = idxScan;
}
/*******************************************************************************
** Fluent setter for idxScan.
*******************************************************************************/
public DbPerfTableSnapshot withIdxScan(Long idxScan)
{
this.idxScan = idxScan;
return (this);
}
/*******************************************************************************
** Getter for idxTupFetch.
*******************************************************************************/
public Long getIdxTupFetch()
{
return (this.idxTupFetch);
}
/*******************************************************************************
** Setter for idxTupFetch.
*******************************************************************************/
public void setIdxTupFetch(Long idxTupFetch)
{
this.idxTupFetch = idxTupFetch;
}
/*******************************************************************************
** Fluent setter for idxTupFetch.
*******************************************************************************/
public DbPerfTableSnapshot withIdxTupFetch(Long idxTupFetch)
{
this.idxTupFetch = idxTupFetch;
return (this);
}
/*******************************************************************************
** Getter for nTupIns.
*******************************************************************************/
public Long getNTupIns()
{
return (this.nTupIns);
}
/*******************************************************************************
** Setter for nTupIns.
*******************************************************************************/
public void setNTupIns(Long nTupIns)
{
this.nTupIns = nTupIns;
}
/*******************************************************************************
** Fluent setter for nTupIns.
*******************************************************************************/
public DbPerfTableSnapshot withNTupIns(Long nTupIns)
{
this.nTupIns = nTupIns;
return (this);
}
/*******************************************************************************
** Getter for nTupUpd.
*******************************************************************************/
public Long getNTupUpd()
{
return (this.nTupUpd);
}
/*******************************************************************************
** Setter for nTupUpd.
*******************************************************************************/
public void setNTupUpd(Long nTupUpd)
{
this.nTupUpd = nTupUpd;
}
/*******************************************************************************
** Fluent setter for nTupUpd.
*******************************************************************************/
public DbPerfTableSnapshot withNTupUpd(Long nTupUpd)
{
this.nTupUpd = nTupUpd;
return (this);
}
/*******************************************************************************
** Getter for nTupDel.
*******************************************************************************/
public Long getNTupDel()
{
return (this.nTupDel);
}
/*******************************************************************************
** Setter for nTupDel.
*******************************************************************************/
public void setNTupDel(Long nTupDel)
{
this.nTupDel = nTupDel;
}
/*******************************************************************************
** Fluent setter for nTupDel.
*******************************************************************************/
public DbPerfTableSnapshot withNTupDel(Long nTupDel)
{
this.nTupDel = nTupDel;
return (this);
}
/*******************************************************************************
** Getter for nTupHotUpd.
*******************************************************************************/
public Long getNTupHotUpd()
{
return (this.nTupHotUpd);
}
/*******************************************************************************
** Setter for nTupHotUpd.
*******************************************************************************/
public void setNTupHotUpd(Long nTupHotUpd)
{
this.nTupHotUpd = nTupHotUpd;
}
/*******************************************************************************
** Fluent setter for nTupHotUpd.
*******************************************************************************/
public DbPerfTableSnapshot withNTupHotUpd(Long nTupHotUpd)
{
this.nTupHotUpd = nTupHotUpd;
return (this);
}
/*******************************************************************************
** Getter for nLiveTup.
*******************************************************************************/
public Long getNLiveTup()
{
return (this.nLiveTup);
}
/*******************************************************************************
** Setter for nLiveTup.
*******************************************************************************/
public void setNLiveTup(Long nLiveTup)
{
this.nLiveTup = nLiveTup;
}
/*******************************************************************************
** Fluent setter for nLiveTup.
*******************************************************************************/
public DbPerfTableSnapshot withNLiveTup(Long nLiveTup)
{
this.nLiveTup = nLiveTup;
return (this);
}
/*******************************************************************************
** Getter for nDeadTup.
*******************************************************************************/
public Long getNDeadTup()
{
return (this.nDeadTup);
}
/*******************************************************************************
** Setter for nDeadTup.
*******************************************************************************/
public void setNDeadTup(Long nDeadTup)
{
this.nDeadTup = nDeadTup;
}
/*******************************************************************************
** Fluent setter for nDeadTup.
*******************************************************************************/
public DbPerfTableSnapshot withNDeadTup(Long nDeadTup)
{
this.nDeadTup = nDeadTup;
return (this);
}
/*******************************************************************************
** Getter for totalBytes.
*******************************************************************************/
public Long getTotalBytes()
{
return (this.totalBytes);
}
/*******************************************************************************
** Setter for totalBytes.
*******************************************************************************/
public void setTotalBytes(Long totalBytes)
{
this.totalBytes = totalBytes;
}
/*******************************************************************************
** Fluent setter for totalBytes.
*******************************************************************************/
public DbPerfTableSnapshot withTotalBytes(Long totalBytes)
{
this.totalBytes = totalBytes;
return (this);
}
/*******************************************************************************
** Getter for tableBytes.
*******************************************************************************/
public Long getTableBytes()
{
return (this.tableBytes);
}
/*******************************************************************************
** Setter for tableBytes.
*******************************************************************************/
public void setTableBytes(Long tableBytes)
{
this.tableBytes = tableBytes;
}
/*******************************************************************************
** Fluent setter for tableBytes.
*******************************************************************************/
public DbPerfTableSnapshot withTableBytes(Long tableBytes)
{
this.tableBytes = tableBytes;
return (this);
}
/*******************************************************************************
** Getter for indexBytes.
*******************************************************************************/
public Long getIndexBytes()
{
return (this.indexBytes);
}
/*******************************************************************************
** Setter for indexBytes.
*******************************************************************************/
public void setIndexBytes(Long indexBytes)
{
this.indexBytes = indexBytes;
}
/*******************************************************************************
** Fluent setter for indexBytes.
*******************************************************************************/
public DbPerfTableSnapshot withIndexBytes(Long indexBytes)
{
this.indexBytes = indexBytes;
return (this);
}
/*******************************************************************************
** Getter for autovacuumCount.
*******************************************************************************/
public Long getAutovacuumCount()
{
return (this.autovacuumCount);
}
/*******************************************************************************
** Setter for autovacuumCount.
*******************************************************************************/
public void setAutovacuumCount(Long autovacuumCount)
{
this.autovacuumCount = autovacuumCount;
}
/*******************************************************************************
** Fluent setter for autovacuumCount.
*******************************************************************************/
public DbPerfTableSnapshot withAutovacuumCount(Long autovacuumCount)
{
this.autovacuumCount = autovacuumCount;
return (this);
}
/*******************************************************************************
** Getter for autoanalyzeCount.
*******************************************************************************/
public Long getAutoanalyzeCount()
{
return (this.autoanalyzeCount);
}
/*******************************************************************************
** Setter for autoanalyzeCount.
*******************************************************************************/
public void setAutoanalyzeCount(Long autoanalyzeCount)
{
this.autoanalyzeCount = autoanalyzeCount;
}
/*******************************************************************************
** Fluent setter for autoanalyzeCount.
*******************************************************************************/
public DbPerfTableSnapshot withAutoanalyzeCount(Long autoanalyzeCount)
{
this.autoanalyzeCount = autoanalyzeCount;
return (this);
}
/*******************************************************************************
** Getter for freezeAge.
*******************************************************************************/
public Long getFreezeAge()
{
return (this.freezeAge);
}
/*******************************************************************************
** Setter for freezeAge.
*******************************************************************************/
public void setFreezeAge(Long freezeAge)
{
this.freezeAge = freezeAge;
}
/*******************************************************************************
** Fluent setter for freezeAge.
*******************************************************************************/
public DbPerfTableSnapshot withFreezeAge(Long freezeAge)
{
this.freezeAge = freezeAge;
return (this);
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,357 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.normalize;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HexFormat;
import java.util.List;
import java.util.Locale;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/*******************************************************************************
** Literal-free SQL shapes shared by PostgreSQL snapshots and QQQ attribution.
*******************************************************************************/
public final class SqlNormalizer
{
private static final Pattern NUMBER = Pattern.compile("(?:0[xX][0-9a-fA-F]+|(?:[0-9]+(?:\\.[0-9]*)?|\\.[0-9]+)(?:[eE][+-]?[0-9]+)?)");
private static final Pattern OPERATOR = Pattern.compile("(?:->>|#>>|!~\\*|\\|\\|/|::|>=|<=|<>|!=|\\|\\||->|#>|!~|~\\*|@>|<@|&&|\\|/|:=|=>|\\?\\||\\?&)");
private static final Pattern DOLLAR = Pattern.compile("\\$(?:[A-Za-z_][A-Za-z_0-9]*)?\\$");
private static final Pattern TABLE = Pattern.compile("(?:^| )(?:(?:from|into|update) )((?:\"(?:[^\"]|\"\")*\"|[a-z_][a-z_0-9$]*)(?: \\. (?:\"(?:[^\"]|\"\")*\"|[a-z_][a-z_0-9$]*))?)");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private SqlNormalizer()
{
}
/*******************************************************************************
** Lex SQL before folding whitespace, so quoted identifiers stay intact.
** Unterminated literals and comments consume their remainder and never leak it.
*******************************************************************************/
public static String normalize(String sql)
{
if(sql == null || sql.isBlank())
{
return ("");
}
StringBuilder out = new StringBuilder();
int i = 0;
while(i < sql.length())
{
char c = sql.charAt(i);
if(Character.isWhitespace(c))
{
i++;
continue;
}
if(sql.startsWith("--", i))
{
int end = sql.indexOf('\n', i);
i = end < 0 ? sql.length() : end + 1;
continue;
}
if(sql.startsWith("/*", i))
{
int depth = 1;
i += 2;
while(i < sql.length() && depth > 0)
{
if(sql.startsWith("/*", i))
{
depth++;
i += 2;
}
else if(sql.startsWith("*/", i))
{
depth--;
i += 2;
}
else
{
i++;
}
}
continue;
}
boolean escape = (c == 'e' || c == 'E') && i + 1 < sql.length() && sql.charAt(i + 1) == '\'';
if(c == '\'' || escape)
{
i = quotedEnd(sql, i + (escape ? 1 : 0), '\'', escape);
out.append("? ");
continue;
}
if(c == '"')
{
int end = quotedEnd(sql, i, '"', false);
out.append(sql, i, end).append(' ');
i = end;
continue;
}
if(c == '$')
{
Matcher dollar = DOLLAR.matcher(sql).region(i, sql.length());
if(dollar.lookingAt())
{
int end = sql.indexOf(dollar.group(), dollar.end());
i = end < 0 ? sql.length() : end + dollar.group().length();
out.append("? ");
continue;
}
int end = i + 1;
while(end < sql.length() && Character.isDigit(sql.charAt(end)))
{
end++;
}
if(end > i + 1)
{
out.append("? ");
i = end;
continue;
}
}
if(Character.isDigit(c) || c == '.' && i + 1 < sql.length() && Character.isDigit(sql.charAt(i + 1)))
{
Matcher number = NUMBER.matcher(sql).region(i, sql.length());
if(number.lookingAt())
{
i = number.end();
out.append("? ");
continue;
}
}
if(Character.isLetter(c) || c == '_')
{
int end = i + 1;
while(end < sql.length() && (Character.isLetterOrDigit(sql.charAt(end)) || sql.charAt(end) == '_' || sql.charAt(end) == '$'))
{
end++;
}
out.append(sql.substring(i, end).toLowerCase(Locale.ROOT)).append(' ');
i = end;
continue;
}
Matcher operator = OPERATOR.matcher(sql).region(i, sql.length());
if(operator.lookingAt())
{
out.append(operator.group()).append(' ');
i = operator.end();
continue;
}
out.append(c).append(' ');
i++;
}
String normalized = collapseOutsideIdentifiers(out.toString().trim());
return (normalized.substring(0, Math.min(10000, normalized.length())));
}
/*******************************************************************************
** List folding applies only outside quoted identifiers, whose contents may
** themselves resemble SQL. Escaped identifier quotes remain untouched.
*******************************************************************************/
private static String collapseOutsideIdentifiers(String sql)
{
StringBuilder result = new StringBuilder();
int start = 0;
while(start < sql.length())
{
int quote = sql.indexOf('"', start);
if(quote < 0)
{
result.append(collapseLists(sql.substring(start)));
break;
}
result.append(collapseLists(sql.substring(start, quote)));
int end = quotedEnd(sql, quote, '"', false);
result.append(sql, quote, end);
start = end;
}
return (result.toString());
}
/*******************************************************************************
** Collapse placeholder repetitions, preserving VALUES column arity.
*******************************************************************************/
private static String collapseLists(String normalized)
{
///////////////////////////////////////////////////////////////////////////
// a linear scan over the space-separated tokens: regular expressions //
// with nested repetition overflow the stack on very long IN and VALUES //
// lists (seen with 60 KB statements in production) //
///////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////
// segments between quoted identifiers keep their edge spaces //
////////////////////////////////////////////////////////////////////////
String trimmed = normalized.strip();
if(trimmed.isEmpty())
{
return (normalized);
}
String leading = normalized.substring(0, normalized.indexOf(trimmed.charAt(0)));
String trailing = normalized.substring(leading.length() + trimmed.length());
String[] tokens = trimmed.split(" ");
List<String> out = new ArrayList<>(tokens.length);
int i = 0;
while(i < tokens.length)
{
String token = tokens[i];
if("in".equals(token))
{
int end = placeholderGroupEnd(tokens, i + 1);
if(end > 0)
{
out.addAll(List.of("in", "(", "?", ")"));
i = end;
continue;
}
}
if("values".equals(token))
{
int first = placeholderGroupEnd(tokens, i + 1);
if(first > 0)
{
int end = first;
while(end + 1 < tokens.length && ",".equals(tokens[end]))
{
int next = placeholderGroupEnd(tokens, end + 1);
if(next < 0)
{
break;
}
end = next;
}
out.add("values");
out.addAll(Arrays.asList(tokens).subList(i + 1, first));
i = end;
continue;
}
}
out.add(token);
i++;
}
return (leading + String.join(" ", out) + trailing);
}
/*******************************************************************************
** If tokens[start] opens "( ? , ? ... )" made only of placeholders, the index
** just after its ")"; otherwise -1.
*******************************************************************************/
private static int placeholderGroupEnd(String[] tokens, int start)
{
if(start + 2 >= tokens.length || !"(".equals(tokens[start]) || !"?".equals(tokens[start + 1]))
{
return (-1);
}
int i = start + 2;
while(i + 1 < tokens.length && ",".equals(tokens[i]) && "?".equals(tokens[i + 1]))
{
i += 2;
}
return (i < tokens.length && ")".equals(tokens[i]) ? i + 1 : -1);
}
/*******************************************************************************
** Consume a quote with SQL doubled-quote and optional escape handling.
*******************************************************************************/
private static int quotedEnd(String sql, int start, char quote, boolean escapes)
{
int i = start + 1;
while(i < sql.length())
{
if(escapes && sql.charAt(i) == '\\')
{
i = Math.min(i + 2, sql.length());
}
else if(sql.charAt(i++) == quote)
{
if(i < sql.length() && sql.charAt(i) == quote)
{
i++;
}
else
{
return (i);
}
}
}
return (i);
}
/*******************************************************************************
** Stable hexadecimal SHA-256; callers supply the normalized shape.
*******************************************************************************/
public static String fingerprint(String normalized)
{
try
{
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(normalized.getBytes(StandardCharsets.UTF_8))));
}
catch(NoSuchAlgorithmException e)
{
throw (new IllegalStateException("SHA-256 is required by Java", e));
}
}
/*******************************************************************************
** Leading operation, including WITH as its own kind.
*******************************************************************************/
public static String operation(String sql)
{
String first = normalize(sql).split(" ", 2)[0].toUpperCase(Locale.ROOT);
return (switch(first)
{
case "SELECT", "INSERT", "UPDATE", "DELETE", "WITH" -> first;
default -> "OTHER";
});
}
/*******************************************************************************
** Best-effort first target table, without schema or identifier quotes.
*******************************************************************************/
public static String mainTable(String sql)
{
Matcher match = TABLE.matcher(normalize(sql));
if(!match.find())
{
return (null);
}
String[] parts = match.group(1).split(" \\. ");
return (parts[parts.length - 1].replace("\"", ""));
}
}
@@ -0,0 +1,76 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.processes;
import java.util.ArrayList;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.analyze.AnalyzeService;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
/*******************************************************************************
** Scheduled analysis and Analyze now use the same stored-data evaluation path.
*******************************************************************************/
public class DbPerfAnalyzeStep implements BackendStep
{
/*******************************************************************************
** Include per-rule counts and failures in the standard Result screen.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".manage_settings"))
{
throw (new QException("Managing settings permission is required"));
}
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
try
{
for(AnalyzeService.Result result : new AnalyzeService(runtime).run(runtime.getConfig().getBackendName()))
{
if(result.error() != null)
{
lines.add(new ProcessSummaryLine(Status.ERROR, null, result.ruleCode() + ": " + result.error()));
}
else
{
lines.add(new ProcessSummaryLine(Status.INFO, result.counts().created(), result.ruleCode() + " findings created"));
lines.add(new ProcessSummaryLine(Status.INFO, result.counts().updated(), result.ruleCode() + " findings updated"));
lines.add(new ProcessSummaryLine(Status.INFO, result.counts().resolved(), result.ruleCode() + " findings resolved"));
}
}
}
catch(Exception e)
{
throw (new QException("Unable to analyze database performance", e));
}
output.addValue("processResults", lines);
}
}
@@ -0,0 +1,85 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.processes;
import java.time.Instant;
import java.util.ArrayList;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.DbPerfSettingsService;
import com.kingsrook.qbits.dbperformance.collect.SnapshotCollector;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Collect on schedule or explicitly from the Snapshots table.
*******************************************************************************/
public class DbPerfCollectStep implements BackendStep
{
/*******************************************************************************
** Respect runtime interval settings on scheduled retries; Collect now forces
** table statistics and always returns the recorded outcome on its result screen.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".manage_settings"))
{
throw (new QException("Managing settings permission is required"));
}
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
boolean force = Boolean.TRUE.equals(input.getValueBoolean("forceTables"));
try
{
QRecord settings = new DbPerfSettingsService(runtime).ensure(runtime.getConfig().getBackendName());
Instant last = settings.getValueInstant("lastSnapshotAt");
if(!force && last != null && last.plusSeconds(60L * SnapshotCollector.positive(settings, "snapshotIntervalMinutes", 5)).isAfter(Instant.now()))
{
lines.add(new ProcessSummaryLine(Status.INFO, 0, "snapshots collected; the next interval is not due"));
}
else
{
QRecord snapshot = new SnapshotCollector(runtime).collect(runtime.getConfig().getBackendName(), force);
output.addRecord(snapshot);
lines.add(new ProcessSummaryLine("FAILED".equals(snapshot.getValueString("status")) ? Status.ERROR : Status.INFO, 1, "snapshot: " + snapshot.getValueString("status")));
lines.add(new ProcessSummaryLine(Status.INFO, snapshot.getValueInteger("storedStatementCount") == null ? 0 : snapshot.getValueInteger("storedStatementCount"), "query details stored"));
if(snapshot.getValueString("errorMessage") != null)
{
lines.add(new ProcessSummaryLine(Status.INFO, 0, snapshot.getValueString("errorMessage")));
}
}
}
catch(Exception e)
{
throw (new QException("Unable to collect database performance snapshot", e));
}
output.addValue("processResults", lines);
}
}
@@ -0,0 +1,132 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.processes;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfPermissions;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qqq.backend.core.jobs.DurableProcessMetaData;
import com.kingsrook.qqq.backend.core.jobs.model.JobIdempotency;
import com.kingsrook.qqq.backend.core.jobs.model.JobSemantics;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionOutputMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Scheduled maintenance and the Snapshot table action; no process navigation.
*******************************************************************************/
public final class DbPerfProcesses
{
public static final String ANALYZE = "dbPerfAnalyze";
public static final String ANALYZE_NOW = "dbPerfAnalyzeNow";
public static final String COLLECT = "dbPerfCollectSnapshot";
public static final String ROLLUP = "dbPerfRollupAndPurge";
public static final String COLLECT_NOW = "dbPerfCollectNow";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfProcesses()
{
}
/*******************************************************************************
** Add schedules only when the host supplies a scheduler name.
*******************************************************************************/
public static void addTo(QInstance instance, DbPerfQBitConfig config)
{
instance.addProcess(scheduled(config, ANALYZE, "Analyze database performance", DbPerfAnalyzeStep.class, config.getAnalyzeCron()));
instance.addProcess(base(config, ANALYZE_NOW, "Analyze now", DbPerfAnalyzeStep.class, false).withTableName("db_perf_finding")
.withStep(new QFrontendStepMetaData().withName("result").withLabel("Result")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.PROCESS_SUMMARY_RESULTS))));
for(String status : List.of("ACKNOWLEDGED", "RESOLVED", "IGNORED", "OPEN"))
{
String label = switch(status)
{
case "ACKNOWLEDGED" -> "Acknowledge";
case "RESOLVED" -> "Mark resolved";
case "IGNORED" -> "Ignore";
default -> "Reopen";
};
QProcessMetaData action = StreamedETLWithFrontendProcess.processMetaDataBuilder()
.withName("dbPerfFinding" + status).withLabel(label).withIcon(new QIcon("done"))
.withTableName("db_perf_finding").withSourceTable("db_perf_finding").withDestinationTable("db_perf_finding")
.withExtractStepClass(ExtractViaQueryStep.class).withTransformStepClass(FindingSteps.Transform.class).withLoadStepClass(FindingSteps.Load.class)
.withSupportsFullValidation(true).withDoFullValidation(true).withMinInputRecords(1)
.withPreviewMessage("Review the selected findings before changing their status.")
.withFields(List.of(new QFieldMetaData(FindingSteps.TARGET, QFieldType.STRING).withDefaultValue(status))).getProcessMetaData();
action.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".manage_findings"))
.withStep(0, new QFrontendStepMetaData().withName("note").withLabel("Note")
.withFormField(new QFieldMetaData(FindingSteps.NOTE, QFieldType.STRING).withLabel("Note").withMaxLength(2000).withIsRequired("IGNORED".equals(status))));
instance.addProcess(action);
}
instance.addProcess(scheduled(config, COLLECT, "Collect database snapshot", DbPerfCollectStep.class, config.getSnapshotCron()));
instance.addProcess(scheduled(config, ROLLUP, "Roll up and purge database performance", DbPerfRollupStep.class, config.getRollupCron()));
instance.addProcess(base(config, COLLECT_NOW, "Collect now", DbPerfCollectStep.class, true)
.withTableName("db_perf_snapshot")
.withStep(new QFrontendStepMetaData().withName("result").withLabel("Result")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.PROCESS_SUMMARY_RESULTS))));
}
/*******************************************************************************
** Hidden durable-eligible maintenance process.
*******************************************************************************/
private static QProcessMetaData scheduled(DbPerfQBitConfig config, String name, String label, Class<?> step, String cron)
{
QProcessMetaData process = base(config, name, label, step, false).withIsHidden(true)
.withSupplementalMetaData(new DurableProcessMetaData().withEligible(true).withSemantics(JobSemantics.ATOMIC).withIdempotency(JobIdempotency.NATURAL).withMaxConcurrentJobs(1));
if(config.getSchedulerName() != null)
{
process.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withCronExpression(cron).withCronTimeZoneId(config.getTimeZone()));
}
return (process);
}
/*******************************************************************************
** One backend step with standard process result lines.
*******************************************************************************/
private static QProcessMetaData base(DbPerfQBitConfig config, String name, String label, Class<?> step, boolean force)
{
return (new QProcessMetaData().withName(name).withLabel(label).withIcon(new QIcon("speed"))
.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".manage_settings"))
.withStep(new QBackendStepMetaData().withName("run").withCode(new QCodeReference(step))
.withInputData(new QFunctionInputMetaData().withField(new QFieldMetaData("forceTables", QFieldType.BOOLEAN).withDefaultValue(force)))
.withOutputMetaData(new QFunctionOutputMetaData().withField(new QFieldMetaData("processResults", QFieldType.STRING)))));
}
}
@@ -0,0 +1,82 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.processes;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.ArrayList;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.collect.RetentionService;
import com.kingsrook.qbits.dbperformance.collect.RollupService;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
/*******************************************************************************
** Roll up yesterday and backfill missing recent days, then enforce retention.
*******************************************************************************/
public class DbPerfRollupStep implements BackendStep
{
/*******************************************************************************
** Yesterday includes late detail; missing older days are repaired only while
** their detail is retained. Daily history outlives the detail-retention window.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".manage_settings"))
{
throw (new QException("Managing settings permission is required"));
}
try
{
LocalDate today = LocalDate.now(ZoneId.of(runtime.getConfig().getTimeZone()));
RollupService rollup = new RollupService(runtime);
int rolledUp = 0;
for(int days = 1; days <= 7; days++)
{
LocalDate day = today.minusDays(days);
if(days == 1 || rollup.needsBackfill(day))
{
rollup.rollupDay(day);
rolledUp++;
}
}
int removed = new RetentionService(runtime).purge();
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
lines.add(new ProcessSummaryLine(Status.INFO, rolledUp, "days rolled up"));
lines.add(new ProcessSummaryLine(Status.INFO, removed, "expired records deleted"));
output.addValue("processResults", lines);
}
catch(Exception e)
{
throw (new QException("Database performance rollup and purge failed", e));
}
}
}
@@ -0,0 +1,212 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.processes;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.analyze.FindingService;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
/*******************************************************************************
** Selected-record streamed actions. Review does not write; Load rechecks access
** under the caller session before narrowly scoped system writes.
*******************************************************************************/
public final class FindingSteps
{
public static final String NOTE = "statusNote";
public static final String TARGET = "findingStatus";
private static final Map<String, String> VERBS = Map.of("ACKNOWLEDGED", "acknowledged", "RESOLVED", "marked resolved", "IGNORED", "ignored", "OPEN", "reopened");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private FindingSteps()
{
}
/*******************************************************************************
** Both phases validate permission, requested state and Ignore's required note.
*******************************************************************************/
private static String validate(RunBackendStepInput input) throws QException
{
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(runtime.getConfig().getPermissionBaseName() + ".manage_findings"))
{
throw (new QException("Managing findings permission is required"));
}
String status = input.getValueString(TARGET);
if(!VERBS.containsKey(status == null ? "" : status))
{
throw (new QException("Invalid finding status"));
}
String note = input.getValueString(NOTE);
if("IGNORED".equals(status) && (note == null || note.isBlank()) || note != null && note.length() > 2000)
{
throw (new QException("Ignore requires a note; notes must be at most 2000 characters"));
}
return (status);
}
/*******************************************************************************
** Standard singular/plural and future/past wording.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String future, String past)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage("finding will be " + future);
line.setPluralFutureMessage("findings will be " + future);
line.setSingularPastMessage("finding was " + past);
line.setPluralPastMessage("findings were " + past);
return (line);
}
/*******************************************************************************
** The Review screen describes precisely the selected records.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private ProcessSummaryLine changes;
/*******************************************************************************
** Validate and forward a bounded page with no mutations.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
String verb = VERBS.get(validate(input));
if(changes == null)
{
changes = line(Status.OK, verb, verb);
}
for(QRecord record : input.getRecords())
{
changes.incrementCountAndAddPrimaryKey(record.getValue("id"));
output.addRecord(record);
}
}
/*******************************************************************************
** The frontend chooses the correct tense and plurality.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
{
return (changes == null ? new ArrayList<>() : new ArrayList<>(List.of(changes)));
}
}
/*******************************************************************************
** Persist the status fields only; supplied record data is never trusted.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private ProcessSummaryLine changes;
private final ProcessSummaryLine errors = line(Status.ERROR, "checked for access", "not updated; unavailable or write failed");
/*******************************************************************************
** Preserve the human user id before entering the system write context.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
String status = validate(input);
String actor = QContext.getQSession().getUser() == null ? null : QContext.getQSession().getUser().getIdReference();
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
if(changes == null)
{
changes = line(Status.OK, VERBS.get(status), VERBS.get(status));
}
for(QRecord selected : input.getRecords())
{
try
{
// Read under the caller context so record security still applies.
QRecord visible = DbPerfStore.first("db_perf_finding", DbPerfStore.eq("id", selected.getValueInteger("id")));
if(visible == null || !runtime.getConfig().getBackendName().equals(visible.getValueString("backendName")))
{
errors.incrementCountAndAddPrimaryKey(selected.getValue("id"));
continue;
}
QRecord patch = new QRecord().withValue("id", visible.getValueInteger("id")).withValue("status", status)
.withValue("statusChangedBy", actor).withValue("resolvedAt", "RESOLVED".equals(status) ? Instant.now() : null);
if(input.getValueString(NOTE) != null && !input.getValueString(NOTE).isBlank())
{
patch.withValue(NOTE, input.getValueString(NOTE));
}
synchronized(runtime)
{
runtime.transaction(() ->
{
FindingService.lockRule(runtime, visible.getValueString("ruleCode"));
DbPerfStore.update("db_perf_finding", List.of(patch));
return (null);
});
}
changes.incrementCountAndAddPrimaryKey(selected.getValue("id"));
}
catch(Exception e)
{
errors.incrementCountAndAddPrimaryKey(selected.getValue("id"));
}
}
}
/*******************************************************************************
** Include failures while allowing other selected records to complete.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
if(changes != null)
{
lines.add(changes);
}
errors.addSelfToListIfAnyCount(lines);
return (lines);
}
}
}
@@ -0,0 +1,31 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
/*******************************************************************************
** ConnectionCount read from database statistics; no persistence or side effects.
*******************************************************************************/
public record ConnectionCount(String dbUser, String applicationName, String state, int connectionCount)
{
}
@@ -0,0 +1,33 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.time.Instant;
import java.util.Map;
/*******************************************************************************
** DatabaseStats read from database statistics; no persistence or side effects.
*******************************************************************************/
public record DatabaseStats(Map<String, Long> counters, Instant statsReset, int maxConnections, long sizeBytes)
{
}
@@ -0,0 +1,93 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.List;
import java.util.Map;
/*******************************************************************************
** Read-only database statistics SPI. Implementations never change the database.
*******************************************************************************/
public interface DbStatsProvider
{
/*******************************************************************************
** Provider name.
*******************************************************************************/
String getName();
/*******************************************************************************
** capabilities using the supplied backend connection.
*******************************************************************************/
ProviderCapabilities capabilities(Connection connection) throws SQLException;
/*******************************************************************************
** readDatabase using the supplied backend connection.
*******************************************************************************/
DatabaseStats readDatabase(Connection connection) throws SQLException;
/*******************************************************************************
** readStatements using the supplied backend connection.
*******************************************************************************/
List<StatementStats> readStatements(Connection connection) throws SQLException;
/*******************************************************************************
** readStatementResetInfo using the supplied backend connection.
*******************************************************************************/
StatementResetInfo readStatementResetInfo(Connection connection) throws SQLException;
/*******************************************************************************
** readTables using the supplied backend connection.
*******************************************************************************/
List<TableStats> readTables(Connection connection) throws SQLException;
/*******************************************************************************
** readIndexes using the supplied backend connection.
*******************************************************************************/
List<IndexStats> readIndexes(Connection connection) throws SQLException;
/*******************************************************************************
** readSessions using the supplied backend connection.
*******************************************************************************/
List<SessionInfo> readSessions(Connection connection) throws SQLException;
/*******************************************************************************
** readConnections using the supplied backend connection.
*******************************************************************************/
List<ConnectionCount> readConnections(Connection connection) throws SQLException;
/*******************************************************************************
** readSettings using the supplied backend connection.
*******************************************************************************/
Map<String, String> readSettings(Connection connection) throws SQLException;
}
@@ -0,0 +1,48 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.sql.Connection;
import java.sql.SQLException;
/*******************************************************************************
** Select a provider from JDBC product identity.
*******************************************************************************/
public final class DbStatsProviders
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbStatsProviders()
{
}
/*******************************************************************************
** PostgreSQL first; other vendors are explicitly unsupported.
*******************************************************************************/
public static DbStatsProvider forConnection(Connection connection) throws SQLException
{
return ("PostgreSQL".equalsIgnoreCase(connection.getMetaData().getDatabaseProductName()) ? new PostgresStatsProvider() : new UnsupportedStatsProvider());
}
}
@@ -0,0 +1,31 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
/*******************************************************************************
** IndexStats read from database statistics; no persistence or side effects.
*******************************************************************************/
public record IndexStats(String schemaName, String tableName, String indexName, long relid, String definition, String columns, boolean unique, boolean primary, boolean valid, long idxScan, long idxTupRead, long idxTupFetch, long sizeBytes, long writeCost)
{
}
@@ -0,0 +1,318 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Timestamp;
import java.time.Instant;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/*******************************************************************************
** PostgreSQL 13+ statistics reader; every statement is a read-only SELECT.
*******************************************************************************/
public class PostgresStatsProvider implements DbStatsProvider
{
/*******************************************************************************
** Provider name.
*******************************************************************************/
@Override
public String getName()
{
return ("POSTGRES");
}
/*******************************************************************************
** Server version and privileges, including explicit installation guidance.
*******************************************************************************/
@Override
public ProviderCapabilities capabilities(Connection connection) throws SQLException
{
int version = version(connection);
boolean installed = scalar(connection, "SELECT count(*) FROM pg_extension WHERE extname = 'pg_stat_statements'") > 0;
boolean privileged = scalar(connection, "SELECT CASE WHEN pg_has_role(current_user, 'pg_read_all_stats', 'MEMBER') OR (SELECT rolsuper FROM pg_roles WHERE rolname=current_user) THEN 1 ELSE 0 END") > 0;
String access = version < 130000 ? "UNSUPPORTED" : !installed ? "NO_PG_STAT_STATEMENTS" : !privileged ? "NO_READ_ALL_STATS" : "FULL";
String message = switch(access)
{
case "UNSUPPORTED" -> "PostgreSQL 13 or newer is required.";
case "NO_PG_STAT_STATEMENTS" -> "Install pg_stat_statements in this database and preload it in shared_preload_libraries.";
case "NO_READ_ALL_STATS" -> "Grant pg_read_all_stats to the application database role to see all statement statistics.";
default -> "Full statistics access.";
};
if(installed && version >= 130000)
{
try
{
scalar(connection, "SELECT count(*) FROM " + statementsView(connection));
}
catch(SQLException e)
{
if("55000".equals(e.getSQLState()) || "42501".equals(e.getSQLState()))
{
installed = false;
access = "NO_PG_STAT_STATEMENTS";
message = "pg_stat_statements must be preloaded and readable by the application database role.";
}
else
{
throw (e);
}
}
}
return (new ProviderCapabilities(version >= 130000, version, installed, privileged, access, message));
}
/*******************************************************************************
** Current database cumulative counters and capacity.
*******************************************************************************/
@Override
public DatabaseStats readDatabase(Connection connection) throws SQLException
{
return (read(connection, "SELECT d.*, pg_database_size(current_database()) AS size_bytes, current_setting('max_connections')::int AS max_connections FROM pg_stat_database d WHERE datname=current_database()", row ->
new DatabaseStats(counters(row, "xact_commit", "xact_rollback", "blks_hit", "blks_read", "temp_files", "temp_bytes", "deadlocks", "conflicts"), instant(row, "stats_reset"), row.getInt("max_connections"), row.getLong("size_bytes"))).getFirst());
}
/*******************************************************************************
** Current database statements only. Version-dependent columns are projected
** as null or true when older PostgreSQL versions do not provide them.
*******************************************************************************/
@Override
public List<StatementStats> readStatements(Connection connection) throws SQLException
{
int version = version(connection);
String sql = "SELECT s.queryid, r.rolname, s.query, s.calls, s.total_exec_time, s.mean_exec_time, s.max_exec_time, s.rows, s.shared_blks_hit, s.shared_blks_read, s.temp_blks_written, "
+ (version >= 140000 ? "s.toplevel" : "true") + " AS top_level, "
+ (version >= 170000 ? "s.stats_since, s.minmax_stats_since" : "NULL::timestamptz AS stats_since, NULL::timestamptz AS minmax_stats_since")
+ " FROM " + statementsView(connection) + " s LEFT JOIN pg_roles r ON r.oid=s.userid WHERE s.dbid=(SELECT oid FROM pg_database WHERE datname=current_database()) AND s.queryid IS NOT NULL AND s.query IS NOT NULL";
return (read(connection, sql, row -> new StatementStats(row.getLong("queryid"), row.getString("rolname"), row.getBoolean("top_level"), row.getString("query"), row.getLong("calls"), row.getBigDecimal("total_exec_time"), row.getBigDecimal("mean_exec_time"), row.getBigDecimal("max_exec_time"), row.getLong("rows"), row.getLong("shared_blks_hit"), row.getLong("shared_blks_read"), row.getLong("temp_blks_written"), instant(row, "stats_since"), instant(row, "minmax_stats_since"))));
}
/*******************************************************************************
** Global statement reset is available starting with PostgreSQL 14.
*******************************************************************************/
@Override
public StatementResetInfo readStatementResetInfo(Connection connection) throws SQLException
{
if(version(connection) < 140000)
{
return (new StatementResetInfo(null, 0));
}
return (read(connection, "SELECT stats_reset, dealloc FROM " + statementsView(connection) + "_info", row -> new StatementResetInfo(instant(row, "stats_reset"), row.getLong("dealloc"))).getFirst());
}
/*******************************************************************************
** Latest table counters, tuple gauges, maintenance times and sizes.
*******************************************************************************/
@Override
public List<TableStats> readTables(Connection connection) throws SQLException
{
String sql = "SELECT s.*, pg_total_relation_size(s.relid) AS total_bytes, pg_relation_size(s.relid) AS table_bytes, pg_indexes_size(s.relid) AS index_bytes, CASE WHEN c.reltoastrelid=0 THEN 0 ELSE pg_total_relation_size(c.reltoastrelid) END AS toast_bytes, age(c.relfrozenxid)::bigint AS freeze_age FROM pg_stat_user_tables s JOIN pg_class c ON c.oid=s.relid";
return (read(connection, sql, row ->
{
Map<String, Instant> maintenance = new LinkedHashMap<>();
for(String field : List.of("last_vacuum", "last_autovacuum", "last_analyze", "last_autoanalyze"))
{
maintenance.put(camel(field), instant(row, field));
}
return (new TableStats(row.getString("schemaname"), row.getString("relname"), row.getLong("relid"), counters(row, "seq_scan", "seq_tup_read", "idx_scan", "idx_tup_fetch", "n_tup_ins", "n_tup_upd", "n_tup_del", "n_tup_hot_upd", "n_live_tup", "n_dead_tup", "total_bytes", "table_bytes", "index_bytes", "toast_bytes", "vacuum_count", "autovacuum_count", "analyze_count", "autoanalyze_count", "freeze_age"), maintenance));
}));
}
/*******************************************************************************
** Index definitions, ordered keys (including expressions) and write cost.
*******************************************************************************/
@Override
public List<IndexStats> readIndexes(Connection connection) throws SQLException
{
String sql = "SELECT s.*, i.indisunique, i.indisprimary, i.indisvalid, pg_get_indexdef(s.indexrelid) AS definition, (SELECT string_agg(pg_get_indexdef(s.indexrelid, k, true), ', ' ORDER BY k) FROM generate_series(1, i.indnatts) AS k) AS column_names, pg_relation_size(s.indexrelid) AS size_bytes, t.n_tup_ins+t.n_tup_upd+t.n_tup_del AS write_cost FROM pg_stat_user_indexes s JOIN pg_index i ON i.indexrelid=s.indexrelid JOIN pg_stat_user_tables t ON t.relid=s.relid";
return (read(connection, sql, row -> new IndexStats(row.getString("schemaname"), row.getString("relname"), row.getString("indexrelname"), row.getLong("indexrelid"), row.getString("definition"), row.getString("column_names"), row.getBoolean("indisunique"), row.getBoolean("indisprimary"), row.getBoolean("indisvalid"), row.getLong("idx_scan"), row.getLong("idx_tup_read"), row.getLong("idx_tup_fetch"), row.getLong("size_bytes"), row.getLong("write_cost"))));
}
/*******************************************************************************
** Current database activity, excluding the collector's own session.
*******************************************************************************/
@Override
public List<SessionInfo> readSessions(Connection connection) throws SQLException
{
String sql = "SELECT pid, usename, application_name, client_addr::text, state, wait_event_type, wait_event, backend_type, xact_start, query_start, state_change, extract(epoch FROM clock_timestamp()-query_start) AS query_seconds, extract(epoch FROM clock_timestamp()-xact_start) AS xact_seconds, pg_blocking_pids(pid) AS blocked_by, "
+ (version(connection) >= 140000 ? "query_id" : "NULL::bigint AS query_id") + ", query FROM pg_stat_activity WHERE datname=current_database() AND pid<>pg_backend_pid()";
return (read(connection, sql, row ->
{
List<Integer> blockedBy = new ArrayList<>();
java.sql.Array array = row.getArray("blocked_by");
if(array != null)
{
try
{
for(Object pid : (Object[]) array.getArray())
{
blockedBy.add(((Number) pid).intValue());
}
}
finally
{
array.free();
}
}
Long queryId = row.getObject("query_id") == null ? null : row.getLong("query_id");
return (new SessionInfo(row.getInt("pid"), row.getString("usename"), row.getString("application_name"), row.getString("client_addr"), row.getString("state"), row.getString("wait_event_type"), row.getString("wait_event"), row.getString("backend_type"), instant(row, "xact_start"), instant(row, "query_start"), instant(row, "state_change"), row.getBigDecimal("query_seconds"), row.getBigDecimal("xact_seconds"), blockedBy, queryId, row.getString("query")));
}));
}
/*******************************************************************************
** Cluster-wide capacity counts, including connections in other databases.
*******************************************************************************/
@Override
public List<ConnectionCount> readConnections(Connection connection) throws SQLException
{
return (read(connection, "SELECT usename, application_name, state, count(*) AS connections FROM pg_stat_activity GROUP BY usename, application_name, state", row -> new ConnectionCount(row.getString("usename"), row.getString("application_name"), row.getString("state"), row.getInt("connections"))));
}
/*******************************************************************************
** Only operational, non-secret server settings relevant to this qbit.
*******************************************************************************/
@Override
public Map<String, String> readSettings(Connection connection) throws SQLException
{
Map<String, String> result = new LinkedHashMap<>();
read(connection, "SELECT name, setting FROM pg_settings WHERE name IN ('server_version_num','max_connections','shared_preload_libraries','pg_stat_statements.track','track_io_timing','block_size')", row -> result.put(row.getString("name"), row.getString("setting")));
return (result);
}
/*******************************************************************************
** Extension schema is obtained from the catalog, then identifier-quoted.
*******************************************************************************/
private String statementsView(Connection connection) throws SQLException
{
List<String> schemas = read(connection, "SELECT n.nspname FROM pg_extension e JOIN pg_namespace n ON n.oid=e.extnamespace WHERE e.extname='pg_stat_statements'", row -> row.getString(1));
if(schemas.isEmpty())
{
throw (new SQLException("pg_stat_statements is not installed", "42P01"));
}
return ("\"" + schemas.getFirst().replace("\"", "\"\"") + "\".pg_stat_statements");
}
/*******************************************************************************
** Numeric server version avoids JDBC minor-version ambiguity.
*******************************************************************************/
private int version(Connection connection) throws SQLException
{
return ((int) scalar(connection, "SELECT current_setting('server_version_num')::int"));
}
/*******************************************************************************
** A single numeric read.
*******************************************************************************/
private long scalar(Connection connection, String sql) throws SQLException
{
return (read(connection, sql, row -> row.getLong(1)).getFirst());
}
/*******************************************************************************
** Timestamp conversion preserves null reset/maintenance values.
*******************************************************************************/
private static Instant instant(ResultSet row, String column) throws SQLException
{
Timestamp timestamp = row.getTimestamp(column);
return (timestamp == null ? null : timestamp.toInstant());
}
/*******************************************************************************
** Copy cumulative counters to the entity's camelCase names.
*******************************************************************************/
private static Map<String, Long> counters(ResultSet row, String... columns) throws SQLException
{
Map<String, Long> counters = new LinkedHashMap<>();
for(String column : columns)
{
counters.put(camel(column), row.getLong(column));
}
return (counters);
}
/*******************************************************************************
** PostgreSQL column to entity field.
*******************************************************************************/
private static String camel(String column)
{
StringBuilder value = new StringBuilder();
for(String part : column.split("_"))
{
value.append(value.isEmpty() ? part : Character.toUpperCase(part.charAt(0)) + part.substring(1));
}
return (value.toString());
}
/*******************************************************************************
** Execute one bounded-time read and close JDBC resources.
*******************************************************************************/
private static <T> List<T> read(Connection connection, String sql, RowReader<T> mapper) throws SQLException
{
List<T> result = new ArrayList<>();
try(Statement statement = connection.createStatement())
{
statement.setQueryTimeout(30);
try(ResultSet rows = statement.executeQuery(sql))
{
while(rows.next())
{
result.add(mapper.read(rows));
}
}
}
return (result);
}
/*******************************************************************************
** Checked JDBC row mapping.
*******************************************************************************/
private interface RowReader<T>
{
/*******************************************************************************
** Map one row.
*******************************************************************************/
T read(ResultSet row) throws SQLException;
}
}
@@ -0,0 +1,31 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
/*******************************************************************************
** ProviderCapabilities read from database statistics; no persistence or side effects.
*******************************************************************************/
public record ProviderCapabilities(boolean supported, int version, boolean statements, boolean readAllStats, String statsAccess, String message)
{
}
@@ -0,0 +1,34 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.List;
/*******************************************************************************
** SessionInfo read from database statistics; no persistence or side effects.
*******************************************************************************/
public record SessionInfo(int pid, String dbUser, String applicationName, String clientAddr, String state, String waitEventType, String waitEvent, String backendType, Instant xactStart, Instant queryStart, Instant stateChange, BigDecimal querySeconds, BigDecimal xactSeconds, List<Integer> blockedBy, Long queryId, String queryText)
{
}
@@ -0,0 +1,32 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.time.Instant;
/*******************************************************************************
** StatementResetInfo read from database statistics; no persistence or side effects.
*******************************************************************************/
public record StatementResetInfo(Instant statsReset, long dealloc)
{
}
@@ -0,0 +1,33 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.math.BigDecimal;
import java.time.Instant;
/*******************************************************************************
** StatementStats read from database statistics; no persistence or side effects.
*******************************************************************************/
public record StatementStats(long queryId, String dbUser, boolean toplevel, String queryText, long calls, BigDecimal totalExecMs, BigDecimal meanExecMs, BigDecimal maxExecMs, long rows, long sharedBlksHit, long sharedBlksRead, long tempBlksWritten, Instant statsSince, Instant minmaxStatsSince)
{
}
@@ -0,0 +1,33 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.time.Instant;
import java.util.Map;
/*******************************************************************************
** TableStats read from database statistics; no persistence or side effects.
*******************************************************************************/
public record TableStats(String schemaName, String tableName, long relid, Map<String, Long> counters, Map<String, Instant> maintenance)
{
}
@@ -0,0 +1,132 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.provider;
import java.sql.Connection;
import java.util.List;
import java.util.Map;
/*******************************************************************************
** Unsupported vendors report reduced capability without reading vendor views.
*******************************************************************************/
public class UnsupportedStatsProvider implements DbStatsProvider
{
/*******************************************************************************
** Provider name.
*******************************************************************************/
@Override
public String getName()
{
return ("UNSUPPORTED");
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public ProviderCapabilities capabilities(Connection connection)
{
return (new ProviderCapabilities(false, 0, false, false, "UNSUPPORTED", "This database vendor is not supported; PostgreSQL 13 or newer is required."));
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public DatabaseStats readDatabase(Connection connection)
{
return (new DatabaseStats(Map.of(), null, 0, 0));
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public List<StatementStats> readStatements(Connection connection)
{
return (List.of());
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public StatementResetInfo readStatementResetInfo(Connection connection)
{
return (new StatementResetInfo(null, 0));
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public List<TableStats> readTables(Connection connection)
{
return (List.of());
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public List<IndexStats> readIndexes(Connection connection)
{
return (List.of());
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public List<SessionInfo> readSessions(Connection connection)
{
return (List.of());
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public List<ConnectionCount> readConnections(Connection connection)
{
return (List.of());
}
/*******************************************************************************
** Empty unsupported result.
*******************************************************************************/
@Override
public Map<String, String> readSettings(Connection connection)
{
return (Map.of());
}
}
@@ -0,0 +1,44 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.widgets;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
/*******************************************************************************
** Snapshot time series rendered by the standard typed line-chart component.
*******************************************************************************/
public class DbPerfTimeChart extends ChartData
{
/*******************************************************************************
** Match the registered metadata type.
*******************************************************************************/
@Override
public String getType()
{
return (WidgetType.LINE_CHART.getType());
}
}
@@ -0,0 +1,494 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.widgets;
import java.sql.Connection;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import com.kingsrook.qbits.dbperformance.DbPerfRuntime;
import com.kingsrook.qbits.dbperformance.api.DbPerformanceData;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qbits.dbperformance.collect.SnapshotCollector;
import com.kingsrook.qbits.dbperformance.provider.ConnectionCount;
import com.kingsrook.qbits.dbperformance.provider.DbStatsProviders;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.QFilterOrderByAggregate;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.CompositeWidgetData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiTableData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ParentWidgetData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.QWidgetData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockLink;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.text.TextBlockData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.text.TextStyles;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.text.TextValues;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import static com.kingsrook.qbits.dbperformance.api.DbPerformanceData.map;
import static com.kingsrook.qbits.dbperformance.api.DbPerformanceData.number;
/*******************************************************************************
** Typed data only: paginated tables, full-width panels and explicit empty guidance.
*******************************************************************************/
public class DbPerfWidgetRenderer extends AbstractWidgetRenderer
{
/*******************************************************************************
** No privileged read context: both widget and service enforce the caller's view grant.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
DbPerfRuntime runtime = DbPerfRuntime.of(QContext.getQInstance());
DbPerformanceData data = new DbPerformanceData(runtime);
data.requireView(null);
try
{
QWidgetData result = switch(input.getWidgetMetaData().getName())
{
case "dbPerfSetup" -> setup(runtime, data);
case "dbPerfDatabaseTime" -> databaseTime(data);
case "dbPerfTopQueries" -> topQueries(data);
case "dbPerfFindings" -> findings(data);
case "dbPerfLiveActivity" -> liveActivity(data);
case "dbPerfConnections" -> connections();
case "dbPerfConnectionStatistics" -> connectionStatistics(data);
case "dbPerfConnectionsByApplication" -> connectionsByApplication(runtime);
case "dbPerfAttribution" -> attribution();
case "dbPerfTables" -> tables(data);
default -> throw new QException("Unknown Database Performance widget.");
};
return (new RenderWidgetOutput(result));
}
catch(QException e)
{
throw (e);
}
catch(Exception e)
{
throw (new QException("Database Performance panel unavailable. Check the backend and statistics access."));
}
}
/*******************************************************************************
** Guided checks do not write settings or require a first successful snapshot.
*******************************************************************************/
public TableData setup(DbPerfRuntime runtime, DbPerformanceData data) throws Exception
{
QRecord settings = data.settings();
String access = settings == null ? "NOT_COLLECTED" : settings.getValueString("statsAccess");
Instant last = settings == null ? null : settings.getValueInstant("lastSnapshotAt");
boolean attribution = settings == null ? Boolean.TRUE.equals(runtime.getConfig().getAttributionEnabled()) : Boolean.TRUE.equals(settings.getValueBoolean("attributionEnabled"));
List<Map<String, Object>> rows = new ArrayList<>();
rows.add(check("Monitored backend", runtime.getConfig().getBackendName(), "Review Settings for this backend."));
rows.add(check("Provider", settings == null ? "Not detected yet" : settings.getValueString("provider"), "PostgreSQL 13+ is supported."));
rows.add(check("pg_stat_statements", access, "Preload and create pg_stat_statements; then run Collect now."));
rows.add(check("pg_read_all_stats", access, settings == null ? "Grant pg_read_all_stats to the application's database role." : settings.getValueString("statsAccessMessage")));
rows.add(check("Last snapshot", last == null ? "Nothing collected yet" : Duration.between(last, Instant.now()).toSeconds() + " seconds ago",
last == null ? "Run Collect now from Snapshots, or wait for the next scheduled collection." : "Collection normally runs every " + SnapshotCollector.positive(settings, "snapshotIntervalMinutes", 5) + " minutes."));
rows.add(check("Attribution", attribution ? "On" : "Off", attribution ? "Attribution starts after the next flush." : "Enable attribution in Settings and the host configuration."));
rows.add(check("Schedules", runtime.getConfig().getSchedulerName() == null ? "Not configured" : "Configured", "Set schedulerName on the qbit and register that scheduler in the host."));
return (table("Run Collect now to start collecting database performance.", rows, "Check", "State", "Next step"));
}
/*******************************************************************************
** Plain text check row, never markup.
*******************************************************************************/
private static Map<String, Object> check(String name, String state, String next)
{
return (map("Check", name, "State", state, "Next step", next));
}
/*******************************************************************************
** Separate labeled time and call series from each snapshot's interval deltas.
*******************************************************************************/
public ChartData databaseTime(DbPerformanceData data) throws QException
{
List<QRecord> snapshots = data.snapshots();
List<String> labels = snapshots.stream().map(row -> row.getValueString("capturedAt")).toList();
List<Number> milliseconds = snapshots.stream().map(row -> (Number) row.getValue("statementTotalMs")).toList();
List<Number> calls = snapshots.stream().map(row -> (Number) row.getValue("statementCalls")).toList();
return (new DbPerfTimeChart().withDescription(snapshots.isEmpty() ? "Database time appears after the next snapshot." : "Last 24 hours; each point is one collection interval.")
.withChartData(new ChartData.Data().withLabels(labels).withDatasets(List.of(
new ChartData.Data.Dataset().withLabel("Database time (ms)").withData(milliseconds).withBackgroundColor("#2563eb"),
new ChartData.Data.Dataset().withLabel("Calls").withData(calls).withBackgroundColor("#16a34a")))));
}
/*******************************************************************************
** Truncated query shapes link to their record and paginate rather than scroll.
*******************************************************************************/
public TableData topQueries(DbPerformanceData data) throws QException
{
List<Map<String, Object>> rows = new ArrayList<>();
for(Map<String, Object> statement : data.statements(100, null))
{
rows.add(map("Query", stack((String) statement.get("queryText"), (String) statement.get("historyPath"),
format(statement.get("calls24h")) + " calls, mean " + format(statement.get("meanMs24h")) + " ms"),
"Total ms (24h)", statement.get("totalMs24h"), "Share of DB time %", statement.get("shareOfDbTime24h"), "Cache hit ratio", statement.get("hitRatio")));
}
TableData table = table("Top queries appear after two snapshots establish interval deltas.", rows, "Query", "Total ms (24h)", "Share of DB time %", "Cache hit ratio");
table.getColumns().getFirst().setType("composite");
return (table);
}
/*******************************************************************************
** Show actionable findings from the parallel findings engine, without changing them.
*******************************************************************************/
public TableData findings(DbPerformanceData data) throws QException
{
List<Map<String, Object>> rows = new ArrayList<>();
for(QRecord row : data.findings(100))
{
rows.add(map("Severity", row.getValueString("severity"), "Title", link(row.getValueString("title"), data.recordPath("db_perf_finding", row.getValueInteger("id"))),
"Rule", row.getValueString("ruleCode"), "Last seen", row.getValue("lastSeenAt")));
}
TableData table = table("Findings appear after the next analysis of collected snapshots.", rows, "Severity", "Title", "Rule", "Last seen");
table.getColumns().get(1).setType("composite");
return (table);
}
/*******************************************************************************
** Even callers with live-text permission see only fingerprinted text on home.
*******************************************************************************/
@SuppressWarnings("unchecked")
public TableData liveActivity(DbPerformanceData data) throws Exception
{
QRecord settings = data.settings();
int longRunning = settings == null ? 30 : SnapshotCollector.positive(settings, "activityLongRunningSeconds", 30);
int idle = settings == null ? 60 : SnapshotCollector.positive(settings, "idleInTransactionSeconds", 60);
List<Map<String, Object>> sessions = (List<Map<String, Object>>) data.activity((String) null, false).get("sessions");
List<Map<String, Object>> rows = sessions.stream().filter(row -> !((List<?>) row.get("blockedBy")).isEmpty() || ((Number) row.get("blockingCount")).intValue() > 0
|| "active".equals(row.get("state")) && numeric(row.get("querySeconds")) >= longRunning
|| Objects.toString(row.get("state"), "").startsWith("idle in transaction") && numeric(row.get("xactSeconds")) >= idle).toList();
List<Map<String, Object>> cells = new ArrayList<>();
for(Map<String, Object> row : rows)
{
boolean blocked = !((List<?>) row.get("blockedBy")).isEmpty();
int blocking = ((Number) row.get("blockingCount")).intValue();
cells.add(map("Session", stack("pid " + row.get("pid") + ", " + Objects.toString(row.get("user"), "?") + ", " + Objects.toString(row.get("application"), ""), null,
Objects.toString(row.get("normalizedText"), "")),
"State", Objects.toString(row.get("state"), "") + (row.get("waitEvent") == null ? "" : " (" + row.get("waitEvent") + ")"),
"Query / transaction s", format(row.get("querySeconds")) + " / " + format(row.get("xactSeconds")),
"Blocking", blocked ? "blocked by " + row.get("blockedBy") : (blocking > 0 ? "blocks " + blocking : "")));
}
TableData table = table("No long-running or blocked sessions right now", cells, "Session", "State", "Query / transaction s", "Blocking");
table.getColumns().getFirst().setType("composite");
return (table);
}
/*******************************************************************************
** Capacity and breakdown are stacked, each occupying the whole row.
*******************************************************************************/
public ParentWidgetData connections()
{
return (new ParentWidgetData().withChildWidgetNameList(List.of("dbPerfConnectionStatistics", "dbPerfConnectionsByApplication")));
}
/*******************************************************************************
** Last observed capacity plus summed 24-hour delta metrics, never cumulative totals.
*******************************************************************************/
public MultiStatisticsData connectionStatistics(DbPerformanceData data) throws QException
{
QRecord latest = data.latest();
List<QRecord> snapshots = data.snapshots();
List<MultiStatisticsData.StatisticsGroupData.Statistic> values = new ArrayList<>();
values.add(stat("Total connections", latest == null ? null : number(latest, "connectionsTotal")));
values.add(stat("Maximum connections", latest == null ? null : number(latest, "maxConnections")));
values.add(stat("Active", latest == null ? null : number(latest, "connectionsActive")));
values.add(stat("Idle", latest == null ? null : number(latest, "connectionsIdle")));
values.add(stat("Idle in transaction", latest == null ? null : number(latest, "connectionsIdleInTx")));
values.add(stat("Cache hit %", latest == null ? null : 100 * number(latest, "cacheHitRatio")));
values.add(stat("Temp MB (24h)", snapshots.isEmpty() ? null : snapshots.stream().mapToDouble(row -> number(row, "tempBytes")).sum() / 1048576));
values.add(stat("Deadlocks (24h)", snapshots.isEmpty() ? null : snapshots.stream().mapToDouble(row -> number(row, "deadlocks")).sum()));
return (new MultiStatisticsData(null, List.of(new MultiStatisticsData.StatisticsGroupData().withHeader("Connection capacity")
.withSubheader(latest == null ? "Capacity appears after the next snapshot." : "Latest snapshot; percentages and MB rounded to whole numbers.").withStatisticList(values))));
}
/*******************************************************************************
** Current connection counts by application, user and state, directly from provider.
*******************************************************************************/
public TableData connectionsByApplication(DbPerfRuntime runtime) throws Exception
{
List<Map<String, Object>> rows = new ArrayList<>();
try(Connection connection = ConnectionManager.getConnection((RDBMSBackendMetaData) runtime.getInstance().getBackend(runtime.getConfig().getBackendName())))
{
for(ConnectionCount count : DbStatsProviders.forConnection(connection).readConnections(connection))
{
rows.add(map("Application", count.applicationName(), "User", count.dbUser(), "State", count.state(), "Connections", count.connectionCount()));
}
}
return (table("No connection counts available for this provider.", rows, "Application", "User", "State", "Connections"));
}
/*******************************************************************************
** Most (unit, table) groups summed for the attribution panel. Production
** shows about a thousand; the cap only guards pathological unit names.
*******************************************************************************/
public static final int MAX_ATTRIBUTION_GROUPS = 5000;
/*******************************************************************************
** Aggregate units over 24 hours and show repeated-statement patterns below them.
** The sums run in the database: a day of attribution detail is hundreds of
** thousands of rows (about 540 MB), and loading it into the heap on a widget
** request ran the web pod out of memory (2026-10-16).
*******************************************************************************/
public MultiTableData attribution() throws QException
{
GroupBy kindGroup = new GroupBy(QFieldType.STRING, "unitKind");
GroupBy nameGroup = new GroupBy(QFieldType.STRING, "unitName");
GroupBy tableGroup = new GroupBy(QFieldType.STRING, "tableName");
Aggregate statementsSum = new Aggregate("statements", AggregateOperator.SUM);
Aggregate totalMsSum = new Aggregate("totalMs", AggregateOperator.SUM);
List<AggregateResult> results = new AggregateAction().execute(new AggregateInput("db_perf_attribution")
.withFilter(since("windowEnd").withOrderBy(new QFilterOrderByAggregate(totalMsSum, false)))
.withGroupBy(kindGroup).withGroupBy(nameGroup).withGroupBy(tableGroup)
.withAggregate(statementsSum).withAggregate(totalMsSum)
.withLimit(MAX_ATTRIBUTION_GROUPS)
.withTransaction(QContext.getQBackendTransaction())).getResults();
Map<String, Map<String, Object>> grouped = new LinkedHashMap<>();
Map<String, Map<String, Double>> tableTimes = new LinkedHashMap<>();
double total = results.stream().mapToDouble(row -> numeric(row.getAggregateValue(totalMsSum))).sum();
for(AggregateResult row : results)
{
String kind = (String) row.getGroupByValue(kindGroup);
String name = (String) row.getGroupByValue(nameGroup);
String key = kind + "\n" + name;
Map<String, Object> group = grouped.computeIfAbsent(key, ignored -> map("Kind", kind, "Name", name, "Statements", 0.0, "Total ms", 0.0));
group.put("Statements", numeric(group.get("Statements")) + numeric(row.getAggregateValue(statementsSum)));
group.put("Total ms", numeric(group.get("Total ms")) + numeric(row.getAggregateValue(totalMsSum)));
tableTimes.computeIfAbsent(key, ignored -> new LinkedHashMap<>()).merge(Objects.toString(row.getGroupByValue(tableGroup), "Unknown"), numeric(row.getAggregateValue(totalMsSum)), Double::sum);
}
grouped.forEach((key, group) ->
{
group.put("Share %", total == 0 ? 0 : 100 * numeric(group.get("Total ms")) / total);
group.put("Top table", tableTimes.get(key).entrySet().stream().max(Map.Entry.comparingByValue()).orElseThrow().getKey());
});
List<Map<String, Object>> units = grouped.values().stream().sorted(Comparator.comparingDouble((Map<String, Object> row) -> numeric(row.get("Total ms"))).reversed())
.map(row -> map("Unit of work", stack(Objects.toString(row.get("Name"), ""), null, kindLabel(row.get("Kind")) + ", top table " + row.get("Top table")),
"Statements", row.get("Statements"), "Total ms (24h)", row.get("Total ms"), "Share %", row.get("Share %"))).toList();
List<Map<String, Object>> repeats = DbPerfStore.query("db_perf_repeat_pattern", since("windowEnd").withOrderBy(new QFilterOrderBy("maxRepeats", false)).withLimit(100)).stream()
.map(row -> map("Unit of work", stack(Objects.toString(row.getValueString("unitName"), ""), null,
kindLabel(row.getValueString("unitKind")) + ": " + truncate(row.getValueString("sqlText"), 160)),
"Most repeats in one unit", row.getValue("maxRepeats"), "Units affected", row.getValue("unitsAffected"), "Total repeats", row.getValue("totalRepeats"))).toList();
TableData unitTable = table("Attribution starts after the next flush", units, "Unit of work", "Statements", "Total ms (24h)", "Share %");
unitTable.getColumns().getFirst().setType("composite");
TableData repeatTable = table("Repeated statements appear after the next attribution flush when the repetition threshold is reached.", repeats,
"Unit of work", "Most repeats in one unit", "Units affected", "Total repeats");
repeatTable.getColumns().getFirst().setType("composite");
return (new MultiTableData(List.of(unitTable, repeatTable)));
}
/*******************************************************************************
** Sequential-scan pressure and seven-day size growth are separate ranked tables.
*******************************************************************************/
public MultiTableData tables(DbPerformanceData data) throws QException
{
List<QRecord> scans = DbPerfStore.query("db_perf_table_stat", data.backendFilter().withOrderBy(new QFilterOrderBy("seqScanShare", false)).withLimit(100));
List<QRecord> growing = DbPerfStore.query("db_perf_table_stat", data.backendFilter().withOrderBy(new QFilterOrderBy("growthBytes7d", false)).withLimit(100));
return (new MultiTableData(List.of(table("Table scans appear after the next table snapshot (normally hourly).", scans.stream()
.map(row -> map("Table", row.getValueString("schemaName") + "." + row.getValueString("tableName"), "Sequential scans", row.getValue("seqScan"),
"Index scans", row.getValue("idxScan"), "Sequential share %", row.getValue("seqScanShare"))).toList(),
"Table", "Sequential scans", "Index scans", "Sequential share %"),
table("Growth appears after table history has accumulated.", growing.stream()
.map(row -> map("Table", row.getValueString("schemaName") + "." + row.getValueString("tableName"), "Size MB", mb(row.getValue("totalBytes")),
"Growth MB (7 days)", mb(row.getValue("growthBytes7d")))).toList(),
"Table", "Size MB", "Growth MB (7 days)"))));
}
/*******************************************************************************
** The core TABLE renderer paginates when fixedHeight is absent.
*******************************************************************************/
private static TableData table(String empty, List<Map<String, Object>> rows, String... columns)
{
return (new TableData(null, Arrays.stream(columns).map(name -> new TableData.Column("default", name, name, null, "left")).toList(), rows)
.withRowsPerPage(10).withHidePaginationDropdown(false).withNoRowsFoundHTML(empty));
}
/*******************************************************************************
** Typed record link, never an HTML fragment.
*******************************************************************************/
private static CompositeWidgetData link(String text, String path)
{
return (new CompositeWidgetData().withLayout(CompositeWidgetData.Layout.FLEX_COLUMN)
.withBlock(new TextBlockData().withValues(new TextValues(text)).withLink(new BlockLink(path))));
}
/*******************************************************************************
** A stacked cell: the main text (a record link when a path is given) over a
** smaller secondary line, so tables keep at most four columns.
*******************************************************************************/
private static CompositeWidgetData stack(String main, String path, String secondary)
{
TextBlockData first = new TextBlockData().withValues(new TextValues(truncate(main, 300)));
if(path != null)
{
first.withLink(new BlockLink(path));
}
CompositeWidgetData cell = new CompositeWidgetData().withLayout(CompositeWidgetData.Layout.FLEX_COLUMN).withBlock(first);
if(secondary != null && !secondary.isBlank())
{
cell.withBlock(new TextBlockData().withValues(new TextValues(truncate(secondary, 300))).withStyles(new TextStyles().withColor("MUTED")));
}
return (cell);
}
/*******************************************************************************
** Plain label for a unit-of-work kind.
*******************************************************************************/
private static String kindLabel(Object kind)
{
return (switch(Objects.toString(kind, ""))
{
case "HTTP_REQUEST" -> "Request";
case "SCHEDULED_JOB" -> "Scheduled job";
case "PROCESS" -> "Process";
default -> "Other";
});
}
/*******************************************************************************
** Text capped with an ellipsis.
*******************************************************************************/
private static String truncate(String value, int max)
{
String text = value == null ? "" : value;
return (text.length() <= max ? text : text.substring(0, max - 1) + "…");
}
/*******************************************************************************
** A number for a secondary line: whole numbers without decimals.
*******************************************************************************/
private static String format(Object value)
{
if(value == null)
{
return ("0");
}
double number = ((Number) value).doubleValue();
return (number == Math.rint(number) ? String.valueOf((long) number) : String.format(Locale.ROOT, "%.1f", number));
}
/*******************************************************************************
** Bytes as whole megabytes.
*******************************************************************************/
private static Long mb(Object bytes)
{
return (bytes == null ? null : Math.round(((Number) bytes).doubleValue() / 1048576));
}
/*******************************************************************************
** Trailing-day filter for time-window records.
*******************************************************************************/
private static QQueryFilter since(String field)
{
return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, Instant.now().minusSeconds(86400))));
}
/*******************************************************************************
** Core multi-statistics values are nullable integers.
*******************************************************************************/
private static MultiStatisticsData.StatisticsGroupData.Statistic stat(String label, Double value)
{
return (new MultiStatisticsData.StatisticsGroupData.Statistic(label, value == null ? null : (int) Math.min(Integer.MAX_VALUE, Math.round(value)), null));
}
/*******************************************************************************
** Numbers in the shared read model retain their numeric type.
*******************************************************************************/
private static double numeric(Object value)
{
return (value == null ? 0 : ((Number) value).doubleValue());
}
}
@@ -0,0 +1,81 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance.widgets;
import java.util.List;
import com.kingsrook.qbits.dbperformance.DbPerfPermissions;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
/*******************************************************************************
** Full-width typed home panels, with one panel per row and bounded refresh rates.
*******************************************************************************/
public final class DbPerfWidgets
{
public static final List<String> HOME = List.of("dbPerfSetup", "dbPerfDatabaseTime", "dbPerfTopQueries", "dbPerfFindings",
"dbPerfLiveActivity", "dbPerfConnections", "dbPerfAttribution", "dbPerfTables");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DbPerfWidgets()
{
}
/*******************************************************************************
** Additive registration, preserving phase A record-child widgets and app sections.
*******************************************************************************/
public static void register(QInstance instance, DbPerfQBitConfig config)
{
add(instance, config, HOME.get(0), "Setup", WidgetType.TABLE, 60);
add(instance, config, HOME.get(1), "Database time", WidgetType.LINE_CHART, 60);
add(instance, config, HOME.get(2), "Top queries", WidgetType.TABLE, 60);
add(instance, config, HOME.get(3), "Findings", WidgetType.TABLE, 60);
add(instance, config, HOME.get(4), "Live activity", WidgetType.TABLE, 30);
add(instance, config, HOME.get(5), "Connections", WidgetType.PARENT_WIDGET, 30);
add(instance, config, "dbPerfConnectionStatistics", "Connection capacity", WidgetType.MULTI_STATISTICS, 30);
add(instance, config, "dbPerfConnectionsByApplication", "Connections by application and user", WidgetType.TABLE, 30);
add(instance, config, HOME.get(6), "QQQ attribution", WidgetType.MULTI_TABLE, 60);
add(instance, config, HOME.get(7), "Tables", WidgetType.MULTI_TABLE, 60);
instance.getApp(config.getAppName()).withWidgets(HOME);
}
/*******************************************************************************
** Permission and responsive width also apply when a widget is requested directly.
*******************************************************************************/
private static void add(QInstance instance, DbPerfQBitConfig config, String name, String label, WidgetType type, int refresh)
{
instance.addWidget(new QWidgetMetaData().withName(name).withLabel(label).withType(type.getType()).withGridColumns(12)
.withCodeReference(new QCodeReference(DbPerfWidgetRenderer.class)).withDefaultValue("refreshSeconds", refresh)
.withPermissionRules(DbPerfPermissions.rules(config.getPermissionBaseName() + ".view")));
}
}
@@ -0,0 +1,611 @@
-- liquibase formatted sql
-- changeset qbit-db-performance:1
-- comment: Settings table.
CREATE TABLE IF NOT EXISTS db_perf_settings (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100) NOT NULL,
enabled BOOLEAN DEFAULT TRUE,
provider VARCHAR(20),
snapshot_interval_minutes INT DEFAULT 5,
table_snapshot_minutes INT DEFAULT 60,
detail_retention_days INT DEFAULT 30,
rollup_retention_days INT DEFAULT 365,
top_statements_per_snapshot INT DEFAULT 200,
activity_long_running_seconds INT DEFAULT 30,
idle_in_transaction_seconds INT DEFAULT 60,
attribution_enabled BOOLEAN DEFAULT TRUE,
attribution_sample_rate DECIMAL(5,4) DEFAULT 1.0,
attribution_flush_seconds INT DEFAULT 60,
attribution_max_keys INT DEFAULT 10000,
n_plus_one_threshold INT DEFAULT 10,
explain_timeout_ms INT DEFAULT 10000,
alerts_enabled BOOLEAN DEFAULT TRUE,
spike_factor DECIMAL(6,2) DEFAULT 3,
spike_min_total_ms INT DEFAULT 5000,
connection_alert_percent INT DEFAULT 80,
blocked_alert_seconds INT DEFAULT 30,
last_snapshot_at TIMESTAMP,
last_table_snapshot_at TIMESTAMP,
stats_access VARCHAR(30),
stats_access_message VARCHAR(500),
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_settings_0 UNIQUE (backend_name)
);
-- changeset qbit-db-performance:2
-- comment: Suggestion Rules table.
CREATE TABLE IF NOT EXISTS db_perf_rule (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
code VARCHAR(40) NOT NULL,
label VARCHAR(100),
description VARCHAR(1000),
enabled BOOLEAN DEFAULT TRUE,
severity VARCHAR(20),
threshold_1 DECIMAL(18,4),
threshold_2 DECIMAL(18,4),
threshold_1_label VARCHAR(100),
threshold_2_label VARCHAR(100),
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_rule_0 UNIQUE (code)
);
-- changeset qbit-db-performance:3
-- comment: Snapshots table.
CREATE TABLE IF NOT EXISTS db_perf_snapshot (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100),
captured_at TIMESTAMP NOT NULL,
interval_seconds INT,
status VARCHAR(20),
duration_ms INT,
error_message VARCHAR(2000),
stats_reset_detected BOOLEAN,
includes_tables BOOLEAN,
xact_commit BIGINT,
xact_rollback BIGINT,
blks_hit BIGINT,
blks_read BIGINT,
cache_hit_ratio DECIMAL(7,4),
temp_files BIGINT,
temp_bytes BIGINT,
deadlocks BIGINT,
conflicts BIGINT,
connections_total INT,
connections_active INT,
connections_idle INT,
connections_idle_in_tx INT,
max_connections INT,
database_size_bytes BIGINT,
statement_count INT,
statement_calls BIGINT,
statement_total_ms DECIMAL(18,3),
stored_statement_count INT,
create_date TIMESTAMP,
raw_xact_commit BIGINT,
raw_xact_rollback BIGINT,
raw_blks_hit BIGINT,
raw_blks_read BIGINT,
raw_temp_files BIGINT,
raw_temp_bytes BIGINT,
raw_deadlocks BIGINT,
raw_conflicts BIGINT,
raw_stats_reset TIMESTAMP,
raw_statement_stats_reset TIMESTAMP,
CONSTRAINT uq_snapshot_0 UNIQUE (backend_name,captured_at)
);
CREATE INDEX IF NOT EXISTS ix_snapshot_0 ON db_perf_snapshot (captured_at);
-- changeset qbit-db-performance:4
-- comment: Queries table.
CREATE TABLE IF NOT EXISTS db_perf_statement (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100),
query_id BIGINT,
db_user VARCHAR(100),
toplevel BOOLEAN,
fingerprint CHAR(64),
query_text TEXT,
operation VARCHAR(20),
main_table VARCHAR(200),
first_seen_at TIMESTAMP,
last_seen_at TIMESTAMP,
last_calls BIGINT,
last_total_exec_ms DECIMAL(18,3),
last_rows BIGINT,
last_shared_blks_hit BIGINT,
last_shared_blks_read BIGINT,
last_temp_blks_written BIGINT,
last_max_exec_ms DECIMAL(18,3),
last_stats_since TIMESTAMP,
calls_24h BIGINT,
total_ms_24h DECIMAL(18,3),
mean_ms_24h DECIMAL(18,3),
share_of_db_time_24h DECIMAL(7,4),
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_statement_0 UNIQUE (backend_name,query_id,db_user,toplevel)
);
CREATE INDEX IF NOT EXISTS ix_statement_0 ON db_perf_statement (fingerprint);
CREATE INDEX IF NOT EXISTS ix_statement_1 ON db_perf_statement (last_seen_at);
CREATE INDEX IF NOT EXISTS ix_statement_2 ON db_perf_statement (total_ms_24h);
-- changeset qbit-db-performance:5
-- comment: Query Snapshots table.
CREATE TABLE IF NOT EXISTS db_perf_query_snapshot (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
snapshot_id BIGINT,
statement_id BIGINT,
captured_at TIMESTAMP,
calls BIGINT,
total_ms DECIMAL(18,3),
mean_ms DECIMAL(18,3),
max_ms DECIMAL(18,3),
rows_count BIGINT,
shared_blks_hit BIGINT,
shared_blks_read BIGINT,
hit_ratio DECIMAL(7,4),
temp_blks_written BIGINT,
share_of_db_time DECIMAL(7,4)
);
CREATE INDEX IF NOT EXISTS ix_query_snapshot_0 ON db_perf_query_snapshot (snapshot_id);
CREATE INDEX IF NOT EXISTS ix_query_snapshot_1 ON db_perf_query_snapshot (statement_id);
CREATE INDEX IF NOT EXISTS ix_query_snapshot_2 ON db_perf_query_snapshot (captured_at);
CREATE INDEX IF NOT EXISTS ix_query_snapshot_3 ON db_perf_query_snapshot (statement_id,captured_at);
-- changeset qbit-db-performance:6
-- comment: Query Daily table.
CREATE TABLE IF NOT EXISTS db_perf_query_daily (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
statement_id BIGINT,
"day" DATE,
calls BIGINT,
total_ms DECIMAL(18,3),
mean_ms DECIMAL(18,3),
max_ms DECIMAL(18,3),
rows_count BIGINT,
hit_ratio DECIMAL(7,4),
share_of_db_time DECIMAL(7,4),
snapshots INT,
CONSTRAINT uq_query_daily_0 UNIQUE (statement_id,"day")
);
CREATE INDEX IF NOT EXISTS ix_query_daily_0 ON db_perf_query_daily (statement_id);
CREATE INDEX IF NOT EXISTS ix_query_daily_1 ON db_perf_query_daily ("day");
-- changeset qbit-db-performance:7
-- comment: Tables table.
CREATE TABLE IF NOT EXISTS db_perf_table_stat (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100),
schema_name VARCHAR(100),
table_name VARCHAR(200),
relid BIGINT,
seq_scan BIGINT,
seq_tup_read BIGINT,
idx_scan BIGINT,
idx_tup_fetch BIGINT,
n_tup_ins BIGINT,
n_tup_upd BIGINT,
n_tup_del BIGINT,
n_tup_hot_upd BIGINT,
n_live_tup BIGINT,
n_dead_tup BIGINT,
dead_ratio DECIMAL(7,4),
total_bytes BIGINT,
table_bytes BIGINT,
index_bytes BIGINT,
toast_bytes BIGINT,
growth_bytes_7d BIGINT,
last_vacuum TIMESTAMP,
last_autovacuum TIMESTAMP,
last_analyze TIMESTAMP,
last_autoanalyze TIMESTAMP,
vacuum_count BIGINT,
autovacuum_count BIGINT,
analyze_count BIGINT,
autoanalyze_count BIGINT,
freeze_age BIGINT,
seq_scan_share DECIMAL(7,4),
last_seen_at TIMESTAMP,
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_table_stat_0 UNIQUE (backend_name,schema_name,table_name)
);
CREATE INDEX IF NOT EXISTS ix_table_stat_0 ON db_perf_table_stat (last_seen_at);
-- changeset qbit-db-performance:8
-- comment: Table Snapshots table.
CREATE TABLE IF NOT EXISTS db_perf_table_snapshot (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
snapshot_id BIGINT,
table_stat_id BIGINT,
captured_at TIMESTAMP,
seq_scan BIGINT,
seq_tup_read BIGINT,
idx_scan BIGINT,
idx_tup_fetch BIGINT,
n_tup_ins BIGINT,
n_tup_upd BIGINT,
n_tup_del BIGINT,
n_tup_hot_upd BIGINT,
n_live_tup BIGINT,
n_dead_tup BIGINT,
total_bytes BIGINT,
table_bytes BIGINT,
index_bytes BIGINT,
autovacuum_count BIGINT,
autoanalyze_count BIGINT,
freeze_age BIGINT
);
CREATE INDEX IF NOT EXISTS ix_table_snapshot_0 ON db_perf_table_snapshot (snapshot_id);
CREATE INDEX IF NOT EXISTS ix_table_snapshot_1 ON db_perf_table_snapshot (table_stat_id);
CREATE INDEX IF NOT EXISTS ix_table_snapshot_2 ON db_perf_table_snapshot (captured_at);
CREATE INDEX IF NOT EXISTS ix_table_snapshot_3 ON db_perf_table_snapshot (table_stat_id,captured_at);
-- changeset qbit-db-performance:9
-- comment: Table Daily table.
CREATE TABLE IF NOT EXISTS db_perf_table_daily (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
table_stat_id BIGINT,
"day" DATE,
seq_scan BIGINT,
idx_scan BIGINT,
seq_tup_read BIGINT,
n_tup_ins BIGINT,
n_tup_upd BIGINT,
n_tup_del BIGINT,
n_live_tup BIGINT,
n_dead_tup BIGINT,
total_bytes BIGINT,
CONSTRAINT uq_table_daily_0 UNIQUE (table_stat_id,"day")
);
CREATE INDEX IF NOT EXISTS ix_table_daily_0 ON db_perf_table_daily (table_stat_id);
CREATE INDEX IF NOT EXISTS ix_table_daily_1 ON db_perf_table_daily ("day");
-- changeset qbit-db-performance:10
-- comment: Indexes table.
CREATE TABLE IF NOT EXISTS db_perf_index_stat (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100),
table_stat_id BIGINT,
schema_name VARCHAR(100),
table_name VARCHAR(200),
index_name VARCHAR(200),
relid BIGINT,
definition VARCHAR(2000),
"columns" VARCHAR(1000),
is_unique BOOLEAN,
is_primary BOOLEAN,
is_valid BOOLEAN,
idx_scan BIGINT,
idx_tup_read BIGINT,
idx_tup_fetch BIGINT,
size_bytes BIGINT,
write_cost BIGINT,
scans_since_reset BIGINT,
last_seen_at TIMESTAMP,
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_index_stat_0 UNIQUE (backend_name,schema_name,index_name)
);
CREATE INDEX IF NOT EXISTS ix_index_stat_0 ON db_perf_index_stat (table_stat_id);
CREATE INDEX IF NOT EXISTS ix_index_stat_1 ON db_perf_index_stat (last_seen_at);
-- changeset qbit-db-performance:11
-- comment: Activity Samples table.
CREATE TABLE IF NOT EXISTS db_perf_activity_sample (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
snapshot_id BIGINT,
captured_at TIMESTAMP,
pid INT,
db_user VARCHAR(100),
application_name VARCHAR(200),
client_addr VARCHAR(100),
state VARCHAR(40),
wait_event_type VARCHAR(60),
wait_event VARCHAR(100),
backend_type VARCHAR(60),
xact_start TIMESTAMP,
query_start TIMESTAMP,
state_change TIMESTAMP,
query_seconds DECIMAL(12,3),
xact_seconds DECIMAL(12,3),
blocked_by VARCHAR(500),
blocking_count INT,
reason VARCHAR(30),
query_id BIGINT,
statement_id BIGINT,
query_fingerprint CHAR(64),
query_text_normalized VARCHAR(4000)
);
CREATE INDEX IF NOT EXISTS ix_activity_sample_0 ON db_perf_activity_sample (snapshot_id);
CREATE INDEX IF NOT EXISTS ix_activity_sample_1 ON db_perf_activity_sample (captured_at);
CREATE INDEX IF NOT EXISTS ix_activity_sample_2 ON db_perf_activity_sample (statement_id);
-- changeset qbit-db-performance:12
-- comment: Connection Samples table.
CREATE TABLE IF NOT EXISTS db_perf_connection_sample (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
snapshot_id BIGINT,
captured_at TIMESTAMP,
db_user VARCHAR(100),
application_name VARCHAR(200),
state VARCHAR(40),
connection_count INT
);
CREATE INDEX IF NOT EXISTS ix_connection_sample_0 ON db_perf_connection_sample (snapshot_id);
CREATE INDEX IF NOT EXISTS ix_connection_sample_1 ON db_perf_connection_sample (captured_at);
-- changeset qbit-db-performance:13
-- comment: QQQ Attribution table.
CREATE TABLE IF NOT EXISTS db_perf_attribution (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
window_start TIMESTAMP,
window_end TIMESTAMP,
pod_name VARCHAR(200),
unit_kind VARCHAR(20),
unit_name VARCHAR(300),
process_name VARCHAR(200),
table_name VARCHAR(200),
operation VARCHAR(20),
fingerprint CHAR(64),
sql_text VARCHAR(4000),
units BIGINT,
statements BIGINT,
total_ms DECIMAL(18,3),
max_ms DECIMAL(18,3),
rows_count BIGINT,
errors BIGINT,
sample_rate DECIMAL(5,4)
);
CREATE INDEX IF NOT EXISTS ix_attribution_0 ON db_perf_attribution (window_start);
CREATE INDEX IF NOT EXISTS ix_attribution_1 ON db_perf_attribution (fingerprint);
CREATE INDEX IF NOT EXISTS ix_attribution_2 ON db_perf_attribution (unit_kind,unit_name);
-- changeset qbit-db-performance:14
-- comment: QQQ Attribution Daily table.
CREATE TABLE IF NOT EXISTS db_perf_attribution_daily (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
"day" DATE,
rollup_key CHAR(64),
unit_kind VARCHAR(20),
unit_name VARCHAR(300),
process_name VARCHAR(200),
table_name VARCHAR(200),
operation VARCHAR(20),
fingerprint CHAR(64),
sql_text VARCHAR(4000),
units BIGINT,
statements BIGINT,
total_ms DECIMAL(18,3),
max_ms DECIMAL(18,3),
rows_count BIGINT,
errors BIGINT,
CONSTRAINT uq_attribution_daily_0 UNIQUE ("day",rollup_key)
);
CREATE INDEX IF NOT EXISTS ix_attribution_daily_0 ON db_perf_attribution_daily ("day");
CREATE INDEX IF NOT EXISTS ix_attribution_daily_1 ON db_perf_attribution_daily (fingerprint);
-- changeset qbit-db-performance:15
-- comment: Repeated Statements table.
CREATE TABLE IF NOT EXISTS db_perf_repeat_pattern (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
window_start TIMESTAMP,
window_end TIMESTAMP,
pod_name VARCHAR(200),
unit_kind VARCHAR(20),
unit_name VARCHAR(300),
process_name VARCHAR(200),
table_name VARCHAR(200),
fingerprint CHAR(64),
sql_text VARCHAR(4000),
units_affected BIGINT,
max_repeats INT,
total_repeats BIGINT
);
CREATE INDEX IF NOT EXISTS ix_repeat_pattern_0 ON db_perf_repeat_pattern (window_start);
CREATE INDEX IF NOT EXISTS ix_repeat_pattern_1 ON db_perf_repeat_pattern (fingerprint);
-- changeset qbit-db-performance:16
-- comment: Findings table.
CREATE TABLE IF NOT EXISTS db_perf_finding (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
backend_name VARCHAR(100),
rule_id BIGINT,
rule_code VARCHAR(40),
dedupe_key VARCHAR(300),
status VARCHAR(20) DEFAULT 'OPEN',
severity VARCHAR(20),
score DECIMAL(12,4),
title VARCHAR(300),
summary VARCHAR(2000),
evidence TEXT,
recommendation TEXT,
migration_snippet TEXT,
subject_table VARCHAR(200),
statement_id BIGINT,
table_stat_id BIGINT,
index_stat_id BIGINT,
unit_name VARCHAR(300),
first_seen_at TIMESTAMP,
last_seen_at TIMESTAMP,
occurrences INT,
resolved_at TIMESTAMP,
status_note VARCHAR(2000),
status_changed_by VARCHAR(100),
create_date TIMESTAMP,
modify_date TIMESTAMP,
CONSTRAINT uq_finding_0 UNIQUE (dedupe_key)
);
CREATE INDEX IF NOT EXISTS ix_finding_0 ON db_perf_finding (rule_id);
CREATE INDEX IF NOT EXISTS ix_finding_1 ON db_perf_finding (statement_id);
CREATE INDEX IF NOT EXISTS ix_finding_2 ON db_perf_finding (table_stat_id);
CREATE INDEX IF NOT EXISTS ix_finding_3 ON db_perf_finding (index_stat_id);
CREATE INDEX IF NOT EXISTS ix_finding_4 ON db_perf_finding (last_seen_at);
CREATE INDEX IF NOT EXISTS ix_finding_5 ON db_perf_finding (resolved_at);
CREATE INDEX IF NOT EXISTS ix_finding_6 ON db_perf_finding (status,score);
-- changeset qbit-db-performance:17
-- comment: Query Plans table.
CREATE TABLE IF NOT EXISTS db_perf_explain (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
statement_id BIGINT,
requested_by VARCHAR(100),
requested_at TIMESTAMP,
mode VARCHAR(20),
sql_fingerprint CHAR(64),
sql_text TEXT,
parameter_count INT,
duration_ms INT,
outcome VARCHAR(20),
error_message VARCHAR(2000),
plan_json TEXT,
plan_text TEXT,
total_cost DECIMAL(18,3)
);
CREATE INDEX IF NOT EXISTS ix_explain_0 ON db_perf_explain (statement_id);
CREATE INDEX IF NOT EXISTS ix_explain_1 ON db_perf_explain (requested_at);
-- changeset qbit-db-performance:18
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_query_snapshot_snapshot_id' AND LOWER(TABLE_NAME) = 'db_perf_query_snapshot' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Query Snapshots reference to Snapshots.
ALTER TABLE db_perf_query_snapshot ADD CONSTRAINT fk_query_snapshot_snapshot_id FOREIGN KEY (snapshot_id) REFERENCES db_perf_snapshot(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:19
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_query_snapshot_statement_id' AND LOWER(TABLE_NAME) = 'db_perf_query_snapshot' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Query Snapshots reference to Queries.
ALTER TABLE db_perf_query_snapshot ADD CONSTRAINT fk_query_snapshot_statement_id FOREIGN KEY (statement_id) REFERENCES db_perf_statement(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:20
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_query_daily_statement_id' AND LOWER(TABLE_NAME) = 'db_perf_query_daily' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Query Daily reference to Queries.
ALTER TABLE db_perf_query_daily ADD CONSTRAINT fk_query_daily_statement_id FOREIGN KEY (statement_id) REFERENCES db_perf_statement(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:21
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_table_snapshot_snapshot_id' AND LOWER(TABLE_NAME) = 'db_perf_table_snapshot' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Table Snapshots reference to Snapshots.
ALTER TABLE db_perf_table_snapshot ADD CONSTRAINT fk_table_snapshot_snapshot_id FOREIGN KEY (snapshot_id) REFERENCES db_perf_snapshot(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:22
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_table_snapshot_table_stat_id' AND LOWER(TABLE_NAME) = 'db_perf_table_snapshot' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Table Snapshots reference to Tables.
ALTER TABLE db_perf_table_snapshot ADD CONSTRAINT fk_table_snapshot_table_stat_id FOREIGN KEY (table_stat_id) REFERENCES db_perf_table_stat(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:23
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_table_daily_table_stat_id' AND LOWER(TABLE_NAME) = 'db_perf_table_daily' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Table Daily reference to Tables.
ALTER TABLE db_perf_table_daily ADD CONSTRAINT fk_table_daily_table_stat_id FOREIGN KEY (table_stat_id) REFERENCES db_perf_table_stat(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:24
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_index_stat_table_stat_id' AND LOWER(TABLE_NAME) = 'db_perf_index_stat' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Indexes reference to Tables.
ALTER TABLE db_perf_index_stat ADD CONSTRAINT fk_index_stat_table_stat_id FOREIGN KEY (table_stat_id) REFERENCES db_perf_table_stat(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:25
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_activity_sample_snapshot_id' AND LOWER(TABLE_NAME) = 'db_perf_activity_sample' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Activity Samples reference to Snapshots.
ALTER TABLE db_perf_activity_sample ADD CONSTRAINT fk_activity_sample_snapshot_id FOREIGN KEY (snapshot_id) REFERENCES db_perf_snapshot(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:26
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_activity_sample_statement_id' AND LOWER(TABLE_NAME) = 'db_perf_activity_sample' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Activity Samples reference to Queries.
ALTER TABLE db_perf_activity_sample ADD CONSTRAINT fk_activity_sample_statement_id FOREIGN KEY (statement_id) REFERENCES db_perf_statement(id) ON DELETE SET NULL;
-- changeset qbit-db-performance:27
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_connection_sample_snapshot_id' AND LOWER(TABLE_NAME) = 'db_perf_connection_sample' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Connection Samples reference to Snapshots.
ALTER TABLE db_perf_connection_sample ADD CONSTRAINT fk_connection_sample_snapshot_id FOREIGN KEY (snapshot_id) REFERENCES db_perf_snapshot(id) ON DELETE CASCADE;
-- changeset qbit-db-performance:28
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_finding_rule_id' AND LOWER(TABLE_NAME) = 'db_perf_finding' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Findings reference to Suggestion Rules.
ALTER TABLE db_perf_finding ADD CONSTRAINT fk_finding_rule_id FOREIGN KEY (rule_id) REFERENCES db_perf_rule(id) ON DELETE RESTRICT;
-- changeset qbit-db-performance:29
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_finding_statement_id' AND LOWER(TABLE_NAME) = 'db_perf_finding' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Findings reference to Queries.
ALTER TABLE db_perf_finding ADD CONSTRAINT fk_finding_statement_id FOREIGN KEY (statement_id) REFERENCES db_perf_statement(id) ON DELETE SET NULL;
-- changeset qbit-db-performance:30
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_finding_table_stat_id' AND LOWER(TABLE_NAME) = 'db_perf_finding' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Findings reference to Tables.
ALTER TABLE db_perf_finding ADD CONSTRAINT fk_finding_table_stat_id FOREIGN KEY (table_stat_id) REFERENCES db_perf_table_stat(id) ON DELETE SET NULL;
-- changeset qbit-db-performance:31
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_finding_index_stat_id' AND LOWER(TABLE_NAME) = 'db_perf_finding' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Findings reference to Indexes.
ALTER TABLE db_perf_finding ADD CONSTRAINT fk_finding_index_stat_id FOREIGN KEY (index_stat_id) REFERENCES db_perf_index_stat(id) ON DELETE SET NULL;
-- changeset qbit-db-performance:32
-- preconditions onFail:MARK_RAN
-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE LOWER(CONSTRAINT_NAME) = 'fk_explain_statement_id' AND LOWER(TABLE_NAME) = 'db_perf_explain' AND LOWER(TABLE_SCHEMA) = LOWER(CURRENT_SCHEMA())
-- comment: Query Plans reference to Queries.
ALTER TABLE db_perf_explain ADD CONSTRAINT fk_explain_statement_id FOREIGN KEY (statement_id) REFERENCES db_perf_statement(id) ON DELETE SET NULL;
-- changeset qbit-db-performance:33
-- comment: Suggestion Rules insert-only defaults; administrator edits survive.
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'SLOW_QUERY', 'Slow query', 'Statements whose average run time is high, or that use a large share of all database time over the last 24 hours. Open the query and run Query plan to see why.', TRUE, 'WARNING', 500, 'Average run time (ms)', 5, 'Share of all database time (%)' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'SLOW_QUERY');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'MISSING_INDEX', 'Possible missing index', 'Tables that are mostly read by full sequential scans of many rows, where the busiest queries filter on columns no index starts with. The finding includes a CREATE INDEX CONCURRENTLY changeset to review and commit.', TRUE, 'WARNING', 50, 'Share of scans that are sequential (%)', 1000000, 'Rows read by sequential scans per day' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'MISSING_INDEX');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'UNUSED_INDEX', 'Unused index', 'Non-unique indexes that no query has used since statistics were last reset. Every write still pays to maintain them. The finding includes a DROP INDEX CONCURRENTLY changeset with a rollback.', TRUE, 'WARNING', 14, 'Days of statistics before judging', 1, 'Smallest index size to report (MB)' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'UNUSED_INDEX');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'DUPLICATE_INDEX', 'Duplicate index', 'Indexes with the same columns as another index on the table, or whose columns are a leading prefix of another non-unique index. Unique and primary key indexes are never proposed for removal.', TRUE, 'WARNING', NULL, NULL, NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'DUPLICATE_INDEX');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'N_PLUS_ONE', 'Repeated statements (N+1)', 'The same statement shape run many times within one request, process or scheduled job, usually a lookup inside a loop. Batch it into one query with IN (...) or a join.', TRUE, 'WARNING', 10, 'Repeats in one unit of work', 5, 'Units of work affected per day' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'N_PLUS_ONE');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'HOT_JOB', 'Expensive job or process', 'Scheduled jobs and processes that account for a large share of the database time QQQ issued in the last 24 hours.', TRUE, 'WARNING', 10, 'Share of attributed database time (%)', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'HOT_JOB');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'OVER_AUDITED', 'High audit volume', 'QQQ audit tables that grow by many rows a day. Lowering the audit level of the noisiest tables reduces writes and storage.', TRUE, 'WARNING', 50000, 'Audit rows inserted per day', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'OVER_AUDITED');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'BLOAT', 'Dead rows (bloat)', 'Tables where a large share of rows are dead versions waiting for vacuum. Bloat slows scans and wastes space. Tune autovacuum for the table.', TRUE, 'WARNING', 20, 'Dead rows as a share of live rows (%)', 10000, 'Fewest dead rows to report' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'BLOAT');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'VACUUM_STALE', 'Vacuum overdue', 'Tables with recent writes that have not been vacuumed for a long time, or whose oldest transaction ID is getting close to the freeze limit.', TRUE, 'WARNING', 7, 'Days since the last vacuum', 150000000, 'Transaction ID age (freeze age)' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'VACUUM_STALE');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'POOL_SIZING', 'Connection pool sizing', 'Peak connections close to max_connections, or many idle connections held by an application. The finding suggests pool sizes per application.', TRUE, 'WARNING', 80, 'Peak connections as a share of max_connections (%)', 50, 'Idle connections as a share of all connections (%)' WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'POOL_SIZING');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'IDLE_IN_TRANSACTION', 'Idle in transaction', 'Sessions that opened a transaction and then sat idle, holding locks and blocking vacuum. Fix the transaction scope in the application.', TRUE, 'WARNING', 60, 'Seconds idle in a transaction', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'IDLE_IN_TRANSACTION');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'CACHE_HIT_LOW', 'Low cache hit ratio', 'The share of block reads served from shared buffers fell below the threshold over the last 24 hours, so more reads go to disk.', TRUE, 'WARNING', 95, 'Cache hit ratio (%)', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'CACHE_HIT_LOW');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'TEMP_SPILL', 'Temporary disk spill', 'Statements that write temporary files because sorts or hashes do not fit in work_mem.', TRUE, 'WARNING', 100, 'Temporary data written per day (MB)', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'TEMP_SPILL');
INSERT INTO db_perf_rule (code,label,description,enabled,severity,threshold_1,threshold_1_label,threshold_2,threshold_2_label)
SELECT 'DEADLOCKS', 'Deadlocks', 'Transactions that deadlocked and were cancelled by PostgreSQL. Usually two code paths that lock rows in a different order.', TRUE, 'WARNING', 1, 'Deadlocks per day', NULL, NULL WHERE NOT EXISTS (SELECT 1 FROM db_perf_rule WHERE code = 'DEADLOCKS');
-- changeset qbit-db-performance:34
-- comment: Repeated Statements: the sampling rate of each window, so repeat counts can be read at scale.
ALTER TABLE db_perf_repeat_pattern ADD COLUMN IF NOT EXISTS sample_rate DECIMAL(5,4);
@@ -0,0 +1 @@
${project.version}
@@ -0,0 +1,972 @@
bundle: qbit-db-performance
version: ${project.version}
layer: QBIT
entries:
- key: app:dbPerformance;slot:help
summary: "Find expensive queries, inspect query plans and review database capacity."
content: |
Start with Setup. Run {{label:dbPerfCollectNow}} from Snapshots to establish the first baseline. Two collections produce useful statement deltas. Table and index data normally refresh hourly; the dashboard refreshes every 30 to 60 seconds.
Use {{label:dbPerfQueryPlan}} on a query to inspect its plan. PLAN does not execute the query; generic plans require PostgreSQL 16+. ANALYZE executes a read-only SELECT with a timeout and always rolls back. Parameters and ANALYZE plans are never stored. Raw live SQL is permission-gated and never persisted.
All data lives in standard QQQ tables. The host owns scheduler registration and permissions. Attribution and findings require their corresponding runtime collectors; empty panels explain when data will arrive.
- key: table:db_perf_activity_sample;slot:help
summary: "Notable session samples with normalized text, never raw query text."
content: |
The collector samples long-running, blocked and idle-in-transaction sessions every collection. Quiet databases can remain empty. Retained for detailRetentionDays (30). Use Live activity for current sessions.
- key: table:db_perf_activity_sample;slot:emptyState
summary: "Notable session samples with normalized text, never raw query text."
content: |
The collector samples long-running, blocked and idle-in-transaction sessions every collection. Quiet databases can remain empty. Retained for detailRetentionDays (30). Use Live activity for current sessions.
- key: table:db_perf_activity_sample;field:snapshotId
summary: "Reference to the related snapshot record."
- key: table:db_perf_activity_sample;field:capturedAt
summary: "Time this database sample was captured."
- key: table:db_perf_activity_sample;field:pid
summary: "Pid reported for this activity sample record."
- key: table:db_perf_activity_sample;field:dbUser
summary: "Database role associated with this statement, session or connection group."
- key: table:db_perf_activity_sample;field:applicationName
summary: "Client application name reported by PostgreSQL."
- key: table:db_perf_activity_sample;field:clientAddr
summary: "Client address reported by the statistics provider."
- key: table:db_perf_activity_sample;field:state
summary: "State reported for this activity sample record."
- key: table:db_perf_activity_sample;field:waitEventType
summary: "PostgreSQL category of the current wait, such as Lock or IO."
- key: table:db_perf_activity_sample;field:waitEvent
summary: "PostgreSQL event on which this backend is currently waiting."
- key: table:db_perf_activity_sample;field:backendType
summary: "Backend type reported for this activity sample record."
- key: table:db_perf_activity_sample;field:xactStart
summary: "Xact start reported for this activity sample record."
- key: table:db_perf_activity_sample;field:queryStart
summary: "Query start reported for this activity sample record."
- key: table:db_perf_activity_sample;field:stateChange
summary: "State change reported for this activity sample record."
- key: table:db_perf_activity_sample;field:querySeconds
summary: "Age of the current query in seconds."
- key: table:db_perf_activity_sample;field:xactSeconds
summary: "Age of the current transaction in seconds."
- key: table:db_perf_activity_sample;field:blockedBy
summary: "Blocking process identifiers reported by PostgreSQL."
- key: table:db_perf_activity_sample;field:blockingCount
summary: "Number of sessions directly blocked by this session."
- key: table:db_perf_activity_sample;field:reason
summary: "Reason reported for this activity sample record."
- key: table:db_perf_activity_sample;field:queryId
summary: "PostgreSQL query identifier; fingerprint provides cross-source normalized matching."
- key: table:db_perf_activity_sample;field:statementId
summary: "Reference to the related statement record."
- key: table:db_perf_activity_sample;field:queryFingerprint
summary: "Query fingerprint reported for this activity sample record."
- key: table:db_perf_activity_sample;field:queryTextNormalized
summary: "Query text normalized reported for this activity sample record."
- key: table:db_perf_attribution;slot:help
summary: "QQQ request, process and job database time by statement shape."
content: |
The host attribution listener flushes one batch per attributionFlushSeconds (60 by default). Retained for detailRetentionDays (30). Attribution starts after the next flush; enable the listener in the host if empty.
- key: table:db_perf_attribution;slot:emptyState
summary: "QQQ request, process and job database time by statement shape."
content: |
The host attribution listener flushes one batch per attributionFlushSeconds (60 by default). Retained for detailRetentionDays (30). Attribution starts after the next flush; enable the listener in the host if empty.
- key: table:db_perf_attribution;field:windowStart
summary: "Beginning of the attribution aggregation window."
- key: table:db_perf_attribution;field:windowEnd
summary: "End of the attribution aggregation window."
- key: table:db_perf_attribution;field:podName
summary: "Application pod that observed and flushed this attribution window."
- key: table:db_perf_attribution;field:unitKind
summary: "Kind of QQQ unit of work, such as route, process or scheduled job."
- key: table:db_perf_attribution;field:unitName
summary: "Name of the QQQ unit responsible for database work."
- key: table:db_perf_attribution;field:processName
summary: "QQQ process responsible for the attributed work, when known."
- key: table:db_perf_attribution;field:tableName
summary: "Table name reported for this attribution record."
- key: table:db_perf_attribution;field:operation
summary: "Normalized SQL operation, such as SELECT, INSERT, UPDATE or DELETE."
- key: table:db_perf_attribution;field:fingerprint
summary: "SHA-256 of the normalized SQL shape; literals and bind values are excluded."
- key: table:db_perf_attribution;field:sqlText
summary: "Normalized SQL shape without literal values. Parameter values are never stored."
- key: table:db_perf_attribution;field:units
summary: "Number of sampled QQQ units contributing to this aggregate."
- key: table:db_perf_attribution;field:statements
summary: "Number of attributed SQL executions during the window."
- key: table:db_perf_attribution;field:totalMs
summary: "Total ms in milliseconds for this measurement."
- key: table:db_perf_attribution;field:maxMs
summary: "Max ms in milliseconds for this measurement."
- key: table:db_perf_attribution;field:rowsCount
summary: "Rows processed during the measured execution window."
- key: table:db_perf_attribution;field:errors
summary: "Number of failed SQL executions observed in the attribution window."
- key: table:db_perf_attribution;field:sampleRate
summary: "Fraction of units sampled in this attribution window."
- key: table:db_perf_attribution_daily;slot:help
summary: "Daily database-time attribution for QQQ units of work."
content: |
The daily rollup aggregates flushed attribution windows at 03:15 by default. Retained for rollupRetentionDays (365). Wait for a flush and the next daily rollup when empty.
- key: table:db_perf_attribution_daily;slot:emptyState
summary: "Daily database-time attribution for QQQ units of work."
content: |
The daily rollup aggregates flushed attribution windows at 03:15 by default. Retained for rollupRetentionDays (365). Wait for a flush and the next daily rollup when empty.
- key: table:db_perf_attribution_daily;field:day
summary: "Calendar day in the configured qbit time zone for this rollup."
- key: table:db_perf_attribution_daily;field:rollupKey
summary: "Stable identity used to replace a daily aggregate idempotently."
- key: table:db_perf_attribution_daily;field:unitKind
summary: "Kind of QQQ unit of work, such as route, process or scheduled job."
- key: table:db_perf_attribution_daily;field:unitName
summary: "Name of the QQQ unit responsible for database work."
- key: table:db_perf_attribution_daily;field:processName
summary: "QQQ process responsible for the attributed work, when known."
- key: table:db_perf_attribution_daily;field:tableName
summary: "Table name reported for this attribution daily record."
- key: table:db_perf_attribution_daily;field:operation
summary: "Normalized SQL operation, such as SELECT, INSERT, UPDATE or DELETE."
- key: table:db_perf_attribution_daily;field:fingerprint
summary: "SHA-256 of the normalized SQL shape; literals and bind values are excluded."
- key: table:db_perf_attribution_daily;field:sqlText
summary: "Normalized SQL shape without literal values. Parameter values are never stored."
- key: table:db_perf_attribution_daily;field:units
summary: "Number of sampled QQQ units contributing to this aggregate."
- key: table:db_perf_attribution_daily;field:statements
summary: "Number of attributed SQL executions during the window."
- key: table:db_perf_attribution_daily;field:totalMs
summary: "Total ms in milliseconds for this measurement."
- key: table:db_perf_attribution_daily;field:maxMs
summary: "Max ms in milliseconds for this measurement."
- key: table:db_perf_attribution_daily;field:rowsCount
summary: "Rows processed during the measured execution window."
- key: table:db_perf_attribution_daily;field:errors
summary: "Number of failed SQL executions observed in the attribution window."
- key: table:db_perf_connection_sample;slot:help
summary: "Connection counts grouped by application, user and state for each snapshot."
content: |
The collector writes these rows every snapshot. Retained for detailRetentionDays (30). Run Collect now or wait for the next scheduled pass when empty.
- key: table:db_perf_connection_sample;slot:emptyState
summary: "Connection counts grouped by application, user and state for each snapshot."
content: |
The collector writes these rows every snapshot. Retained for detailRetentionDays (30). Run Collect now or wait for the next scheduled pass when empty.
- key: table:db_perf_connection_sample;field:snapshotId
summary: "Reference to the related snapshot record."
- key: table:db_perf_connection_sample;field:capturedAt
summary: "Time this database sample was captured."
- key: table:db_perf_connection_sample;field:dbUser
summary: "Database role associated with this statement, session or connection group."
- key: table:db_perf_connection_sample;field:applicationName
summary: "Client application name reported by PostgreSQL."
- key: table:db_perf_connection_sample;field:state
summary: "State reported for this connection sample record."
- key: table:db_perf_connection_sample;field:connectionCount
summary: "Connection count reported for this connection sample record."
- key: table:db_perf_explain;slot:help
summary: "Audited plan requests, including denied, failed and timed-out attempts."
content: |
Query plan and the explain UI API write one audit row per service request. Only literal-free PLAN or GENERIC results may be saved; ANALYZE plans and all parameter values remain transient. Retained for detailRetentionDays (30). Run Query plan on a statement when empty.
- key: table:db_perf_explain;slot:emptyState
summary: "Audited plan requests, including denied, failed and timed-out attempts."
content: |
Query plan and the explain UI API write one audit row per service request. Only literal-free PLAN or GENERIC results may be saved; ANALYZE plans and all parameter values remain transient. Retained for detailRetentionDays (30). Run Query plan on a statement when empty.
- key: table:db_perf_explain;field:statementId
summary: "Reference to the related statement record."
- key: table:db_perf_explain;field:requestedBy
summary: "Session user identifier that requested the plan, even for a denied request."
- key: table:db_perf_explain;field:requestedAt
summary: "When the plan request was received."
- key: table:db_perf_explain;field:mode
summary: "PLAN, automatically selected GENERIC, or explicitly authorized ANALYZE."
- key: table:db_perf_explain;field:sqlFingerprint
summary: "SHA-256 of the normalized query in this explain audit."
- key: table:db_perf_explain;field:sqlText
summary: "Normalized SQL shape without literal values. Parameter values are never stored."
- key: table:db_perf_explain;field:parameterCount
summary: "Number of positional parameter slots; values are never stored."
- key: table:db_perf_explain;field:durationMs
summary: "Duration ms in milliseconds for this measurement."
- key: table:db_perf_explain;field:outcome
summary: "OK, ERROR, DENIED or TIMEOUT; every explain service request is audited."
- key: table:db_perf_explain;field:errorMessage
summary: "Sanitized failure description. Database SQL errors and parameter values are excluded."
- key: table:db_perf_explain;field:planJson
summary: "Saved PostgreSQL JSON plan, only for eligible literal-free PLAN or GENERIC requests."
- key: table:db_perf_explain;field:planText
summary: "Readable indented plan tree; never stored for ANALYZE."
- key: table:db_perf_explain;field:totalCost
summary: "Total cost reported for this explain record."
- key: table:db_perf_finding;slot:help
summary: "Ranked, deduplicated suggestions with supporting evidence and proposed fixes."
content: |
The findings engine analyzes collected data hourly. OPEN and ACKNOWLEDGED findings remain actionable; resolved or ignored stale findings are purged after rollupRetentionDays (365). Wait for collection and the next analysis when empty. Proposed migrations require developer review; this qbit never applies them.
- key: table:db_perf_finding;slot:emptyState
summary: "Ranked, deduplicated suggestions with supporting evidence and proposed fixes."
content: |
The findings engine analyzes collected data hourly. OPEN and ACKNOWLEDGED findings remain actionable; resolved or ignored stale findings are purged after rollupRetentionDays (365). Wait for collection and the next analysis when empty. Proposed migrations require developer review; this qbit never applies them.
- key: table:db_perf_finding;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_finding;field:ruleId
summary: "Reference to the related rule record."
- key: table:db_perf_finding;field:ruleCode
summary: "Rule code reported for this finding record."
- key: table:db_perf_finding;field:dedupeKey
summary: "Stable rule-plus-subject identity that updates an existing finding instead of creating duplicates."
- key: table:db_perf_finding;field:status
summary: "Current lifecycle state. Review the status-specific guidance and related messages."
- key: table:db_perf_finding;field:severity
summary: "Configured importance of a finding or rule; contributes to the ranking score."
- key: table:db_perf_finding;field:score
summary: "Severity weight multiplied by normalized impact; larger scores rank first."
- key: table:db_perf_finding;field:title
summary: "Title reported for this finding record."
- key: table:db_perf_finding;field:summary
summary: "Summary reported for this finding record."
- key: table:db_perf_finding;field:evidence
summary: "Measurements supporting the finding, without raw SQL values."
- key: table:db_perf_finding;field:recommendation
summary: "Suggested developer or operator action; the qbit never applies it automatically."
- key: table:db_perf_finding;field:migrationSnippet
summary: "Proposed Liquibase migration text for review and source control; never executed by this qbit."
- key: table:db_perf_finding;field:subjectTable
summary: "Subject table associated with the finding."
- key: table:db_perf_finding;field:statementId
summary: "Reference to the related statement record."
- key: table:db_perf_finding;field:tableStatId
summary: "Reference to the related table stat record."
- key: table:db_perf_finding;field:indexStatId
summary: "Reference to the related index stat record."
- key: table:db_perf_finding;field:unitName
summary: "Name of the QQQ unit responsible for database work."
- key: table:db_perf_finding;field:firstSeenAt
summary: "First recorded observation of this statement or finding."
- key: table:db_perf_finding;field:lastSeenAt
summary: "Most recent observation of this statement, table, index or finding."
- key: table:db_perf_finding;field:occurrences
summary: "Number of analyses in which this finding has been observed."
- key: table:db_perf_finding;field:resolvedAt
summary: "When this finding was marked resolved."
- key: table:db_perf_finding;field:statusNote
summary: "Human explanation for the latest finding status change."
- key: table:db_perf_finding;field:statusChangedBy
summary: "User identifier responsible for the latest manual status change."
- key: table:db_perf_index_stat;slot:help
summary: "Current index definitions, usage, size and uniqueness."
content: |
The collector upserts indexes with the hourly table pass. Stale indexes are purged after rollupRetentionDays (365). Run Collect now to force collection when empty.
- key: table:db_perf_index_stat;slot:emptyState
summary: "Current index definitions, usage, size and uniqueness."
content: |
The collector upserts indexes with the hourly table pass. Stale indexes are purged after rollupRetentionDays (365). Run Collect now to force collection when empty.
- key: table:db_perf_index_stat;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_index_stat;field:tableStatId
summary: "Reference to the related table stat record."
- key: table:db_perf_index_stat;field:schemaName
summary: "Schema name reported for this index stat record."
- key: table:db_perf_index_stat;field:tableName
summary: "Table name reported for this index stat record."
- key: table:db_perf_index_stat;field:indexName
summary: "Index name reported for this index stat record."
- key: table:db_perf_index_stat;field:relid
summary: "Relid reported for this index stat record."
- key: table:db_perf_index_stat;field:definition
summary: "Definition reported for this index stat record."
- key: table:db_perf_index_stat;field:columns
summary: "Columns reported for this index stat record."
- key: table:db_perf_index_stat;field:isUnique
summary: "Whether this index enforces uniqueness."
- key: table:db_perf_index_stat;field:isPrimary
summary: "Whether this index enforces a primary key."
- key: table:db_perf_index_stat;field:isValid
summary: "Whether PostgreSQL considers the index valid for queries."
- key: table:db_perf_index_stat;field:idxScan
summary: "Idx scan reported for this index stat record."
- key: table:db_perf_index_stat;field:idxTupRead
summary: "Idx tup read reported for this index stat record."
- key: table:db_perf_index_stat;field:idxTupFetch
summary: "Idx tup fetch reported for this index stat record."
- key: table:db_perf_index_stat;field:sizeBytes
summary: "Size bytes; size measured in bytes."
- key: table:db_perf_index_stat;field:writeCost
summary: "Write cost reported for this index stat record."
- key: table:db_perf_index_stat;field:scansSinceReset
summary: "Scans since reset reported for this index stat record."
- key: table:db_perf_index_stat;field:lastSeenAt
summary: "Most recent observation of this statement, table, index or finding."
- key: table:db_perf_query_daily;slot:help
summary: "Daily totals from retained statement snapshot details."
content: |
Roll up and purge writes completed days, normally at 03:15 in the configured time zone. Retained for rollupRetentionDays (365). Wait for the next daily run after query snapshots exist.
- key: table:db_perf_query_daily;slot:emptyState
summary: "Daily totals from retained statement snapshot details."
content: |
Roll up and purge writes completed days, normally at 03:15 in the configured time zone. Retained for rollupRetentionDays (365). Wait for the next daily run after query snapshots exist.
- key: table:db_perf_query_daily;field:statementId
summary: "Reference to the related statement record."
- key: table:db_perf_query_daily;field:day
summary: "Calendar day in the configured qbit time zone for this rollup."
- key: table:db_perf_query_daily;field:calls
summary: "Calls reported for this query daily record."
- key: table:db_perf_query_daily;field:totalMs
summary: "Total ms in milliseconds for this measurement."
- key: table:db_perf_query_daily;field:meanMs
summary: "Mean ms in milliseconds for this measurement."
- key: table:db_perf_query_daily;field:maxMs
summary: "Max ms in milliseconds for this measurement."
- key: table:db_perf_query_daily;field:rowsCount
summary: "Rows processed during the measured execution window."
- key: table:db_perf_query_daily;field:hitRatio
summary: "Shared block hits divided by hits plus reads, from 0 to 1; null when no blocks were accessed."
- key: table:db_perf_query_daily;field:shareOfDbTime
summary: "Percentage of all database statement time in this interval, from 0 to 100."
- key: table:db_perf_query_daily;field:snapshots
summary: "Snapshots reported for this query daily record."
- key: table:db_perf_query_snapshot;slot:help
summary: "Interval deltas for the highest-cost statements in each collection."
content: |
The collector writes the top configured number of statement deltas every collection. New or reset counters establish baselines instead. Retained for detailRetentionDays (30). Collect twice when empty.
- key: table:db_perf_query_snapshot;slot:emptyState
summary: "Interval deltas for the highest-cost statements in each collection."
content: |
The collector writes the top configured number of statement deltas every collection. New or reset counters establish baselines instead. Retained for detailRetentionDays (30). Collect twice when empty.
- key: table:db_perf_query_snapshot;field:snapshotId
summary: "Reference to the related snapshot record."
- key: table:db_perf_query_snapshot;field:statementId
summary: "Reference to the related statement record."
- key: table:db_perf_query_snapshot;field:capturedAt
summary: "Time this database sample was captured."
- key: table:db_perf_query_snapshot;field:calls
summary: "Calls reported for this query snapshot record."
- key: table:db_perf_query_snapshot;field:totalMs
summary: "Total ms in milliseconds for this measurement."
- key: table:db_perf_query_snapshot;field:meanMs
summary: "Mean ms in milliseconds for this measurement."
- key: table:db_perf_query_snapshot;field:maxMs
summary: "Max ms in milliseconds for this measurement."
- key: table:db_perf_query_snapshot;field:rowsCount
summary: "Rows processed during the measured execution window."
- key: table:db_perf_query_snapshot;field:sharedBlksHit
summary: "Shared blks hit reported for this query snapshot record."
- key: table:db_perf_query_snapshot;field:sharedBlksRead
summary: "Shared blks read reported for this query snapshot record."
- key: table:db_perf_query_snapshot;field:hitRatio
summary: "Shared block hits divided by hits plus reads, from 0 to 1; null when no blocks were accessed."
- key: table:db_perf_query_snapshot;field:tempBlksWritten
summary: "Temp blks written reported for this query snapshot record."
- key: table:db_perf_query_snapshot;field:shareOfDbTime
summary: "Percentage of all database statement time in this interval, from 0 to 100."
- key: table:db_perf_repeat_pattern;slot:help
summary: "Repeated statement shapes within a QQQ unit of work, indicating possible N+1 queries."
content: |
The attribution listener flushes patterns that reach the configured repetition threshold. Retained for detailRetentionDays (30). Empty is normal when no repeated-statement pattern meets the threshold.
- key: table:db_perf_repeat_pattern;slot:emptyState
summary: "Repeated statement shapes within a QQQ unit of work, indicating possible N+1 queries."
content: |
The attribution listener flushes patterns that reach the configured repetition threshold. Retained for detailRetentionDays (30). Empty is normal when no repeated-statement pattern meets the threshold.
- key: table:db_perf_repeat_pattern;field:windowStart
summary: "Beginning of the attribution aggregation window."
- key: table:db_perf_repeat_pattern;field:windowEnd
summary: "End of the attribution aggregation window."
- key: table:db_perf_repeat_pattern;field:podName
summary: "Application pod that observed and flushed this attribution window."
- key: table:db_perf_repeat_pattern;field:unitKind
summary: "Kind of QQQ unit of work, such as route, process or scheduled job."
- key: table:db_perf_repeat_pattern;field:unitName
summary: "Name of the QQQ unit responsible for database work."
- key: table:db_perf_repeat_pattern;field:processName
summary: "QQQ process responsible for the attributed work, when known."
- key: table:db_perf_repeat_pattern;field:tableName
summary: "Table name reported for this repeat pattern record."
- key: table:db_perf_repeat_pattern;field:fingerprint
summary: "SHA-256 of the normalized SQL shape; literals and bind values are excluded."
- key: table:db_perf_repeat_pattern;field:sqlText
summary: "Normalized SQL shape without literal values. Parameter values are never stored."
- key: table:db_perf_repeat_pattern;field:unitsAffected
summary: "Number of sampled units that exceeded the repeated-statement threshold."
- key: table:db_perf_repeat_pattern;field:maxRepeats
summary: "Largest repetition count for one fingerprint in a single sampled unit of work."
- key: table:db_perf_repeat_pattern;field:totalRepeats
summary: "Total repeats reported for this repeat pattern record."
- key: table:db_perf_repeat_pattern;field:sampleRate
summary: "Share of units of work that were measured in this window (1 means all)."
- key: table:db_perf_rule;slot:help
summary: "Editable thresholds and severity for each suggestion rule."
content: |
Seeded insert-only by the Liquibase changelog; existing edits are preserved. Changes affect the next analysis. Rules are retained indefinitely. Apply the changelog when the list is empty.
Threshold guide (stored labels are authoritative):
- DUPLICATE_INDEX: no numeric thresholds; compares index definitions for identical or prefix-redundant coverage.
- SLOW_QUERY: threshold 1 = Mean execution time (ms) (default 500); threshold 2 = Database time (%) (default 5).
- MISSING_INDEX: threshold 1 = Sequential scan share (%) (default 50); threshold 2 = Tuples read per "day" (default 1000000).
- UNUSED_INDEX: threshold 1 = Days since statistics reset (default 14); threshold 2 = Index size (MB) (default 1).
- N_PLUS_ONE: threshold 1 = Repeats per unit (default 10); threshold 2 = Affected units (default 5).
- HOT_JOB: threshold 1 = Database time (%) (default 10); threshold 2 is unused.
- OVER_AUDITED: threshold 1 = Audit inserts per "day" (default 50000); threshold 2 is unused.
- BLOAT: threshold 1 = Dead tuple ratio (%) (default 20); threshold 2 = Dead tuples (default 10000).
- VACUUM_STALE: threshold 1 = Days since vacuum (default 7); threshold 2 = Transaction freeze age (default 150000000).
- POOL_SIZING: threshold 1 = Connections used (%) (default 80); threshold 2 = Idle connections (%) (default 50).
- IDLE_IN_TRANSACTION: threshold 1 = Idle seconds (default 60); threshold 2 is unused.
- CACHE_HIT_LOW: threshold 1 = Cache hit ratio (%) (default 95); threshold 2 is unused.
- TEMP_SPILL: threshold 1 = Temporary data per "day" (MB) (default 100); threshold 2 is unused.
- DEADLOCKS: threshold 1 = Deadlocks per "day" (default 1); threshold 2 is unused.
- key: table:db_perf_rule;slot:emptyState
summary: "Editable thresholds and severity for each suggestion rule."
content: |
Seeded insert-only by the Liquibase changelog; existing edits are preserved. Changes affect the next analysis. Rules are retained indefinitely. Apply the changelog when the list is empty.
- key: table:db_perf_rule;field:code
summary: "Code reported for this rule record."
- key: table:db_perf_rule;field:label
summary: "Label reported for this rule record."
- key: table:db_perf_rule;field:description
summary: "Description reported for this rule record."
- key: table:db_perf_rule;field:enabled
summary: "Whether collection or this suggestion rule participates in the next scheduled run."
- key: table:db_perf_rule;field:severity
summary: "Configured importance of a finding or rule; contributes to the ranking score."
- key: table:db_perf_rule;field:threshold1
summary: "Numeric rule threshold; its label defines the unit and meaning. See the rule threshold guide."
content: |
Rule thresholds use the units named on each rule row.
- SLOW_QUERY: threshold 1 = Mean execution time (ms) (default 500); threshold 2 = Database time (%) (default 5).
- MISSING_INDEX: threshold 1 = Sequential scan share (%) (default 50); threshold 2 = Tuples read per "day" (default 1000000).
- UNUSED_INDEX: threshold 1 = Days since statistics reset (default 14); threshold 2 = Index size (MB) (default 1).
- N_PLUS_ONE: threshold 1 = Repeats per unit (default 10); threshold 2 = Affected units (default 5).
- HOT_JOB: threshold 1 = Database time (%) (default 10); threshold 2 is unused.
- OVER_AUDITED: threshold 1 = Audit inserts per "day" (default 50000); threshold 2 is unused.
- BLOAT: threshold 1 = Dead tuple ratio (%) (default 20); threshold 2 = Dead tuples (default 10000).
- VACUUM_STALE: threshold 1 = Days since vacuum (default 7); threshold 2 = Transaction freeze age (default 150000000).
- POOL_SIZING: threshold 1 = Connections used (%) (default 80); threshold 2 = Idle connections (%) (default 50).
- IDLE_IN_TRANSACTION: threshold 1 = Idle seconds (default 60); threshold 2 is unused.
- CACHE_HIT_LOW: threshold 1 = Cache hit ratio (%) (default 95); threshold 2 is unused.
- TEMP_SPILL: threshold 1 = Temporary data per "day" (MB) (default 100); threshold 2 is unused.
- DEADLOCKS: threshold 1 = Deadlocks per "day" (default 1); threshold 2 is unused.
- key: table:db_perf_rule;field:threshold2
summary: "Numeric rule threshold; its label defines the unit and meaning. See the rule threshold guide."
content: |
Rule thresholds use the units named on each rule row.
- SLOW_QUERY: threshold 1 = Mean execution time (ms) (default 500); threshold 2 = Database time (%) (default 5).
- MISSING_INDEX: threshold 1 = Sequential scan share (%) (default 50); threshold 2 = Tuples read per "day" (default 1000000).
- UNUSED_INDEX: threshold 1 = Days since statistics reset (default 14); threshold 2 = Index size (MB) (default 1).
- N_PLUS_ONE: threshold 1 = Repeats per unit (default 10); threshold 2 = Affected units (default 5).
- HOT_JOB: threshold 1 = Database time (%) (default 10); threshold 2 is unused.
- OVER_AUDITED: threshold 1 = Audit inserts per "day" (default 50000); threshold 2 is unused.
- BLOAT: threshold 1 = Dead tuple ratio (%) (default 20); threshold 2 = Dead tuples (default 10000).
- VACUUM_STALE: threshold 1 = Days since vacuum (default 7); threshold 2 = Transaction freeze age (default 150000000).
- POOL_SIZING: threshold 1 = Connections used (%) (default 80); threshold 2 = Idle connections (%) (default 50).
- IDLE_IN_TRANSACTION: threshold 1 = Idle seconds (default 60); threshold 2 is unused.
- CACHE_HIT_LOW: threshold 1 = Cache hit ratio (%) (default 95); threshold 2 is unused.
- TEMP_SPILL: threshold 1 = Temporary data per "day" (MB) (default 100); threshold 2 is unused.
- DEADLOCKS: threshold 1 = Deadlocks per "day" (default 1); threshold 2 is unused.
- key: table:db_perf_rule;field:threshold1Label
summary: "Threshold1 label reported for this rule record."
- key: table:db_perf_rule;field:threshold2Label
summary: "Threshold2 label reported for this rule record."
- key: table:db_perf_settings;slot:help
summary: "Per-backend collection, retention, attribution and query-plan settings."
content: |
Created on the first collection or explain request. Administrators edit these values; dashboard reads do not create settings. Settings are retained until removed by the host. Run Collect now when empty.
- key: table:db_perf_settings;slot:emptyState
summary: "Per-backend collection, retention, attribution and query-plan settings."
content: |
Created on the first collection or explain request. Administrators edit these values; dashboard reads do not create settings. Settings are retained until removed by the host. Run Collect now when empty.
- key: table:db_perf_settings;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_settings;field:enabled
summary: "Whether collection or this suggestion rule participates in the next scheduled run."
- key: table:db_perf_settings;field:provider
summary: "Detected statistics provider; unsupported vendors report a setup message."
- key: table:db_perf_settings;field:snapshotIntervalMinutes
summary: "Minimum minutes between scheduled snapshots; default 5. Collect now forces collection."
- key: table:db_perf_settings;field:tableSnapshotMinutes
summary: "Minutes between table and index collection passes; default 60."
- key: table:db_perf_settings;field:detailRetentionDays
summary: "Age in days before detailed samples and plan audits are purged; default 30."
- key: table:db_perf_settings;field:rollupRetentionDays
summary: "Retention in days for daily rows and stale inventory or closed findings; default 365."
- key: table:db_perf_settings;field:topStatementsPerSnapshot
summary: "Maximum statement details saved per pass, ranked by database time; default 200."
- key: table:db_perf_settings;field:activityLongRunningSeconds
summary: "Active query age in seconds that makes a session notable; default 30."
- key: table:db_perf_settings;field:idleInTransactionSeconds
summary: "Idle transaction age in seconds that makes a session notable; default 60."
- key: table:db_perf_settings;field:attributionEnabled
summary: "Enable QQQ statement attribution; the host listener must also be enabled."
- key: table:db_perf_settings;field:attributionSampleRate
summary: "Fraction of QQQ units sampled, from 0 to 1; default 1 captures every unit."
- key: table:db_perf_settings;field:attributionFlushSeconds
summary: "Seconds between attribution batch flushes; default 60. A crash can lose one window."
- key: table:db_perf_settings;field:attributionMaxKeys
summary: "Maximum in-memory attribution groups per pod; overflow goes to an other group. Default 10000."
- key: table:db_perf_settings;field:nPlusOneThreshold
summary: "Repeated occurrences of a statement shape in one unit before recording a pattern; default 10."
- key: table:db_perf_settings;field:explainTimeoutMs
summary: "Maximum query-plan execution time in milliseconds; default 10000 for PLAN and ANALYZE."
- key: table:db_perf_settings;field:alertsEnabled
summary: "Enable database performance alert emission when the alert integration is installed."
- key: table:db_perf_settings;field:spikeFactor
summary: "Multiplier above the seven-day interval median needed for a query-spike alert; default 3."
- key: table:db_perf_settings;field:spikeMinTotalMs
summary: "Minimum interval query time in milliseconds before a spike can alert; default 5000."
- key: table:db_perf_settings;field:connectionAlertPercent
summary: "Percentage of max connections at which saturation can alert; default 80."
- key: table:db_perf_settings;field:blockedAlertSeconds
summary: "Blocked session duration in seconds required for a blocking-chain alert; default 30."
- key: table:db_perf_settings;field:lastSnapshotAt
summary: "Time of the latest collection tracked by the backend settings."
- key: table:db_perf_settings;field:lastTableSnapshotAt
summary: "Time of the latest table and index collection pass."
- key: table:db_perf_settings;field:statsAccess
summary: "Statistics capability: FULL, reduced access, missing extension, or unsupported provider."
- key: table:db_perf_settings;field:statsAccessMessage
summary: "The collector explains missing capabilities and the next grant or setup step."
- key: table:db_perf_snapshot;slot:help
summary: "One collection pass and its database-wide interval totals."
content: |
The collector writes this table every five minutes by default, including partial and failed passes. The first pass establishes a baseline. Retained for detailRetentionDays (30 by default). Run Collect now when empty.
- key: table:db_perf_snapshot;slot:emptyState
summary: "One collection pass and its database-wide interval totals."
content: |
The collector writes this table every five minutes by default, including partial and failed passes. The first pass establishes a baseline. Retained for detailRetentionDays (30 by default). Run Collect now when empty.
- key: table:db_perf_snapshot;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_snapshot;field:capturedAt
summary: "Time this database sample was captured."
- key: table:db_perf_snapshot;field:intervalSeconds
summary: "Interval seconds reported for this snapshot record."
- key: table:db_perf_snapshot;field:status
summary: "Current lifecycle state. Review the status-specific guidance and related messages."
- key: table:db_perf_snapshot;field:durationMs
summary: "Duration ms in milliseconds for this measurement."
- key: table:db_perf_snapshot;field:errorMessage
summary: "Sanitized failure description. Database SQL errors and parameter values are excluded."
- key: table:db_perf_snapshot;field:statsResetDetected
summary: "Counters restarted or decreased; that pass establishes a new baseline."
- key: table:db_perf_snapshot;field:includesTables
summary: "Includes tables reported for this snapshot record."
- key: table:db_perf_snapshot;field:xactCommit
summary: "Xact commit reported for this snapshot record."
- key: table:db_perf_snapshot;field:xactRollback
summary: "Xact rollback reported for this snapshot record."
- key: table:db_perf_snapshot;field:blksHit
summary: "Blks hit reported for this snapshot record."
- key: table:db_perf_snapshot;field:blksRead
summary: "Blks read reported for this snapshot record."
- key: table:db_perf_snapshot;field:cacheHitRatio
summary: "Database cache hits divided by hits plus reads, from 0 to 1."
- key: table:db_perf_snapshot;field:tempFiles
summary: "Temp files reported for this snapshot record."
- key: table:db_perf_snapshot;field:tempBytes
summary: "Temp bytes; size measured in bytes."
- key: table:db_perf_snapshot;field:deadlocks
summary: "Deadlocks reported for this snapshot record."
- key: table:db_perf_snapshot;field:conflicts
summary: "Conflicts reported for this snapshot record."
- key: table:db_perf_snapshot;field:connectionsTotal
summary: "Connections total reported for this snapshot record."
- key: table:db_perf_snapshot;field:connectionsActive
summary: "Connections active reported for this snapshot record."
- key: table:db_perf_snapshot;field:connectionsIdle
summary: "Connections idle reported for this snapshot record."
- key: table:db_perf_snapshot;field:connectionsIdleInTx
summary: "Connections idle in tx reported for this snapshot record."
- key: table:db_perf_snapshot;field:maxConnections
summary: "Max connections reported for this snapshot record."
- key: table:db_perf_snapshot;field:databaseSizeBytes
summary: "Database size bytes; size measured in bytes."
- key: table:db_perf_snapshot;field:statementCount
summary: "Statement count reported for this snapshot record."
- key: table:db_perf_snapshot;field:statementCalls
summary: "Statement calls reported for this snapshot record."
- key: table:db_perf_snapshot;field:statementTotalMs
summary: "Statement total ms in milliseconds for this measurement."
- key: table:db_perf_snapshot;field:storedStatementCount
summary: "Stored statement count reported for this snapshot record."
- key: table:db_perf_snapshot;field:rawXactCommit
summary: "Previous cumulative xact commit counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawXactRollback
summary: "Previous cumulative xact rollback counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawBlksHit
summary: "Previous cumulative blks hit counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawBlksRead
summary: "Previous cumulative blks read counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawTempFiles
summary: "Previous cumulative temp files counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawTempBytes
summary: "Previous cumulative temp bytes counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawDeadlocks
summary: "Previous cumulative deadlocks counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawConflicts
summary: "Previous cumulative conflicts counter used for reset-safe deltas."
- key: table:db_perf_snapshot;field:rawStatsReset
summary: "Previous database counter reset timestamp used for delta calculations."
- key: table:db_perf_snapshot;field:rawStatementStatsReset
summary: "Previous provider reset marker used to detect counter resets, not raw SQL."
- key: table:db_perf_statement;slot:help
summary: "Normalized query shapes with the latest counters and retained 24-hour totals."
content: |
The collector upserts statements observed in pg_stat_statements each collection. Two snapshots establish deltas. Retained while observed; stale records are purged after rollupRetentionDays (365 by default). Enable pg_stat_statements and collect twice when empty.
- key: table:db_perf_statement;slot:emptyState
summary: "Normalized query shapes with the latest counters and retained 24-hour totals."
content: |
The collector upserts statements observed in pg_stat_statements each collection. Two snapshots establish deltas. Retained while observed; stale records are purged after rollupRetentionDays (365 by default). Enable pg_stat_statements and collect twice when empty.
- key: table:db_perf_statement;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_statement;field:queryId
summary: "PostgreSQL query identifier; fingerprint provides cross-source normalized matching."
- key: table:db_perf_statement;field:dbUser
summary: "Database role associated with this statement, session or connection group."
- key: table:db_perf_statement;field:toplevel
summary: "Toplevel reported for this statement record."
- key: table:db_perf_statement;field:fingerprint
summary: "SHA-256 of the normalized SQL shape; literals and bind values are excluded."
- key: table:db_perf_statement;field:queryText
summary: "Normalized query text with literals replaced by placeholders; never raw live text."
- key: table:db_perf_statement;field:operation
summary: "Normalized SQL operation, such as SELECT, INSERT, UPDATE or DELETE."
- key: table:db_perf_statement;field:mainTable
summary: "Best-effort main table extracted from normalized SQL."
- key: table:db_perf_statement;field:firstSeenAt
summary: "First recorded observation of this statement or finding."
- key: table:db_perf_statement;field:lastSeenAt
summary: "Most recent observation of this statement, table, index or finding."
- key: table:db_perf_statement;field:lastCalls
summary: "Last calls reported for this statement record."
- key: table:db_perf_statement;field:lastTotalExecMs
summary: "Last total exec ms in milliseconds for this measurement."
- key: table:db_perf_statement;field:lastRows
summary: "Last rows reported for this statement record."
- key: table:db_perf_statement;field:lastSharedBlksHit
summary: "Last shared blks hit reported for this statement record."
- key: table:db_perf_statement;field:lastSharedBlksRead
summary: "Last shared blks read reported for this statement record."
- key: table:db_perf_statement;field:lastTempBlksWritten
summary: "Last temp blks written reported for this statement record."
- key: table:db_perf_statement;field:lastMaxExecMs
summary: "Latest cumulative maximum execution time in milliseconds; not an interval maximum."
- key: table:db_perf_statement;field:lastStatsSince
summary: "Per-statement counter start time when the PostgreSQL version supplies it."
- key: table:db_perf_statement;field:calls24h
summary: "Calls in retained statement details from the last 24 hours."
- key: table:db_perf_statement;field:totalMs24h
summary: "Execution time in milliseconds in retained statement details from the last 24 hours."
- key: table:db_perf_statement;field:meanMs24h
summary: "Total retained execution milliseconds divided by retained calls over 24 hours."
- key: table:db_perf_statement;field:shareOfDbTime24h
summary: "Percentage of database time over 24 hours; numerator uses retained statement details."
- key: table:db_perf_table_daily;slot:help
summary: "Daily table scan deltas and ending size used to track growth."
content: |
The daily rollup writes these rows from retained table samples, normally at 03:15. Retained for rollupRetentionDays (365). Wait for the daily rollup after table snapshots exist.
- key: table:db_perf_table_daily;slot:emptyState
summary: "Daily table scan deltas and ending size used to track growth."
content: |
The daily rollup writes these rows from retained table samples, normally at 03:15. Retained for rollupRetentionDays (365). Wait for the daily rollup after table snapshots exist.
- key: table:db_perf_table_daily;field:tableStatId
summary: "Reference to the related table stat record."
- key: table:db_perf_table_daily;field:day
summary: "Calendar day in the configured qbit time zone for this rollup."
- key: table:db_perf_table_daily;field:seqScan
summary: "Seq scan reported for this table daily record."
- key: table:db_perf_table_daily;field:idxScan
summary: "Idx scan reported for this table daily record."
- key: table:db_perf_table_daily;field:seqTupRead
summary: "Seq tup read reported for this table daily record."
- key: table:db_perf_table_daily;field:nTupIns
summary: "PostgreSQL n tup ins tuple statistic for the sampled table."
- key: table:db_perf_table_daily;field:nTupUpd
summary: "PostgreSQL n tup upd tuple statistic for the sampled table."
- key: table:db_perf_table_daily;field:nTupDel
summary: "PostgreSQL n tup del tuple statistic for the sampled table."
- key: table:db_perf_table_daily;field:nLiveTup
summary: "PostgreSQL n live tup tuple statistic for the sampled table."
- key: table:db_perf_table_daily;field:nDeadTup
summary: "PostgreSQL n dead tup tuple statistic for the sampled table."
- key: table:db_perf_table_daily;field:totalBytes
summary: "Total bytes; size measured in bytes."
- key: table:db_perf_table_snapshot;slot:help
summary: "Historical samples of table scans, tuples, size and maintenance."
content: |
The collector writes table snapshots every tableSnapshotMinutes (60). Retained for detailRetentionDays (30). Run Collect now or wait for the next hourly table pass.
- key: table:db_perf_table_snapshot;slot:emptyState
summary: "Historical samples of table scans, tuples, size and maintenance."
content: |
The collector writes table snapshots every tableSnapshotMinutes (60). Retained for detailRetentionDays (30). Run Collect now or wait for the next hourly table pass.
- key: table:db_perf_table_snapshot;field:snapshotId
summary: "Reference to the related snapshot record."
- key: table:db_perf_table_snapshot;field:tableStatId
summary: "Reference to the related table stat record."
- key: table:db_perf_table_snapshot;field:capturedAt
summary: "Time this database sample was captured."
- key: table:db_perf_table_snapshot;field:seqScan
summary: "Seq scan reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:seqTupRead
summary: "Seq tup read reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:idxScan
summary: "Idx scan reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:idxTupFetch
summary: "Idx tup fetch reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:nTupIns
summary: "PostgreSQL n tup ins tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:nTupUpd
summary: "PostgreSQL n tup upd tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:nTupDel
summary: "PostgreSQL n tup del tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:nTupHotUpd
summary: "PostgreSQL n tup hot upd tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:nLiveTup
summary: "PostgreSQL n live tup tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:nDeadTup
summary: "PostgreSQL n dead tup tuple statistic for the sampled table."
- key: table:db_perf_table_snapshot;field:totalBytes
summary: "Total bytes; size measured in bytes."
- key: table:db_perf_table_snapshot;field:tableBytes
summary: "Table bytes; size measured in bytes."
- key: table:db_perf_table_snapshot;field:indexBytes
summary: "Index bytes; size measured in bytes."
- key: table:db_perf_table_snapshot;field:autovacuumCount
summary: "Autovacuum count reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:autoanalyzeCount
summary: "Autoanalyze count reported for this table snapshot record."
- key: table:db_perf_table_snapshot;field:freezeAge
summary: "Transaction-ID age used to assess vacuum and wraparound risk."
- key: table:db_perf_table_stat;slot:help
summary: "Current table size, scans, tuple counts and maintenance statistics."
content: |
The collector upserts table data every tableSnapshotMinutes (60). Stale records are purged after rollupRetentionDays (365). Run Collect now to force table collection when empty.
- key: table:db_perf_table_stat;slot:emptyState
summary: "Current table size, scans, tuple counts and maintenance statistics."
content: |
The collector upserts table data every tableSnapshotMinutes (60). Stale records are purged after rollupRetentionDays (365). Run Collect now to force table collection when empty.
- key: table:db_perf_table_stat;field:backendName
summary: "The configured QQQ backend whose database is monitored."
- key: table:db_perf_table_stat;field:schemaName
summary: "Schema name reported for this table stat record."
- key: table:db_perf_table_stat;field:tableName
summary: "Table name reported for this table stat record."
- key: table:db_perf_table_stat;field:relid
summary: "Relid reported for this table stat record."
- key: table:db_perf_table_stat;field:seqScan
summary: "Seq scan reported for this table stat record."
- key: table:db_perf_table_stat;field:seqTupRead
summary: "Seq tup read reported for this table stat record."
- key: table:db_perf_table_stat;field:idxScan
summary: "Idx scan reported for this table stat record."
- key: table:db_perf_table_stat;field:idxTupFetch
summary: "Idx tup fetch reported for this table stat record."
- key: table:db_perf_table_stat;field:nTupIns
summary: "PostgreSQL n tup ins tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:nTupUpd
summary: "PostgreSQL n tup upd tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:nTupDel
summary: "PostgreSQL n tup del tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:nTupHotUpd
summary: "PostgreSQL n tup hot upd tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:nLiveTup
summary: "PostgreSQL n live tup tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:nDeadTup
summary: "PostgreSQL n dead tup tuple statistic for the sampled table."
- key: table:db_perf_table_stat;field:deadRatio
summary: "Dead tuples divided by live plus dead tuples, from 0 to 1; an estimate used for maintenance review."
- key: table:db_perf_table_stat;field:totalBytes
summary: "Total bytes; size measured in bytes."
- key: table:db_perf_table_stat;field:tableBytes
summary: "Table bytes; size measured in bytes."
- key: table:db_perf_table_stat;field:indexBytes
summary: "Index bytes; size measured in bytes."
- key: table:db_perf_table_stat;field:toastBytes
summary: "Toast bytes; size measured in bytes."
- key: table:db_perf_table_stat;field:growthBytes7d
summary: "Change in table size in bytes over seven days, once enough daily history exists."
- key: table:db_perf_table_stat;field:lastVacuum
summary: "Last manual vacuum reported by PostgreSQL."
- key: table:db_perf_table_stat;field:lastAutovacuum
summary: "Last automatic vacuum reported by PostgreSQL."
- key: table:db_perf_table_stat;field:lastAnalyze
summary: "Last manual statistics analyze reported by PostgreSQL."
- key: table:db_perf_table_stat;field:lastAutoanalyze
summary: "Last automatic statistics analyze reported by PostgreSQL."
- key: table:db_perf_table_stat;field:vacuumCount
summary: "Vacuum count reported for this table stat record."
- key: table:db_perf_table_stat;field:autovacuumCount
summary: "Autovacuum count reported for this table stat record."
- key: table:db_perf_table_stat;field:analyzeCount
summary: "Analyze count reported for this table stat record."
- key: table:db_perf_table_stat;field:autoanalyzeCount
summary: "Autoanalyze count reported for this table stat record."
- key: table:db_perf_table_stat;field:freezeAge
summary: "Transaction-ID age used to assess vacuum and wraparound risk."
- key: table:db_perf_table_stat;field:seqScanShare
summary: "Fraction of table scans that are sequential, from 0 to 1; high values merit query and index review."
- key: table:db_perf_table_stat;field:lastSeenAt
summary: "Most recent observation of this statement, table, index or finding."
- key: process:dbPerfCollectSnapshot;slot:help
summary: "Collect database snapshots on the host schedule."
content: |
Runs at the configured snapshot cadence, normally every five minutes. Reads statistics and writes interval deltas and notable activity; table/index passes normally run hourly. Requires manage settings.
- key: process:dbPerfCollectNow;slot:help
summary: "Collect an immediate snapshot and force table and index statistics."
content: |
Run from the Snapshots table. Requires manage settings. The Result screen reports success, partial access or collection failure. The first pass establishes counter baselines.
- key: process:dbPerfRollupAndPurge;slot:help
summary: "Roll up completed days and remove expired details."
content: |
Runs daily at 03:15 by default. Uses the configured time zone, detail retention and rollup retention. Reruns replace daily aggregates instead of duplicating them.
- key: process:dbPerfQueryPlan;slot:help
summary: "Inspect the plan of one statement, with optional read-only execution."
content: |
Choose PLAN (default) or ANALYZE. PLAN uses a generic plan for parameterized SQL on PostgreSQL 16+. ANALYZE requires explain analyze permission and exactly one SELECT with no modifying CTEs, INTO or row locks. Enter parameter values only for ANALYZE, one per line in placeholder order. Values are sent transiently; they and the ANALYZE plan are never stored. Every request is audited. Results include copyable plan text, cost, a SQL Workbench link and a literal-free assistant prompt. No model is called by this qbit.
- key: widget:dbPerfSetup;slot:label
summary: "Check backend, statistics access, collection, attribution and scheduler setup."
- key: widget:dbPerfDatabaseTime;slot:label
summary: "Database execution milliseconds and call counts per snapshot over the last 24 hours."
- key: widget:dbPerfTopQueries;slot:label
summary: "Most expensive retained query shapes over 24 hours; open a query to inspect its history."
- key: widget:dbPerfFindings;slot:label
summary: "Open and acknowledged findings ranked by impact."
- key: widget:dbPerfLiveActivity;slot:label
summary: "Current notable sessions with normalized text, blocking and transaction ages."
- key: widget:dbPerfConnections;slot:label
summary: "Connection capacity followed by a live application and user breakdown."
- key: widget:dbPerfConnectionStatistics;slot:label
summary: "Latest snapshot capacity and cache hit percentage, with 24-hour temp and deadlock totals."
- key: widget:dbPerfConnectionsByApplication;slot:label
summary: "Current connections grouped by application, database user and state."
- key: widget:dbPerfAttribution;slot:label
summary: "QQQ units ranked by database time, followed by repeated-statement patterns."
- key: widget:dbPerfTables;slot:label
summary: "Sequential-scan-heavy tables and tables with the fastest seven-day size growth."
- key: widget:db_perf_snapshot_db_perf_query_snapshot;slot:label
summary: "Related interval deltas for the highest-cost statements in each collection."
- key: widget:db_perf_statement_db_perf_query_snapshot;slot:label
summary: "Related interval deltas for the highest-cost statements in each collection."
- key: widget:db_perf_statement_db_perf_query_daily;slot:label
summary: "Related daily totals from retained statement snapshot details."
- key: widget:db_perf_snapshot_db_perf_table_snapshot;slot:label
summary: "Related historical samples of table scans, tuples, size and maintenance."
- key: widget:db_perf_table_stat_db_perf_table_snapshot;slot:label
summary: "Related historical samples of table scans, tuples, size and maintenance."
- key: widget:db_perf_table_stat_db_perf_table_daily;slot:label
summary: "Related daily table scan deltas and ending size used to track growth."
- key: widget:db_perf_table_stat_db_perf_index_stat;slot:label
summary: "Related current index definitions, usage, size and uniqueness."
- key: widget:db_perf_snapshot_db_perf_activity_sample;slot:label
summary: "Related notable session samples with normalized text, never raw query text."
- key: widget:db_perf_snapshot_db_perf_connection_sample;slot:label
summary: "Related connection counts grouped by application, user and state for each snapshot."
- key: widget:db_perf_rule_db_perf_finding;slot:label
summary: "Related ranked, deduplicated suggestions with supporting evidence and proposed fixes."
- key: widget:db_perf_statement_db_perf_finding;slot:label
summary: "Related ranked, deduplicated suggestions with supporting evidence and proposed fixes."
- key: widget:db_perf_table_stat_db_perf_finding;slot:label
summary: "Related ranked, deduplicated suggestions with supporting evidence and proposed fixes."
- key: widget:db_perf_statement_db_perf_explain;slot:label
summary: "Related audited plan requests, including denied, failed and timed-out attempts."
- key: possibleValueSource:db_perf_ExplainMode;slot:help
summary: "Available values for explainmode."
- key: possibleValueSource:db_perf_FindingStatus;slot:help
summary: "Available values for findingstatus."
- key: possibleValueSource:db_perf_Operation;slot:help
summary: "Available values for operation."
- key: possibleValueSource:db_perf_Outcome;slot:help
summary: "Available values for outcome."
- key: possibleValueSource:db_perf_Provider;slot:help
summary: "Available values for provider."
- key: possibleValueSource:db_perf_Reason;slot:help
summary: "Available values for reason."
- key: possibleValueSource:db_perf_Severity;slot:help
summary: "Available values for severity."
- key: possibleValueSource:db_perf_SnapshotStatus;slot:help
summary: "Available values for snapshotstatus."
- key: possibleValueSource:db_perf_State;slot:help
summary: "Available values for state."
- key: possibleValueSource:db_perf_StatsAccess;slot:help
summary: "Available values for statsaccess."
- key: possibleValueSource:db_perf_UnitKind;slot:help
summary: "Available values for unitkind."
- key: possibleValueSource:db_perf_activity_sample;slot:help
summary: "Available values for activity sample."
- key: possibleValueSource:db_perf_attribution;slot:help
summary: "Available values for attribution."
- key: possibleValueSource:db_perf_attribution_daily;slot:help
summary: "Available values for attribution daily."
- key: possibleValueSource:db_perf_connection_sample;slot:help
summary: "Available values for connection sample."
- key: possibleValueSource:db_perf_explain;slot:help
summary: "Available values for explain."
- key: possibleValueSource:db_perf_finding;slot:help
summary: "Available values for finding."
- key: possibleValueSource:db_perf_index_stat;slot:help
summary: "Available values for index stat."
- key: possibleValueSource:db_perf_query_daily;slot:help
summary: "Available values for query daily."
- key: possibleValueSource:db_perf_query_snapshot;slot:help
summary: "Available values for query snapshot."
- key: possibleValueSource:db_perf_repeat_pattern;slot:help
summary: "Available values for repeat pattern."
- key: possibleValueSource:db_perf_rule;slot:help
summary: "Available values for rule."
- key: possibleValueSource:db_perf_settings;slot:help
summary: "Available values for settings."
- key: possibleValueSource:db_perf_snapshot;slot:help
summary: "Available values for snapshot."
- key: possibleValueSource:db_perf_statement;slot:help
summary: "Available values for statement."
- key: possibleValueSource:db_perf_table_daily;slot:help
summary: "Available values for table daily."
- key: possibleValueSource:db_perf_table_snapshot;slot:help
summary: "Available values for table snapshot."
- key: possibleValueSource:db_perf_table_stat;slot:help
summary: "Available values for table stat."
- key: process:db_perf_settings.bulkEdit;slot:help
summary: Review and apply changes to database performance settings records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: process:db_perf_settings.bulkEditWithFile;slot:help
summary: Review and apply changes to database performance settings records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: process:db_perf_rule.bulkInsert;slot:help
summary: Review and apply changes to database performance rule records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: process:db_perf_rule.bulkEdit;slot:help
summary: Review and apply changes to database performance rule records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: process:db_perf_rule.bulkEditWithFile;slot:help
summary: Review and apply changes to database performance rule records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: process:db_perf_rule.bulkDelete;slot:help
summary: Review and apply changes to database performance rule records.
content: |
Requires manage settings permission. Review every changed value before applying the standard QQQ action. Changes affect subsequent collection or analysis; they do not change the monitored database schema.
- key: possibleValueSource:db_perf_ExplainRequestMode;slot:help
summary: PLAN inspects the query; ANALYZE executes a guarded read-only SELECT with separate permission.
- key: process:dbPerfAnalyze;slot:help
summary: "Scheduled run of the suggestion rules that creates, updates and resolves findings."
content: |
Runs hourly by default. Each enabled rule in Suggestion Rules reads the stored statistics and produces findings with evidence. A rerun updates existing findings instead of adding new ones, resolves findings that are no longer detected, and never changes Ignored findings. It never changes the live database.
- key: process:dbPerfAnalyzeNow;slot:help
summary: "Run the suggestion rules now instead of waiting for the hourly run."
content: |
Needs the manage settings permission. The result shows new, updated and resolved findings per rule. Run {{label:dbPerfCollectNow}} first if the statistics are old.
- key: process:dbPerfFindingACKNOWLEDGED;slot:help
summary: "Mark the selected findings as acknowledged: seen and being worked on."
content: |
Acknowledged findings stay on the list and are still updated with new evidence. They resolve automatically when no longer detected.
- key: process:dbPerfFindingRESOLVED;slot:help
summary: "Mark the selected findings as resolved."
content: |
Use after shipping the fix. If the rule detects the problem again, the finding reopens.
- key: process:dbPerfFindingIGNORED;slot:help
summary: "Ignore the selected findings; a note saying why is required."
content: |
Ignored findings are never reopened or resolved automatically. Use Reopen to bring one back.
- key: process:dbPerfFindingOPEN;slot:help
summary: "Reopen the selected findings."
content: |
Returns acknowledged, resolved or ignored findings to Open so the rules manage them again.
@@ -0,0 +1,434 @@
components:
schemas:
DbPerformanceActivityV1:
properties:
blockingChains:
items:
$ref: "#/components/schemas/DbPerformanceBlockingChainV1"
type: "array"
liveTextRedacted:
type: "boolean"
sessions:
items:
$ref: "#/components/schemas/DbPerformanceSessionV1"
type: "array"
type: "object"
DbPerformanceBlockingChainV1:
properties:
blockedPids:
items:
nullable: true
type: "integer"
type: "array"
rootPid:
nullable: true
type: "integer"
type: "object"
DbPerformanceErrorV1:
properties:
error:
nullable: true
type: "string"
type: "object"
DbPerformanceExplainRequestV1:
description: "Supply exactly one of statementId or sql. Parameters are strings,\
\ accepted only for ANALYZE and never stored."
properties:
backend:
nullable: true
type: "string"
mode:
enum:
- "PLAN"
- "ANALYZE"
type: "string"
parameters:
items:
nullable: true
type: "string"
type: "array"
sql:
nullable: true
type: "string"
statementId:
nullable: true
type: "integer"
type: "object"
DbPerformanceExplainResultV1:
properties:
actualTotalMs:
nullable: true
type: "number"
assistantPrompt:
nullable: true
type: "string"
explainId:
nullable: true
type: "integer"
mode:
nullable: true
type: "string"
planJson:
nullable: true
type: "string"
planText:
nullable: true
type: "string"
sqlWorkbenchUrl:
nullable: true
type: "string"
totalCost:
nullable: true
type: "number"
warnings:
items:
nullable: true
type: "string"
type: "array"
type: "object"
DbPerformanceFindingV1:
properties:
id:
nullable: true
type: "integer"
score:
nullable: true
type: "number"
severity:
nullable: true
type: "string"
title:
nullable: true
type: "string"
type: "object"
DbPerformanceLookupV1:
properties:
fingerprint:
nullable: true
type: "string"
matches:
items:
$ref: "#/components/schemas/DbPerformanceStatementV1"
type: "array"
normalizedSql:
nullable: true
type: "string"
type: "object"
DbPerformanceOverviewV1:
properties:
findingsBySeverity:
additionalProperties:
nullable: true
type: "integer"
type: "object"
settings:
$ref: "#/components/schemas/DbPerformanceSettingsV1"
snapshot:
$ref: "#/components/schemas/DbPerformanceSnapshotV1"
topFindings:
items:
$ref: "#/components/schemas/DbPerformanceFindingV1"
type: "array"
topStatements:
items:
$ref: "#/components/schemas/DbPerformanceStatementV1"
type: "array"
type: "object"
DbPerformanceSessionV1:
properties:
application:
nullable: true
type: "string"
blockedBy:
items:
nullable: true
type: "integer"
type: "array"
blockingCount:
nullable: true
type: "integer"
fingerprint:
nullable: true
type: "string"
liveTextRedacted:
type: "boolean"
normalizedText:
nullable: true
type: "string"
pid:
nullable: true
type: "integer"
query:
description: "Raw live SQL only with view_live_text; never persisted."
nullable: true
type: "string"
querySeconds:
nullable: true
type: "number"
state:
nullable: true
type: "string"
user:
nullable: true
type: "string"
waitEvent:
nullable: true
type: "string"
xactSeconds:
nullable: true
type: "number"
type: "object"
DbPerformanceSettingsV1:
properties:
lastSnapshotAt:
nullable: true
type: "string"
provider:
nullable: true
type: "string"
statsAccess:
nullable: true
type: "string"
statsAccessMessage:
nullable: true
type: "string"
type: "object"
DbPerformanceSnapshotV1:
properties:
cacheHitRatio:
nullable: true
type: "number"
capturedAt:
nullable: true
type: "string"
connectionsTotal:
nullable: true
type: "integer"
deadlocks:
nullable: true
type: "integer"
id:
nullable: true
type: "integer"
maxConnections:
nullable: true
type: "integer"
statementCalls:
nullable: true
type: "integer"
statementTotalMs:
nullable: true
type: "number"
status:
nullable: true
type: "string"
tempBytes:
nullable: true
type: "integer"
type: "object"
DbPerformanceStatementV1:
properties:
calls24h:
nullable: true
type: "integer"
historyPath:
nullable: true
type: "string"
hitRatio:
nullable: true
type: "number"
id:
nullable: true
type: "integer"
meanMs24h:
nullable: true
type: "number"
queryText:
description: "Normalized SQL, truncated to 300 characters."
nullable: true
type: "string"
shareOfDbTime24h:
nullable: true
type: "number"
totalMs24h:
nullable: true
type: "number"
type: "object"
securitySchemes:
sessionUuidCookie:
in: "cookie"
name: "sessionUUID"
type: "apiKey"
info:
contact:
email: "contact@kingsrook.com"
description: "Database performance, live activity and audited query plans"
title: "QQQ Middleware API extension: db-performance"
version: "1.0.0"
openapi: "3.0.3"
paths:
/qqq/v1/qbit/db-performance/overview:
get:
description: "Requires db_performance.view and uses the caller's session."
operationId: "dbPerformanceOverview"
parameters:
- description: "Configured monitored backend; defaults to the qbit backend."
in: "query"
name: "backend"
required: false
schema:
type: "string"
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Invalid request"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Permission denied"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceOverviewV1"
description: "Successful Response"
security:
- sessionUuidCookie:
- "N/A"
summary: "Database Performance overview"
/qqq/v1/qbit/db-performance/activity:
get:
description: "Requires db_performance.view and uses the caller's session."
operationId: "dbPerformanceActivity"
parameters:
- description: "Configured monitored backend; defaults to the qbit backend."
in: "query"
name: "backend"
required: false
schema:
type: "string"
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Invalid request"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Permission denied"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceActivityV1"
description: "Successful Response"
security:
- sessionUuidCookie:
- "N/A"
summary: "Database Performance activity"
/qqq/v1/qbit/db-performance/explain:
post:
description: "PLAN requires db_performance.explain; ANALYZE requires db_performance.explain_analyze.\
\ Values and ANALYZE plans are never persisted."
operationId: "dbPerformanceExplain"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceExplainRequestV1"
required: true
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Invalid request"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Permission denied"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceExplainResultV1"
description: "Successful Response"
security:
- sessionUuidCookie:
- "N/A"
summary: "Explain a PostgreSQL query"
/qqq/v1/qbit/db-performance/statements/lookup:
get:
description: "Requires db_performance.view and uses the caller's session."
operationId: "dbPerformanceLookup"
parameters:
- description: "Configured monitored backend; defaults to the qbit backend."
in: "query"
name: "backend"
required: false
schema:
type: "string"
- in: "query"
name: "sql"
required: true
schema:
type: "string"
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Invalid request"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceErrorV1"
description: "Permission denied"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/DbPerformanceLookupV1"
description: "Successful Response"
security:
- sessionUuidCookie:
- "N/A"
summary: "Database Performance lookup"
@@ -0,0 +1,239 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.math.BigDecimal;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import com.kingsrook.qbits.dbperformance.analyze.RuleContext;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qbits.dbperformance.collect.RollupService;
import com.kingsrook.qbits.dbperformance.widgets.DbPerfWidgetRenderer;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiTableData;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Attribution detail grows by hundreds of thousands of rows a day. On
** 2026-10-16 the home widget loaded a whole day of it into the heap and ran
** the kof22 web pod out of memory, and the hourly analysis filled the worker
** heap the same way. These tests count every attribution detail record QQQ
** materializes, so the widget, the analysis and the daily rollup cannot go
** back to reading the table in one unbounded query.
*******************************************************************************/
class AttributionBoundedReadTest
{
private static final AtomicInteger LOADED = new AtomicInteger();
private static final AtomicInteger MAX_PAGE = new AtomicInteger();
private DbPerfRuntime runtime;
/*******************************************************************************
** Counts the detail records each QueryAction hands back.
*******************************************************************************/
public static class CountingCustomizer implements TableCustomizerInterface
{
/***************************************************************************
** Record the page size, change nothing.
***************************************************************************/
@Override
public List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records)
{
LOADED.addAndGet(records.size());
MAX_PAGE.accumulateAndGet(records.size(), Math::max);
return (records);
}
}
/*******************************************************************************
** Fresh H2 instance with the counter on the detail table.
*******************************************************************************/
@BeforeEach
void setup() throws Exception
{
runtime = DbPerfRuntime.of(DbPerfTestSupport.instance());
runtime.getInstance().getTable("db_perf_attribution").withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(CountingCustomizer.class));
LOADED.set(0);
MAX_PAGE.set(0);
}
/*******************************************************************************
** The widget sums in the database: no detail rows reach the heap, and the
** totals still match the detail.
*******************************************************************************/
@Test
void widgetSumsInTheDatabase() throws Exception
{
Instant now = Instant.now();
insert(3000, i -> row(now.minusSeconds(60), now, i));
LOADED.set(0);
MultiTableData attribution = new DbPerfWidgetRenderer().attribution();
assertThat(LOADED.get()).as("attribution detail rows loaded by the widget").isZero();
List<Map<String, Object>> units = attribution.getTableDataList().getFirst().getRows();
assertThat(units).hasSize(3);
// unit-0 holds i = 0, 3, 6 ... (1000 rows, 2 statements each), mostly against "patient"
assertThat(((Number) units.getFirst().get("Statements")).doubleValue() + ((Number) units.get(1).get("Statements")).doubleValue()
+ ((Number) units.get(2).get("Statements")).doubleValue()).isEqualTo(6000.0);
assertThat(units).allSatisfy(unit -> assertThat(((Number) unit.get("Statements")).doubleValue()).isEqualTo(2000.0));
assertThat(JsonUtils.toJson(units.getFirst())).contains("top table patient");
assertThat(units.stream().mapToDouble(unit -> ((Number) unit.get("Share %")).doubleValue()).sum()).isCloseTo(100.0, org.assertj.core.data.Offset.offset(0.001));
}
/*******************************************************************************
** Detail older than 24 hours is left out of the sums.
*******************************************************************************/
@Test
void widgetKeepsTheDayWindow() throws Exception
{
Instant now = Instant.now();
insert(3, i -> row(now.minusSeconds(60), now, i));
insert(3, i -> row(now.minusSeconds(3 * 86400), now.minusSeconds(2 * 86400), i));
List<Map<String, Object>> units = new DbPerfWidgetRenderer().attribution().getTableDataList().getFirst().getRows();
assertThat(units).hasSize(3).allSatisfy(unit -> assertThat(((Number) unit.get("Statements")).doubleValue()).isEqualTo(2.0));
}
/*******************************************************************************
** The hourly analysis reads attribution as database sums per unit, table
** and operation: no detail rows, same totals.
*******************************************************************************/
@Test
void analysisSumsInTheDatabase() throws Exception
{
Instant now = Instant.now();
insert(3000, i -> row(now.minusSeconds(60), now, i));
LOADED.set(0);
RuleContext context = new RuleContext(runtime, runtime.getConfig().getBackendName(), new QRecord(), new QRecord(), now.plusSeconds(1));
List<QRecord> rows = context.window("attribution", 1);
assertThat(LOADED.get()).as("attribution detail rows loaded by the analysis").isZero();
assertThat(rows).hasSize(6);
assertThat(rows.stream().mapToDouble(row -> RuleContext.number(row, "statements")).sum()).isEqualTo(6000.0);
assertThat(rows.stream().mapToDouble(row -> RuleContext.number(row, "totalMs")).sum()).isEqualTo(1500 * 10.0 + 1500 * 1.0);
assertThat(rows).allSatisfy(row -> assertThat(row.getValueString("operation")).isEqualTo("SELECT"));
}
/*******************************************************************************
** Keyset pages never exceed the page size, and every row is seen once.
*******************************************************************************/
@Test
void forEachPageIsBounded() throws Exception
{
Instant now = Instant.now();
insert(1050, i -> row(now.minusSeconds(60), now, i));
LOADED.set(0);
List<Integer> sizes = new ArrayList<>();
List<Object> ids = new ArrayList<>();
DbPerfStore.forEachPage("db_perf_attribution", new QQueryFilter(), 100, page ->
{
sizes.add(page.size());
page.forEach(row -> ids.add(row.getValue("id")));
});
assertThat(sizes).hasSize(11).allSatisfy(size -> assertThat(size).isLessThanOrEqualTo(100));
assertThat(ids).hasSize(1050).doesNotHaveDuplicates();
assertThat(MAX_PAGE.get()).isLessThanOrEqualTo(100);
}
/*******************************************************************************
** The daily rollup reads the day in BATCH_SIZE pages and its sums still
** cover every detail row across page boundaries.
*******************************************************************************/
@Test
void rollupReadsTheDayInPages() throws Exception
{
ZoneId zone = ZoneId.of(runtime.getConfig().getTimeZone());
LocalDate day = LocalDate.now(zone).minusDays(1);
Instant start = day.atStartOfDay(zone).toInstant().plusSeconds(3600);
int rows = DbPerfStore.BATCH_SIZE * 2 + 500;
insert(rows, i -> row(start, start.plusSeconds(60), i));
MAX_PAGE.set(0);
LOADED.set(0);
new RollupService(runtime).rollupDay(day);
assertThat(MAX_PAGE.get()).as("largest page of attribution detail held at once").isLessThanOrEqualTo(DbPerfStore.BATCH_SIZE);
assertThat(LOADED.get()).isEqualTo(rows);
List<QRecord> daily = DbPerfStore.query("db_perf_attribution_daily", new QQueryFilter());
assertThat(daily).hasSize(6);
assertThat(daily.stream().mapToLong(row -> row.getValueLong("statements")).sum()).isEqualTo(2L * rows);
}
/*******************************************************************************
** Insert generated rows in batches.
*******************************************************************************/
private static void insert(int count, java.util.function.IntFunction<QRecord> factory) throws Exception
{
List<QRecord> rows = new ArrayList<>();
for(int i = 0; i < count; i++)
{
rows.add(factory.apply(i));
}
DbPerfStore.insert("db_perf_attribution", rows);
}
/*******************************************************************************
** Three units; each unit spends most of its time on "patient".
*******************************************************************************/
private static QRecord row(Instant windowStart, Instant windowEnd, int i)
{
boolean patient = i % 6 < 3;
return (new QRecord().withValue("windowStart", windowStart).withValue("windowEnd", windowEnd).withValue("podName", "pod-" + (i % 2))
.withValue("unitKind", "PROCESS").withValue("unitName", "unit-" + (i % 3)).withValue("tableName", patient ? "patient" : "visit")
.withValue("operation", "SELECT").withValue("fingerprint", "f".repeat(64)).withValue("sqlText", "select 1")
.withValue("units", 1L).withValue("statements", 2L).withValue("rowsCount", 1L).withValue("errors", 0L)
.withValue("totalMs", patient ? BigDecimal.TEN : BigDecimal.ONE).withValue("maxMs", BigDecimal.ONE));
}
}
@@ -0,0 +1,121 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.time.Clock;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.List;
import com.kingsrook.qbits.dbperformance.analyze.RuleContext;
import com.kingsrook.qbits.dbperformance.analyze.StoredRule;
import com.kingsrook.qbits.dbperformance.collect.DbPerfSettingsService;
import com.kingsrook.qbits.dbperformance.collect.DbPerfStore;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Real H2/QRecord fixtures shared by rule, alert and action integration tests.
*******************************************************************************/
class B1Fixture
{
static final Instant NOW = Instant.parse("2026-10-15T15:00:00Z");
static final Clock CLOCK = Clock.fixed(NOW, ZoneOffset.UTC);
final DbPerfRuntime runtime;
final String backend;
final QRecord settings;
final QRecord snapshot;
final QRecord table;
final QRecord statement;
/*******************************************************************************
** Evidence crossing every seeded threshold without a live statistics probe.
*******************************************************************************/
B1Fixture() throws Exception
{
runtime = DbPerfRuntime.of(DbPerfTestSupport.instance());
backend = runtime.getConfig().getBackendName();
settings = new DbPerfSettingsService(runtime).ensure(backend);
snapshot = insert("snapshot", new QRecord().withValue("backendName", backend).withValue("capturedAt", NOW).withValue("status", "OK")
.withValue("rawStatsReset", NOW.minusSeconds(30 * 86400L)).withValue("cacheHitRatio", 90).withValue("deadlocks", 2L)
.withValue("connectionsTotal", 90).withValue("connectionsIdle", 60).withValue("maxConnections", 100).withValue("tempBytes", 200 * 1048576L));
table = insert("table_stat", new QRecord().withValue("backendName", backend).withValue("schemaName", "public").withValue("tableName", "audit_orders")
.withValue("lastSeenAt", NOW).withValue("seqScanShare", 80).withValue("seqScan", 800L).withValue("idxScan", 200L)
.withValue("nTupIns", 200000L).withValue("nDeadTup", 30000L).withValue("nLiveTup", 70000L).withValue("deadRatio", 30)
.withValue("lastVacuum", NOW.minusSeconds(10 * 86400L)).withValue("freezeAge", 160000000L));
insert("table_snapshot", new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("tableStatId", table.getValueInteger("id"))
.withValue("capturedAt", NOW).withValue("seqTupRead", 100000L).withValue("nTupIns", 10000L));
statement = insert("statement", new QRecord().withValue("backendName", backend).withValue("queryId", 1L).withValue("dbUser", "app").withValue("toplevel", true)
.withValue("fingerprint", "a".repeat(64)).withValue("queryText", "select * from audit_orders where customer_id = ? and status in (?)")
.withValue("mainTable", "audit_orders").withValue("lastSeenAt", NOW).withValue("calls24h", 100L).withValue("meanMs24h", 600)
.withValue("totalMs24h", 60000).withValue("shareOfDbTime24h", 25));
insert("query_snapshot", new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("statementId", statement.getValueInteger("id"))
.withValue("capturedAt", NOW).withValue("totalMs", 10000).withValue("tempBlksWritten", 150 * 128L));
index("idx_customer", "customer_id", false, 0);
index("idx_customer_copy", "customer_id", false, 0);
index("idx_customer_wide", "customer_id, created_at", false, 20);
insert("repeat_pattern", new QRecord().withValue("windowStart", NOW.minusSeconds(60)).withValue("windowEnd", NOW).withValue("unitKind", "PROCESS")
.withValue("unitName", "shipOrders").withValue("tableName", "orders").withValue("fingerprint", "a".repeat(64))
.withValue("maxRepeats", 20).withValue("unitsAffected", 3L).withValue("totalRepeats", 60L));
insert("repeat_pattern", new QRecord().withValue("windowStart", NOW.minusSeconds(60)).withValue("windowEnd", NOW).withValue("unitKind", "PROCESS")
.withValue("unitName", "shipOrders").withValue("tableName", "orders").withValue("fingerprint", "a".repeat(64))
.withValue("maxRepeats", 10).withValue("unitsAffected", 3L).withValue("totalRepeats", 30L));
insert("attribution", new QRecord().withValue("windowStart", NOW.minusSeconds(60)).withValue("windowEnd", NOW).withValue("unitKind", "PROCESS")
.withValue("unitName", "shipOrders").withValue("tableName", "audit_orders").withValue("operation", "INSERT").withValue("totalMs", 1000).withValue("statements", 300L));
insert("connection_sample", new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", NOW)
.withValue("applicationName", "orders-app").withValue("state", "active").withValue("connectionCount", 30));
insert("connection_sample", new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", NOW)
.withValue("applicationName", "orders-app").withValue("state", "idle").withValue("connectionCount", 60));
insert("activity_sample", new QRecord().withValue("snapshotId", snapshot.getValueInteger("id")).withValue("capturedAt", NOW)
.withValue("applicationName", "orders-app").withValue("reason", "IDLE_IN_TRANSACTION").withValue("xactSeconds", 120));
}
/*******************************************************************************
** Consistent index metadata with collected definition.
*******************************************************************************/
QRecord index(String name, String columns, boolean unique, long scans) throws Exception
{
return (insert("index_stat", new QRecord().withValue("backendName", backend).withValue("schemaName", "public").withValue("tableName", "audit_orders")
.withValue("tableStatId", table.getValueInteger("id")).withValue("indexName", name).withValue("columns", columns)
.withValue("definition", "CREATE " + (unique ? "UNIQUE " : "") + "INDEX " + name + " ON public.audit_orders USING btree (" + columns + ")")
.withValue("isValid", true).withValue("isUnique", unique).withValue("isPrimary", false).withValue("idxScan", scans).withValue("sizeBytes", 2 * 1048576L).withValue("lastSeenAt", NOW)));
}
/*******************************************************************************
** A fresh context observes edits to threshold rows on every call.
*******************************************************************************/
RuleContext context(StoredRule rule) throws Exception
{
return (new RuleContext(runtime, backend, DbPerfStore.first("db_perf_rule", DbPerfStore.eq("code", rule.code())), settings, NOW));
}
/*******************************************************************************
** Use normal QQQ inserts and return the generated identity.
*******************************************************************************/
static QRecord insert(String suffix, QRecord row) throws Exception
{
return (DbPerfStore.insert("db_perf_" + suffix, List.of(row)).getFirst());
}
}
@@ -0,0 +1,66 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.nio.file.Files;
import java.nio.file.Path;
import com.kingsrook.qbits.dbperformance.api.DbPerformanceUiApi;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData;
import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1;
import com.kingsrook.qqq.middleware.javalin.tools.UiApiContract;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The committed fragment is generated from exactly the registered UI specs.
*******************************************************************************/
class DbPerfApiContractTest
{
/*******************************************************************************
** Explicit regeneration flag; normal builds detect contract drift.
*******************************************************************************/
@Test
void contract() throws Exception
{
var extension = DbPerformanceUiApi.extension();
var version = new MiddlewareVersionV1();
String generated = UiApiContract.generateExtensionYaml(version, extension);
Path path = Path.of("src/main/resources/openapi/qqq-v1/db-performance.yaml");
if(Boolean.getBoolean("dbperf.openapi.update"))
{
Files.createDirectories(path.getParent());
Files.writeString(path, generated);
}
assertThat(UiApiContract.check(generated, Files.readString(path), null, 1, "mvnw -Dtest=DbPerfApiContractTest -Ddbperf.openapi.update=true test")).isEmpty();
var model = version.generateExtensionOpenAPIModel("qqq", extension);
assertThat(model.getComponents().getSchemas().keySet()).allMatch(name -> name.startsWith("DbPerformance") && name.endsWith("V1"));
assertThat(model.getPaths().keySet()).containsExactlyInAnyOrder("/qqq/v1/qbit/db-performance/overview", "/qqq/v1/qbit/db-performance/activity",
"/qqq/v1/qbit/db-performance/explain", "/qqq/v1/qbit/db-performance/statements/lookup");
assertThat(extension.getSemanticVersion()).isEqualTo("1.0.0");
assertThat(MiddlewareSpecExtensionsMetaData.of(DbPerfTestSupport.instance()).getExtensions()).containsOnlyKeys("db-performance");
assertThat(generated).contains("liveTextRedacted", "parameters", "assistantPrompt", "403");
}
}
@@ -0,0 +1,79 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2026. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.dbperformance;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import com.kingsrook.qbits.dbperformance.metadata.DbPerfMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Real Liquibase migration and insert-only defaults on H2.
*******************************************************************************/
class DbPerfChangelogTest
{
/*******************************************************************************
** Tables exist, all defaults are seeded and reapplying preserves edits.
*******************************************************************************/
@Test
void migrateTwice() throws Exception
{
QInstance instance = DbPerfTestSupport.instance();
RDBMSBackendMetaData backend = (RDBMSBackendMetaData) instance.getBackends().values().iterator().next();
try(Connection connection = DriverManager.getConnection(backend.getJdbcUrl(), "sa", ""); Statement statement = connection.createStatement())
{
for(String table : DbPerfMetaData.TABLES)
{
try(ResultSet rows = statement.executeQuery("SELECT count(*) FROM " + table))
{
rows.next();
assertThat(rows.getInt(1)).isEqualTo(table.equals("db_perf_rule") ? 14 : 0);
}
}
statement.executeUpdate("UPDATE db_perf_rule SET threshold_1=123 WHERE code='SLOW_QUERY'");
DbPerfTestSupport.migrate(backend);
try(var resource = getClass().getClassLoader().getResourceAsStream(DbPerfMetaData.CHANGELOG))
{
String script = new String(resource.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8);
String seeds = script.substring(script.indexOf("INSERT INTO db_perf_rule"));
for(String sql : seeds.split(";"))
{
if(!sql.isBlank())
{
statement.executeUpdate(sql);
}
}
}
try(ResultSet rows = statement.executeQuery("SELECT threshold_1 FROM db_perf_rule WHERE code='SLOW_QUERY'"))
{
rows.next();
assertThat(rows.getInt(1)).isEqualTo(123);
}
}
}
}
Loaded 100 of 125 files, more files were not shown because too many files have changed in this diff. Show more