feat(store): version the journal's schema and migrate it once, when the service starts (#49)
This commit was merged in pull request #49.
This commit is contained in:
35 files changed
+2899
-267
No files matched your search
@@ -2,6 +2,26 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
The journal has a schema version (2.0). Printing, pairing and history are unchanged.
|
||||
|
||||
- The service brings the journal up to date once, when it starts, in one transaction: either every pending change
|
||||
is made or none is, and a failed start leaves the journal as it was. Until now each part of the service that
|
||||
opened the journal created and altered tables as it went. A journal left by any 1.x release, and by the
|
||||
prereleases before it, is brought to the same schema; no row is changed, except that job events written before
|
||||
events recorded which pairing they belong to are given the saved pairing's, as 1.x did each time it started.
|
||||
- When the journal cannot be brought up to date the service does not start, as before. The log now has one line
|
||||
for the operator: the journal's folder, the kind of failure (the file could not be opened or written, another
|
||||
program has it in use, it is damaged, the disk is full), that the journal was not changed, and what to check.
|
||||
Until now the log had only SQLite's own message.
|
||||
- An older version installed over data that a newer version wrote no longer runs on it. This can happen between
|
||||
release candidates, which share a version number. The service leaves the data exactly as it found it, contacts
|
||||
no server, and reports "A newer version of this application wrote the print data on this computer, and this
|
||||
version cannot use it. Nothing was changed. Install the newer version again to continue printing." Status, log
|
||||
settings and updates stay available, so the newer version can be installed again; every other operation is
|
||||
refused with that sentence. The log has the reason once, as an error written when the service starts. A refused
|
||||
operation is recorded only as debug detail, so a companion that keeps asking does not push the reason out of
|
||||
the log entries it shows.
|
||||
|
||||
Servers that run as several nodes (2.0; the 1.2.x fix carried over). A load balancer in front of such a server
|
||||
names the node it chose in a cookie; the client attached only the session cookie, so its realtime socket could open
|
||||
on one node while its claims and downloads went to another, and a server whose claim gate needs the socket on the
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"Qqq.PrintAgent.Core": 100,
|
||||
"Qqq.PrintAgent.Presentation": 100,
|
||||
"Qqq.PrintAgent.Windows": 86,
|
||||
"Qqq.PrintAgent.Windows": 87,
|
||||
"qqq-print-agent": 100
|
||||
}
|
||||
@@ -134,7 +134,7 @@ Severity is the reviewer's, in the terms above. Status: open, in progress with t
|
||||
| W1 | High | A job notification arriving while settings are applied can be discarded with no later wake (read, not reproduced) | Open |
|
||||
| W2 | High | Printer serialization lives inside the one runtime, keyed by job only | Open (ADR-0004) |
|
||||
| W3 | High | The store's scope is implicit and printer sharing is machine-wide | Open (ADR-0004) |
|
||||
| W4 | High | The schema is migrated on every store open, under a process-wide lock, with no version | Open (ADR-0004) |
|
||||
| W4 | High | The schema is migrated on every store open, under a process-wide lock, with no version | Closed by the change that gives the journal a schema version and a one-time migration (plan step 5 of the 2.0 build): the journal carries a version (`PRAGMA user_version`), the service migrates it once at start in one transaction, and opening a store checks the version and changes nothing |
|
||||
| W5 | High | Any of four tasks ending stops the service; the pipe is created and accepted outside the guarded region | Open (ADR-0004) |
|
||||
| W6 | High | The pipe contract has no version, no server identity and no typed reply; replies are matched by prefix at ten sites | Open (ADR-0004). The version 2 contract exists as types since pull request 37; nothing serves or sends it, and the service still speaks version 1 |
|
||||
| W7 | Medium | Log-file and SQLite I/O run under the state lock | Open |
|
||||
|
||||
+15
-2
@@ -42,9 +42,22 @@ This performs locked restore, Release solution build, core tests, Windows/CLI te
|
||||
|
||||
Wait for the event (a recorded message, a log line, a signal from a fake) with `fixture.WaitAsync`, never for a length of time; its 30-second limit applies to that one wait, and the runtime lives until the fixture is disposed. The runtime reads these from its time provider: the heartbeat interval, session renewal, the reconnect backoff, the update pause and its preparation limit, the catalog refresh interval, command expiry and download pacing. Pass a `ManualTimeProvider` and let a heartbeat interval pass with `fixture.HeartbeatAsync`, or a backoff with `fixture.BackoffPassAsync`. The WebSocket handshake timeout, the acknowledgement timeout and pipe command timeouts still use real time. To show that something did not happen, find an event that can only follow it: commands are handled in arrival order, claims are made one at a time, and frames on one socket arrive in the order they were sent. Where no such event exists (a claim wave or a synchronization pass that decides to do nothing leaves no trace), say so in the test and give the wrong action heartbeat rounds in which to show.
|
||||
|
||||
A test opens a store with `TestState.OpenStore`, the only place the suite constructs one, declares it as `JournalStore` (the test project's alias for the store's type, declared in `TestState.cs`), and deletes its folder with `JournalFiles.Delete`, which empties the SQLite connection pool of that journal only. Do not call `SqliteConnection.ClearAllPools()` in this suite: the classes run in parallel, and clearing every pool disposes a connection that a store in another test has just taken from its pool, which that test sees as `ObjectDisposedException: SQLitePCL.sqlite3`.
|
||||
A test opens a store with `TestState.OpenStore`, the only place the suite constructs one, declares it as `JournalStore` (the test project's alias for the store's type, declared in `TestState.cs`), and deletes its folder with `JournalFiles.Delete`, which empties the SQLite connection pool of that journal only. `TestState.OpenStore` migrates the journal before it opens the store, as the service does when it starts, and `SingleServerHarness` migrates when it is built, because many tests call the runtime without running its loop; a test whose subject is the unmigrated journal uses `TestState.OpenStoreAsFound`. Do not call `SqliteConnection.ClearAllPools()` in this suite: the classes run in parallel, and clearing every pool disposes a connection that a store in another test has just taken from its pool, which that test sees as `ObjectDisposedException: SQLitePCL.sqlite3`.
|
||||
|
||||
`AgentStore` opens its SQLite connection inside the journal's schema lock, and the open must stay there. Microsoft.Data.Sqlite 10.0.0 takes a pooled connection under the pool's lock but records which `SqliteConnection` owns it only afterwards (`SqliteConnectionFactory.GetConnection`, then `SqliteConnectionInternal.Activate`). A second open of the same journal that finds the pool empty in that gap takes the connection for abandoned (`SqliteConnectionPool.ReclaimLeakedConnections`) and is handed it too, so two stores share one native handle and one store's transaction can swallow or roll back the other's writes. Every pooled open of a journal goes through the store's constructor, so the lock keeps any opener out of that gap while another asks the pool. A test that needs SQL of its own against a journal takes its connection from `TestState.OpenJournal`, which opens it with pooling off: a pooled connection opened anywhere else shares the journal's pool without that lock and can fall into the same gap. `AgentStoreOpenRaceTests` fails when the open is moved out of the lock. Read those three library methods again before changing the package version or the pooling of the journal's connection.
|
||||
A store takes its SQLite connection (`JournalDatabase.Open`) inside the journal's lock, and the open must stay there. Microsoft.Data.Sqlite 10.0.0 takes a pooled connection under the pool's lock but records which `SqliteConnection` owns it only afterwards (`SqliteConnectionFactory.GetConnection`, then `SqliteConnectionInternal.Activate`). A second open of the same journal that finds the pool empty in that gap takes the connection for abandoned (`SqliteConnectionPool.ReclaimLeakedConnections`) and is handed it too, so two stores share one native handle and one store's transaction can swallow or roll back the other's writes. Every pooled open of a journal goes through `JournalDatabase.Open`, so the lock keeps any opener out of that gap while another asks the pool. The migration and the version read use connections that are not pooled, and so does a test that needs SQL of its own against a journal: it takes its connection from `TestState.OpenJournal`. A pooled connection opened anywhere else shares the journal's pool without that lock and can fall into the same gap. `AgentStoreOpenRaceTests` fails when the open is moved out of the lock. Read those three library methods again before changing the package version or the pooling of the journal's connection.
|
||||
|
||||
## The journal's schema
|
||||
|
||||
The journal carries a schema version in SQLite's `PRAGMA user_version`: 0 for a journal no versioned build has migrated (a new one, or one a 1.x release left), and from then on the number of migration steps it has passed. `JournalSchema.Version` is the version this build works with, which is the number of steps in `JournalMigration.Steps`.
|
||||
|
||||
- **Migration happens once, when the service starts.** `AgentRuntime.RunAsync` calls `JournalSchema.Migrate` before anything opens a store. It reads the version, and when steps are pending it runs all of them in one transaction on a connection of its own and stamps the new version in that same transaction. A step that fails rolls back every step of that run and the version with them; the service fails to start, as it did when the first store open failed, and the next start tries again. Before the failure is passed on, the runtime logs one line for the operator (`JournalSchema.StartFailureText`): the journal's folder, the kind of failure, that the journal was not changed and what to check. That line never repeats the failure's own message, so it is safe to show whatever the journal holds. Two callers in one process are serialized by the journal's lock, and the second finds nothing to do. Across processes only the service's `worker.lock` keeps a second migration away, so the version is read again inside the transaction, under SQLite's write lock, and that reading decides which steps run: a journal another process migrated in between has no step run on it twice.
|
||||
- **Opening a store never changes the schema.** The store's constructor opens the journal read-write without create and checks the version (`JournalSchema.OpenCurrent`). A missing journal or one behind the version fails with "The print journal has not been brought up to date"; nothing is created. There is no `CREATE TABLE` or `ALTER TABLE` anywhere outside the migration steps. `JournalOpenTests` calls every method of the store and compares `sqlite_master` before and after; its list of method names fails the test when a method is added without being called there. That comparison cannot see a statement that creates a table which already exists, so a second test removes tables from a migrated journal and shows that the store calls which need them fail with "no such table" and put nothing back.
|
||||
- **A step is never edited once it has shipped.** A journal that has passed a step does not run it again, so a changed step would leave old and new installations with different schemas. A schema change is a new file `JournalMigration.V<n>.cs` and a new entry at the end of `Steps`. A step works only through the `JournalMigrationContext` it is given, which belongs to the migration's transaction. Its statements are written out in the step, not built from code that queries the tables: the five `jobs_display_*` indexes repeat the sort expressions of `JobSorting`, and a test compares the two, so changing a sort expression requires a step that replaces its index.
|
||||
- **Step 1 is every schema change of the 1.x releases.** Those releases had no version: each store open created what was missing, and three tables were created on first use. Step 1 creates each table, column and index only when it is missing, with the statements those releases used, so a journal in any 1.x shape, and an empty one, ends in the same schema (`Migrate_EveryShape_EndsInOneSchema`). Rows from before events carried a connection scope get the scope of the pairing saved at that moment, as every 1.x open did.
|
||||
- **A journal from a newer build is not touched.** When the version is higher than this build knows, `Migrate` returns `StateStatus.NewerBuild` with both versions and opens nothing for writing. The service then starts no connection and serves only its pipe: `status` and `snapshot` carry the one sentence in `StateStatus.NewerBuildText`, `transport`, the three log operations and the two update operations work, and every other operation is refused with that sentence. The runtime's `OpenStore` refuses as well, so no path can open a store on that state. The reason is logged once, as an ERROR line when the service starts; each refused operation is logged at DEBUG, to the file and only while debug logging is on, so a client that keeps asking does not fill the 100 entries the snapshot shows. This is the case of an older release candidate installed over a newer one.
|
||||
- **The version is read without writing.** `JournalDatabase.ReadVersion` uses a read-only connection. A service process that was killed leaves its write-ahead log on disk with its latest changes, and a version stamped by a newer build may be only there, so a log with content is read through an ordinary read-only connection. (A service that is stopped in the ordinary way is a different case: Microsoft.Data.Sqlite closes its pooled connections when the process exits, and SQLite folds the log into the journal when the last connection closes.) For a log with content SQLite then creates or rebuilds `journal.db-shm`, its index of the log, which holds no data. The journal and the log are not written. With no log, or an empty one, the journal is opened as immutable, which creates no file at all; an ordinary read-only connection would leave an empty log and an index behind.
|
||||
|
||||
Tests build the state an earlier release left with `LegacyStateBuilder`: the journal from a script of that release's own statements (`tests/Qqq.PrintAgent.Cli.Tests/Fixtures/journal-1.2.1.sql`, transcribed from tag `v1.2.1`, and `journal-oldest.sql`, the oldest shape the history shows), and the pairing file in the serialization and protection 1.x used. `JournalProbe` answers what the tests ask from outside the product: the version, the schema as SQLite records it, a digest of each table's rows and a hash of every file. A new release that changes what is stored gets a fixture of the journal the release before it leaves. `JournalUpgradeTests` runs the upgrade on what an installation can leave besides a cleanly closed journal: a 1.2.1 journal whose rows are only in the log of a killed service (`LegacyStateBuilder.WithJournalLeftByAKill`), each shape between the oldest and 1.2.1, including 1.0.x without the certificate exception table, and a 1.2.1 state folder under the service loop with one job left in SUBMITTING and one in RECEIVED, which the start records as OUTCOME_UNKNOWN and FAILED.
|
||||
|
||||
## Local server and CLI
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# ADR-0004: Multi-server runtime, stored state and the management pipe contract
|
||||
|
||||
Status: accepted for 2.0 under the owner's direction of 2026-11-10 ("take the recommended option, record it, keep going"); not yet implemented. Refines [ADR-0003](0003-version-2.md), which decided that the service keeps up to eight server connections. Revised on 2026-11-10 after an independent review of the first draft; the revision is what is to be built. Revised again on 2026-11-10 after two retrospective reviews of merged work: what changed, and why, is in "Resolved after the retrospective reviews" below.
|
||||
Status: accepted for 2.0 under the owner's direction of 2026-11-10 ("take the recommended option, record it, keep going"); not yet implemented. Refines [ADR-0003](0003-version-2.md), which decided that the service keeps up to eight server connections. Revised on 2026-11-10 after an independent review of the first draft; the revision is what is to be built. Revised again on 2026-11-10 after two retrospective reviews of merged work: what changed, and why, is in "Resolved after the retrospective reviews" below. One sentence about journal rows without a scope was corrected on 2026-11-10 when the journal's schema version was built: see "Resolved while building the journal's schema version".
|
||||
|
||||
## Context
|
||||
|
||||
@@ -217,7 +217,7 @@ A planning pass over the code on 2026-11-10 found places where this record was s
|
||||
|
||||
- **Where the pipe contract lives.** The envelope, error codes, operation names, arguments and results are one set of types in a `Management` folder of the Windows assembly, which the service, the command line and the presentation assembly already reference. A separate assembly would add a project and installer payload for no new boundary.
|
||||
- **The companion between now and the tree.** The wizard and tree are a later stage, and until then the companion in this branch speaks version 1 of the pipe and knows one server. While the service can hold only one server, that server follows the stored files after each version 1 sign-in, enrollment and sign-out. Once it can hold several, an interim adapter serves version 1 requests against the first server in the list: a version 1 sign-in to a new address adds that server and moves it first, and signs the others out, because the old window cannot show a second live connection. The adapter is deleted when the companion speaks version 2. Only then does a request without `v` get the legacy error described above. *Changed on 2026-11-10: see "The interim companion" under "Resolved after the retrospective reviews".*
|
||||
- **An unpaired server has no scope.** In 1.x an unpaired store uses one fixed scope name, which two signed-in, unpaired servers would share. An unpaired server reads defaults and cannot write to the journal. Rows the migration finds under that name, under an "unreadable" name or under no scope belong to no server.
|
||||
- **An unpaired server has no scope.** In 1.x an unpaired store uses one fixed scope name, which two signed-in, unpaired servers would share. An unpaired server reads defaults and cannot write to the journal. Rows the migration finds under that name, under an "unreadable" name or under no scope belong to no server. *Changed on 2026-11-10 for rows under no scope: see "Rows found under no scope" under "Resolved while building the journal's schema version".*
|
||||
- **When a scope becomes current.** The scope's row is inserted, not current, before any file is written. It becomes current in one transaction after the pairing file is in place. On open, a readable pairing whose row is not current is made current, which covers a stop between the two; an unreadable pairing never changes it. *Changed on 2026-11-10: see "When a scope becomes current" under "Resolved after the retrospective reviews".*
|
||||
- **A server's own print limit and the shared printer queue.** The queue skips a server whose head job is not yet staged, and also one that cannot take another print at that moment under its own policy. Both are decided together under the queue's lock, so a server busy on one printer neither holds another printer against everyone nor has its job for an idle printer held by someone else's long print. *Changed on 2026-11-10: see "Print capacity and the shared queue" under "Resolved after the retrospective reviews".*
|
||||
- **The staging ceiling.** 8 GiB for the computer, so each server's guaranteed eighth is 1 GiB, above the default policy of 512 MiB. A server's own limit still applies to its own total. A document larger than a server could ever be given is refused before download and recorded FAILED, instead of waiting forever. *Changed on 2026-11-10: see "The staging ceiling" under "Resolved after the retrospective reviews".*
|
||||
@@ -337,6 +337,19 @@ A fifth gap comes from this section's own decision, not from the review's probe:
|
||||
|
||||
- `ServersAddArgs` has no argument for the sign-in provider's origin, and its comment says the provider's address "is not an argument" (`src/Qqq.PrintAgent.Windows/Management/ServersAddArgs.cs:9-16`). "Who names the sign-in provider" above now says `servers.add` carries it for comparison, and leaves the refusal's error code unsettled. Nothing is blocked today, because no handler exists. The code must be settled when the argument is added: the code table and the golden files are closed by test, and the wizard may not decide by matching text (audit W56).
|
||||
|
||||
## Resolved while building the journal's schema version, 2026-11-10
|
||||
|
||||
The journal's schema version and its one-time migration (plan step 5 of the build) were reviewed independently before they merged. The review found one place where the code and this record disagreed. It is settled here, on 2026-11-10, under the owner's standing instruction for this work: when a decision comes up, take the recommended option, record it and keep going. It follows the reviewer's recommended resolution. It is not the owner's own words, and the owner can overrule it, but only before version 1 of the schema ships: a migration step that has shipped is never edited. The earlier sentence stays where it was written, marked as changed.
|
||||
|
||||
**Rows found under no scope.** This corrects the last sentence of "An unpaired server has no scope" under "Resolved while planning the build", for rows under no scope. What that sentence says about rows under the unpaired name and under an "unreadable" name stands.
|
||||
|
||||
- What the code does: version 1 of the journal's schema gives job events that were written before events recorded their pairing the scope of the pairing that is saved when the migration runs. With no pairing saved that is the unpaired name, and with a pairing that cannot be read it is a new "unreadable" name (`JournalMigration.V1` and `AgentStore.ScopeForUnscopedRows`). Every 1.x start did exactly this: the store ran `UPDATE events SET connection_id=$scope WHERE connection_id IS NULL` each time it was opened, at every tag from `v0.1.6-rc.1` to `v1.2.1`.
|
||||
- What was wrong: the earlier sentence said that rows found under no scope belong to no server. Built as written, version 1 would have had to leave those rows without a scope, or set them apart. A journal with such rows would then show fewer jobs after the upgrade to 2.0 than any 1.x release shows on the same journal.
|
||||
- The decision: the code stays as it is, and this record now says what it does. Rows found under no scope are given the saved pairing's scope by version 1, as every 1.x start gave it to them.
|
||||
- Why. Only a journal last written by the first release candidate (`v0.1.0-rc.1`), or by a build from before it, has such rows: events carry a scope from the next tagged build (`v0.1.6-rc.1`) on, and every start of that build and of every later one adopted the unscoped rows it found. So no 1.x installation reaches version 1 with any, and for the journals that do have them, adopting keeps what 1.x showed the operator: the same jobs, under the same pairing.
|
||||
- The consequence, stated plainly: after version 1 no journal has rows without a scope. The later step that introduces several servers will find none, and needs no rule for them. Rows adopted under a readable pairing become that pairing's rows, which are the first server's after the upgrade, and cannot be told apart afterwards from rows that pairing wrote itself. Rows adopted under the unpaired name or an "unreadable" name belong to no server, as the earlier sentence says of those names.
|
||||
- Considered and not taken: taking the adoption out of version 1, so that the rows stay under no scope until the step that introduces several servers assigns them to no server. It would hide jobs that every build since `v0.1.6-rc.1` shows on the same journal, and the only journals it would treat differently are ones no 1.x release leaves behind.
|
||||
|
||||
## Decisions taken while working, per the owner's direction
|
||||
|
||||
About the runtime:
|
||||
|
||||
@@ -71,7 +71,7 @@ The QQQ server supplies assigned document bytes over HTTPS. The Windows LocalSys
|
||||
|
||||
## Data handling
|
||||
|
||||
The service owns `connection.dpapi`, `journal.db` and its WAL. The MSI retains state through uninstall/reinstall. Revoke/decommission the device and resolve pending jobs before deliberately deleting retained state. Only delivered terminal detail whose failures were reviewed is pruned; final-state evidence is retained. Never publish raw state or customer documents.
|
||||
The service owns `connection.dpapi`, `journal.db` and its WAL. The journal carries a schema version, and only the service changes its schema: once, at start, in one transaction, before anything else opens it. A build that finds a version newer than it knows (an older release candidate installed over a newer one) does not open the journal for writing, contacts no server and reads no protected file; it serves its pipe so that an administrator can see why and install the newer build again. The MSI retains state through uninstall/reinstall. Revoke/decommission the device and resolve pending jobs before deliberately deleting retained state. Only delivered terminal detail whose failures were reviewed is pruned; final-state evidence is retained. Never publish raw state or customer documents.
|
||||
|
||||
Service and companion persist serialized, credential-redacted AppKit logs with 30-day retention, rolling a day's file over to numbered segments at a configurable size (default 100 MiB, at most 10 segments kept per day) rather than suppressing entries; recent job snapshots come from durable connection-scoped metadata. Tray colors describe observed connectivity/queue state, not a guarantee of physical printer readiness or completed output.
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ public sealed partial class AgentRuntime(string directory, Action<string> log, I
|
||||
private DateTimeOffset? _lastServerContact;
|
||||
private IReadOnlyList<JobSnapshot> _queuedJobs = [];
|
||||
private long _queueGeneration;
|
||||
private StateStatus _storedState = new StateStatus.Ready();
|
||||
private TimeProvider Clock => timeProvider ?? TimeProvider.System;
|
||||
private DateTimeOffset ObservationTime => Clock.GetUtcNow();
|
||||
|
||||
@@ -52,24 +53,61 @@ public sealed partial class AgentRuntime(string directory, Action<string> log, I
|
||||
|
||||
|
||||
|
||||
/// <summary>Owns the data directory until Windows requests shutdown.</summary>
|
||||
/// <summary>
|
||||
/// Owns the data directory until Windows requests shutdown. The journal is migrated first, once, before anything opens
|
||||
/// a store. When the stored state cannot be used (a newer build wrote it), it is left exactly as it was found, no
|
||||
/// connection to a server is started, and only the management pipe is served, so that an operator can see why and the
|
||||
/// newer build can be installed again.
|
||||
/// </summary>
|
||||
public async Task RunAsync(CancellationToken token)
|
||||
{
|
||||
Directory.CreateDirectory(directory);
|
||||
using FileStream instance = new(Path.Combine(directory, "worker.lock"), FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
|
||||
_log.Write("INFO", "Print service started.");
|
||||
_storedState = MigrateJournal();
|
||||
if(_storedState.Refusal is string refusal)
|
||||
{
|
||||
_log.Write("ERROR", refusal + (_storedState is StateStatus.NewerBuild newer ? $" The journal is at schema version {newer.FoundVersion}; this version supports up to {newer.SupportedVersion}." : string.Empty)
|
||||
+ " No server connection is started; status, log settings and updates stay available.");
|
||||
await ServePipeAsync(token).ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
await RunRealtimeAsync(token).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Brings the journal up to date, which is the first thing the service does with its stored state. A failure stops
|
||||
/// the service from starting, as a failed first open of the store did before there was a migration. Before the
|
||||
/// failure is passed on, one line is logged for the operator: which journal it was, that it was not changed, and
|
||||
/// what to check.
|
||||
/// </summary>
|
||||
private StateStatus MigrateJournal()
|
||||
{
|
||||
try
|
||||
{
|
||||
return JournalSchema.Migrate(directory);
|
||||
}
|
||||
catch(Exception exception)
|
||||
{
|
||||
_log.Write("ERROR", JournalSchema.StartFailureText(directory, exception));
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Opens the journal and the protected files of the state directory for one unit of work. Every part of the runtime
|
||||
/// opens its store here, so what a store is opened for is decided in one place. With <paramref name="recover"/> a saved
|
||||
/// connection that can no longer be read is set aside instead of failing the open, which the paths that must keep
|
||||
/// working without a readable pairing (status, sign-in, enrollment, settings) ask for.
|
||||
/// working without a readable pairing (status, sign-in, enrollment, settings) ask for. While the stored state cannot be
|
||||
/// used no store is opened at all, whatever asked for one: that state belongs to another build.
|
||||
/// </summary>
|
||||
private AgentStore OpenStore(bool recover) => new(directory, recoverUnreadableConfiguration: recover);
|
||||
private AgentStore OpenStore(bool recover) => _storedState.Refusal is string refusal
|
||||
? throw new InvalidOperationException(refusal)
|
||||
: new(directory, recoverUnreadableConfiguration: recover);
|
||||
|
||||
|
||||
|
||||
@@ -117,9 +155,16 @@ public sealed partial class AgentRuntime(string directory, Action<string> log, I
|
||||
|
||||
|
||||
|
||||
/// <summary>Current local observations, with no token or document content.</summary>
|
||||
/// <summary>
|
||||
/// Current local observations, with no token or document content. While the stored state cannot be used the snapshot
|
||||
/// is built without it: red, with the reason as its status, no pairing, no printers and no jobs.
|
||||
/// </summary>
|
||||
public AgentSnapshot GetSnapshot()
|
||||
{
|
||||
if(_storedState.Refusal is string refusal)
|
||||
{
|
||||
return new AgentSnapshot(Enrolled: false, Connected: false, "RED", refusal, ServerUrl: null, DeviceId: null, LastSuccessfulPoll: null, [], [], _log.Entries(), Transport: GetTransportSnapshot());
|
||||
}
|
||||
using AgentStore store = OpenStore(recover: true);
|
||||
AgentConfiguration? configuration = store.LoadConfiguration();
|
||||
IReadOnlySet<string> exposed = store.GetExposedPrinterIds();
|
||||
@@ -228,10 +273,16 @@ public sealed partial class AgentRuntime(string directory, Action<string> log, I
|
||||
|
||||
/// <summary>
|
||||
/// Dispatches one management-pipe operation by name to its handler and returns the text sent back to the caller. An
|
||||
/// unknown operation is refused, and exposure changes validate their arguments before touching the store.
|
||||
/// unknown operation is refused, and exposure changes validate their arguments before touching the store. While the
|
||||
/// stored state cannot be used, only the operations that need none of it are dispatched and every other one is refused
|
||||
/// with the reason.
|
||||
/// </summary>
|
||||
private async Task<string> HandleCommandAsync(PipeRequest request, CancellationToken token)
|
||||
{
|
||||
if(_storedState.Refusal is string refusal && !AnsweredWithoutStoredState(request.Operation))
|
||||
{
|
||||
return RefuseWithoutStoredState(request.Operation, refusal);
|
||||
}
|
||||
switch(request.Operation)
|
||||
{
|
||||
case "set-device-profile":
|
||||
@@ -286,5 +337,28 @@ public sealed partial class AgentRuntime(string directory, Action<string> log, I
|
||||
throw new InvalidOperationException("Unknown command.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Answers an operation that is refused because the stored state cannot be used. The caller gets the reason as an
|
||||
/// error, in the form a failed command has. The reason was logged once, as an error, when the service started; each
|
||||
/// refusal is logged as debug detail only, to the file and only while debug logging is on. A companion that keeps
|
||||
/// asking would otherwise add an error line for every request and push the reason out of the entries it shows.
|
||||
/// </summary>
|
||||
private string RefuseWithoutStoredState(string operation, string refusal)
|
||||
{
|
||||
_log.Write("DEBUG", $"Refused '{operation}': the stored state cannot be used.");
|
||||
return "Error: " + refusal;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The operations the service still answers while the stored state cannot be used: what it is doing and why (status,
|
||||
/// snapshot, and the transport read the companion repeats every second), the log operations, and the two update
|
||||
/// operations, without which the build that can use the state could not be installed. None of them opens a store.
|
||||
/// </summary>
|
||||
private static bool AnsweredWithoutStoredState(string operation)
|
||||
=> operation is "status" or "snapshot" or "transport" or "get-log-settings" or "set-log-settings" or "clear-logs" or "prepare-update" or "resume-after-update";
|
||||
}
|
||||
@@ -17,21 +17,6 @@ namespace Qqq.PrintAgent.Windows;
|
||||
/// </summary>
|
||||
public sealed partial class AgentStore
|
||||
{
|
||||
/// <summary>Creates the control tables and their indexes when they do not exist yet; existing data is untouched.</summary>
|
||||
private void InitializeControls()
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = """
|
||||
CREATE TABLE IF NOT EXISTS control_state(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL,updated_at TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS control_commands(connection_id TEXT NOT NULL,command_id TEXT NOT NULL,command_json TEXT NOT NULL,receipt_json TEXT,acknowledged INTEGER NOT NULL DEFAULT 0,expires_at TEXT NOT NULL,PRIMARY KEY(connection_id,command_id));
|
||||
CREATE INDEX IF NOT EXISTS control_outbox ON control_commands(connection_id,acknowledged);
|
||||
CREATE INDEX IF NOT EXISTS control_expiry ON control_commands(expires_at);
|
||||
""";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Reads this connection's saved control state, or the initial (unrestricted-but-unsynchronized) state when none was saved or it cannot be read.</summary>
|
||||
public AgentControlState LoadControlState()
|
||||
{
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Globalization;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// Creation-time migration of the job table. It exists so jobs saved before the column was added still sort by when they were created,
|
||||
/// instead of by their last update.
|
||||
/// </summary>
|
||||
public sealed partial class AgentStore
|
||||
{
|
||||
/// <summary>Adds the created-at column if missing, filling it from the first recorded event (or the update time), and ensures one sort index per display field. It runs in a single transaction so a failure leaves the old table unchanged.</summary>
|
||||
private void MigrateJobCreationTime()
|
||||
{
|
||||
using SqliteTransaction transaction = _database.BeginTransaction();
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.Transaction = transaction;
|
||||
command.CommandText = "SELECT COUNT(*) FROM pragma_table_info('local_jobs') WHERE name='created_at'";
|
||||
if(Convert.ToInt64(command.ExecuteScalar(), CultureInfo.InvariantCulture) == 0)
|
||||
{
|
||||
command.CommandText = """
|
||||
ALTER TABLE local_jobs ADD COLUMN created_at TEXT;
|
||||
UPDATE local_jobs SET created_at=COALESCE(
|
||||
(SELECT recorded_at FROM events WHERE events.connection_id=local_jobs.connection_id
|
||||
AND events.job_id=local_jobs.job_id AND recorded_at IS NOT NULL ORDER BY sequence LIMIT 1),updated_at);
|
||||
""";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
foreach(string field in JobSorting.Fields)
|
||||
{
|
||||
command.CommandText = "CREATE INDEX IF NOT EXISTS jobs_display_" + field + " ON local_jobs(connection_id," + JobSorting.SqlColumn(field) + " COLLATE NOCASE DESC,job_id DESC)";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
transaction.Commit();
|
||||
}
|
||||
}
|
||||
@@ -31,27 +31,6 @@ public sealed partial class AgentStore
|
||||
|
||||
|
||||
|
||||
/// <summary>Creates the indexes that keep outbox drains, status and printer filters, the attention list, retention and prefix searches fast as history grows; each is created only if missing.</summary>
|
||||
private void CreateJobIndexes()
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = """
|
||||
CREATE INDEX IF NOT EXISTS events_outbox ON events(connection_id,acknowledged,sequence);
|
||||
CREATE INDEX IF NOT EXISTS events_job ON events(connection_id,job_id,sequence);
|
||||
CREATE INDEX IF NOT EXISTS jobs_recent ON local_jobs(connection_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_status ON local_jobs(connection_id,state,failure_acknowledged,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer ON local_jobs(connection_id,printer_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_attention ON local_jobs(connection_id,CASE WHEN state IN ('FAILED','OUTCOME_UNKNOWN') AND failure_acknowledged=0 THEN 0 ELSE 1 END,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_retention ON local_jobs(connection_id,history_pruned,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_title_search ON local_jobs(connection_id,title COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer_search ON local_jobs(connection_id,printer_name COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_id_search ON local_jobs(connection_id,job_id COLLATE NOCASE);
|
||||
""";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Returns one page of this connection's jobs in one read transaction, so the count and the rows agree. The query is
|
||||
/// validated first; search is an escaped prefix match on job identifier, title or printer name; status, printer and
|
||||
|
||||
@@ -44,10 +44,9 @@ public sealed partial class AgentStore
|
||||
|
||||
|
||||
|
||||
/// <summary>Reads one stored profile by key as JSON, creating the table first when this is a store from before profiles existed; a missing key yields the default.</summary>
|
||||
/// <summary>Reads one stored profile by key as JSON; a missing key yields the default.</summary>
|
||||
private T? ReadProfile<T>(string key)
|
||||
{
|
||||
EnsureMetadataTable();
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "SELECT json FROM local_metadata WHERE profile_key=$key";
|
||||
command.Parameters.AddWithValue("$key", key);
|
||||
@@ -59,7 +58,6 @@ public sealed partial class AgentStore
|
||||
/// <summary>Inserts or replaces one profile as JSON in a single statement, so a save is all-or-nothing.</summary>
|
||||
private void WriteProfile<T>(string key, T profile)
|
||||
{
|
||||
EnsureMetadataTable();
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "INSERT INTO local_metadata(profile_key,json) VALUES($key,$json) ON CONFLICT(profile_key) DO UPDATE SET json=excluded.json";
|
||||
command.Parameters.AddWithValue("$key", key);
|
||||
@@ -69,19 +67,6 @@ public sealed partial class AgentStore
|
||||
|
||||
|
||||
|
||||
/// <summary>Creates the profile table on first use under this journal's schema lock, so concurrent stores of the same journal cannot race creating it.</summary>
|
||||
private void EnsureMetadataTable()
|
||||
{
|
||||
lock(SchemaLock)
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS local_metadata(profile_key TEXT PRIMARY KEY,json TEXT NOT NULL)";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Trims a field and rejects it when it exceeds the maximum or contains control characters. Only the description may
|
||||
/// span lines (carriage return, line feed and tab are allowed there), so single-line fields cannot smuggle line breaks into logs or tables.
|
||||
|
||||
@@ -19,7 +19,7 @@ public sealed partial class AgentStore
|
||||
public RealtimeSettings LoadRealtimeSettings()
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS realtime_settings(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL); SELECT json FROM realtime_settings WHERE connection_id=$scope";
|
||||
command.CommandText = "SELECT json FROM realtime_settings WHERE connection_id=$scope";
|
||||
command.Parameters.AddWithValue("$scope", _connectionScope);
|
||||
RealtimeSettings settings = command.ExecuteScalar() is string json ? JsonSerializer.Deserialize<RealtimeSettings>(json, PrintApiClient.JsonOptions) ?? new() : new();
|
||||
settings.Validate();
|
||||
@@ -33,7 +33,7 @@ public sealed partial class AgentStore
|
||||
{
|
||||
settings.Validate();
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS realtime_settings(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL); INSERT INTO realtime_settings(connection_id,json) VALUES($scope,$json) ON CONFLICT(connection_id) DO UPDATE SET json=$json";
|
||||
command.CommandText = "INSERT INTO realtime_settings(connection_id,json) VALUES($scope,$json) ON CONFLICT(connection_id) DO UPDATE SET json=$json";
|
||||
command.Parameters.AddWithValue("$scope", _connectionScope);
|
||||
command.Parameters.AddWithValue("$json", JsonSerializer.Serialize(settings, PrintApiClient.JsonOptions));
|
||||
command.ExecuteNonQuery();
|
||||
|
||||
@@ -19,7 +19,7 @@ public sealed partial class AgentStore
|
||||
public ServerTrust LoadServerTrust()
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS server_trust(scope TEXT PRIMARY KEY,json TEXT NOT NULL); SELECT json FROM server_trust WHERE scope='machine'";
|
||||
command.CommandText = "SELECT json FROM server_trust WHERE scope='machine'";
|
||||
try
|
||||
{ return command.ExecuteScalar() is string json ? JsonSerializer.Deserialize<ServerTrust>(json, PrintApiClient.JsonOptions) ?? new() : new(); }
|
||||
catch(JsonException)
|
||||
@@ -33,7 +33,7 @@ public sealed partial class AgentStore
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(trust);
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS server_trust(scope TEXT PRIMARY KEY,json TEXT NOT NULL); INSERT INTO server_trust(scope,json) VALUES('machine',$json) ON CONFLICT(scope) DO UPDATE SET json=$json";
|
||||
command.CommandText = "INSERT INTO server_trust(scope,json) VALUES('machine',$json) ON CONFLICT(scope) DO UPDATE SET json=$json";
|
||||
command.Parameters.AddWithValue("$json", JsonSerializer.Serialize(trust, PrintApiClient.JsonOptions));
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
@@ -16,84 +15,35 @@ namespace Qqq.PrintAgent.Windows;
|
||||
/// The service's durable local state in one SQLite journal plus a DPAPI-protected connection file. It records every job
|
||||
/// event before it is reported (so a crash never loses an outcome), keeps events and job summaries scoped to the server and
|
||||
/// device they belong to, and holds the printer exposure choices. Other parts of this class add controls, metadata,
|
||||
/// authentication, paging and retention. One instance owns one connection and is not shared across threads.
|
||||
/// authentication, paging and retention. One instance owns one connection and is not shared across threads. The store
|
||||
/// works on the journal's schema and never changes it: <see cref="JournalSchema"/> owns the version and the migration.
|
||||
/// </summary>
|
||||
public sealed partial class AgentStore : IJobJournal, IDisposable
|
||||
{
|
||||
/// <summary>Selects the Windows system SQLite library once, so the agent ships no native SQLite binary of its own.</summary>
|
||||
static AgentStore() => SQLitePCL.raw.SetProvider(new SQLitePCL.SQLite3Provider_winsqlite3());
|
||||
|
||||
|
||||
|
||||
private const string UnenrolledScope = "unenrolled";
|
||||
private readonly SqliteConnection _database;
|
||||
private readonly string _directory;
|
||||
private static readonly ConcurrentDictionary<string, Lock> SchemaLocks = new(StringComparer.OrdinalIgnoreCase);
|
||||
private string _connectionScope = "unenrolled";
|
||||
private string _connectionScope = UnenrolledScope;
|
||||
private bool _configurationRecoveryRequired;
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The lock that serializes schema creation and migration for the journal in a folder. Every store this process opens on
|
||||
/// that folder shares it, so two of them cannot both add a column or a table. Stores on different folders do not wait for
|
||||
/// each other: their journals are separate files, and a single lock for the whole process made every open queue behind
|
||||
/// every other one. The lock is found by the folder's full path without a trailing separator, compared without regard to
|
||||
/// letter case. That covers one path written relative or absolute, in another letter case, or with a trailing separator.
|
||||
/// It does not cover other routes to the same folder: a short (8.3) name, a junction or a substituted drive letter each
|
||||
/// get a lock of their own.
|
||||
/// </summary>
|
||||
internal static Lock SchemaLockFor(string directory)
|
||||
=> SchemaLocks.GetOrAdd(Path.TrimEndingDirectorySeparator(Path.GetFullPath(directory)), static _ => new Lock());
|
||||
|
||||
|
||||
|
||||
/// <summary>The schema lock of this store's journal.</summary>
|
||||
private Lock SchemaLock => SchemaLockFor(_directory);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The connection string every store uses for the journal in a folder. SQLite keeps one pool of idle connections per
|
||||
/// connection string, so this text also names the journal's pool; a test that must delete a journal releases that pool by
|
||||
/// it, and no other.
|
||||
/// </summary>
|
||||
internal static string JournalConnectionString(string directory)
|
||||
=> new SqliteConnectionStringBuilder { DataSource = Path.Combine(Path.GetFullPath(directory), "journal.db") }.ToString();
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Opens (creating if needed) the journal under the protected state directory and brings its schema up to date. With
|
||||
/// recovery enabled, an unreadable saved connection no longer blocks opening: the store switches to an isolated scope so
|
||||
/// old evidence is kept but never attached to a replacement identity. On any failure the connection is closed. The
|
||||
/// connection is opened under the journal's schema lock, which keeps two stores from being handed one database connection.
|
||||
/// Opens the journal under the protected state directory. The journal must already be at this build's schema version:
|
||||
/// the service migrates it when it starts (<see cref="JournalSchema.Migrate(string)"/>), and a store neither creates
|
||||
/// the journal nor changes its schema, so opening one before that fails. With recovery enabled, an unreadable saved
|
||||
/// connection no longer blocks opening: the store switches to an isolated scope so old evidence is kept but never
|
||||
/// attached to a replacement identity. On any failure the connection is closed. The connection is taken under the
|
||||
/// journal's lock, which keeps two stores from being handed one database connection.
|
||||
/// </summary>
|
||||
public AgentStore(string directory, bool recoverUnreadableConfiguration = false)
|
||||
{
|
||||
_directory = Path.GetFullPath(directory);
|
||||
Qqq.PrintAgent.Core.MachineStateDirectory.EnsureForPath(_directory);
|
||||
Directory.CreateDirectory(_directory);
|
||||
_database = new SqliteConnection(JournalConnectionString(_directory));
|
||||
_database = JournalSchema.OpenCurrent(_directory);
|
||||
try
|
||||
{
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
// Open must stay inside this lock. Microsoft.Data.Sqlite 10.0.0 //
|
||||
// takes a pooled connection and only afterwards, outside the //
|
||||
// pool's own lock, records which SqliteConnection owns it. When //
|
||||
// another thread opens the same journal in that gap and finds //
|
||||
// the pool empty, the pool takes the first connection for //
|
||||
// abandoned and hands it to the second thread as well. Both //
|
||||
// stores then share one native handle: one store's transaction //
|
||||
// can swallow or roll back the other's writes. Every open of one //
|
||||
// journal goes through this constructor, and the lock is at //
|
||||
// least as wide as the pool, so with Open inside it no opener //
|
||||
// can be in that gap while another asks the pool. //
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
lock(SchemaLock)
|
||||
{
|
||||
_database.Open();
|
||||
}
|
||||
InitializeDatabase(recoverUnreadableConfiguration);
|
||||
ResolveScope(recoverUnreadableConfiguration);
|
||||
}
|
||||
catch
|
||||
{
|
||||
@@ -105,60 +55,46 @@ public sealed partial class AgentStore : IJobJournal, IDisposable
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Creates or migrates the schema under this journal's schema lock: events, exposure, per-job summaries, and the added
|
||||
/// timestamp and scope columns, with legacy rows assigned to the current scope. Job summaries are rebuilt from the latest
|
||||
/// event only the first time that table appears. The migration runs in a transaction so a failure leaves the old schema usable.
|
||||
/// Sets the connection scope from the saved pairing. A pairing that cannot be read fails the open, unless recovery was
|
||||
/// asked for: then the store works in a scope of its own that nothing else uses.
|
||||
/// </summary>
|
||||
private void InitializeDatabase(bool recoverUnreadableConfiguration)
|
||||
private void ResolveScope(bool recoverUnreadableConfiguration)
|
||||
{
|
||||
lock(SchemaLock)
|
||||
try
|
||||
{
|
||||
using SqliteCommand command = _database.CreateCommand();
|
||||
command.CommandText = "PRAGMA journal_mode=WAL; CREATE TABLE IF NOT EXISTS events(sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL, event_id TEXT NOT NULL UNIQUE, state TEXT NOT NULL, json TEXT NOT NULL, acknowledged INTEGER NOT NULL DEFAULT 0);";
|
||||
command.ExecuteNonQuery();
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS exposed_printers(printer_id TEXT PRIMARY KEY); SELECT COUNT(*) FROM pragma_table_info('events') WHERE name='recorded_at';";
|
||||
long hasTimestamp = Convert.ToInt64(command.ExecuteScalar(), System.Globalization.CultureInfo.InvariantCulture);
|
||||
if(hasTimestamp == 0)
|
||||
{
|
||||
command.CommandText = "ALTER TABLE events ADD COLUMN recorded_at TEXT";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
command.CommandText = "SELECT COUNT(*) FROM pragma_table_info('events') WHERE name='connection_id'";
|
||||
if(Convert.ToInt64(command.ExecuteScalar(), System.Globalization.CultureInfo.InvariantCulture) == 0)
|
||||
{
|
||||
command.CommandText = "ALTER TABLE events ADD COLUMN connection_id TEXT";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
try
|
||||
{
|
||||
_ = LoadConfiguration();
|
||||
}
|
||||
catch(Exception exception) when(recoverUnreadableConfiguration && exception is CryptographicException or JsonException or ArgumentException or UriFormatException)
|
||||
{
|
||||
// Only explicit re-enrollment may replace unreadable credentials. Never attach
|
||||
// unscoped legacy evidence to the new identity when its original scope is unknown.
|
||||
_configurationRecoveryRequired = true;
|
||||
_connectionScope = "unreadable-" + Guid.NewGuid().ToString("N");
|
||||
}
|
||||
command.CommandText = "UPDATE events SET connection_id=$scope WHERE connection_id IS NULL";
|
||||
command.Parameters.AddWithValue("$scope", _connectionScope);
|
||||
command.ExecuteNonQuery();
|
||||
command.CommandText = "SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='local_jobs'";
|
||||
bool migrateJobs = Convert.ToInt64(command.ExecuteScalar(), System.Globalization.CultureInfo.InvariantCulture) == 0;
|
||||
using SqliteTransaction migration = _database.BeginTransaction();
|
||||
command.Transaction = migration;
|
||||
command.CommandText = "CREATE TABLE IF NOT EXISTS local_jobs(connection_id TEXT NOT NULL, job_id TEXT NOT NULL, printer_id TEXT, printer_name TEXT, title TEXT, format TEXT, state TEXT, last_event_json TEXT, updated_at TEXT, failure_acknowledged INTEGER NOT NULL DEFAULT 0, history_pruned INTEGER NOT NULL DEFAULT 0, PRIMARY KEY(connection_id,job_id));";
|
||||
command.ExecuteNonQuery();
|
||||
if(migrateJobs)
|
||||
{
|
||||
command.CommandText = "INSERT OR IGNORE INTO local_jobs(connection_id,job_id,state,last_event_json,updated_at) SELECT connection_id,job_id,state,json,recorded_at FROM events WHERE sequence IN (SELECT MAX(sequence) FROM events GROUP BY connection_id,job_id);";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
migration.Commit();
|
||||
command.Transaction = null;
|
||||
CreateJobIndexes();
|
||||
MigrateJobCreationTime();
|
||||
InitializeControls();
|
||||
_ = LoadConfiguration();
|
||||
}
|
||||
catch(Exception exception) when(recoverUnreadableConfiguration && exception is CryptographicException or JsonException or ArgumentException or UriFormatException)
|
||||
{
|
||||
// Only explicit re-enrollment may replace unreadable credentials. Never attach
|
||||
// unscoped legacy evidence to the new identity when its original scope is unknown.
|
||||
_configurationRecoveryRequired = true;
|
||||
_connectionScope = UnreadableScope();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>A scope name for a pairing that cannot be read. It is new each time, so nothing recorded under it is ever found under a readable pairing.</summary>
|
||||
private static string UnreadableScope() => "unreadable-" + Guid.NewGuid().ToString("N");
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The scope that journal rows without one belong to: the scope of the pairing saved in the state folder, the
|
||||
/// unenrolled scope when there is none, and a scope of their own when the pairing cannot be read. The 1.x releases gave
|
||||
/// such rows this scope each time the service opened its store; the migration does it once, and is the only caller.
|
||||
/// </summary>
|
||||
internal static string ScopeForUnscopedRows(string directory)
|
||||
{
|
||||
string path = Path.Combine(Path.GetFullPath(directory), "connection.dpapi");
|
||||
try
|
||||
{
|
||||
return File.Exists(path) ? ConnectionScope(ReadConfiguration(path)) : UnenrolledScope;
|
||||
}
|
||||
catch(Exception exception) when(exception is CryptographicException or JsonException or ArgumentException or UriFormatException)
|
||||
{
|
||||
return UnreadableScope();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -213,13 +149,20 @@ public sealed partial class AgentStore : IJobJournal, IDisposable
|
||||
{
|
||||
return null;
|
||||
}
|
||||
AgentConfiguration? configuration = ReadConfiguration(path);
|
||||
_connectionScope = ConnectionScope(configuration);
|
||||
return configuration;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Decrypts and parses the pairing file at a path, which must exist. The plaintext buffer is zeroed on every path, and a file that cannot be decrypted or parsed throws.</summary>
|
||||
private static AgentConfiguration? ReadConfiguration(string path)
|
||||
{
|
||||
byte[] plaintext = ProtectedData.Unprotect(File.ReadAllBytes(path), optionalEntropy: null, DataProtectionScope.CurrentUser);
|
||||
try
|
||||
{
|
||||
AgentConfiguration? configuration = JsonSerializer.Deserialize<AgentConfiguration>(plaintext, PrintApiClient.JsonOptions);
|
||||
_connectionScope = ConnectionScope(configuration);
|
||||
return configuration;
|
||||
return JsonSerializer.Deserialize<AgentConfiguration>(plaintext, PrintApiClient.JsonOptions);
|
||||
}
|
||||
finally { CryptographicOperations.ZeroMemory(plaintext); }
|
||||
}
|
||||
@@ -472,12 +415,10 @@ public sealed partial class AgentStore : IJobJournal, IDisposable
|
||||
{
|
||||
if(configuration is null)
|
||||
{
|
||||
return "unenrolled";
|
||||
return UnenrolledScope;
|
||||
}
|
||||
string address = new Uri(configuration.ServerUrl).AbsoluteUri.TrimEnd('/');
|
||||
return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(address + "\n" + configuration.DeviceId)));
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
@@ -11,6 +11,12 @@ namespace Qqq.PrintAgent.Windows;
|
||||
/// <summary>One whitelisted order for SQLite pages and the bounded in-memory cloud queue merge.</summary>
|
||||
internal static class JobSorting
|
||||
{
|
||||
/// <summary>
|
||||
/// The fields the job list can be sorted by. The journal has one index for each, named <c>jobs_display_</c> plus the
|
||||
/// field and built on the expression <see cref="SqlColumn"/> returns for it. Those indexes are written out in the
|
||||
/// journal's first migration step, so a field added here, or an expression changed below, needs a new migration step
|
||||
/// that creates or replaces its index; a test compares the two and fails until it does.
|
||||
/// </summary>
|
||||
internal static readonly string[] Fields = ["createdAt", "title", "printerName", "status", "format"];
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
using System.Globalization;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// The journal file of a state folder and the three ways it is opened: by a store, to work with it; by the migration, to
|
||||
/// change its schema; and read-only, to learn its schema version without touching it. It also owns the lock of each
|
||||
/// folder's journal, which a store holds while it takes its connection and the migration holds while it runs. Nothing
|
||||
/// here knows what the journal contains; <see cref="JournalSchema"/> does.
|
||||
/// </summary>
|
||||
internal static class JournalDatabase
|
||||
{
|
||||
/// <summary>The journal's file name inside the state folder.</summary>
|
||||
internal const string FileName = "journal.db";
|
||||
|
||||
|
||||
|
||||
private static readonly ConcurrentDictionary<string, Lock> Locks = new(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
|
||||
|
||||
/// <summary>Selects the Windows system SQLite library once, so the agent ships no native SQLite binary of its own. Every way of opening a journal passes through this type, so the library is selected before the first one.</summary>
|
||||
static JournalDatabase() => SQLitePCL.raw.SetProvider(new SQLitePCL.SQLite3Provider_winsqlite3());
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The lock of the journal in a folder. Every store this process opens on that folder takes it while it opens its
|
||||
/// connection, and the migration holds it while it changes the schema, so two callers cannot both migrate one journal.
|
||||
/// Journals in different folders do not wait for each other: they are separate files, and a single lock for the whole
|
||||
/// process made every open queue behind every other one. The lock is found by the folder's full path without a
|
||||
/// trailing separator, compared without regard to letter case. That covers one path written relative or absolute, in
|
||||
/// another letter case, or with a trailing separator. It does not cover other routes to the same folder: a short (8.3)
|
||||
/// name, a junction or a substituted drive letter each get a lock of their own.
|
||||
/// </summary>
|
||||
internal static Lock LockFor(string directory)
|
||||
=> Locks.GetOrAdd(Path.TrimEndingDirectorySeparator(Path.GetFullPath(directory)), static _ => new Lock());
|
||||
|
||||
|
||||
|
||||
/// <summary>The full path of the journal file in a folder.</summary>
|
||||
internal static string PathOf(string directory) => Path.Combine(Path.GetFullPath(directory), FileName);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The connection string every store uses for the journal in a folder. SQLite keeps one pool of idle connections per
|
||||
/// connection string, so this text also names the journal's pool; a test that must delete a journal releases that pool by
|
||||
/// it, and no other. The mode is read-write without create: a store works on a journal the migration made and never
|
||||
/// makes one itself.
|
||||
/// </summary>
|
||||
internal static string ConnectionString(string directory)
|
||||
=> new SqliteConnectionStringBuilder { DataSource = PathOf(directory), Mode = SqliteOpenMode.ReadWrite }.ToString();
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Opens a store's connection to the journal in a folder, from the journal's pool, while holding the journal's lock.
|
||||
/// The caller owns the connection. One that could not be opened is disposed before the failure is passed on.
|
||||
/// </summary>
|
||||
internal static SqliteConnection Open(string directory)
|
||||
{
|
||||
SqliteConnection connection = new(ConnectionString(directory));
|
||||
try
|
||||
{
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
// Open must stay inside this lock. Microsoft.Data.Sqlite 10.0.0 //
|
||||
// takes a pooled connection and only afterwards, outside the //
|
||||
// pool's own lock, records which SqliteConnection owns it. When //
|
||||
// another thread opens the same journal in that gap and finds //
|
||||
// the pool empty, the pool takes the first connection for //
|
||||
// abandoned and hands it to the second thread as well. Both //
|
||||
// stores then share one native handle: one store's transaction //
|
||||
// can swallow or roll back the other's writes. Every pooled open //
|
||||
// of one journal goes through this method, and the lock is at //
|
||||
// least as wide as the pool, so with Open inside it no opener //
|
||||
// can be in that gap while another asks the pool. //
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
lock(LockFor(directory))
|
||||
{
|
||||
connection.Open();
|
||||
}
|
||||
return connection;
|
||||
}
|
||||
catch
|
||||
{
|
||||
connection.Dispose();
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Opens a connection to the journal in a folder that is not pooled, creating the file when it is missing. The migration
|
||||
/// works on one, and so does a test that needs SQL of its own. Unpooled, it has a native handle that no store can be
|
||||
/// handed by the pool, and closing it releases the file at once.
|
||||
/// </summary>
|
||||
internal static SqliteConnection OpenUnpooled(string directory)
|
||||
{
|
||||
SqliteConnection connection = new(new SqliteConnectionStringBuilder { DataSource = PathOf(directory), Pooling = false }.ToString());
|
||||
try
|
||||
{
|
||||
connection.Open();
|
||||
return connection;
|
||||
}
|
||||
catch
|
||||
{
|
||||
connection.Dispose();
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Reads the schema version of the journal in a folder (<c>PRAGMA user_version</c>) through a read-only connection, or
|
||||
/// returns 0 when there is no journal. It never writes to the journal or to its write-ahead log, so it is safe to ask of
|
||||
/// a journal this build must not touch.
|
||||
/// </summary>
|
||||
internal static int ReadVersion(string directory)
|
||||
{
|
||||
string path = PathOf(directory);
|
||||
if(!File.Exists(path))
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
using SqliteConnection connection = new(ReadOnlyConnectionString(path));
|
||||
connection.Open();
|
||||
using SqliteCommand command = connection.CreateCommand();
|
||||
command.CommandText = "PRAGMA user_version";
|
||||
return Convert.ToInt32(command.ExecuteScalar(), CultureInfo.InvariantCulture);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The connection string for reading a journal without changing anything on disk. Which one that is depends on the
|
||||
/// write-ahead log. A process that was killed leaves its log behind with the latest changes still in it, and the
|
||||
/// version may be among them, so a log with content has to be read: SQLite does that through an index file
|
||||
/// (<c>journal.db-shm</c>), which it creates or rebuilds beside the journal. That file holds no data, and whichever build
|
||||
/// opens the journal next rebuilds it; the journal and its log are not written. A process that exits in the ordinary
|
||||
/// way, as a stopped service does, is a different case: Microsoft.Data.Sqlite closes its pooled connections when the
|
||||
/// process exits, and SQLite folds the log into the journal when the last connection closes. With no log, or an
|
||||
/// empty one, the journal file alone is the truth, and it is opened as immutable: SQLite then takes no lock and
|
||||
/// creates no file, where an ordinary read-only connection would leave an empty log and an index behind.
|
||||
/// </summary>
|
||||
private static string ReadOnlyConnectionString(string path)
|
||||
{
|
||||
FileInfo log = new(path + "-wal");
|
||||
string source = log.Exists && log.Length > 0 ? path : ImmutableAddress(path);
|
||||
return new SqliteConnectionStringBuilder { DataSource = source, Mode = SqliteOpenMode.ReadOnly, Pooling = false }.ToString();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The address by which SQLite opens a journal file as immutable: the file's full path as a <c>file:</c> URI with the
|
||||
/// <c>immutable</c> parameter, which only a URI can carry. Each part of the path is escaped by itself, so every
|
||||
/// character of a folder name reaches SQLite as the character it is. A name may hold a percent sign, a number sign,
|
||||
/// a space or a letter outside ASCII, and a name that already looks escaped (<c>c%20d</c>) is escaped again, not taken
|
||||
/// for an escape. The drive's colon is escaped with the rest; SQLite reads the escaped form as the same path.
|
||||
/// </summary>
|
||||
private static string ImmutableAddress(string path)
|
||||
=> "file:///" + string.Join('/', path.Split(Path.DirectorySeparatorChar).Select(Uri.EscapeDataString)) + "?immutable=1";
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// Step 1: every schema change the 1.x releases made, as one step. Those releases had no schema version. Each store they
|
||||
/// opened created what was missing and added the columns a later release had introduced, and three tables were created
|
||||
/// only when first used. A journal can therefore arrive in any shape between a single six-column events table and the
|
||||
/// full 1.2.1 schema, with any of the late tables absent. This step creates each table, column and index only when it is
|
||||
/// missing, with the statements those releases used, so every journal leaves it in one shape whatever shape it came in.
|
||||
/// <para>
|
||||
/// The statements are written out here and not built from the code that queries the tables. A migration step has to mean
|
||||
/// the same thing for as long as it exists; if it borrowed, say, the sort expressions of the job list, a later change to
|
||||
/// those would silently change what a new installation gets while existing ones kept the old index.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal static partial class JournalMigration
|
||||
{
|
||||
/// <summary>Brings a journal of any 1.x shape, or an empty one, to schema version 1.</summary>
|
||||
internal static void V1(JournalMigrationContext context)
|
||||
{
|
||||
CreateEvents(context);
|
||||
CreateJobSummaries(context);
|
||||
CreateJobIndexes(context);
|
||||
CreateControlAndSettings(context);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The event outbox and the shared-printer list, as the first release made them, then the two columns later releases
|
||||
/// added to events. Rows written before events carried a connection scope get the scope of the pairing that is saved
|
||||
/// now, which is what every 1.x store did on its next open; only a journal last opened by a release from before the
|
||||
/// scope column can still hold such rows.
|
||||
/// </summary>
|
||||
private static void CreateEvents(JournalMigrationContext context)
|
||||
{
|
||||
context.Execute("""
|
||||
CREATE TABLE IF NOT EXISTS events(sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL, event_id TEXT NOT NULL UNIQUE, state TEXT NOT NULL, json TEXT NOT NULL, acknowledged INTEGER NOT NULL DEFAULT 0);
|
||||
CREATE TABLE IF NOT EXISTS exposed_printers(printer_id TEXT PRIMARY KEY);
|
||||
""");
|
||||
if(!context.HasColumn("events", "recorded_at"))
|
||||
{
|
||||
context.Execute("ALTER TABLE events ADD COLUMN recorded_at TEXT");
|
||||
}
|
||||
if(!context.HasColumn("events", "connection_id"))
|
||||
{
|
||||
context.Execute("ALTER TABLE events ADD COLUMN connection_id TEXT");
|
||||
}
|
||||
if(context.Scalar("SELECT EXISTS(SELECT 1 FROM events WHERE connection_id IS NULL)") != 0)
|
||||
{
|
||||
context.Execute("UPDATE events SET connection_id=$scope WHERE connection_id IS NULL", ("$scope", AgentStore.ScopeForUnscopedRows(context.Directory)));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The per-job summary table. When it did not exist, each job's summary is built from its latest event; that happens
|
||||
/// only then, so summaries a release has since maintained are never overwritten from the outbox. The creation time is a
|
||||
/// later column: where it is missing it is added and filled from the job's first recorded event, or from the summary's
|
||||
/// own time when the events are gone.
|
||||
/// </summary>
|
||||
private static void CreateJobSummaries(JournalMigrationContext context)
|
||||
{
|
||||
bool summariesMissing = !context.HasTable("local_jobs");
|
||||
context.Execute("CREATE TABLE IF NOT EXISTS local_jobs(connection_id TEXT NOT NULL, job_id TEXT NOT NULL, printer_id TEXT, printer_name TEXT, title TEXT, format TEXT, state TEXT, last_event_json TEXT, updated_at TEXT, failure_acknowledged INTEGER NOT NULL DEFAULT 0, history_pruned INTEGER NOT NULL DEFAULT 0, PRIMARY KEY(connection_id,job_id));");
|
||||
if(summariesMissing)
|
||||
{
|
||||
context.Execute("INSERT OR IGNORE INTO local_jobs(connection_id,job_id,state,last_event_json,updated_at) SELECT connection_id,job_id,state,json,recorded_at FROM events WHERE sequence IN (SELECT MAX(sequence) FROM events GROUP BY connection_id,job_id);");
|
||||
}
|
||||
if(!context.HasColumn("local_jobs", "created_at"))
|
||||
{
|
||||
context.Execute("""
|
||||
ALTER TABLE local_jobs ADD COLUMN created_at TEXT;
|
||||
UPDATE local_jobs SET created_at=COALESCE(
|
||||
(SELECT recorded_at FROM events WHERE events.connection_id=local_jobs.connection_id
|
||||
AND events.job_id=local_jobs.job_id AND recorded_at IS NOT NULL ORDER BY sequence LIMIT 1),updated_at);
|
||||
""");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The indexes that keep outbox drains, the status and printer filters, the attention list, retention, the prefix
|
||||
/// searches and each sort order of the job list fast as history grows. The five display indexes repeat the expressions
|
||||
/// the job list sorts by; a test compares the two, so a change to a sort expression cannot leave its index behind.
|
||||
/// </summary>
|
||||
private static void CreateJobIndexes(JournalMigrationContext context)
|
||||
{
|
||||
context.Execute("""
|
||||
CREATE INDEX IF NOT EXISTS events_outbox ON events(connection_id,acknowledged,sequence);
|
||||
CREATE INDEX IF NOT EXISTS events_job ON events(connection_id,job_id,sequence);
|
||||
CREATE INDEX IF NOT EXISTS jobs_recent ON local_jobs(connection_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_status ON local_jobs(connection_id,state,failure_acknowledged,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer ON local_jobs(connection_id,printer_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_attention ON local_jobs(connection_id,CASE WHEN state IN ('FAILED','OUTCOME_UNKNOWN') AND failure_acknowledged=0 THEN 0 ELSE 1 END,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_retention ON local_jobs(connection_id,history_pruned,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_title_search ON local_jobs(connection_id,title COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer_search ON local_jobs(connection_id,printer_name COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_id_search ON local_jobs(connection_id,job_id COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_createdAt ON local_jobs(connection_id,created_at COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_title ON local_jobs(connection_id,COALESCE(title,job_id) COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_printerName ON local_jobs(connection_id,COALESCE(printer_name,printer_id,'Not recorded') COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_status ON local_jobs(connection_id,(CASE COALESCE(state,'CLAIMED') WHEN 'READY' THEN 'Ready' WHEN 'OFFLINE' THEN 'Offline' WHEN 'PAUSED' THEN 'Paused' WHEN 'ERROR' THEN 'Needs attention' WHEN 'QUEUED' THEN 'Queued' WHEN 'CLAIMED' THEN 'Assigned to this computer' WHEN 'RECEIVED' THEN 'Received' WHEN 'SUBMITTING' THEN 'Sending to printer' WHEN 'SPOOLED' THEN 'Sent to printer' WHEN 'SPOOLER_COMPLETED' THEN 'Windows completed the job' WHEN 'FAILED' THEN 'Failed' WHEN 'OUTCOME_UNKNOWN' THEN 'Check printer before retrying' ELSE 'Status unavailable' END || CASE WHEN failure_acknowledged<>0 THEN ' · Reviewed' ELSE '' END) COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_format ON local_jobs(connection_id,COALESCE(format,'—') COLLATE NOCASE DESC,job_id DESC);
|
||||
""");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The remote-control tables and the three tables 1.x created only on first use: local names and descriptions,
|
||||
/// realtime settings per connection, and the certificate exception. An installation that never used one of them has
|
||||
/// none, so each is created here when missing; existing rows are untouched.
|
||||
/// </summary>
|
||||
private static void CreateControlAndSettings(JournalMigrationContext context)
|
||||
{
|
||||
context.Execute("""
|
||||
CREATE TABLE IF NOT EXISTS control_state(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL,updated_at TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS control_commands(connection_id TEXT NOT NULL,command_id TEXT NOT NULL,command_json TEXT NOT NULL,receipt_json TEXT,acknowledged INTEGER NOT NULL DEFAULT 0,expires_at TEXT NOT NULL,PRIMARY KEY(connection_id,command_id));
|
||||
CREATE INDEX IF NOT EXISTS control_outbox ON control_commands(connection_id,acknowledged);
|
||||
CREATE INDEX IF NOT EXISTS control_expiry ON control_commands(expires_at);
|
||||
CREATE TABLE IF NOT EXISTS local_metadata(profile_key TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS realtime_settings(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS server_trust(scope TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
""");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// The steps that bring a journal's schema from one version to the next, in order. Each step lives in a file of its own
|
||||
/// (<c>JournalMigration.V1.cs</c> and so on). A step that has shipped is never edited: a journal that has passed it will
|
||||
/// not run it again, so a change to it would leave old and new installations with different schemas. A change to the
|
||||
/// schema is a new step at the end of the list.
|
||||
/// </summary>
|
||||
internal static partial class JournalMigration
|
||||
{
|
||||
/// <summary>
|
||||
/// Every step, oldest first. The step at position n brings a journal from version n to version n + 1, so the number of
|
||||
/// steps is the schema version this build works with. <see cref="JournalSchema.Migrate(string)"/> runs the ones a
|
||||
/// journal has not passed yet.
|
||||
/// </summary>
|
||||
internal static IReadOnlyList<Action<JournalMigrationContext>> Steps { get; } = [V1];
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Globalization;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// What one migration step works with: the journal, through a command that belongs to the migration's one transaction,
|
||||
/// and the state folder the journal lives in. A step never opens a connection or a transaction of its own, so everything
|
||||
/// it does is committed or rolled back together with the steps around it. Later steps need more than the journal (what
|
||||
/// the previous version left in the folder, the time of the migration); those are added here, not to each step.
|
||||
/// </summary>
|
||||
internal sealed class JournalMigrationContext(SqliteCommand command, string directory)
|
||||
{
|
||||
/// <summary>The state folder whose journal is being migrated, as a full path.</summary>
|
||||
internal string Directory => directory;
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs SQL that returns nothing, inside the migration's transaction. Values are passed as named parameters and never written into the text.</summary>
|
||||
internal void Execute(string sql, params (string Name, object Value)[] parameters)
|
||||
{
|
||||
Prepare(sql, parameters);
|
||||
_ = command.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs a query that returns one whole number, inside the migration's transaction.</summary>
|
||||
internal long Scalar(string sql, params (string Name, object Value)[] parameters)
|
||||
{
|
||||
Prepare(sql, parameters);
|
||||
return Convert.ToInt64(command.ExecuteScalar(), CultureInfo.InvariantCulture);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>True when the journal has a table of this name.</summary>
|
||||
internal bool HasTable(string table) => Scalar("SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=$table", ("$table", table)) != 0;
|
||||
|
||||
|
||||
|
||||
/// <summary>True when a table of the journal has a column of this name; false also when the table does not exist.</summary>
|
||||
internal bool HasColumn(string table, string column)
|
||||
=> Scalar("SELECT COUNT(*) FROM pragma_table_info($table) WHERE name=$column", ("$table", table), ("$column", column)) != 0;
|
||||
|
||||
|
||||
|
||||
/// <summary>Puts the text and the parameters of the next statement on the one command, replacing those of the statement before.</summary>
|
||||
private void Prepare(string sql, (string Name, object Value)[] parameters)
|
||||
{
|
||||
command.CommandText = sql;
|
||||
command.Parameters.Clear();
|
||||
foreach((string name, object value) in parameters)
|
||||
{
|
||||
_ = command.Parameters.AddWithValue(name, value);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Globalization;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// The journal's schema version and the one place it changes. The version is SQLite's <c>PRAGMA user_version</c>: 0 for a
|
||||
/// journal no versioned build has migrated (a new one, or one a 1.x release left), and from then on the number of
|
||||
/// migration steps the journal has passed. The service migrates once, when it starts, before anything opens a store.
|
||||
/// Opening a store only checks the version; it never creates or alters a table.
|
||||
/// </summary>
|
||||
internal static class JournalSchema
|
||||
{
|
||||
/// <summary>What a store says when it is opened on a journal the migration has not brought to this build's version.</summary>
|
||||
internal const string NotMigratedText = "The print journal has not been brought up to date. The service does that when it starts, and nothing may open the journal before it has.";
|
||||
|
||||
|
||||
|
||||
/// <summary>The schema version this build works with: the number of migration steps it knows.</summary>
|
||||
internal static int Version => JournalMigration.Steps.Count;
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// What the service logs when the journal in a state folder could not be brought up to date and the service therefore
|
||||
/// does not start: which journal, the kind of failure, that the journal was not changed, and what an operator can
|
||||
/// check. A migration changes the journal in one transaction or not at all, so whatever failed, the journal holds
|
||||
/// what it held. The failure is named by its kind only. Its own message is not repeated: a message can quote what a
|
||||
/// statement was working on, and this line must be safe to show whatever the journal and the pairing file contain.
|
||||
/// </summary>
|
||||
internal static string StartFailureText(string directory, Exception failure)
|
||||
=> $"The print journal in {Path.GetFullPath(directory)} could not be brought up to date: {KindOf(failure)}. The journal was not changed; its changes are made together or not at all. "
|
||||
+ "Printing does not start. Check that the print service can read and write that folder and the file journal.db in it, that no other program has the file open, "
|
||||
+ "and that the disk has free space, then start the service again. Every start tries again.";
|
||||
|
||||
|
||||
|
||||
/// <summary>The kind of a migration failure in an operator's words, with SQLite's error number where SQLite reported it, and the failure's type where it did not.</summary>
|
||||
private static string KindOf(Exception failure)
|
||||
{
|
||||
if(failure is not SqliteException sqlite)
|
||||
{
|
||||
return "the migration stopped with " + failure.GetType().Name;
|
||||
}
|
||||
string number = " (SQLite error " + sqlite.SqliteErrorCode.ToString(CultureInfo.InvariantCulture) + ")";
|
||||
return sqlite.SqliteErrorCode switch
|
||||
{
|
||||
3 or 8 or 10 or 14 => "the journal file could not be opened or written" + number,
|
||||
5 or 6 => "another program has the journal file in use" + number,
|
||||
11 or 26 => "the journal file is damaged or is not a journal" + number,
|
||||
13 => "the disk is full" + number,
|
||||
_ => "SQLite refused a change" + number,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Brings the journal in a state folder to this build's schema version, creating the folder and the journal when they are missing, and says whether the stored state may be used.</summary>
|
||||
internal static StateStatus Migrate(string directory) => Migrate(directory, JournalMigration.Steps);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Brings the journal in a state folder to the version of the last of the given steps, under the journal's lock, so
|
||||
/// that of several callers one migrates and the others find it done. The version is read first, read-only. A journal
|
||||
/// already at the version is ready. One at a higher version was written by a newer build and is left exactly as it is:
|
||||
/// it is not opened for writing, and the caller is told both versions. A negative version is one no build writes; such
|
||||
/// a journal is refused, because no step list says how to continue from it. Only a journal behind the version is
|
||||
/// opened for writing and migrated. The step list is a parameter so that a test can add a step of its own.
|
||||
/// </summary>
|
||||
internal static StateStatus Migrate(string directory, IReadOnlyList<Action<JournalMigrationContext>> steps)
|
||||
{
|
||||
string folder = Path.GetFullPath(directory);
|
||||
Qqq.PrintAgent.Core.MachineStateDirectory.EnsureForPath(folder);
|
||||
_ = Directory.CreateDirectory(folder);
|
||||
lock(JournalDatabase.LockFor(folder))
|
||||
{
|
||||
return Settled(JournalDatabase.ReadVersion(folder), steps.Count) ?? Apply(folder, steps);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// What a journal at the version found is to a build that knows the given number of steps, when no step has to run:
|
||||
/// ready at that number, a newer build's above it, and refused below zero. Null means the journal is behind and has
|
||||
/// steps to run. Both readings of the version, the read-only one and the one inside the transaction, are judged here.
|
||||
/// </summary>
|
||||
private static StateStatus? Settled(int found, int supported)
|
||||
{
|
||||
if(found < 0)
|
||||
{
|
||||
throw new InvalidDataException("The print journal carries a schema version no version of this application writes; it was not changed.");
|
||||
}
|
||||
if(found > supported)
|
||||
{
|
||||
return new StateStatus.NewerBuild(found, supported);
|
||||
}
|
||||
return found == supported ? new StateStatus.Ready() : null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Opens a store's connection to the journal in a state folder and checks that the journal is at this build's schema
|
||||
/// version. A missing journal, or one behind the version, has not been migrated, and one ahead of it belongs to a newer
|
||||
/// build; either way the connection is closed and the caller is told which. Nothing is created and nothing is written.
|
||||
/// </summary>
|
||||
internal static SqliteConnection OpenCurrent(string directory)
|
||||
{
|
||||
if(!File.Exists(JournalDatabase.PathOf(directory)))
|
||||
{
|
||||
throw new InvalidOperationException(NotMigratedText);
|
||||
}
|
||||
SqliteConnection connection = JournalDatabase.Open(directory);
|
||||
try
|
||||
{
|
||||
using SqliteCommand command = connection.CreateCommand();
|
||||
command.CommandText = "PRAGMA user_version";
|
||||
int found = Convert.ToInt32(command.ExecuteScalar(), CultureInfo.InvariantCulture);
|
||||
if(found != Version)
|
||||
{
|
||||
throw new InvalidOperationException(found > Version ? StateStatus.NewerBuildText : NotMigratedText);
|
||||
}
|
||||
return connection;
|
||||
}
|
||||
catch
|
||||
{
|
||||
connection.Dispose();
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Runs the steps a journal has not passed, all in one transaction, and stamps the new version in that same
|
||||
/// transaction. A step that fails rolls back every step of this run and the version with them, so the journal is as
|
||||
/// it was and the next start tries again. The connection is not pooled: it is closed, and the file released, the
|
||||
/// moment the migration ends. Write-ahead logging is switched on before the transaction because SQLite cannot change
|
||||
/// the journal mode inside one; it is a property of the file, set once here for every later connection.
|
||||
/// <para>
|
||||
/// The transaction takes SQLite's write lock when it begins, and the version is read again under that lock. That
|
||||
/// reading, not the caller's earlier one, decides which steps run. The journal's lock keeps the callers of this
|
||||
/// process apart; a second process does not hold it, and could have migrated the journal between the caller's reading
|
||||
/// and this transaction. The steps it ran are then not run a second time, and when it left a version above this
|
||||
/// build's, the journal is reported as a newer build's and nothing is written to it.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private static StateStatus Apply(string folder, IReadOnlyList<Action<JournalMigrationContext>> steps)
|
||||
{
|
||||
using SqliteConnection connection = JournalDatabase.OpenUnpooled(folder);
|
||||
using SqliteCommand command = connection.CreateCommand();
|
||||
command.CommandText = "PRAGMA journal_mode=WAL";
|
||||
_ = command.ExecuteScalar();
|
||||
using SqliteTransaction transaction = connection.BeginTransaction(deferred: false);
|
||||
command.Transaction = transaction;
|
||||
JournalMigrationContext context = new(command, folder);
|
||||
int found = Convert.ToInt32(context.Scalar("PRAGMA user_version"), CultureInfo.InvariantCulture);
|
||||
if(Settled(found, steps.Count) is StateStatus settled)
|
||||
{
|
||||
return settled;
|
||||
}
|
||||
for(int step = found; step < steps.Count; step++)
|
||||
{
|
||||
steps[step](context);
|
||||
}
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
// PRAGMA takes no parameter, so the version is written into the //
|
||||
// text. It is a count of steps, formatted here, never input. //
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
context.Execute("PRAGMA user_version=" + steps.Count.ToString(CultureInfo.InvariantCulture));
|
||||
transaction.Commit();
|
||||
return new StateStatus.Ready();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using Qqq.PrintAgent.Windows.Management;
|
||||
|
||||
namespace Qqq.PrintAgent.Windows;
|
||||
|
||||
/// <summary>
|
||||
/// Whether the service may use the state it has stored, as it found out when it started. It is one of three cases and
|
||||
/// each carries only what belongs to it, so a caller cannot read a file name from a ready state or a version from an
|
||||
/// interrupted upgrade. While the state is not ready the service runs no connection and answers only status, the log
|
||||
/// operations and the update operations; <see cref="Refusal"/> is what it tells everything else.
|
||||
/// </summary>
|
||||
internal abstract record StateStatus
|
||||
{
|
||||
/// <summary>
|
||||
/// What the service answers, in the one case of a newer build's data, to every operation it will not carry out, and
|
||||
/// shows as its status. A later step gives the refusal an error code; until then clients have this text, so it is kept
|
||||
/// in one place.
|
||||
/// </summary>
|
||||
internal const string NewerBuildText = "A newer version of this application wrote the print data on this computer, and this version cannot use it. Nothing was changed. Install the newer version again to continue printing.";
|
||||
|
||||
|
||||
|
||||
/// <summary>What the service answers while an upgrade from 1.x could not finish. The snapshot names the file. The wording is provisional: the step that moves the files settles it.</summary>
|
||||
internal const string UpgradeIncompleteText = "The upgrade of the print data on this computer could not finish. Nothing was lost. Restart the print service to try again.";
|
||||
|
||||
|
||||
|
||||
/// <summary>Closed to this file: the three cases below are the only ones.</summary>
|
||||
private StateStatus()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The sentence the service answers refused operations with, or null when the state is ready and nothing is refused.</summary>
|
||||
internal abstract string? Refusal { get; }
|
||||
|
||||
|
||||
|
||||
/// <summary>The same case in the form the management contract's snapshot carries.</summary>
|
||||
internal abstract StoredState ToStoredState();
|
||||
|
||||
|
||||
|
||||
/// <summary>The stored state is current: the journal is at the schema version this build works with.</summary>
|
||||
internal sealed record Ready : StateStatus
|
||||
{
|
||||
/// <summary>Nothing is refused.</summary>
|
||||
internal override string? Refusal => null;
|
||||
|
||||
|
||||
|
||||
/// <summary>Ready carries no detail.</summary>
|
||||
internal override StoredState ToStoredState() => new(StoredState.Ready);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The upgrade from 1.x stopped at a file it could not move; <paramref name="File"/> is that file, relative to the state folder. No step produces this yet: the moves arrive with the per-server folders.</summary>
|
||||
internal sealed record UpgradeIncomplete(string File) : StateStatus
|
||||
{
|
||||
/// <summary>The upgrade sentence.</summary>
|
||||
internal override string? Refusal => UpgradeIncompleteText;
|
||||
|
||||
|
||||
|
||||
/// <summary>Carries the file the upgrade stopped at.</summary>
|
||||
internal override StoredState ToStoredState() => new(StoredState.UpgradeIncomplete, File: File);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>A newer build wrote the journal: it is at schema version <paramref name="FoundVersion"/> and this build knows versions up to <paramref name="SupportedVersion"/>. The journal was only read.</summary>
|
||||
internal sealed record NewerBuild(int FoundVersion, int SupportedVersion) : StateStatus
|
||||
{
|
||||
/// <summary>The newer-build sentence.</summary>
|
||||
internal override string? Refusal => NewerBuildText;
|
||||
|
||||
|
||||
|
||||
/// <summary>Carries the version found and the newest this build knows.</summary>
|
||||
internal override StoredState ToStoredState() => new(StoredState.NewerBuild, FoundVersion: FoundVersion, SupportedVersion: SupportedVersion);
|
||||
}
|
||||
}
|
||||
@@ -24,15 +24,17 @@ public sealed class AgentStoreOpenRaceTests
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A store takes its database connection only while it holds the journal's schema lock. This thread holds that lock for a
|
||||
/// journal that does not exist yet and lets another thread construct a store for it. The constructor creates the folder,
|
||||
/// then must wait for the lock; while it waits, the journal file must not exist, because opening the connection is what
|
||||
/// creates it. With the open outside the lock the file appears before the constructor waits, and the test fails every
|
||||
/// time. This is the property that keeps two stores from being handed one connection: Microsoft.Data.Sqlite 10.0.0 records
|
||||
/// a pooled connection's owner only after it has left the pool's lock, and another open of the same journal in that gap can
|
||||
/// be given the same connection. There is no event for "the constructor is now waiting", so the test waits until the
|
||||
/// folder exists and the thread is blocked, and then gives it a further fifth of a second in which a constructor that
|
||||
/// opened first would have created the file.
|
||||
/// A store takes its database connection only while it holds the journal's lock. The journal is migrated first, which
|
||||
/// leaves it closed: the migration's connection is not pooled. This thread then holds the journal's lock and lets
|
||||
/// another thread construct a store on it. The constructor must wait for the lock, and while it waits nothing may have
|
||||
/// the journal file open, which the test shows by opening the file for itself alone. With the open outside the lock the
|
||||
/// store's connection has the file before the constructor could be seen waiting, and the test fails every time. (Before
|
||||
/// the journal carried a schema version this test used a journal that did not exist yet and looked for the file the open
|
||||
/// created; a store no longer creates a journal, so it looks for the open handle instead.) This is
|
||||
/// the property that keeps two stores from being handed one connection: Microsoft.Data.Sqlite 10.0.0 records a pooled
|
||||
/// connection's owner only after it has left the pool's lock, and another open of the same journal in that gap can be
|
||||
/// given the same connection. There is no event for "the constructor is now waiting", so the test waits until the thread
|
||||
/// is blocked, and then gives it a further fifth of a second in which a constructor that did not wait would have finished.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void StoreOpensItsJournalOnlyWhileItHoldsTheJournalsLock()
|
||||
@@ -45,12 +47,13 @@ public sealed class AgentStoreOpenRaceTests
|
||||
// Loads the SQLite provider and the store's types, so the thread below has nothing else to wait for.
|
||||
using(JournalStore warm = TestState.OpenStore(Path.Combine(root, "warm")))
|
||||
{ Assert.Null(warm.GetState("none")); }
|
||||
_ = TestState.Migrate(folder);
|
||||
Exception? failure = null;
|
||||
var opening = new Thread(() =>
|
||||
{
|
||||
try
|
||||
{
|
||||
using JournalStore store = TestState.OpenStore(folder);
|
||||
using JournalStore store = TestState.OpenStoreAsFound(folder);
|
||||
_ = store.GetState("none");
|
||||
}
|
||||
catch(Exception exception) when(exception is SqliteException or IOException or InvalidOperationException)
|
||||
@@ -62,16 +65,18 @@ public sealed class AgentStoreOpenRaceTests
|
||||
lock(TestState.SchemaLockFor(folder))
|
||||
{
|
||||
opening.Start();
|
||||
reachedTheLock = SpinWait.SpinUntil(() => Directory.Exists(folder) && opening.ThreadState.HasFlag(ThreadState.WaitSleepJoin), TimeSpan.FromSeconds(30));
|
||||
reachedTheLock = SpinWait.SpinUntil(() => opening.ThreadState.HasFlag(ThreadState.WaitSleepJoin), TimeSpan.FromSeconds(30));
|
||||
finishedWhileHeld = opening.Join(TimeSpan.FromMilliseconds(200));
|
||||
openedWhileHeld = File.Exists(file);
|
||||
// A store that had opened would hold the file for as long as it waits here, so trying for a while costs a correct
|
||||
// run nothing and lets a virus scanner that looked at the new file for a moment get out of the way.
|
||||
openedWhileHeld = !SpinWait.SpinUntil(() => CanBeOpenedAlone(file), TimeSpan.FromSeconds(5));
|
||||
}
|
||||
opening.Join();
|
||||
Assert.True(reachedTheLock, "The constructor never came to wait for the journal's lock.");
|
||||
Assert.False(finishedWhileHeld, "The constructor finished while another thread held the journal's lock.");
|
||||
Assert.False(openedWhileHeld, "The store opened its journal before it held the journal's lock.");
|
||||
Assert.Null(failure);
|
||||
Assert.True(File.Exists(file));
|
||||
Assert.False(CanBeOpenedAlone(file), "The store that opened after the lock was released left no connection in the journal's pool, so the check above proved nothing.");
|
||||
}
|
||||
finally
|
||||
{
|
||||
@@ -81,6 +86,22 @@ public sealed class AgentStoreOpenRaceTests
|
||||
|
||||
|
||||
|
||||
/// <summary>True when nothing has the file open: opening it with no sharing succeeds only then.</summary>
|
||||
private static bool CanBeOpenedAlone(string file)
|
||||
{
|
||||
try
|
||||
{
|
||||
using FileStream alone = new(file, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
|
||||
return true;
|
||||
}
|
||||
catch(IOException)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Eight threads open a store on one journal at the same instant, round after round, read from it inside a transaction
|
||||
/// and close it. Every fiftieth round a thread also records a job event (an explicit transaction) and saves a printer
|
||||
|
||||
@@ -17,7 +17,11 @@ namespace Qqq.PrintAgent.Cli.Tests;
|
||||
/// </summary>
|
||||
public sealed class AgentStoreTests
|
||||
{
|
||||
/// <summary>A database with an unexpected schema makes the constructor fail, and the failure must release the file so the damaged journal can be inspected or replaced rather than left locked by a half-built store.</summary>
|
||||
/// <summary>
|
||||
/// A journal with an unexpected schema cannot be opened, and the failure must release the file so the damaged journal can be
|
||||
/// inspected or replaced rather than left locked. The failure now comes from the migration, which runs before a store opens
|
||||
/// and cannot build on a table it does not recognize; it closes its connection, and no store is constructed at all.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void InvalidSchema_DoesNotKeepDatabaseOpenAfterConstructorFailure()
|
||||
{
|
||||
@@ -442,6 +446,8 @@ public sealed class AgentStoreTests
|
||||
/// <summary>
|
||||
/// A journal written before connection scoping existed is adopted by the connection that was saved at the time, so its
|
||||
/// incomplete job is still visible to that device. Enrolling elsewhere hides it, and re-enrolling the original device brings it back.
|
||||
/// The migration does the adopting, once; a journal that old carries no schema version, so the test takes the version
|
||||
/// away together with the scope column.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void LegacyJournal_IsBoundToSavedConnectionBeforeReenrollment()
|
||||
@@ -459,7 +465,7 @@ public sealed class AgentStoreTests
|
||||
using(Microsoft.Data.Sqlite.SqliteConnection connection = TestState.OpenJournal(directory))
|
||||
{
|
||||
using Microsoft.Data.Sqlite.SqliteCommand command = connection.CreateCommand();
|
||||
command.CommandText = "DROP INDEX IF EXISTS events_outbox; DROP INDEX IF EXISTS events_job; DROP TABLE local_jobs; ALTER TABLE events DROP COLUMN connection_id";
|
||||
command.CommandText = "DROP INDEX IF EXISTS events_outbox; DROP INDEX IF EXISTS events_job; DROP TABLE local_jobs; ALTER TABLE events DROP COLUMN connection_id; PRAGMA user_version=0";
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
using(JournalStore migrated = TestState.OpenStore(directory))
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
-- Copyright (c) 2026 James Maes
|
||||
-- SPDX-License-Identifier: MIT
|
||||
--
|
||||
-- The journal as an installation of 1.2.1 leaves it: the last release without a schema version (user_version stays 0).
|
||||
--
|
||||
-- The schema is transcribed from tag v1.2.1, src/Qqq.PrintAgent.Windows/AgentStore*.cs, statement by statement and in
|
||||
-- the order that release runs them on a new installation: InitializeDatabase (events, exposed_printers, the two
|
||||
-- columns added to events, local_jobs), CreateJobIndexes, MigrateJobCreationTime (the column added to local_jobs and
|
||||
-- the five display indexes, whose expressions come from JobSorting.SqlColumn at that tag), InitializeControls, and
|
||||
-- then the three tables created on first use: realtime_settings (read when the service starts), local_metadata (read
|
||||
-- by the first snapshot) and server_trust (read by the first sign-in). SQLite keeps each table's text as it was
|
||||
-- created plus the columns added later, so creating and altering here, as 1.2.1 did, gives the stored schema text of
|
||||
-- a real installation.
|
||||
--
|
||||
-- The rows are synthetic. Two connection scopes appear: the current pairing (https://synthetic.invalid, device
|
||||
-- "device") and an earlier one (https://earlier.invalid, device "device-before"). A scope is the upper-case
|
||||
-- hexadecimal SHA-256 of the server address, a line feed and the device identifier. The JSON is what 1.2.1 wrote,
|
||||
-- including the authorizationRequired member of the control state that later versions no longer have.
|
||||
|
||||
PRAGMA journal_mode=WAL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events(sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL, event_id TEXT NOT NULL UNIQUE, state TEXT NOT NULL, json TEXT NOT NULL, acknowledged INTEGER NOT NULL DEFAULT 0);
|
||||
CREATE TABLE IF NOT EXISTS exposed_printers(printer_id TEXT PRIMARY KEY);
|
||||
ALTER TABLE events ADD COLUMN recorded_at TEXT;
|
||||
ALTER TABLE events ADD COLUMN connection_id TEXT;
|
||||
CREATE TABLE IF NOT EXISTS local_jobs(connection_id TEXT NOT NULL, job_id TEXT NOT NULL, printer_id TEXT, printer_name TEXT, title TEXT, format TEXT, state TEXT, last_event_json TEXT, updated_at TEXT, failure_acknowledged INTEGER NOT NULL DEFAULT 0, history_pruned INTEGER NOT NULL DEFAULT 0, PRIMARY KEY(connection_id,job_id));
|
||||
|
||||
CREATE INDEX IF NOT EXISTS events_outbox ON events(connection_id,acknowledged,sequence);
|
||||
CREATE INDEX IF NOT EXISTS events_job ON events(connection_id,job_id,sequence);
|
||||
CREATE INDEX IF NOT EXISTS jobs_recent ON local_jobs(connection_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_status ON local_jobs(connection_id,state,failure_acknowledged,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer ON local_jobs(connection_id,printer_id,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_attention ON local_jobs(connection_id,CASE WHEN state IN ('FAILED','OUTCOME_UNKNOWN') AND failure_acknowledged=0 THEN 0 ELSE 1 END,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_retention ON local_jobs(connection_id,history_pruned,updated_at DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_title_search ON local_jobs(connection_id,title COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_printer_search ON local_jobs(connection_id,printer_name COLLATE NOCASE);
|
||||
CREATE INDEX IF NOT EXISTS jobs_id_search ON local_jobs(connection_id,job_id COLLATE NOCASE);
|
||||
|
||||
ALTER TABLE local_jobs ADD COLUMN created_at TEXT;
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_createdAt ON local_jobs(connection_id,created_at COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_title ON local_jobs(connection_id,COALESCE(title,job_id) COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_printerName ON local_jobs(connection_id,COALESCE(printer_name,printer_id,'Not recorded') COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_status ON local_jobs(connection_id,(CASE COALESCE(state,'CLAIMED') WHEN 'READY' THEN 'Ready' WHEN 'OFFLINE' THEN 'Offline' WHEN 'PAUSED' THEN 'Paused' WHEN 'ERROR' THEN 'Needs attention' WHEN 'QUEUED' THEN 'Queued' WHEN 'CLAIMED' THEN 'Assigned to this computer' WHEN 'RECEIVED' THEN 'Received' WHEN 'SUBMITTING' THEN 'Sending to printer' WHEN 'SPOOLED' THEN 'Sent to printer' WHEN 'SPOOLER_COMPLETED' THEN 'Windows completed the job' WHEN 'FAILED' THEN 'Failed' WHEN 'OUTCOME_UNKNOWN' THEN 'Check printer before retrying' ELSE 'Status unavailable' END || CASE WHEN failure_acknowledged<>0 THEN ' · Reviewed' ELSE '' END) COLLATE NOCASE DESC,job_id DESC);
|
||||
CREATE INDEX IF NOT EXISTS jobs_display_format ON local_jobs(connection_id,COALESCE(format,'—') COLLATE NOCASE DESC,job_id DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS control_state(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL,updated_at TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS control_commands(connection_id TEXT NOT NULL,command_id TEXT NOT NULL,command_json TEXT NOT NULL,receipt_json TEXT,acknowledged INTEGER NOT NULL DEFAULT 0,expires_at TEXT NOT NULL,PRIMARY KEY(connection_id,command_id));
|
||||
CREATE INDEX IF NOT EXISTS control_outbox ON control_commands(connection_id,acknowledged);
|
||||
CREATE INDEX IF NOT EXISTS control_expiry ON control_commands(expires_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS realtime_settings(connection_id TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS local_metadata(profile_key TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
CREATE TABLE IF NOT EXISTS server_trust(scope TEXT PRIMARY KEY,json TEXT NOT NULL);
|
||||
|
||||
-- Job events. Sequence 5 and 9 have not been delivered to the server (acknowledged = 0); sequence 7 belongs to the
|
||||
-- earlier pairing.
|
||||
INSERT INTO events(sequence,job_id,event_id,state,json,acknowledged,recorded_at,connection_id) VALUES
|
||||
(1,'job-1001','ev-1001-received','RECEIVED','{"eventId":"ev-1001-received","status":"RECEIVED"}',1,'2026-11-06T14:03:20.1000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(2,'job-1001','ev-1001-submitting','SUBMITTING','{"eventId":"ev-1001-submitting","status":"SUBMITTING"}',1,'2026-11-06T14:03:21.2000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(3,'job-1001','ev-1001-spooled','SPOOLED','{"eventId":"ev-1001-spooled","status":"SPOOLED","spoolerJobId":"41"}',1,'2026-11-06T14:03:22.3000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(4,'job-1002','ev-1002-received','RECEIVED','{"eventId":"ev-1002-received","status":"RECEIVED"}',1,'2026-11-06T14:10:00.0000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(5,'job-1002','ev-1002-failed','FAILED','{"eventId":"ev-1002-failed","status":"FAILED","message":"The printer is offline. No bytes were submitted."}',0,'2026-11-06T14:10:01.5000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(6,'job-1003','ev-1003-failed','FAILED','{"eventId":"ev-1003-failed","status":"FAILED","message":"The document did not match its checksum. No bytes were submitted."}',1,'2026-11-06T14:20:00.0000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(7,'job-0900','ev-0900-spooled','SPOOLED','{"eventId":"ev-0900-spooled","status":"SPOOLED","spoolerJobId":"12"}',1,'2026-10-20T09:00:00.0000000+00:00','5F2379F644BF0ADC1CB267E7C0BDE2DE1E2CABEDC247DC7CE598E73E67854221'),
|
||||
(8,'job-1004','ev-1004-received','RECEIVED','{"eventId":"ev-1004-received","status":"RECEIVED"}',1,'2026-11-06T15:00:00.0000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED'),
|
||||
(9,'job-1004','ev-1004-unknown','OUTCOME_UNKNOWN','{"eventId":"ev-1004-unknown","status":"OUTCOME_UNKNOWN","message":"Agent stopped during native submission; inspect the printer before retrying."}',0,'2026-11-06T15:02:00.0000000+00:00','B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED');
|
||||
|
||||
-- Job summaries: one sent to the printer, one failed and not reviewed, one failed and reviewed, one whose outcome is
|
||||
-- unknown, one whose event detail retention has removed (history_pruned = 1, no events), and one of the earlier
|
||||
-- pairing.
|
||||
INSERT INTO local_jobs(connection_id,job_id,printer_id,printer_name,title,format,state,last_event_json,updated_at,failure_acknowledged,history_pruned,created_at) VALUES
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','job-1001','printer-a','Front desk laser','Packing slip 1001','PDF','SPOOLED','{"eventId":"ev-1001-spooled","status":"SPOOLED","spoolerJobId":"41"}','2026-11-06T14:03:22.3000000+00:00',0,0,'2026-11-06T14:03:19.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','job-1002','printer-a','Front desk laser','Packing slip 1002','PDF','FAILED','{"eventId":"ev-1002-failed","status":"FAILED","message":"The printer is offline. No bytes were submitted."}','2026-11-06T14:10:01.5000000+00:00',0,0,'2026-11-06T14:09:58.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','job-1003','printer-zpl','Label printer','Shipping label 1003','ZPL','FAILED','{"eventId":"ev-1003-failed","status":"FAILED","message":"The document did not match its checksum. No bytes were submitted."}','2026-11-06T14:20:00.0000000+00:00',1,0,'2026-11-06T14:19:55.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','job-1004','printer-a','Front desk laser','Packing slip 1004','PDF','OUTCOME_UNKNOWN','{"eventId":"ev-1004-unknown","status":"OUTCOME_UNKNOWN","message":"Agent stopped during native submission; inspect the printer before retrying."}','2026-11-06T15:02:00.0000000+00:00',0,0,'2026-11-06T14:59:50.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','job-0950','printer-a','Front desk laser','Packing slip 0950','PDF','SPOOLED','{"eventId":"ev-0950-spooled","status":"SPOOLED","spoolerJobId":"30"}','2026-10-01T08:00:00.0000000+00:00',0,1,'2026-10-01T07:59:58.0000000+00:00'),
|
||||
('5F2379F644BF0ADC1CB267E7C0BDE2DE1E2CABEDC247DC7CE598E73E67854221','job-0900','printer-a','Front desk laser','Packing slip 0900','PDF','SPOOLED','{"eventId":"ev-0900-spooled","status":"SPOOLED","spoolerJobId":"12"}','2026-10-20T09:00:00.0000000+00:00',0,0,'2026-10-20T08:59:57.0000000+00:00');
|
||||
|
||||
-- Sharing and the operator's own names.
|
||||
INSERT INTO exposed_printers(printer_id) VALUES ('printer-a'),('printer-zpl');
|
||||
INSERT INTO local_metadata(profile_key,json) VALUES
|
||||
('device','{"name":"Front desk PC","description":"Shipping station\nsecond shift","location":"Dock 2"}'),
|
||||
('printer:printer-a','{"displayName":"Front desk laser","description":"Tray 2 holds label stock","location":"Dock 2"}'),
|
||||
('printer:printer-zpl','{"displayName":"Label printer","description":"","location":""}');
|
||||
|
||||
-- Remote control: the desired state of each pairing, a delivered command, a receipt that has not been delivered and a
|
||||
-- command whose result was never recorded.
|
||||
INSERT INTO control_state(connection_id,json,updated_at) VALUES
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','{"support":"SUPPORTED","desiredState":{"revision":7,"dispatchPaused":true,"connectionState":"CONNECTED"},"authorizationRequired":false,"lastContact":"2026-11-06T15:04:00.0000000Z","lastCommand":"CANCEL_SPOOL","lastOutcome":"ALREADY_ABSENT","lastCommandAt":"2026-11-06T14:31:00.0000000Z"}','2026-11-06T15:04:00.0000000+00:00'),
|
||||
('5F2379F644BF0ADC1CB267E7C0BDE2DE1E2CABEDC247DC7CE598E73E67854221','{"support":"SUPPORTED","desiredState":{"revision":2,"dispatchPaused":false,"connectionState":"CONNECTED"},"authorizationRequired":false,"lastContact":"2026-10-20T09:05:00.0000000Z"}','2026-10-20T09:05:00.0000000+00:00');
|
||||
INSERT INTO control_commands(connection_id,command_id,command_json,receipt_json,acknowledged,expires_at) VALUES
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','cmd-refresh','{"commandId":"cmd-refresh","requestId":"req-1","action":"REFRESH_INVENTORY","createdAt":"2026-11-06T14:25:00Z","expiresAt":"2026-11-06T14:30:00Z"}','{"receiptId":"receipt-1","outcome":"INVENTORY_REFRESHED","completedAt":"2026-11-06T14:25:01.0000000Z"}',1,'2026-11-06T14:30:00.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','cmd-cancel','{"commandId":"cmd-cancel","requestId":"req-2","action":"CANCEL_SPOOL","createdAt":"2026-11-06T14:30:30Z","expiresAt":"2026-11-06T14:35:30Z","jobId":"job-1001"}','{"receiptId":"receipt-2","outcome":"ALREADY_ABSENT","completedAt":"2026-11-06T14:31:00.0000000Z","message":"The document was no longer in the Windows queue."}',0,'2026-11-06T14:35:30.0000000+00:00'),
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','cmd-interrupted','{"commandId":"cmd-interrupted","requestId":"req-3","action":"REFRESH_INVENTORY","createdAt":"2026-11-06T15:03:00Z","expiresAt":"2026-11-06T15:08:00Z"}',NULL,0,'2026-11-06T15:08:00.0000000+00:00');
|
||||
|
||||
-- The server's policy as cached for the current pairing, and the development certificate exception.
|
||||
INSERT INTO realtime_settings(connection_id,json) VALUES
|
||||
('B80C4B521AC51BDCD7D606BAAFDBFE37BE3A7E34B3FBF39B5FBEC8DA5F1029ED','{"revision":3,"maxPdfBytes":104857600,"maxZplBytes":20971520,"maxFrameBytes":1048576,"maxQueuedFrames":64,"maxQueuedFrameBytes":4194304,"maxConcurrentDownloads":2,"maxConcurrentPrints":1,"maxStagedBytes":536870912,"maxDownloadBytesPerSecond":0,"requestTimeoutMillis":30000,"downloadConnectTimeoutMillis":30000,"webSocketHandshakeTimeoutMillis":30000,"ackTimeoutMillis":15000,"downloadIdleTimeoutMillis":30000,"downloadTimeoutMillis":600000,"heartbeatIntervalSeconds":20,"heartbeatTimeoutSeconds":60,"reconnectMinDelayMillis":1000,"reconnectMaxDelayMillis":30000,"reconnectJitterPercent":20,"maxCopies":100,"nativeSubmissionTimeoutMillis":120000}');
|
||||
INSERT INTO server_trust(scope,json) VALUES ('machine','{"allowUntrustedCertificate":true,"origin":"https://synthetic.invalid"}');
|
||||
@@ -0,0 +1,26 @@
|
||||
-- Copyright (c) 2026 James Maes
|
||||
-- SPDX-License-Identifier: MIT
|
||||
--
|
||||
-- The oldest journal the repository's history can establish: one events table of six columns and nothing else.
|
||||
--
|
||||
-- The first commit of the store (461a3c0, 2026-10-29, in tag v0.1.0-rc.1) creates events with these six columns and
|
||||
-- then adds recorded_at when it is missing. That check can only be for a journal made before the repository began, so
|
||||
-- a journal without the column existed. Whether such a journal had the exposed_printers table is not recorded; the
|
||||
-- one here does not, which is the harder case. Every later table and column is absent: the scope column and the job
|
||||
-- summaries (6e0476e), the indexes and local names (b45a906), the creation time (d9033ae), the control tables
|
||||
-- (2c1c5f1), realtime settings (8a3b7c1) and the certificate exception (aaa5051).
|
||||
--
|
||||
-- The rows are synthetic: one job sent to the printer, one received and never finished, and one failed whose last
|
||||
-- event has not been delivered to the server.
|
||||
|
||||
PRAGMA journal_mode=WAL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events(sequence INTEGER PRIMARY KEY AUTOINCREMENT, job_id TEXT NOT NULL, event_id TEXT NOT NULL UNIQUE, state TEXT NOT NULL, json TEXT NOT NULL, acknowledged INTEGER NOT NULL DEFAULT 0);
|
||||
|
||||
INSERT INTO events(sequence,job_id,event_id,state,json,acknowledged) VALUES
|
||||
(1,'job-a','ev-a-received','RECEIVED','{"eventId":"ev-a-received","status":"RECEIVED"}',1),
|
||||
(2,'job-a','ev-a-submitting','SUBMITTING','{"eventId":"ev-a-submitting","status":"SUBMITTING"}',1),
|
||||
(3,'job-a','ev-a-spooled','SPOOLED','{"eventId":"ev-a-spooled","status":"SPOOLED","spoolerJobId":"7"}',1),
|
||||
(4,'job-b','ev-b-received','RECEIVED','{"eventId":"ev-b-received","status":"RECEIVED"}',1),
|
||||
(5,'job-c','ev-c-received','RECEIVED','{"eventId":"ev-c-received","status":"RECEIVED"}',1),
|
||||
(6,'job-c','ev-c-failed','FAILED','{"eventId":"ev-c-failed","status":"FAILED","message":"The printer is offline."}',0);
|
||||
@@ -9,7 +9,7 @@ using Qqq.PrintAgent.Windows;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>The job list sorts by when the server created each job. These tests pin how that creation time is stored: normalized to UTC, kept stable when a job is replayed or changes status, and filled in for journals written before the column existed.</summary>
|
||||
/// <summary>The job list sorts by when the server created each job. These tests pin how that creation time is stored: normalized to UTC, kept stable when a job is replayed or changes status, and filled in by the migration for journals written before the column existed.</summary>
|
||||
public sealed class JobCreationTimeTests
|
||||
{
|
||||
/// <summary>A server creation time with an offset is stored as UTC and never replaced by a later claim carrying a different time, nor by status updates (which change the updated time only). Sorting by creation then orders the jobs by their server creation times in both directions.</summary>
|
||||
@@ -43,9 +43,10 @@ public sealed class JobCreationTimeTests
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// For a journal created before the creation-time column existed, opening the store fills it from the job's first event, or from
|
||||
/// the last-updated time when its events were pruned. The fill happens once: changing every timestamp afterwards and reopening
|
||||
/// leaves the earlier values alone, so the migration is not repeated on each open.
|
||||
/// For a journal created before the creation-time column existed, the migration fills it from the job's first event, or from
|
||||
/// the last-updated time when its events were pruned. Such a journal carries no schema version, which is what makes the
|
||||
/// migration run on it; the test takes the version away together with the column. The fill happens once: changing every
|
||||
/// timestamp afterwards and reopening leaves the earlier values alone, because a migrated journal is not migrated again.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void LegacyBackfill_UsesFirstEventThenSummaryFallbackAndRunsOnlyOnce()
|
||||
@@ -66,6 +67,7 @@ public sealed class JobCreationTimeTests
|
||||
UPDATE local_jobs SET updated_at='2021-01-01T00:00:00.0000000+00:00' WHERE job_id='pruned';
|
||||
DROP INDEX jobs_display_createdAt;
|
||||
ALTER TABLE local_jobs DROP COLUMN created_at;
|
||||
PRAGMA user_version=0;
|
||||
""");
|
||||
using(JournalStore migrated = TestState.OpenStore(directory))
|
||||
{
|
||||
|
||||
@@ -0,0 +1,843 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Buffers.Binary;
|
||||
using System.Text.Json;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Qqq.PrintAgent.Core.Printing;
|
||||
using Qqq.PrintAgent.Windows;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// The journal's schema version and the migration that sets it. The service migrates once, when it starts: every pending
|
||||
/// step in one transaction, or none of them. These tests run that migration on the journals earlier releases left (built
|
||||
/// by <see cref="LegacyStateBuilder"/> from the scripts in <c>Fixtures</c>), and show what it keeps, what it adds, what a
|
||||
/// failing step leaves behind, and that a journal from a newer build is not touched in any way.
|
||||
/// </summary>
|
||||
public sealed class JournalMigrationTests
|
||||
{
|
||||
/// <summary>The pairing whose scope most rows of the 1.2.1 fixture carry.</summary>
|
||||
private static readonly AgentConfiguration CurrentPairing = new("https://synthetic.invalid", "device", "synthetic-token");
|
||||
|
||||
|
||||
|
||||
/// <summary>A folder that does not exist yet gets a journal at the current version with every table, and a store then opens on it.</summary>
|
||||
[Fact]
|
||||
public void Migrate_EmptyFolder_Version1()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
Assert.False(Directory.Exists(directory));
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
Assert.Equal(1, TestState.SchemaVersion);
|
||||
Assert.Equal(1, JournalProbe.Version(directory));
|
||||
Assert.Equal(["control_commands", "control_state", "events", "exposed_printers", "local_jobs", "local_metadata", "realtime_settings", "server_trust", "sqlite_sequence"],
|
||||
JournalProbe.Tables(directory));
|
||||
Assert.Equal("wal", JournalProbe.Scalar(directory, "PRAGMA journal_mode"));
|
||||
using JournalStore store = TestState.OpenStoreAsFound(directory);
|
||||
Assert.Empty(store.RecentJobs());
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A 1.2.1 journal already has everything step 1 would add, so migrating it changes one thing: the version. Every
|
||||
/// table's rows are the same before and after (a digest of each is compared), and so is every table and index. The
|
||||
/// current store then reads those rows as 1.2.1 wrote them: the jobs of the current pairing with their states, the two
|
||||
/// events not yet delivered, the sharing, the names, the control state with a member this version no longer has, the
|
||||
/// undelivered receipt, the cached policy and the certificate exception. The earlier pairing's job stays out of sight.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_Journal121_KeepsEveryRow()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithPairing(CurrentPairing);
|
||||
IReadOnlyDictionary<string, string> rowsBefore = JournalProbe.RowDigests(directory);
|
||||
IReadOnlyList<string> schemaBefore = JournalProbe.Schema(directory);
|
||||
Assert.Equal(0, JournalProbe.Version(directory));
|
||||
Assert.Equal(9, rowsBefore.Count);
|
||||
Assert.DoesNotContain(rowsBefore, table => table.Value.StartsWith("0 rows", StringComparison.Ordinal));
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(1, JournalProbe.Version(directory));
|
||||
Assert.Equal(rowsBefore, JournalProbe.RowDigests(directory));
|
||||
Assert.Equal(schemaBefore, JournalProbe.Schema(directory));
|
||||
using JournalStore store = TestState.OpenStoreAsFound(directory);
|
||||
AssertReads121(store);
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>What the current store must read from the migrated 1.2.1 fixture; each line names a row of the script.</summary>
|
||||
private static void AssertReads121(JournalStore store)
|
||||
{
|
||||
Assert.Equal(["job-0950", "job-1001", "job-1002", "job-1003", "job-1004"], store.RecentJobs().Select(job => job.JobId).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal("SPOOLED", store.GetState("job-1001"));
|
||||
Assert.Null(store.GetState("job-0900"));
|
||||
Assert.Equal(["ev-1002-failed", "ev-1004-unknown"], store.Pending().Select(pending => pending.Event.EventId), StringComparer.Ordinal);
|
||||
Assert.Equal(2, store.NeedsAttentionCount());
|
||||
JobDetailsSnapshot details = store.GetJobDetails("job-1001")!;
|
||||
Assert.Equal(["RECEIVED", "SUBMITTING", "SPOOLED"], details.Events.Select(item => item.Status), StringComparer.Ordinal);
|
||||
Assert.Equal("41", details.Job.SpoolerJobId);
|
||||
Assert.Equal("2026-11-06T14:03:19.0000000+00:00", details.Job.CreatedAt);
|
||||
Assert.True(store.GetJobDetails("job-0950")!.HistoryPruned);
|
||||
Assert.Equal(["printer-a", "printer-zpl"], store.GetExposedPrinterIds().Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(new DeviceProfile("Front desk PC", "Shipping station\nsecond shift", "Dock 2"), store.GetDeviceProfile());
|
||||
Assert.Equal("Front desk laser", store.GetPrinterProfile("printer-a").DisplayName);
|
||||
AgentControlState control = store.LoadControlState();
|
||||
Assert.Equal(("SUPPORTED", 7L, true, "CANCEL_SPOOL"), (control.Support, control.DesiredState.Revision, control.DesiredState.DispatchPaused, control.LastCommand));
|
||||
Assert.Equal("cmd-cancel", Assert.Single(store.PendingControlReceipts()).CommandId);
|
||||
Assert.Equal("INVENTORY_REFRESHED", store.GetControlReceipt("cmd-refresh")!.Outcome);
|
||||
Assert.Null(store.GetControlReceipt("cmd-interrupted"));
|
||||
RealtimeSettings settings = store.LoadRealtimeSettings();
|
||||
Assert.Equal((3L, 2), (settings.Revision, settings.MaxConcurrentDownloads));
|
||||
Assert.Equal(new ServerTrust(true, "https://synthetic.invalid"), store.LoadServerTrust());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The oldest journal there is evidence of holds one events table of six columns. Migrating it adds the two later
|
||||
/// columns, every later table and index, and a summary for each job built from its latest event. Its events are all
|
||||
/// still there, unchanged, and now carry a scope: with no pairing saved, the scope a computer has before it is paired.
|
||||
/// A store on that computer then shows the three jobs, the one that never finished and the event still to be delivered.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_OldestShape_AddsColumnsAndJobSummaries()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest);
|
||||
Assert.Equal(["events", "sqlite_sequence"], JournalProbe.Tables(directory));
|
||||
IReadOnlyList<string> eventsBefore = JournalProbe.Rows(directory, "SELECT sequence,job_id,event_id,state,json,acknowledged FROM events ORDER BY sequence");
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(1, JournalProbe.Version(directory));
|
||||
Assert.Equal(eventsBefore, JournalProbe.Rows(directory, "SELECT sequence,job_id,event_id,state,json,acknowledged FROM events ORDER BY sequence"));
|
||||
Assert.Equal(["connection_id=unenrolled | recorded_at=NULL"], JournalProbe.Rows(directory, "SELECT DISTINCT connection_id,recorded_at FROM events"));
|
||||
Assert.Equal(
|
||||
[
|
||||
"job_id=job-a | state=SPOOLED | connection_id=unenrolled | failure_acknowledged=0 | history_pruned=0 | updated_at=NULL | created_at=NULL",
|
||||
"job_id=job-b | state=RECEIVED | connection_id=unenrolled | failure_acknowledged=0 | history_pruned=0 | updated_at=NULL | created_at=NULL",
|
||||
"job_id=job-c | state=FAILED | connection_id=unenrolled | failure_acknowledged=0 | history_pruned=0 | updated_at=NULL | created_at=NULL",
|
||||
], JournalProbe.Rows(directory, "SELECT job_id,state,connection_id,failure_acknowledged,history_pruned,updated_at,created_at FROM local_jobs ORDER BY job_id"));
|
||||
using JournalStore store = TestState.OpenStoreAsFound(directory);
|
||||
Assert.Equal(["job-a", "job-b", "job-c"], store.RecentJobs().Select(job => job.JobId).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(("job-b", "RECEIVED"), Assert.Single(store.Incomplete()));
|
||||
Assert.Equal("ev-c-failed", Assert.Single(store.Pending()).Event.EventId);
|
||||
Assert.Equal("7", store.GetJobDetails("job-a")!.Job.SpoolerJobId);
|
||||
Assert.Equal(1, store.NeedsAttentionCount());
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Whatever shape a journal came in, it leaves the migration in one: the same tables, columns and indexes, written with
|
||||
/// the same statements, whether it was new, the oldest shape, the oldest shape with the shared-printer table of the
|
||||
/// first release, or a full 1.2.1 journal. Later steps can then be written against one schema.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_EveryShape_EndsInOneSchema()
|
||||
{
|
||||
string root = NewFolder();
|
||||
try
|
||||
{
|
||||
string empty = Path.Combine(root, "empty");
|
||||
string oldest = new LegacyStateBuilder(Path.Combine(root, "oldest")).WithJournal(LegacyStateBuilder.JournalOldest).DirectoryPath;
|
||||
string first = new LegacyStateBuilder(Path.Combine(root, "first")).WithJournal(LegacyStateBuilder.JournalOldest)
|
||||
.WithJournalSql("CREATE TABLE exposed_printers(printer_id TEXT PRIMARY KEY); ALTER TABLE events ADD COLUMN recorded_at TEXT").DirectoryPath;
|
||||
string latest = new LegacyStateBuilder(Path.Combine(root, "latest")).WithJournal(LegacyStateBuilder.Journal121).DirectoryPath;
|
||||
foreach(string directory in (string[])[empty, oldest, first, latest])
|
||||
{
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
}
|
||||
IReadOnlyList<string> schema = JournalProbe.Schema(empty);
|
||||
// Nine tables, the seventeen indexes the steps name, and the eight SQLite makes for primary keys and the unique event identifier.
|
||||
Assert.Equal(34, schema.Count);
|
||||
Assert.Equal(schema, JournalProbe.Schema(oldest));
|
||||
Assert.Equal(schema, JournalProbe.Schema(first));
|
||||
Assert.Equal(schema, JournalProbe.Schema(latest));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The five display indexes are written out in step 1, not built from the job list's sort expressions, so that the step
|
||||
/// keeps its meaning. This holds the two together: for each sortable field the index in a migrated journal is on
|
||||
/// exactly the expression the job list orders by. A sort expression changed without a migration step that replaces its
|
||||
/// index fails here.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void DisplayIndexes_AreOnTheExpressionsTheJobListSortsBy()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = TestState.Migrate(directory);
|
||||
IReadOnlyList<string> expected = [.. JobSorting.Fields.Order(StringComparer.Ordinal).Select(field
|
||||
=> "name=jobs_display_" + field + " | sql=CREATE INDEX jobs_display_" + field + " ON local_jobs(connection_id," + JobSorting.SqlColumn(field) + " COLLATE NOCASE DESC,job_id DESC)")];
|
||||
Assert.Equal(expected, JournalProbe.Rows(directory, "SELECT name,sql FROM sqlite_master WHERE name LIKE 'jobs!_display!_%' ESCAPE '!' ORDER BY name"));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Rows from before events carried a scope get a scope of their own when the saved pairing cannot be read: nobody can
|
||||
/// say which pairing they belonged to, so they must not appear under one that is saved later. The pairing file here is
|
||||
/// not something DPAPI wrote. The events are kept, all under one scope that starts "unreadable-", and a store opened
|
||||
/// for recovery, as the service opens it, shows none of them.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_OldestShape_UnreadablePairing_SetsTheRowsApart()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest).WithFile("connection.dpapi", [1, 2, 3, 4]);
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
|
||||
string scope = Assert.Single(JournalProbe.Rows(directory, "SELECT DISTINCT connection_id FROM events"));
|
||||
Assert.StartsWith("connection_id=unreadable-", scope, StringComparison.Ordinal);
|
||||
Assert.Equal(6L, JournalProbe.Scalar(directory, "SELECT COUNT(*) FROM events"));
|
||||
Assert.Equal(3L, JournalProbe.Scalar(directory, "SELECT COUNT(*) FROM local_jobs"));
|
||||
using JournalStore store = TestState.OpenStore(directory, recoverUnreadableConfiguration: true);
|
||||
Assert.Empty(store.RecentJobs());
|
||||
Assert.Empty(store.Pending());
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A step that fails takes the whole migration with it. Step 1 runs first here and does all its work on the oldest
|
||||
/// journal; a second step, added by the test, creates a table and then throws. Afterwards the journal is as it was:
|
||||
/// version 0, the one original table, the same rows. The failure reaches the caller. Migrating again with the product's
|
||||
/// steps then succeeds, which is what the next start of the service does.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_StepThrows_VersionAndTablesUnchanged()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest);
|
||||
IReadOnlyList<string> schemaBefore = JournalProbe.Schema(directory);
|
||||
IReadOnlyDictionary<string, string> rowsBefore = JournalProbe.RowDigests(directory);
|
||||
bool firstStepRan = false;
|
||||
IReadOnlyList<Action<JournalMigrationContext>> steps =
|
||||
[
|
||||
context =>
|
||||
{
|
||||
TestState.MigrationSteps[0](context);
|
||||
firstStepRan = context.HasTable("local_jobs");
|
||||
},
|
||||
context =>
|
||||
{
|
||||
context.Execute("CREATE TABLE added_by_the_failing_step(value TEXT)");
|
||||
throw new InvalidOperationException("Synthetic failure in a migration step.");
|
||||
},
|
||||
];
|
||||
|
||||
InvalidOperationException failure = Assert.Throws<InvalidOperationException>(() => TestState.Migrate(directory, steps));
|
||||
|
||||
Assert.Equal("Synthetic failure in a migration step.", failure.Message);
|
||||
Assert.True(firstStepRan);
|
||||
Assert.Equal(0, JournalProbe.Version(directory));
|
||||
Assert.Equal(schemaBefore, JournalProbe.Schema(directory));
|
||||
Assert.Equal(rowsBefore, JournalProbe.RowDigests(directory));
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
Assert.Equal(3L, JournalProbe.Scalar(directory, "SELECT COUNT(*) FROM local_jobs"));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The steps and the version stamp are one transaction. Were the version stamped in a second one, a service that died
|
||||
/// between the two would leave a journal with every step done and the old version on it, and the next start would run
|
||||
/// the steps again. The write-ahead log shows how many transactions the migration made: SQLite marks the end of each
|
||||
/// one there, and migrating the oldest journal, which makes every change step 1 has, leaves one mark. A second
|
||||
/// connection stays open on the journal throughout. Without it the migration's connection would be the last to
|
||||
/// close, and SQLite would fold the log into the journal and delete it before the test could read it.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_StepsAndVersion_AreOneTransactionInTheLog()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest);
|
||||
using SqliteConnection keepsTheLog = TestState.OpenJournal(directory);
|
||||
using SqliteCommand command = keepsTheLog.CreateCommand();
|
||||
command.CommandText = "SELECT COUNT(*) FROM events";
|
||||
Assert.Equal(6L, command.ExecuteScalar());
|
||||
Assert.Equal(0, JournalProbe.CommitsInLog(directory));
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(1, JournalProbe.CommitsInLog(directory));
|
||||
command.CommandText = "PRAGMA user_version";
|
||||
Assert.Equal(1L, command.ExecuteScalar());
|
||||
command.CommandText = "SELECT COUNT(*) FROM local_jobs";
|
||||
Assert.Equal(3L, command.ExecuteScalar());
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A journal at a higher version than this build knows belongs to a newer build. The migration says so, with both
|
||||
/// versions, and touches nothing: every file in the folder (the journal, the pairing, a session file, a staged
|
||||
/// document in a folder beneath) has the same bytes and the same last-write time afterwards, and no file has appeared.
|
||||
/// The version is read through a read-only connection; an ordinary one would leave a log and an index file behind.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_NewerVersion_FileBytesUnchanged()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
int newer = TestState.SchemaVersion + 1;
|
||||
LegacyStateBuilder state = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithPairing(CurrentPairing)
|
||||
.WithFile("qqq-session.dpapi", [9, 8, 7]).WithJournalSql("CREATE TABLE added_by_the_newer_build(value TEXT); PRAGMA user_version=" + newer.ToString(System.Globalization.CultureInfo.InvariantCulture));
|
||||
_ = Directory.CreateDirectory(Path.Combine(directory, "staging"));
|
||||
_ = state.WithFile(Path.Combine("staging", "document.pdf"), [37, 80, 68, 70]);
|
||||
IReadOnlyDictionary<string, string> hashesBefore = JournalProbe.FileHashes(directory);
|
||||
IReadOnlyDictionary<string, DateTime> writtenBefore = LastWrites(directory);
|
||||
Assert.Equal(["connection.dpapi", "journal.db", "qqq-session.dpapi", Path.Combine("staging", "document.pdf")], hashesBefore.Keys, StringComparer.Ordinal);
|
||||
|
||||
StateStatus status = TestState.Migrate(directory);
|
||||
|
||||
Assert.Equal(new StateStatus.NewerBuild(FoundVersion: newer, SupportedVersion: TestState.SchemaVersion), status);
|
||||
Assert.Equal(StateStatus.NewerBuildText, status.Refusal);
|
||||
Assert.Equal(hashesBefore, JournalProbe.FileHashes(directory));
|
||||
Assert.Equal(writtenBefore, LastWrites(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A service that is killed leaves its write-ahead log behind, and what it wrote last may be only there; one that is
|
||||
/// stopped in the ordinary way closes its connections as it exits, and SQLite folds the log into the journal. Here a newer
|
||||
/// build raised the version and was killed: the journal file alone still says this build's version (its header is read
|
||||
/// to show that) and the log says the next. The migration must find the newer version all the same, so it reads the log;
|
||||
/// that makes SQLite create its index file beside the journal, which holds no data. The journal and the log keep their bytes.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_NewerVersionOnlyInTheLog_IsFound_JournalAndLogUnchanged()
|
||||
{
|
||||
string root = NewFolder();
|
||||
string live = Path.Combine(root, "live");
|
||||
string killed = Path.Combine(root, "killed");
|
||||
int newer = TestState.SchemaVersion + 1;
|
||||
try
|
||||
{
|
||||
_ = TestState.Migrate(live);
|
||||
_ = Directory.CreateDirectory(killed);
|
||||
using(SqliteConnection newerBuild = TestState.OpenJournal(live))
|
||||
{
|
||||
using SqliteCommand command = newerBuild.CreateCommand();
|
||||
command.CommandText = "CREATE TABLE added_by_the_newer_build(value TEXT); PRAGMA user_version=" + newer.ToString(System.Globalization.CultureInfo.InvariantCulture);
|
||||
_ = command.ExecuteNonQuery();
|
||||
// The files as they are while the newer build still has them open: what is on disk if it never closes them.
|
||||
File.WriteAllBytes(Path.Combine(killed, "journal.db"), JournalProbe.ReadShared(Path.Combine(live, "journal.db")));
|
||||
File.WriteAllBytes(Path.Combine(killed, "journal.db-wal"), JournalProbe.ReadShared(Path.Combine(live, "journal.db-wal")));
|
||||
}
|
||||
IReadOnlyDictionary<string, string> hashesBefore = JournalProbe.FileHashes(killed);
|
||||
Assert.Equal(TestState.SchemaVersion, BinaryPrimitives.ReadInt32BigEndian(File.ReadAllBytes(Path.Combine(killed, "journal.db")).AsSpan(60, 4)));
|
||||
|
||||
StateStatus status = TestState.Migrate(killed);
|
||||
|
||||
Assert.Equal(new StateStatus.NewerBuild(FoundVersion: newer, SupportedVersion: TestState.SchemaVersion), status);
|
||||
IReadOnlyDictionary<string, string> hashesAfter = JournalProbe.FileHashes(killed);
|
||||
Assert.Equal(["journal.db", "journal.db-shm", "journal.db-wal"], hashesAfter.Keys, StringComparer.Ordinal);
|
||||
Assert.Equal(hashesBefore["journal.db"], hashesAfter["journal.db"]);
|
||||
Assert.Equal(hashesBefore["journal.db-wal"], hashesAfter["journal.db-wal"]);
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Migrating a migrated journal does nothing. The second call is given the product's own steps, each wrapped to count
|
||||
/// its runs, and none runs; the journal file has the same bytes afterwards and no other file has appeared.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_Twice_SecondDoesNothing()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest);
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
IReadOnlyDictionary<string, string> hashesBefore = JournalProbe.FileHashes(directory);
|
||||
int runs = 0;
|
||||
IReadOnlyList<Action<JournalMigrationContext>> counting = [.. TestState.MigrationSteps.Select(step => (Action<JournalMigrationContext>)(context =>
|
||||
{
|
||||
runs++;
|
||||
step(context);
|
||||
}))];
|
||||
|
||||
StateStatus second = TestState.Migrate(directory, counting);
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(second);
|
||||
Assert.Equal(0, runs);
|
||||
Assert.Equal(["journal.db"], hashesBefore.Keys, StringComparer.Ordinal);
|
||||
Assert.Equal(hashesBefore, JournalProbe.FileHashes(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The state folder's name can hold characters that mean something in the address SQLite is given for the read-only
|
||||
/// version read: a space, a number sign, a percent sign, a name that looks like an escape already, and letters outside
|
||||
/// ASCII, alone and together. For each such folder the journal is migrated, and the second call then has to read the
|
||||
/// version of that same journal: it finds it migrated, runs no step and leaves no file beside the journal. A journal a
|
||||
/// newer build stamped is recognized through the same read. The third name is the one that used to fail: a literal
|
||||
/// percent-escape beside a letter outside ASCII was passed on unescaped, and SQLite looked for another folder.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_FolderNameWithReservedCharacters_VersionIsStillRead()
|
||||
{
|
||||
string root = NewFolder();
|
||||
try
|
||||
{
|
||||
foreach(string name in (string[])["a b", "a#b", "c%20d é", "a b#c%20d é", "100%", "x%2523y", "é ü", "+&=;,'"])
|
||||
{
|
||||
string directory = Path.Combine(root, name);
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
int runs = 0;
|
||||
IReadOnlyList<Action<JournalMigrationContext>> counting = [context =>
|
||||
{
|
||||
runs++;
|
||||
TestState.MigrationSteps[0](context);
|
||||
}];
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory, counting));
|
||||
|
||||
Assert.Equal((name, 0), (name, runs));
|
||||
Assert.Equal(["journal.db"], JournalProbe.FileHashes(directory).Keys, StringComparer.Ordinal);
|
||||
_ = new LegacyStateBuilder(directory).WithJournalSql("PRAGMA user_version=5");
|
||||
Assert.Equal(new StateStatus.NewerBuild(FoundVersion: 5, SupportedVersion: 1), TestState.Migrate(directory, counting));
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Two callers migrate one journal at the same moment and the step runs once. The first caller's step signals that it
|
||||
/// has started and then waits to be released, while it holds the journal's lock and its open transaction. The second
|
||||
/// caller is started only then, and is seen to block: the only thing it can be waiting for is that lock. It has not
|
||||
/// returned while the first is held. Once the first is released and has committed, the second reads the version, finds
|
||||
/// the journal migrated and returns without running anything. Nothing here waits for a length of time.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_TwoCallersAtOnce_RunsOnce()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
using ManualResetEventSlim started = new();
|
||||
using ManualResetEventSlim released = new();
|
||||
int runs = 0;
|
||||
IReadOnlyList<Action<JournalMigrationContext>> steps =
|
||||
[
|
||||
context =>
|
||||
{
|
||||
_ = Interlocked.Increment(ref runs);
|
||||
started.Set();
|
||||
if(!released.Wait(TimeSpan.FromSeconds(30)))
|
||||
{ throw new TimeoutException("The test never released the first migration."); }
|
||||
TestState.MigrationSteps[0](context);
|
||||
},
|
||||
];
|
||||
var results = new StateStatus?[2];
|
||||
var failures = new Exception?[2];
|
||||
Thread first = MigratingThread(directory, steps, results, failures, 0);
|
||||
Thread second = MigratingThread(directory, steps, results, failures, 1);
|
||||
first.Start();
|
||||
Assert.True(started.Wait(TimeSpan.FromSeconds(30)), "The first migration never reached its step.");
|
||||
second.Start();
|
||||
bool secondWaited = SpinWait.SpinUntil(() => second.ThreadState.HasFlag(ThreadState.WaitSleepJoin), TimeSpan.FromSeconds(30));
|
||||
StateStatus? secondWhileFirstIsHeld = results[1];
|
||||
released.Set();
|
||||
first.Join();
|
||||
second.Join();
|
||||
|
||||
Assert.True(secondWaited, "The second migration never came to wait for the journal's lock.");
|
||||
Assert.Null(secondWhileFirstIsHeld);
|
||||
Assert.Equal([null, null], failures);
|
||||
Assert.IsType<StateStatus.Ready>(results[0]);
|
||||
Assert.IsType<StateStatus.Ready>(results[1]);
|
||||
Assert.Equal(1, runs);
|
||||
Assert.Equal(1, JournalProbe.Version(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The version is read twice: first without opening the journal for writing, and again inside the migration's
|
||||
/// transaction, under SQLite's write lock. The second reading decides which steps run. Inside one process the
|
||||
/// journal's lock makes the two agree; a second process holds no such lock and could migrate the journal between
|
||||
/// them. Here that happens. The list of steps given to the migration acts the first time it is asked for its length,
|
||||
/// which is after the first reading and before the transaction, and what it does is migrate the journal through a
|
||||
/// connection of its own, as that second process would. The migration then finds the journal at the current version
|
||||
/// and runs no step. In the second folder the other process is a newer build, which leaves a table of its own and the
|
||||
/// next version: the migration reports a newer build's journal, runs no step and leaves that version and that table.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_JournalMigratedByAnotherProcessAfterTheFirstReading_RunsNoStep()
|
||||
{
|
||||
string root = NewFolder();
|
||||
try
|
||||
{
|
||||
string same = new LegacyStateBuilder(Path.Combine(root, "same")).WithJournal(LegacyStateBuilder.JournalOldest).DirectoryPath;
|
||||
string newer = new LegacyStateBuilder(Path.Combine(root, "newer")).WithJournal(LegacyStateBuilder.JournalOldest).DirectoryPath;
|
||||
int runs = 0;
|
||||
Action<JournalMigrationContext> counted = context =>
|
||||
{
|
||||
runs++;
|
||||
TestState.MigrationSteps[0](context);
|
||||
};
|
||||
StepsThatActWhenCounted thisBuild = new([counted], () => Assert.IsType<StateStatus.Ready>(TestState.Migrate(same)));
|
||||
StepsThatActWhenCounted newerBuild = new([counted], () =>
|
||||
{
|
||||
_ = TestState.Migrate(newer);
|
||||
_ = new LegacyStateBuilder(newer).WithJournalSql("CREATE TABLE added_by_the_newer_build(value TEXT); PRAGMA user_version=2");
|
||||
});
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(same, thisBuild));
|
||||
Assert.Equal(new StateStatus.NewerBuild(FoundVersion: 2, SupportedVersion: 1), TestState.Migrate(newer, newerBuild));
|
||||
|
||||
Assert.True(thisBuild.Acted && newerBuild.Acted);
|
||||
Assert.Equal(0, runs);
|
||||
Assert.Equal(1, JournalProbe.Version(same));
|
||||
Assert.Equal(3L, JournalProbe.Scalar(same, "SELECT COUNT(*) FROM local_jobs"));
|
||||
Assert.Equal(2, JournalProbe.Version(newer));
|
||||
Assert.Contains("added_by_the_newer_build", JournalProbe.Tables(newer), StringComparer.Ordinal);
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A journal whose version is below zero was not written by any build, and no list of steps says how to go on from
|
||||
/// there. The migration refuses it and leaves it alone, where counting on from a negative number would have run steps
|
||||
/// that do not exist.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_NegativeVersion_IsRefused_JournalUnchanged()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.JournalOldest).WithJournalSql("PRAGMA user_version=-3");
|
||||
IReadOnlyDictionary<string, string> hashesBefore = JournalProbe.FileHashes(directory);
|
||||
|
||||
_ = Assert.Throws<InvalidDataException>(() => TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(hashesBefore, JournalProbe.FileHashes(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A journal that cannot be opened fails the migration with SQLite's own error, which is what stops the service from
|
||||
/// starting on state it cannot read. Here the journal's name is taken by a folder. Nothing is left open: the folder can
|
||||
/// be deleted at once.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Migrate_JournalCannotBeOpened_Fails()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
try
|
||||
{
|
||||
_ = Directory.CreateDirectory(Path.Combine(directory, "journal.db"));
|
||||
|
||||
SqliteException failure = Assert.Throws<SqliteException>(() => TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(14, failure.SqliteErrorCode);
|
||||
Directory.Delete(directory, recursive: true);
|
||||
Assert.False(Directory.Exists(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The service migrates the journal itself when it starts. The harness migrates when it is built, for the tests that
|
||||
/// never run the service loop, so the journal is deleted again here before the loop starts, and a store then refuses to
|
||||
/// open. The loop must make the journal and bring it to the current version before it answers anything: the snapshot
|
||||
/// it gives says the computer is paired, which it can only know by opening a store.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task ServiceStart_MigratesTheJournalBeforeItOpensAStore()
|
||||
{
|
||||
using SessionFixture fixture = new();
|
||||
string directory = fixture.Local.DirectoryPath;
|
||||
using SingleServerHarness runtime = fixture.Runtime();
|
||||
JournalFiles.Release(directory);
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "journal.db*"))
|
||||
{ File.Delete(file); }
|
||||
Assert.Equal(JournalSchema.NotMigratedText, Assert.Throws<InvalidOperationException>(runtime.GetSnapshot).Message);
|
||||
using CancellationTokenSource stop = new(TimeSpan.FromSeconds(60));
|
||||
Task running = runtime.RunAsync(stop.Token);
|
||||
try
|
||||
{
|
||||
string reply = await runtime.PipeAsync(new PipeRequest("snapshot"), stop.Token);
|
||||
Assert.True(JsonSerializer.Deserialize<AgentSnapshot>(reply, PrintApiClient.JsonOptions)!.Enrolled);
|
||||
Assert.Equal(TestState.SchemaVersion, JournalProbe.Version(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
await stop.CancelAsync();
|
||||
try
|
||||
{ await running; }
|
||||
catch(OperationCanceledException) { }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// When the journal cannot be brought up to date the service does not start, and before it gives up it logs one line
|
||||
/// an operator can act on. Here the journal of a paired, signed-in computer has been replaced by a file that is not a
|
||||
/// database. The start fails with SQLite's error. The line names the journal's folder and the kind of failure, says
|
||||
/// that the journal was not changed and what to check, and has nothing in it of SQLite's own text, the pairing or the
|
||||
/// session. The file has the bytes it had, and no session was created.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task ServiceStart_JournalCannotBeMigrated_LogsWhichJournalAndThatItWasNotChanged()
|
||||
{
|
||||
using SessionFixture fixture = new();
|
||||
fixture.SaveSession();
|
||||
string directory = fixture.Local.DirectoryPath;
|
||||
using SingleServerHarness runtime = fixture.Runtime();
|
||||
JournalFiles.Release(directory);
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "journal.db*"))
|
||||
{ File.Delete(file); }
|
||||
byte[] notAJournal = [.. Enumerable.Range(0, 8192).Select(position => (byte)(position * 31))];
|
||||
File.WriteAllBytes(Path.Combine(directory, "journal.db"), notAJournal);
|
||||
using CancellationTokenSource stop = new(TimeSpan.FromSeconds(60));
|
||||
|
||||
SqliteException failure = await Assert.ThrowsAsync<SqliteException>(() => runtime.RunAsync(stop.Token));
|
||||
|
||||
Assert.Equal(26, failure.SqliteErrorCode);
|
||||
string line = Assert.Single(fixture.Logs, entry => entry.Contains("could not be brought up to date", StringComparison.Ordinal));
|
||||
Assert.StartsWith("The print journal in " + Path.GetFullPath(directory) + " could not be brought up to date: the journal file is damaged or is not a journal (SQLite error 26). The journal was not changed;", line, StringComparison.Ordinal);
|
||||
Assert.Contains("Check that the print service can read and write that folder", line, StringComparison.Ordinal);
|
||||
Assert.All((string[])["file is not a database", "synthetic-token", "synthetic-qqq-cookie", "synthetic.invalid", "principal"], secret => Assert.DoesNotContain(secret, line, StringComparison.Ordinal));
|
||||
Assert.Equal(notAJournal, File.ReadAllBytes(Path.Combine(directory, "journal.db")));
|
||||
Assert.Equal(["connection.dpapi", "journal.db", "qqq-session.dpapi", "worker.lock"], Directory.EnumerateFiles(directory).Select(file => Path.GetFileName(file)).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(0, fixture.Created);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The line logged for a failed start names the failure by its kind: what an operator would call it, with SQLite's
|
||||
/// error number, or the failure's type when SQLite did not report it. It never repeats the failure's message, which
|
||||
/// here carries text shaped like a credential and like a row of the journal.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void StartFailureText_NamesTheKindOfFailure_AndNeverItsMessage()
|
||||
{
|
||||
const string Message = "token=synthetic-secret {\"eventId\":\"ev-1001-received\"}";
|
||||
(Exception Failure, string Kind)[] failures =
|
||||
[
|
||||
(new SqliteException(Message, 14), "the journal file could not be opened or written (SQLite error 14)"),
|
||||
(new SqliteException(Message, 8), "the journal file could not be opened or written (SQLite error 8)"),
|
||||
(new SqliteException(Message, 5), "another program has the journal file in use (SQLite error 5)"),
|
||||
(new SqliteException(Message, 11), "the journal file is damaged or is not a journal (SQLite error 11)"),
|
||||
(new SqliteException(Message, 13), "the disk is full (SQLite error 13)"),
|
||||
(new SqliteException(Message, 1), "SQLite refused a change (SQLite error 1)"),
|
||||
(new InvalidDataException(Message), "the migration stopped with InvalidDataException"),
|
||||
];
|
||||
string folder = NewFolder();
|
||||
foreach((Exception failure, string kind) in failures)
|
||||
{
|
||||
string text = JournalSchema.StartFailureText(folder, failure);
|
||||
Assert.StartsWith("The print journal in " + folder + " could not be brought up to date: " + kind + ". The journal was not changed; its changes are made together or not at all. Printing does not start.", text, StringComparison.Ordinal);
|
||||
Assert.EndsWith("then start the service again. Every start tries again.", text, StringComparison.Ordinal);
|
||||
Assert.DoesNotContain("synthetic-secret", text, StringComparison.Ordinal);
|
||||
Assert.DoesNotContain("ev-1001-received", text, StringComparison.Ordinal);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The three cases of the stored state say the same thing in the form the management contract's snapshot carries, and only a ready state refuses nothing.</summary>
|
||||
[Fact]
|
||||
public void StateStatus_MapsToTheSnapshotsStoredState()
|
||||
{
|
||||
Assert.Equal(new Windows.Management.StoredState(Windows.Management.StoredState.Ready), new StateStatus.Ready().ToStoredState());
|
||||
Assert.Null(new StateStatus.Ready().Refusal);
|
||||
StateStatus incomplete = new StateStatus.UpgradeIncomplete("connection.dpapi");
|
||||
Assert.Equal(new Windows.Management.StoredState(Windows.Management.StoredState.UpgradeIncomplete, File: "connection.dpapi"), incomplete.ToStoredState());
|
||||
Assert.Equal(StateStatus.UpgradeIncompleteText, incomplete.Refusal);
|
||||
StateStatus newer = new StateStatus.NewerBuild(4, 1);
|
||||
Assert.Equal(new Windows.Management.StoredState(Windows.Management.StoredState.NewerBuild, FoundVersion: 4, SupportedVersion: 1), newer.ToStoredState());
|
||||
Assert.Equal(StateStatus.NewerBuildText, newer.Refusal);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>A thread that migrates the folder with the given steps and records what it returned or how it failed, at its own position.</summary>
|
||||
private static Thread MigratingThread(string directory, IReadOnlyList<Action<JournalMigrationContext>> steps, StateStatus?[] results, Exception?[] failures, int position) => new(() =>
|
||||
{
|
||||
try
|
||||
{
|
||||
results[position] = TestState.Migrate(directory, steps);
|
||||
}
|
||||
catch(Exception exception) when(exception is SqliteException or IOException or InvalidOperationException or TimeoutException)
|
||||
{
|
||||
failures[position] = exception;
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A list of migration steps that does something once, the first time it is asked how many steps it has. The migration
|
||||
/// asks that only after it has read the journal's version for the first time, so this is how a test acts between that
|
||||
/// reading and the migration's transaction, on the migrating thread and without waiting for anything.
|
||||
/// </summary>
|
||||
private sealed class StepsThatActWhenCounted(IReadOnlyList<Action<JournalMigrationContext>> steps, Action act) : IReadOnlyList<Action<JournalMigrationContext>>
|
||||
{
|
||||
/// <summary>True once the list has been asked for its length and has acted.</summary>
|
||||
internal bool Acted { get; private set; }
|
||||
|
||||
|
||||
|
||||
/// <summary>The number of steps; the first call acts before it answers.</summary>
|
||||
public int Count
|
||||
{
|
||||
get
|
||||
{
|
||||
if(!Acted)
|
||||
{
|
||||
Acted = true;
|
||||
act();
|
||||
}
|
||||
return steps.Count;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The step at a position.</summary>
|
||||
public Action<JournalMigrationContext> this[int index] => steps[index];
|
||||
|
||||
|
||||
|
||||
/// <summary>The steps in order.</summary>
|
||||
public IEnumerator<Action<JournalMigrationContext>> GetEnumerator() => steps.GetEnumerator();
|
||||
|
||||
|
||||
|
||||
/// <summary>The steps in order, for a caller that does not name their type.</summary>
|
||||
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The last-write time of every file in the folder and beneath it, by relative path.</summary>
|
||||
private static SortedDictionary<string, DateTime> LastWrites(string directory)
|
||||
{
|
||||
SortedDictionary<string, DateTime> written = new(StringComparer.OrdinalIgnoreCase);
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "*", SearchOption.AllDirectories))
|
||||
{
|
||||
written[Path.GetRelativePath(directory, file)] = File.GetLastWriteTimeUtc(file);
|
||||
}
|
||||
return written;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>A path for a state folder of this test's own; nothing is created.</summary>
|
||||
private static string NewFolder() => Path.Combine(Path.GetTempPath(), "qqq-journal-migration-" + Guid.NewGuid().ToString("N"));
|
||||
}
|
||||
@@ -0,0 +1,281 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Globalization;
|
||||
using System.Reflection;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Qqq.PrintAgent.Core.Authentication;
|
||||
using Qqq.PrintAgent.Core.Printing;
|
||||
using Qqq.PrintAgent.Windows;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// What opening a store does to the journal's schema: nothing. A store works only on a journal the migration has brought
|
||||
/// to this build's version, says so plainly when it is given any other, and neither creates nor alters a table whatever
|
||||
/// is asked of it afterwards.
|
||||
/// </summary>
|
||||
public sealed class JournalOpenTests
|
||||
{
|
||||
/// <summary>
|
||||
/// Every public and internal method of the store, by name. The test below calls each of them. A method added to the
|
||||
/// store makes this list wrong, and the test fails until the new method is called there too, so "every store call" stays true.
|
||||
/// </summary>
|
||||
private static readonly string[] StoreMethods =
|
||||
[
|
||||
"Acknowledge", "AcknowledgeControlReceipt", "AcknowledgeFailure", "BeginControlCommand", "CompleteStagedPairing", "Dispose", "GetControlReceipt",
|
||||
"GetDeviceProfile", "GetExposedPrinterIds", "GetJobDetails", "GetPrinterProfile", "GetState", "HasJob", "Incomplete", "LoadAuthentication",
|
||||
"LoadAuthenticationForRecovery", "LoadConfiguration", "LoadControlState", "LoadRealtimeSettings", "LoadServerTrust", "NeedsAttentionCount", "Pending",
|
||||
"PendingControlReceipts", "PruneHistory", "QuarantineUnreadableAuthentication", "QueryJobs", "RecentJobs", "Record", "RecoverInterruptedControlCommands",
|
||||
"RetireExpiredControlReceipt", "SaveAuthentication", "SaveConfiguration", "SaveControlReceipt", "SaveControlState", "SaveDeviceProfile", "SavePrinterProfile",
|
||||
"SaveRealtimeSettings", "SaveServerTrust", "SetExposure", "StagePairing", "TrackJob",
|
||||
];
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A store opened before the journal has been migrated fails, and says that the journal has not been brought up to
|
||||
/// date: for a folder that does not exist (which the store does not create), for a folder with no journal in it (it
|
||||
/// creates none), and for a journal that has the complete 1.2.1 schema but no version, because the version is what a
|
||||
/// store goes by. A journal from a newer build is refused with the newer-build sentence. The failed open changes
|
||||
/// nothing in the journal and leaves no connection behind: once the pool is emptied the file can be opened alone.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Open_BeforeMigration_Throws()
|
||||
{
|
||||
string root = Path.Combine(Path.GetTempPath(), "qqq-journal-open-" + Guid.NewGuid().ToString("N"));
|
||||
try
|
||||
{
|
||||
string missing = Path.Combine(root, "missing");
|
||||
Assert.Equal(JournalSchema.NotMigratedText, Assert.Throws<InvalidOperationException>(() => TestState.OpenStoreAsFound(missing)).Message);
|
||||
Assert.False(Directory.Exists(missing));
|
||||
|
||||
string empty = Directory.CreateDirectory(Path.Combine(root, "empty")).FullName;
|
||||
Assert.Equal(JournalSchema.NotMigratedText, Assert.Throws<InvalidOperationException>(() => TestState.OpenStoreAsFound(empty)).Message);
|
||||
Assert.Empty(Directory.EnumerateFileSystemEntries(empty));
|
||||
|
||||
string unversioned = new LegacyStateBuilder(Path.Combine(root, "unversioned")).WithJournal(LegacyStateBuilder.Journal121).DirectoryPath;
|
||||
IReadOnlyList<string> schemaBefore = JournalProbe.Schema(unversioned);
|
||||
Assert.Equal(JournalSchema.NotMigratedText, Assert.Throws<InvalidOperationException>(() => TestState.OpenStoreAsFound(unversioned)).Message);
|
||||
Assert.Equal(0, JournalProbe.Version(unversioned));
|
||||
Assert.Equal(schemaBefore, JournalProbe.Schema(unversioned));
|
||||
JournalFiles.Release(unversioned);
|
||||
using(FileStream alone = new(Path.Combine(unversioned, "journal.db"), FileMode.Open, FileAccess.ReadWrite, FileShare.None))
|
||||
{ Assert.True(alone.Length > 0); }
|
||||
|
||||
string newer = new LegacyStateBuilder(Path.Combine(root, "newer")).WithJournal(LegacyStateBuilder.Journal121)
|
||||
.WithJournalSql("PRAGMA user_version=" + (TestState.SchemaVersion + 1).ToString(CultureInfo.InvariantCulture)).DirectoryPath;
|
||||
Assert.Equal(StateStatus.NewerBuildText, Assert.Throws<InvalidOperationException>(() => TestState.OpenStoreAsFound(newer)).Message);
|
||||
Assert.Equal(StateStatus.NewerBuildText, Assert.Throws<InvalidOperationException>(() => TestState.OpenStore(newer)).Message);
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A store whose journal cannot be opened fails with SQLite's own error and leaves nothing behind. Another program
|
||||
/// has the migrated journal open here and shares it with nobody, as a backup or a scanner can. SQLite tries for about
|
||||
/// a second and gives up with "unable to open database file". The connection that could not be opened is disposed
|
||||
/// before the failure is passed on, and once the other program lets go, a store opens on the journal as usual.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Open_JournalHeldByAnotherProgramWithoutSharing_FailsWithSqlitesError()
|
||||
{
|
||||
string directory = Path.Combine(Path.GetTempPath(), "qqq-journal-open-" + Guid.NewGuid().ToString("N"));
|
||||
try
|
||||
{
|
||||
_ = TestState.Migrate(directory);
|
||||
using(FileStream held = new(Path.Combine(directory, "journal.db"), FileMode.Open, FileAccess.ReadWrite, FileShare.None))
|
||||
{
|
||||
SqliteException failure = Assert.Throws<SqliteException>(() => TestState.OpenStoreAsFound(directory));
|
||||
Assert.Equal(14, failure.SqliteErrorCode);
|
||||
}
|
||||
using JournalStore store = TestState.OpenStoreAsFound(directory);
|
||||
Assert.Empty(store.RecentJobs());
|
||||
Assert.Equal(TestState.SchemaVersion, JournalProbe.Version(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// After the migration, nothing a store does changes the schema. The journal's tables and indexes are recorded, then a
|
||||
/// store is opened several times, with and without recovery, and every one of its methods is called, reads and writes
|
||||
/// alike, including the ones that 1.x used to create a table on first use (names, realtime settings, the certificate
|
||||
/// exception). The tables and indexes are then exactly what they were, and so is the version.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Open_AndEveryStoreCall_LeaveSqliteMasterUnchanged()
|
||||
{
|
||||
string directory = Path.Combine(Path.GetTempPath(), "qqq-journal-open-" + Guid.NewGuid().ToString("N"));
|
||||
try
|
||||
{
|
||||
_ = TestState.Migrate(directory);
|
||||
IReadOnlyList<string> schemaBefore = JournalProbe.Schema(directory);
|
||||
Assert.Equal(StoreMethods, typeof(JournalStore).GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.DeclaredOnly)
|
||||
.Where(method => (method.IsPublic || method.IsAssembly) && !method.IsSpecialName).Select(method => method.Name).Distinct(StringComparer.Ordinal).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
|
||||
using(JournalStore store = TestState.OpenStoreAsFound(directory))
|
||||
{
|
||||
UseJobs(store);
|
||||
UseSharingAndNames(store);
|
||||
}
|
||||
using(JournalStore store = TestState.OpenStore(directory, recoverUnreadableConfiguration: true))
|
||||
{
|
||||
UseControls(store);
|
||||
UseSettings(store);
|
||||
}
|
||||
using(JournalStore store = TestState.OpenStoreAsFound(directory))
|
||||
{
|
||||
UseProtectedFiles(store);
|
||||
}
|
||||
|
||||
Assert.Equal(schemaBefore, JournalProbe.Schema(directory));
|
||||
Assert.Equal(TestState.SchemaVersion, JournalProbe.Version(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The test above cannot see a statement that creates a table which is already there. This one can: the tables 1.x
|
||||
/// created on first use, the control state and an index are removed from a migrated journal, behind the migration's
|
||||
/// back. A store still opens, because opening looks at the version and at nothing else, and each call that needs a
|
||||
/// removed table fails with SQLite's "no such table" instead of quietly making it. The schema afterwards is exactly the
|
||||
/// damaged one. Only the migration puts a table in the journal, and it does not run again on a journal at the current version.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Open_AndStoreCalls_OnAJournalWithTablesRemoved_RecreateNothing()
|
||||
{
|
||||
string directory = Path.Combine(Path.GetTempPath(), "qqq-journal-open-" + Guid.NewGuid().ToString("N"));
|
||||
try
|
||||
{
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithPairing(new AgentConfiguration("https://synthetic.invalid", "device", "synthetic-token"));
|
||||
_ = TestState.Migrate(directory);
|
||||
_ = new LegacyStateBuilder(directory).WithJournalSql("DROP TABLE local_metadata; DROP TABLE realtime_settings; DROP TABLE server_trust; DROP TABLE control_state; DROP INDEX events_outbox");
|
||||
IReadOnlyList<string> damaged = JournalProbe.Schema(directory);
|
||||
Assert.Equal(34 - 4 - 4 - 1, damaged.Count);
|
||||
|
||||
using(JournalStore store = TestState.OpenStore(directory))
|
||||
{
|
||||
Assert.Contains("no such table: local_metadata", Assert.Throws<SqliteException>(store.GetDeviceProfile).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: local_metadata", Assert.Throws<SqliteException>(() => store.SavePrinterProfile("printer", new PrinterProfile("Queue"))).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: realtime_settings", Assert.Throws<SqliteException>(store.LoadRealtimeSettings).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: realtime_settings", Assert.Throws<SqliteException>(() => store.SaveRealtimeSettings(new RealtimeSettings())).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: server_trust", Assert.Throws<SqliteException>(store.LoadServerTrust).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: server_trust", Assert.Throws<SqliteException>(() => store.SaveServerTrust(new ServerTrust())).Message, StringComparison.Ordinal);
|
||||
Assert.Contains("no such table: control_state", Assert.Throws<SqliteException>(store.LoadControlState).Message, StringComparison.Ordinal);
|
||||
// What was not removed still works, without the index that served it.
|
||||
Assert.Equal(2, store.Pending().Count);
|
||||
}
|
||||
|
||||
Assert.Equal(damaged, JournalProbe.Schema(directory));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The job journal: events, summaries, delivery, review, paging and retention.</summary>
|
||||
private static void UseJobs(JournalStore store)
|
||||
{
|
||||
PrintJob job = new("job", "printer", "Synthetic", "PDF", "00", 3, new PrintOptions(), "2026-11-06T14:00:00Z", "CLAIMED");
|
||||
store.TrackJob(job, "Synthetic printer");
|
||||
store.Record(job.JobId, new PrintEvent("received", "RECEIVED"));
|
||||
Assert.Equal("RECEIVED", store.GetState(job.JobId));
|
||||
Assert.True(store.HasJob(job.JobId));
|
||||
Assert.Single(store.Incomplete());
|
||||
Assert.Single(store.Pending());
|
||||
store.Acknowledge("received");
|
||||
store.Record(job.JobId, new PrintEvent("failed", "FAILED", "Synthetic failure"));
|
||||
Assert.Equal(1, store.NeedsAttentionCount());
|
||||
store.AcknowledgeFailure(job.JobId);
|
||||
Assert.Single(store.RecentJobs());
|
||||
Assert.Equal(2, store.GetJobDetails(job.JobId)!.Events.Count);
|
||||
Assert.Equal(1, store.QueryJobs(new JobQuery(Search: "Synth", Status: "FAILED", PrinterId: "printer")).TotalCount);
|
||||
store.Acknowledge("failed");
|
||||
Assert.Equal(0, store.PruneHistory());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Sharing and the operator's own names for the computer and a printer.</summary>
|
||||
private static void UseSharingAndNames(JournalStore store)
|
||||
{
|
||||
store.SetExposure("printer", true, [new PrinterInfo("printer", "Synthetic printer", "Fixture", "None", "READY", ["PDF"])]);
|
||||
Assert.Single(store.GetExposedPrinterIds());
|
||||
store.SetExposure("printer", false, []);
|
||||
store.SaveDeviceProfile(new DeviceProfile("Computer", "Description", "Location"));
|
||||
Assert.Equal("Computer", store.GetDeviceProfile().Name);
|
||||
store.SavePrinterProfile("printer", new PrinterProfile("Queue"));
|
||||
Assert.Equal("Queue", store.GetPrinterProfile("printer").DisplayName);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Remote control: the desired state, a command from intent to delivered receipt, and one interrupted before its result.</summary>
|
||||
private static void UseControls(JournalStore store)
|
||||
{
|
||||
store.SaveControlState(AgentControlState.Initial with { Support = "SUPPORTED" });
|
||||
Assert.Equal("SUPPORTED", store.LoadControlState().Support);
|
||||
string expires = DateTimeOffset.UtcNow.AddMinutes(5).ToString("O", CultureInfo.InvariantCulture);
|
||||
Assert.True(store.BeginControlCommand(new DeviceControlCommand("interrupted", "request-1", "REFRESH_INVENTORY", expires, expires)));
|
||||
store.RecoverInterruptedControlCommands();
|
||||
Assert.Equal("OUTCOME_UNKNOWN", store.GetControlReceipt("interrupted")!.Outcome);
|
||||
Assert.True(store.BeginControlCommand(new DeviceControlCommand("finished", "request-2", "REFRESH_INVENTORY", expires, expires)));
|
||||
store.SaveControlReceipt("finished", new DeviceControlReceipt("receipt", "INVENTORY_REFRESHED", expires));
|
||||
Assert.Equal(2, store.PendingControlReceipts().Count);
|
||||
store.AcknowledgeControlReceipt("finished");
|
||||
Assert.False(store.RetireExpiredControlReceipt("interrupted"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The two settings 1.x created a table for on first use.</summary>
|
||||
private static void UseSettings(JournalStore store)
|
||||
{
|
||||
Assert.Equal(0, store.LoadRealtimeSettings().Revision);
|
||||
store.SaveRealtimeSettings(new RealtimeSettings(Revision: 2));
|
||||
Assert.Equal(2, store.LoadRealtimeSettings().Revision);
|
||||
Assert.False(store.LoadServerTrust().AllowUntrustedCertificate);
|
||||
store.SaveServerTrust(new ServerTrust(true, "https://synthetic.invalid"));
|
||||
Assert.True(store.LoadServerTrust().AllowUntrustedCertificate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The pairing and the session, which live in protected files beside the journal, including a staged pairing change.</summary>
|
||||
private static void UseProtectedFiles(JournalStore store)
|
||||
{
|
||||
Uri origin = new("https://synthetic.invalid/");
|
||||
AgentAuthentication session = new(new QqqSessionBinding(origin, "principal", "tenant", "device"), new QqqSessionCredential(origin, "synthetic-qqq-cookie"));
|
||||
Assert.Null(store.LoadConfiguration());
|
||||
Assert.Null(store.LoadAuthenticationForRecovery());
|
||||
Assert.False(store.QuarantineUnreadableAuthentication());
|
||||
store.SaveConfiguration(new AgentConfiguration("https://synthetic.invalid", "device", "synthetic-token"));
|
||||
store.SaveAuthentication(session);
|
||||
Assert.Equal("device", store.LoadConfiguration()!.DeviceId);
|
||||
store.StagePairing(new AgentConfiguration("https://synthetic.invalid", "device", "rotated-token"), session);
|
||||
Assert.Equal(StagedPairingOutcome.Discarded, store.CompleteStagedPairing());
|
||||
Assert.Equal(StagedPairingOutcome.Discarded, store.StagedPairing);
|
||||
Assert.Null(store.StagedPairingFailure);
|
||||
Assert.Equal("principal", store.LoadAuthentication()!.Binding.PrincipalId);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Buffers.Binary;
|
||||
using System.Globalization;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// What the migration tests ask of a state folder from outside the product: the journal's schema version, its schema as
|
||||
/// SQLite records it, a digest of each table's rows, and a hash of every file in the folder. The journal questions use
|
||||
/// the test project's own unpooled connection, which closes the journal when it is done; closing the last connection
|
||||
/// makes SQLite fold the write-ahead log into the journal file, so a test that compares file bytes takes its hashes
|
||||
/// with <see cref="FileHashes"/> and does not ask a journal question in between.
|
||||
/// </summary>
|
||||
internal static class JournalProbe
|
||||
{
|
||||
/// <summary>The journal's schema version (<c>PRAGMA user_version</c>).</summary>
|
||||
internal static int Version(string directory) => Convert.ToInt32(Scalar(directory, "PRAGMA user_version"), CultureInfo.InvariantCulture);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The journal's schema: one line per table and index, with the statement that made it, in name order. SQLite keeps
|
||||
/// that text as the object was created plus any column added later, so two journals with equal lines have the same
|
||||
/// schema in every detail.
|
||||
/// </summary>
|
||||
internal static IReadOnlyList<string> Schema(string directory)
|
||||
=> Rows(directory, "SELECT type,name,tbl_name,sql FROM sqlite_master ORDER BY name");
|
||||
|
||||
|
||||
|
||||
/// <summary>The names of the journal's tables, in name order.</summary>
|
||||
internal static IReadOnlyList<string> Tables(string directory)
|
||||
=> [.. Rows(directory, "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name").Select(row => row["name=".Length..])];
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A SHA-256 digest of each table's rows, by table name. A row is its column names and values, so a renamed, added or
|
||||
/// removed column changes the digest as surely as a changed value does; the rows are put in order before they are
|
||||
/// hashed, so only their content counts and not where SQLite keeps them.
|
||||
/// </summary>
|
||||
internal static IReadOnlyDictionary<string, string> RowDigests(string directory)
|
||||
{
|
||||
SortedDictionary<string, string> digests = new(StringComparer.Ordinal);
|
||||
foreach(string table in Tables(directory))
|
||||
{
|
||||
//////////////////////////////////////////////////////////////////////////
|
||||
// the table's name comes from the journal's own catalog and is quoted, //
|
||||
// which is the one place a name, not a value, has to go into the text //
|
||||
//////////////////////////////////////////////////////////////////////////
|
||||
List<string> rows = [.. Rows(directory, "SELECT * FROM \"" + table.Replace("\"", "\"\"", StringComparison.Ordinal) + "\"")];
|
||||
rows.Sort(StringComparer.Ordinal);
|
||||
digests[table] = rows.Count.ToString(CultureInfo.InvariantCulture) + " rows " + Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(string.Join('\n', rows))));
|
||||
}
|
||||
return digests;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The SHA-256 of every file in the folder and beneath it, by path relative to the folder. The files are read with
|
||||
/// full sharing, so a file another handle has open can still be hashed, and nothing is opened for writing.
|
||||
/// </summary>
|
||||
internal static IReadOnlyDictionary<string, string> FileHashes(string directory)
|
||||
{
|
||||
SortedDictionary<string, string> hashes = new(StringComparer.OrdinalIgnoreCase);
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "*", SearchOption.AllDirectories))
|
||||
{
|
||||
hashes[Path.GetRelativePath(directory, file)] = Convert.ToHexString(SHA256.HashData(ReadShared(file)));
|
||||
}
|
||||
return hashes;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// How many transactions the journal's write-ahead log holds. SQLite writes a transaction to the log as a run of
|
||||
/// frames, one for each page it changed, and marks the last frame as the commit by recording the size of the database
|
||||
/// there; every other frame has zero in that place. A frame counts only while it carries the salt in the log's
|
||||
/// header, which is how SQLite tells the frames of the current log from what an earlier one left in the file. A
|
||||
/// folder with no log, or an empty one, holds none. The log is read with full sharing, while SQLite has it open.
|
||||
/// </summary>
|
||||
internal static int CommitsInLog(string directory)
|
||||
{
|
||||
const int HeaderBytes = 32;
|
||||
const int FrameHeaderBytes = 24;
|
||||
string path = Path.Combine(directory, "journal.db-wal");
|
||||
byte[] log = File.Exists(path) ? ReadShared(path) : [];
|
||||
if(log.Length < HeaderBytes)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
int frameBytes = FrameHeaderBytes + (int)BinaryPrimitives.ReadUInt32BigEndian(log.AsSpan(8, 4));
|
||||
int commits = 0;
|
||||
for(int frame = HeaderBytes; frame + frameBytes <= log.Length && log.AsSpan(frame + 8, 8).SequenceEqual(log.AsSpan(16, 8)); frame += frameBytes)
|
||||
{
|
||||
if(BinaryPrimitives.ReadUInt32BigEndian(log.AsSpan(frame + 4, 4)) != 0)
|
||||
{
|
||||
commits++;
|
||||
}
|
||||
}
|
||||
return commits;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The bytes of a file that another handle may have open for writing, as SQLite has its journal and log.</summary>
|
||||
internal static byte[] ReadShared(string file)
|
||||
{
|
||||
using FileStream stream = new(file, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
|
||||
using MemoryStream bytes = new();
|
||||
stream.CopyTo(bytes);
|
||||
return bytes.ToArray();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs a query that returns one value.</summary>
|
||||
internal static object? Scalar(string directory, string sql)
|
||||
{
|
||||
using SqliteConnection journal = TestState.OpenJournal(directory);
|
||||
using SqliteCommand command = journal.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
return command.ExecuteScalar();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs a query and returns each row as one line of <c>column=value</c> pairs, a missing value written as <c>NULL</c>.</summary>
|
||||
internal static IReadOnlyList<string> Rows(string directory, string sql)
|
||||
{
|
||||
using SqliteConnection journal = TestState.OpenJournal(directory);
|
||||
using SqliteCommand command = journal.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
using SqliteDataReader reader = command.ExecuteReader();
|
||||
List<string> rows = [];
|
||||
while(reader.Read())
|
||||
{
|
||||
rows.Add(string.Join(" | ", Enumerable.Range(0, reader.FieldCount).Select(column
|
||||
=> reader.GetName(column) + "=" + (reader.IsDBNull(column) ? "NULL" : Convert.ToString(reader.GetValue(column), CultureInfo.InvariantCulture)))));
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.Json;
|
||||
using Qqq.PrintAgent.Core.Printing;
|
||||
using Qqq.PrintAgent.Windows;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// An installation of an earlier release, upgraded in place. The release depends on that: a computer running 1.2.x
|
||||
/// must come out of the first start of this build with its pairing, its job history and its settings as it left them.
|
||||
/// <see cref="JournalMigrationTests"/> shows it for a journal that was closed cleanly. These tests show it for the
|
||||
/// states a real installation can also be in: a journal whose latest rows are only in the write-ahead log, a journal in
|
||||
/// each shape between the oldest and 1.2.1, and a whole state folder under the service loop with jobs left in flight.
|
||||
/// </summary>
|
||||
public sealed class JournalUpgradeTests
|
||||
{
|
||||
/// <summary>The pairing whose scope most rows of the 1.2.1 fixture carry.</summary>
|
||||
private static readonly AgentConfiguration Pairing = new("https://synthetic.invalid", "device", "synthetic-token");
|
||||
|
||||
|
||||
|
||||
/// <summary>A 1.0.x journal: no certificate exception table, which arrived in 1.1.0.</summary>
|
||||
private const string WithoutTrust = "DROP TABLE server_trust;";
|
||||
|
||||
|
||||
|
||||
/// <summary>A journal from before realtime settings were stored.</summary>
|
||||
private const string WithoutRealtime = WithoutTrust + " DROP TABLE realtime_settings;";
|
||||
|
||||
|
||||
|
||||
/// <summary>A journal from before remote control: no control state and no control commands, with their two indexes.</summary>
|
||||
private const string WithoutControl = WithoutRealtime + " DROP TABLE control_commands; DROP TABLE control_state;";
|
||||
|
||||
|
||||
|
||||
/// <summary>A journal from before job summaries had a creation time: without that column and the five indexes the job list sorts by.</summary>
|
||||
private const string WithoutCreationTime = WithoutControl
|
||||
+ " DROP INDEX jobs_display_createdAt; DROP INDEX jobs_display_title; DROP INDEX jobs_display_printerName; DROP INDEX jobs_display_status; DROP INDEX jobs_display_format;"
|
||||
+ " ALTER TABLE local_jobs DROP COLUMN created_at;";
|
||||
|
||||
|
||||
|
||||
/// <summary>A journal from before the first indexes and the operator's own names: the events and summaries of the release that introduced the scope, and the shared-printer list.</summary>
|
||||
private const string WithoutIndexes = WithoutCreationTime
|
||||
+ " DROP TABLE local_metadata; DROP INDEX events_outbox; DROP INDEX events_job; DROP INDEX jobs_recent; DROP INDEX jobs_status; DROP INDEX jobs_printer;"
|
||||
+ " DROP INDEX jobs_attention; DROP INDEX jobs_retention; DROP INDEX jobs_title_search; DROP INDEX jobs_printer_search; DROP INDEX jobs_id_search;";
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A 1.2.1 service was killed with everything it had written still in the write-ahead log: the journal file by itself
|
||||
/// has no table at all. The migration reads the log, and the journal comes out with the schema and, table by table,
|
||||
/// the rows of a 1.2.1 journal that was closed cleanly.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Journal121_KilledWithItsRowsOnlyInTheLog_KeepsEveryRow()
|
||||
{
|
||||
string root = NewFolder();
|
||||
try
|
||||
{
|
||||
string clean = new LegacyStateBuilder(Path.Combine(root, "clean")).WithJournal(LegacyStateBuilder.Journal121).DirectoryPath;
|
||||
string killed = new LegacyStateBuilder(Path.Combine(root, "killed")).WithJournalLeftByAKill(LegacyStateBuilder.Journal121).DirectoryPath;
|
||||
string fileAlone = new LegacyStateBuilder(Path.Combine(root, "file-alone")).WithFile("journal.db", File.ReadAllBytes(Path.Combine(killed, "journal.db"))).DirectoryPath;
|
||||
Assert.Empty(JournalProbe.Tables(fileAlone));
|
||||
Assert.True(new FileInfo(Path.Combine(killed, "journal.db-wal")).Length > 0);
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(killed));
|
||||
|
||||
Assert.Equal(1, JournalProbe.Version(killed));
|
||||
Assert.Equal(JournalProbe.RowDigests(clean), JournalProbe.RowDigests(killed));
|
||||
Assert.Equal(JournalProbe.Schema(clean), JournalProbe.Schema(killed));
|
||||
Assert.Equal(9, JournalProbe.RowDigests(killed).Count);
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The shapes a journal had between the oldest and 1.2.1, each made by taking away from the 1.2.1 journal what the
|
||||
/// later releases added: the certificate exception table (so, a 1.0.x journal), a 1.2.1 journal that never used any of
|
||||
/// the three tables created on first use, the realtime settings, the control tables, the creation time with its five
|
||||
/// indexes, and the earlier indexes with the local names. Each ends in the one schema a new journal has, keeps every
|
||||
/// row it had, and is read by the current store. A summary that had no creation time gets one.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void IntermediateShapes_EndInOneSchema_AndKeepTheirRows()
|
||||
{
|
||||
(string Sql, bool HasCreationTime)[] shapes =
|
||||
[
|
||||
(WithoutTrust, true),
|
||||
(WithoutTrust + " DROP TABLE realtime_settings; DROP TABLE local_metadata;", true),
|
||||
(WithoutRealtime, true),
|
||||
(WithoutControl, true),
|
||||
(WithoutCreationTime, false),
|
||||
(WithoutIndexes, false),
|
||||
];
|
||||
string root = NewFolder();
|
||||
try
|
||||
{
|
||||
string reference = Path.Combine(root, "reference");
|
||||
_ = TestState.Migrate(reference);
|
||||
IReadOnlyList<string> schema = JournalProbe.Schema(reference);
|
||||
// What each shape has left of the 34 tables and indexes of a 1.2.1 journal; the last is four tables and the three indexes SQLite makes for their keys.
|
||||
Assert.Equal([32, 28, 30, 24, 19, 7], shapes.Select((shape, position) => SchemaObjects(Path.Combine(root, "count-" + position), shape.Sql)));
|
||||
foreach((string sql, bool hasCreationTime) in shapes)
|
||||
{
|
||||
AssertUpgrades(Path.Combine(root, Guid.NewGuid().ToString("N")), sql, hasCreationTime, schema);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(root);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A 1.2.1 state folder as a killed service left it: the pairing, the full journal, a job whose submission to Windows
|
||||
/// had begun (SUBMITTING) and a job that had only been received. This build's service loop starts on it. It migrates
|
||||
/// the journal and then settles the two jobs by the rule that has always held: the interrupted submission becomes
|
||||
/// OUTCOME_UNKNOWN, because only an operator can tell whether paper came out, and the job Windows was never asked to
|
||||
/// print becomes FAILED. Nothing is printed again. The snapshot shows the pairing, every job of the pairing with the
|
||||
/// state 1.2.1 left it in, the shared printers with the operator's names, the computer's own name and the control
|
||||
/// state; the certificate exception is still there. No session is created, because none was saved. The events 1.2.1
|
||||
/// wrote are unchanged, and the only new ones are the two that settle those jobs.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Installation121_FirstStart_KeepsPairingHistoryAndSettings_AndSettlesJobsLeftInFlight()
|
||||
{
|
||||
string directory = NewFolder();
|
||||
ConcurrentQueue<string> logs = new();
|
||||
int sessions = 0;
|
||||
try
|
||||
{
|
||||
using SingleServerHarness runtime = new(directory, logs.Enqueue, new EnrolledStateFixture.SyntheticPrinter(), sessionFactory: _ =>
|
||||
{
|
||||
sessions++;
|
||||
throw new InvalidOperationException("No session may be created: none is saved.");
|
||||
});
|
||||
Installation121WithJobsInFlight(directory);
|
||||
IReadOnlyDictionary<string, string> rowsBefore = JournalProbe.RowDigests(directory);
|
||||
IReadOnlyList<string> eventsBefore = JournalProbe.Rows(directory, "SELECT * FROM events ORDER BY sequence");
|
||||
using CancellationTokenSource stop = new(TimeSpan.FromSeconds(60));
|
||||
Task running = runtime.RunAsync(stop.Token);
|
||||
try
|
||||
{
|
||||
AssertKept(JsonSerializer.Deserialize<AgentSnapshot>(await runtime.PipeAsync(new PipeRequest("snapshot"), stop.Token), PrintApiClient.JsonOptions)!);
|
||||
Assert.Equal(new ServerTrust(true, "https://synthetic.invalid"), JsonSerializer.Deserialize<ServerTrust>(await runtime.PipeAsync(new PipeRequest("get-server-trust"), stop.Token), PrintApiClient.JsonOptions));
|
||||
Assert.Contains("Agent stopped during native submission; inspect the printer before retrying.", await runtime.PipeAsync(new PipeRequest("job-details", JobId: "job-1005"), stop.Token), StringComparison.Ordinal);
|
||||
Assert.Contains("Agent stopped before native submission.", await runtime.PipeAsync(new PipeRequest("job-details", JobId: "job-1006"), stop.Token), StringComparison.Ordinal);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await stop.CancelAsync();
|
||||
try
|
||||
{ await running; }
|
||||
catch(OperationCanceledException) { }
|
||||
}
|
||||
Assert.Equal(0, sessions);
|
||||
Assert.Equal(1, JournalProbe.Version(directory));
|
||||
Assert.Equal(eventsBefore, JournalProbe.Rows(directory, "SELECT * FROM events WHERE sequence<=12 ORDER BY sequence"));
|
||||
Assert.Equal(["job_id=job-1005 | state=OUTCOME_UNKNOWN | acknowledged=0", "job_id=job-1006 | state=FAILED | acknowledged=0"],
|
||||
JournalProbe.Rows(directory, "SELECT job_id,state,acknowledged FROM events WHERE sequence>12 ORDER BY job_id"));
|
||||
IReadOnlyDictionary<string, string> rowsAfter = JournalProbe.RowDigests(directory);
|
||||
Assert.All((string[])["exposed_printers", "local_metadata", "realtime_settings", "server_trust", "control_state"], table => Assert.Equal((table, rowsBefore[table]), (table, rowsAfter[table])));
|
||||
}
|
||||
finally
|
||||
{
|
||||
JournalFiles.Delete(directory);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Replaces the journal the harness made in the folder with a 1.2.1 installation's, so that the service loop is the
|
||||
/// first thing of this build to touch it, and saves the pairing as 1.2.1 did. Two jobs of that pairing are added in
|
||||
/// the states a killed service leaves: job-1005 with its SUBMITTING event recorded, job-1006 received and no further.
|
||||
/// </summary>
|
||||
private static void Installation121WithJobsInFlight(string directory)
|
||||
{
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "journal.db*"))
|
||||
{ File.Delete(file); }
|
||||
LegacyStateBuilder state = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithPairing(Pairing);
|
||||
string scope = (string)JournalProbe.Scalar(directory, "SELECT connection_id FROM local_jobs WHERE job_id='job-1001'")!;
|
||||
_ = state.WithJournalSql($$"""
|
||||
INSERT INTO events(job_id,event_id,state,json,acknowledged,recorded_at,connection_id) VALUES
|
||||
('job-1005','ev-1005-received','RECEIVED','{"eventId":"ev-1005-received","status":"RECEIVED"}',1,'2026-11-06T16:00:00.0000000+00:00','{{scope}}'),
|
||||
('job-1005','ev-1005-submitting','SUBMITTING','{"eventId":"ev-1005-submitting","status":"SUBMITTING"}',1,'2026-11-06T16:00:01.0000000+00:00','{{scope}}'),
|
||||
('job-1006','ev-1006-received','RECEIVED','{"eventId":"ev-1006-received","status":"RECEIVED"}',1,'2026-11-06T16:01:00.0000000+00:00','{{scope}}');
|
||||
INSERT INTO local_jobs(connection_id,job_id,printer_id,printer_name,title,format,state,last_event_json,updated_at,failure_acknowledged,history_pruned,created_at) VALUES
|
||||
('{{scope}}','job-1005','printer-a','Front desk laser','Packing slip 1005','PDF','SUBMITTING','{"eventId":"ev-1005-submitting","status":"SUBMITTING"}','2026-11-06T16:00:01.0000000+00:00',0,0,'2026-11-06T15:59:58.0000000+00:00'),
|
||||
('{{scope}}','job-1006','printer-a','Front desk laser','Packing slip 1006','PDF','RECEIVED','{"eventId":"ev-1006-received","status":"RECEIVED"}','2026-11-06T16:01:00.0000000+00:00',0,0,'2026-11-06T16:00:58.0000000+00:00');
|
||||
""");
|
||||
Assert.Equal(0, JournalProbe.Version(directory));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>What the first snapshot after the upgrade must show of the 1.2.1 installation; each line names rows of the fixture or the two jobs left in flight.</summary>
|
||||
private static void AssertKept(AgentSnapshot snapshot)
|
||||
{
|
||||
Assert.Equal((true, "https://synthetic.invalid", "device"), (snapshot.Enrolled, snapshot.ServerUrl, snapshot.DeviceId));
|
||||
Assert.Equal(
|
||||
[
|
||||
"job-0950=SPOOLED", "job-1001=SPOOLED", "job-1002=FAILED", "job-1003=FAILED reviewed", "job-1004=OUTCOME_UNKNOWN", "job-1005=OUTCOME_UNKNOWN", "job-1006=FAILED",
|
||||
], snapshot.Jobs.Select(job => job.JobId + "=" + job.Status + (job.FailureAcknowledged ? " reviewed" : string.Empty)).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(["printer-a=Front desk laser", "printer-zpl=Label printer"],
|
||||
snapshot.Printers.Where(printer => printer.Exposed).Select(printer => printer.PrinterId + "=" + printer.DisplayName).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(new DeviceProfile("Front desk PC", "Shipping station\nsecond shift", "Dock 2"), snapshot.DeviceProfile);
|
||||
Assert.Equal((7L, true, "CANCEL_SPOOL"), (snapshot.Control!.DesiredState.Revision, snapshot.Control.DesiredState.DispatchPaused, snapshot.Control.LastCommand));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Migrates a 1.2.1 journal that was first put into an earlier shape, and requires the one schema, the rows it had and
|
||||
/// a store that reads them. The rows of every table the shape had are compared by digest. Where the shape had no
|
||||
/// creation time, the summaries are compared by the columns they had, and every summary must have got a creation time.
|
||||
/// </summary>
|
||||
private static void AssertUpgrades(string directory, string shape, bool hasCreationTime, IReadOnlyList<string> schema)
|
||||
{
|
||||
const string Summaries = "SELECT connection_id,job_id,printer_id,printer_name,title,format,state,last_event_json,updated_at,failure_acknowledged,history_pruned FROM local_jobs ORDER BY connection_id,job_id";
|
||||
_ = new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithPairing(Pairing).WithJournalSql(shape);
|
||||
IReadOnlyDictionary<string, string> rowsBefore = JournalProbe.RowDigests(directory);
|
||||
IReadOnlyList<string> summariesBefore = JournalProbe.Rows(directory, Summaries);
|
||||
|
||||
Assert.IsType<StateStatus.Ready>(TestState.Migrate(directory));
|
||||
|
||||
Assert.Equal(schema, JournalProbe.Schema(directory));
|
||||
IReadOnlyDictionary<string, string> rowsAfter = JournalProbe.RowDigests(directory);
|
||||
Assert.All(rowsBefore.Keys.Where(table => hasCreationTime || !string.Equals(table, "local_jobs", StringComparison.Ordinal)), table => Assert.Equal((table, rowsBefore[table]), (table, rowsAfter[table])));
|
||||
Assert.Equal(summariesBefore, JournalProbe.Rows(directory, Summaries));
|
||||
Assert.Equal(0L, JournalProbe.Scalar(directory, "SELECT COUNT(*) FROM local_jobs WHERE created_at IS NULL"));
|
||||
using JournalStore store = TestState.OpenStoreAsFound(directory);
|
||||
Assert.Equal(["job-0950", "job-1001", "job-1002", "job-1003", "job-1004"], store.RecentJobs().Select(job => job.JobId).Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
Assert.Equal(["ev-1002-failed", "ev-1004-unknown"], store.Pending().Select(pending => pending.Event.EventId), StringComparer.Ordinal);
|
||||
Assert.Equal(["printer-a", "printer-zpl"], store.GetExposedPrinterIds().Order(StringComparer.Ordinal), StringComparer.Ordinal);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>How many tables and indexes a 1.2.1 journal has once it is put into a shape, which shows that the shape took away what it says.</summary>
|
||||
private static int SchemaObjects(string directory, string shape)
|
||||
=> JournalProbe.Schema(new LegacyStateBuilder(directory).WithJournal(LegacyStateBuilder.Journal121).WithJournalSql(shape).DirectoryPath).Count;
|
||||
|
||||
|
||||
|
||||
/// <summary>A path for a state folder of this test's own; nothing is created.</summary>
|
||||
private static string NewFolder() => Path.Combine(Path.GetTempPath(), "qqq-journal-upgrade-" + Guid.NewGuid().ToString("N"));
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Security.Cryptography;
|
||||
using System.Text.Json;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Qqq.PrintAgent.Core.Printing;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Builds a state folder as an earlier release left it, so that what this build does with such a folder can be tested
|
||||
/// without installing that release. Each call adds one thing the old release would have written, in the form it wrote it,
|
||||
/// and nothing here goes through the current store: the journal is made by running a script of the old release's own
|
||||
/// statements (<c>Fixtures/journal-*.sql</c>), and a protected file is written with the serialization and protection that
|
||||
/// release used. The calls can be made in any order and repeated. The upgrade from 1.x moves the protected files as well
|
||||
/// as migrating the journal; the session, a staged session and the set-aside recovery files are added here, each as a
|
||||
/// method of its own, when the tests of those moves need them.
|
||||
/// </summary>
|
||||
internal sealed class LegacyStateBuilder
|
||||
{
|
||||
/// <summary>The journal of a 1.2.1 installation: the full schema of the last release without a schema version, with rows in every table.</summary>
|
||||
internal const string Journal121 = "journal-1.2.1.sql";
|
||||
|
||||
|
||||
|
||||
/// <summary>The oldest journal history can establish: one six-column events table.</summary>
|
||||
internal const string JournalOldest = "journal-oldest.sql";
|
||||
|
||||
|
||||
|
||||
/// <summary>The state folder being built, as a full path. The constructor creates it.</summary>
|
||||
internal string DirectoryPath { get; }
|
||||
|
||||
|
||||
|
||||
/// <summary>Starts a state folder at the given path; the folder is created empty when it does not exist.</summary>
|
||||
internal LegacyStateBuilder(string directory)
|
||||
{
|
||||
DirectoryPath = Path.GetFullPath(directory);
|
||||
_ = Directory.CreateDirectory(DirectoryPath);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Makes the journal by running one of the fixture scripts against it. The connection is the test project's unpooled
|
||||
/// one, so when this returns the journal is closed, its write-ahead log is folded in and removed, and the folder holds
|
||||
/// <c>journal.db</c> alone, as after a clean stop.
|
||||
/// </summary>
|
||||
internal LegacyStateBuilder WithJournal(string script)
|
||||
{
|
||||
return WithJournalSql(Script(script));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Makes the journal as a service that was killed leaves it: everything the script wrote is in the write-ahead log,
|
||||
/// and the journal file itself holds none of it. The script runs in a folder beside this one, and the journal and its
|
||||
/// log are copied here while the connection that wrote them is still open, which is what is on disk when a process
|
||||
/// never closes them. The index file is not copied: SQLite discards an index that no process has open and builds it
|
||||
/// again from the log. The folder beside this one is removed.
|
||||
/// </summary>
|
||||
internal LegacyStateBuilder WithJournalLeftByAKill(string script)
|
||||
{
|
||||
string live = DirectoryPath + "-live";
|
||||
_ = Directory.CreateDirectory(live);
|
||||
try
|
||||
{
|
||||
using SqliteConnection journal = TestState.OpenJournal(live);
|
||||
using SqliteCommand command = journal.CreateCommand();
|
||||
command.CommandText = Script(script);
|
||||
_ = command.ExecuteNonQuery();
|
||||
foreach(string file in (string[])["journal.db", "journal.db-wal"])
|
||||
{
|
||||
File.WriteAllBytes(Path.Combine(DirectoryPath, file), JournalProbe.ReadShared(Path.Combine(live, file)));
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
Directory.Delete(live, recursive: true);
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The text of a fixture script: the statements an earlier release ran, for a test that needs some of them by themselves.</summary>
|
||||
internal static string Script(string script) => File.ReadAllText(Path.Combine(AppContext.BaseDirectory, "Fixtures", script));
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs further SQL against the journal: a row the scripts do not have, or damage a test wants to see handled.</summary>
|
||||
internal LegacyStateBuilder WithJournalSql(string sql)
|
||||
{
|
||||
using SqliteConnection journal = TestState.OpenJournal(DirectoryPath);
|
||||
using SqliteCommand command = journal.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
_ = command.ExecuteNonQuery();
|
||||
return this;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Writes the pairing as 1.x did (transcribed from the store's <c>SaveConfiguration</c> at tag v1.2.1): the configuration
|
||||
/// as JSON with the print API's serializer options, protected with DPAPI for the current account, in
|
||||
/// <c>connection.dpapi</c>. The service account's protection is the test account's here.
|
||||
/// </summary>
|
||||
internal LegacyStateBuilder WithPairing(AgentConfiguration pairing)
|
||||
{
|
||||
byte[] plaintext = JsonSerializer.SerializeToUtf8Bytes(pairing, PrintApiClient.JsonOptions);
|
||||
return WithFile("connection.dpapi", ProtectedData.Protect(plaintext, optionalEntropy: null, DataProtectionScope.CurrentUser));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Writes a file into the folder byte for byte: a leftover, or a protected file damaged so that it cannot be read.</summary>
|
||||
internal LegacyStateBuilder WithFile(string name, byte[] content)
|
||||
{
|
||||
File.WriteAllBytes(Path.Combine(DirectoryPath, name), content);
|
||||
return this;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Globalization;
|
||||
using System.Text.Json;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Qqq.PrintAgent.Core.Printing;
|
||||
using Qqq.PrintAgent.Windows;
|
||||
|
||||
namespace Qqq.PrintAgent.Cli.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// The service on state it must not use. An older build can be installed over a newer one between release candidates;
|
||||
/// it then finds a journal at a version it does not know. It must leave that state
|
||||
/// exactly as it is, contact no server, and still answer enough over its pipe for an operator to see why and to install
|
||||
/// the newer build again.
|
||||
/// </summary>
|
||||
public sealed class NewerStateTests
|
||||
{
|
||||
/// <summary>The operations the service refuses while a newer build's data is in place: everything that would read or change that data or reach a server, and a name it does not know.</summary>
|
||||
private static readonly PipeRequest[] Refused =
|
||||
[
|
||||
new("printers"),
|
||||
new("set-exposure", PrinterId: "printer", Exposed: false),
|
||||
new("enroll", ServerUrl: "https://synthetic.invalid", EnrollmentCode: "synthetic-code"),
|
||||
new("sign-in"),
|
||||
new("sign-out"),
|
||||
new("reconnect"),
|
||||
new("test-print", PrinterId: "printer", Format: "PDF"),
|
||||
new("retry-job", JobId: "job"),
|
||||
new("cancel-job", JobId: "job"),
|
||||
new("acknowledge-job", JobId: "job"),
|
||||
new("job-details", JobId: "job"),
|
||||
new("query-jobs", JobQuery: new JobQuery()),
|
||||
new("get-device-profile"),
|
||||
new("set-device-profile", DeviceProfile: new DeviceProfile("Computer")),
|
||||
new("set-printer-profile", PrinterId: "printer", PrinterProfile: new PrinterProfile("Queue")),
|
||||
new("get-server-trust"),
|
||||
new("set-server-trust", ServerTrust: new ServerTrust()),
|
||||
new("no-such-operation"),
|
||||
];
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A paired, signed-in computer whose journal a newer build has since migrated. The service starts, runs no connection
|
||||
/// and serves its pipe. It answers status and the snapshot with the sentence that says a newer build wrote the data and
|
||||
/// must be installed again, the transport read, the three log operations and the two update operations; each of those
|
||||
/// is asked by name below. Every other operation, each also by name, gets that one sentence as an error. No session is
|
||||
/// created, so no server is contacted, and the journal and the protected files have the same bytes at the end.
|
||||
/// The reason is in the service's log once, as the error written at the start. A refusal is not an error there: the
|
||||
/// entries the snapshot shows have none after all eighteen, and each refusal is in the log file as debug detail,
|
||||
/// which is on at that point because the log settings were changed above.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task NewerState_ServiceAnswersStatusLogsUpdates_NothingElse()
|
||||
{
|
||||
using SessionFixture fixture = new();
|
||||
fixture.SaveSession();
|
||||
string directory = fixture.Local.DirectoryPath;
|
||||
JournalFiles.Release(directory);
|
||||
using(SqliteConnection newerBuild = TestState.OpenJournal(directory))
|
||||
{
|
||||
using SqliteCommand command = newerBuild.CreateCommand();
|
||||
command.CommandText = "PRAGMA user_version=" + (TestState.SchemaVersion + 1).ToString(CultureInfo.InvariantCulture);
|
||||
_ = command.ExecuteNonQuery();
|
||||
}
|
||||
IReadOnlyDictionary<string, string> stateBefore = StoredFiles(directory);
|
||||
Assert.Equal(["connection.dpapi", "journal.db", "qqq-session.dpapi"], stateBefore.Keys, StringComparer.Ordinal);
|
||||
using SingleServerHarness runtime = fixture.Runtime();
|
||||
using CancellationTokenSource stop = new(TimeSpan.FromSeconds(60));
|
||||
Task running = runtime.RunAsync(stop.Token);
|
||||
try
|
||||
{
|
||||
await AssertAnsweredAsync(runtime, stop.Token);
|
||||
foreach(PipeRequest request in Refused)
|
||||
{
|
||||
Assert.Equal((request.Operation, "Error: " + StateStatus.NewerBuildText), (request.Operation, await runtime.PipeAsync(request, stop.Token)));
|
||||
}
|
||||
Assert.DoesNotContain(JsonSerializer.Deserialize<AgentSnapshot>(await runtime.PipeAsync(new PipeRequest("snapshot"), stop.Token), PrintApiClient.JsonOptions)!.Logs, entry => string.Equals(entry.Level, "ERROR", StringComparison.Ordinal));
|
||||
string logFiles = string.Concat(Directory.EnumerateFiles(Path.Combine(directory, "logs")).Select(file => System.Text.Encoding.UTF8.GetString(JournalProbe.ReadShared(file))));
|
||||
Assert.All(Refused, request => Assert.Contains("Refused '" + request.Operation + "': the stored state cannot be used.", logFiles, StringComparison.Ordinal));
|
||||
Assert.Equal("Ready for update.", await runtime.PipeAsync(new PipeRequest("prepare-update"), stop.Token));
|
||||
Assert.Equal("Printing resumed.", await runtime.PipeAsync(new PipeRequest("resume-after-update"), stop.Token));
|
||||
// Past the pipe, the runtime itself opens no store on this state: the authorization check, which starts by reading the saved session, is refused before it reads anything.
|
||||
Assert.Equal(StateStatus.NewerBuildText, (await Assert.ThrowsAsync<InvalidOperationException>(() => runtime.EnsureAuthorizationAsync(stop.Token))).Message);
|
||||
Assert.False(running.IsCompleted);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await StopAsync(stop, running);
|
||||
}
|
||||
Assert.Equal(0, fixture.Created);
|
||||
fixture.AssertSentNothing(fixture.Session);
|
||||
Assert.Contains("No server connection is started", Assert.Single(fixture.Logs, line => line.Contains(StateStatus.NewerBuildText, StringComparison.Ordinal)), StringComparison.Ordinal);
|
||||
Assert.Equal(stateBefore, StoredFiles(directory));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The operations that are answered, asked one by one: status, snapshot, transport and the three log operations.</summary>
|
||||
private static async Task AssertAnsweredAsync(SingleServerHarness runtime, CancellationToken token)
|
||||
{
|
||||
Assert.Equal(StateStatus.NewerBuildText, await runtime.PipeAsync(new PipeRequest("status"), token));
|
||||
AgentSnapshot snapshot = JsonSerializer.Deserialize<AgentSnapshot>(await runtime.PipeAsync(new PipeRequest("snapshot"), token), PrintApiClient.JsonOptions)!;
|
||||
Assert.Equal((false, false, "RED", StateStatus.NewerBuildText), (snapshot.Enrolled, snapshot.Connected, snapshot.Health, snapshot.Status));
|
||||
Assert.Null(snapshot.ServerUrl);
|
||||
Assert.Null(snapshot.DeviceId);
|
||||
Assert.Empty(snapshot.Printers);
|
||||
Assert.Empty(snapshot.Jobs);
|
||||
Assert.Equal("DISCONNECTED", snapshot.Transport!.State);
|
||||
AgentTransportSnapshot transport = JsonSerializer.Deserialize<AgentTransportSnapshot>(await runtime.PipeAsync(new PipeRequest("transport"), token), PrintApiClient.JsonOptions)!;
|
||||
Assert.Equal(("NONE", "DISCONNECTED"), (transport.Protocol, transport.State));
|
||||
LogSettings read = JsonSerializer.Deserialize<LogSettings>(await runtime.PipeAsync(new PipeRequest("get-log-settings"), token), PrintApiClient.JsonOptions)!;
|
||||
Assert.False(read.DebugLogging);
|
||||
LogSettings saved = JsonSerializer.Deserialize<LogSettings>(await runtime.PipeAsync(new PipeRequest("set-log-settings", LogSettings: read with { DebugLogging = true }), token), PrintApiClient.JsonOptions)!;
|
||||
Assert.True(saved.DebugLogging);
|
||||
Assert.StartsWith("Cleared ", await runtime.PipeAsync(new PipeRequest("clear-logs"), token), StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Stops the service loop as Windows does and waits for it to end. The loop ends either by returning or with the cancellation, and the service host accepts both.</summary>
|
||||
private static async Task StopAsync(CancellationTokenSource stop, Task running)
|
||||
{
|
||||
await stop.CancelAsync();
|
||||
try
|
||||
{
|
||||
await running;
|
||||
}
|
||||
catch(OperationCanceledException)
|
||||
{
|
||||
// The loop was waiting for a pipe client when it was stopped.
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The hashes of the files that are the stored state: the journal with its log and index, and the protected files. The activity log and the service's own lock file are not state and are written while the service runs.</summary>
|
||||
private static SortedDictionary<string, string> StoredFiles(string directory)
|
||||
{
|
||||
SortedDictionary<string, string> state = new(StringComparer.OrdinalIgnoreCase);
|
||||
foreach(string file in Directory.EnumerateFiles(directory, "journal.db*").Concat(Directory.EnumerateFiles(directory, "*.dpapi")))
|
||||
{
|
||||
state[Path.GetFileName(file)] = Convert.ToHexString(System.Security.Cryptography.SHA256.HashData(JournalProbe.ReadShared(file)));
|
||||
}
|
||||
return state;
|
||||
}
|
||||
}
|
||||
@@ -24,6 +24,9 @@
|
||||
failure reply per error code that carries details. -->
|
||||
<None Remove="Fixtures\management\*.json" />
|
||||
<Content Include="Fixtures\management\*.json" CopyToOutputDirectory="PreserveNewest" />
|
||||
<!-- Journals in the shapes earlier releases left, as SQL scripts LegacyStateBuilder runs to build a state folder. -->
|
||||
<None Remove="Fixtures\*.sql" />
|
||||
<Content Include="Fixtures\*.sql" CopyToOutputDirectory="PreserveNewest" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
@@ -42,4 +45,3 @@
|
||||
<Copy SourceFiles="@(WorkerFixtureRuntime)" DestinationFolder="$(OutDir)" />
|
||||
</Target>
|
||||
</Project>
|
||||
|
||||
@@ -68,7 +68,9 @@ internal sealed class SingleServerHarness : IDisposable
|
||||
/// management pipe itself, one instance for the current user on a private name, so no test decides how the service listens;
|
||||
/// a buffer size is given only by the test that needs a reply to outgrow the pipe's buffer. The last argument runs on the
|
||||
/// runtime's command thread just before it saves a command's result, while that thread holds the control state's lock.
|
||||
/// The installed service's pipe name is refused, in any letter case, before anything is built.
|
||||
/// The installed service's pipe name is refused, in any letter case, before anything is built. The journal in the folder
|
||||
/// is migrated here, as the service does at the start of its loop, because many tests call the runtime without ever
|
||||
/// running that loop and a store opens only on a migrated journal.
|
||||
/// </summary>
|
||||
internal SingleServerHarness(string directory, Action<string> log, IPrinterBackend printerBackend, Func<Uri, IQqqSessionClient>? sessionFactory = null,
|
||||
TimeProvider? timeProvider = null, string? pipeName = null, int pipeBufferSize = 0, Action? controlResultSaving = null)
|
||||
@@ -76,6 +78,7 @@ internal sealed class SingleServerHarness : IDisposable
|
||||
if(string.Equals(pipeName, InstalledPipeName, StringComparison.OrdinalIgnoreCase))
|
||||
{ throw new ArgumentException("A test must not listen on the installed service's management pipe.", nameof(pipeName)); }
|
||||
string pipe = PipeName = pipeName ?? "qqq-single-server-" + Guid.NewGuid().ToString("N");
|
||||
_ = TestState.Migrate(directory);
|
||||
_runtime = new AgentRuntime(directory, log, printerBackend, pipeName: pipe,
|
||||
pipeFactory: () => new NamedPipeServerStream(pipe, PipeDirection.InOut, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous | PipeOptions.CurrentUserOnly, pipeBufferSize, pipeBufferSize),
|
||||
sessionFactory: sessionFactory, timeProvider: timeProvider)
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
// The store's type under the name tests use for it. Every test declares a store as JournalStore, so when the product type
|
||||
// is renamed, this line and OpenStore below change and no test does.
|
||||
global using JournalStore = Qqq.PrintAgent.Windows.AgentStore;
|
||||
using System.Runtime.CompilerServices;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Qqq.PrintAgent.Windows;
|
||||
|
||||
@@ -17,15 +16,45 @@ namespace Qqq.PrintAgent.Cli.Tests;
|
||||
/// the store is opened for one server out of a registry, the way a test names "the store in this folder" changes here and
|
||||
/// nowhere else. Tests name the store's type by the alias declared above, never by the product's name. This is also the
|
||||
/// only place a test gets a connection of its own to a journal, because such a connection has to stay out of the pool the
|
||||
/// stores of that journal share.
|
||||
/// stores of that journal share, and the only place a test migrates a journal, which the service does when it starts.
|
||||
/// </summary>
|
||||
internal static class TestState
|
||||
{
|
||||
/// <summary>
|
||||
/// Opens the store on a state folder, creating the folder and the journal when they are missing, exactly as the service
|
||||
/// does. With recovery asked for, a pairing file that cannot be read does not stop the open. The caller disposes the store.
|
||||
/// Opens the store on a state folder. The journal is migrated first, as the service does when it starts and before it
|
||||
/// opens any store, so a test may name a folder that does not exist yet; a store by itself creates nothing. With
|
||||
/// recovery asked for, a pairing file that cannot be read does not stop the open. The caller disposes the store.
|
||||
/// </summary>
|
||||
internal static AgentStore OpenStore(string directory, bool recoverUnreadableConfiguration = false) => new(directory, recoverUnreadableConfiguration);
|
||||
internal static AgentStore OpenStore(string directory, bool recoverUnreadableConfiguration = false)
|
||||
{
|
||||
_ = JournalSchema.Migrate(directory);
|
||||
return new(directory, recoverUnreadableConfiguration);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Opens the store on a state folder without migrating its journal first, for the tests whose subject is what a store does with a journal as it finds it.</summary>
|
||||
internal static AgentStore OpenStoreAsFound(string directory) => new(directory);
|
||||
|
||||
|
||||
|
||||
/// <summary>Brings the journal in a state folder to the schema version this build works with, as the service does when it starts, and returns what the service would then know about its stored state.</summary>
|
||||
internal static StateStatus Migrate(string directory) => JournalSchema.Migrate(directory);
|
||||
|
||||
|
||||
|
||||
/// <summary>Migrates with a step list of the test's own: the product's steps with one added, or one wrapped, to show what a failing or a slow step does.</summary>
|
||||
internal static StateStatus Migrate(string directory, IReadOnlyList<Action<JournalMigrationContext>> steps) => JournalSchema.Migrate(directory, steps);
|
||||
|
||||
|
||||
|
||||
/// <summary>The product's migration steps, oldest first, for a test to build its own list from.</summary>
|
||||
internal static IReadOnlyList<Action<JournalMigrationContext>> MigrationSteps => JournalMigration.Steps;
|
||||
|
||||
|
||||
|
||||
/// <summary>The schema version this build works with.</summary>
|
||||
internal static int SchemaVersion => JournalSchema.Version;
|
||||
|
||||
|
||||
|
||||
@@ -34,16 +63,10 @@ internal static class TestState
|
||||
/// a question the store has no call for. The connection is not pooled. A pooled one would share the pool of the stores on
|
||||
/// that journal without taking the journal's lock as they do, and the SQLite library can then hand one connection to two
|
||||
/// openers (see "Testing the service runtime" in docs/DEVELOPMENT.md). Unpooled, it has a native handle no store can be
|
||||
/// given, and closing it releases the file at once. The store's type is initialized first because that is what selects
|
||||
/// the SQLite library, which a test that opens a journal before any store would otherwise find missing.
|
||||
/// given, and closing it releases the file at once. It creates the journal file when there is none, which is how a test
|
||||
/// builds a journal in the shape an earlier release left.
|
||||
/// </summary>
|
||||
internal static SqliteConnection OpenJournal(string directory)
|
||||
{
|
||||
RuntimeHelpers.RunClassConstructor(typeof(AgentStore).TypeHandle);
|
||||
SqliteConnection journal = new(new SqliteConnectionStringBuilder { DataSource = Path.Combine(directory, "journal.db"), Pooling = false }.ToString());
|
||||
journal.Open();
|
||||
return journal;
|
||||
}
|
||||
internal static SqliteConnection OpenJournal(string directory) => JournalDatabase.OpenUnpooled(directory);
|
||||
|
||||
|
||||
|
||||
@@ -51,10 +74,10 @@ internal static class TestState
|
||||
/// The connection string the stores use for the journal in a folder. It names the journal's pool of idle connections, which
|
||||
/// is how <see cref="JournalFiles"/> empties that pool and no other before a folder is deleted.
|
||||
/// </summary>
|
||||
internal static string JournalConnectionString(string directory) => AgentStore.JournalConnectionString(directory);
|
||||
internal static string JournalConnectionString(string directory) => JournalDatabase.ConnectionString(directory);
|
||||
|
||||
|
||||
|
||||
/// <summary>The lock every store takes while it opens the journal in a folder, for the tests that hold it to show what an open waits for.</summary>
|
||||
internal static Lock SchemaLockFor(string directory) => AgentStore.SchemaLockFor(directory);
|
||||
/// <summary>The lock every store takes while it opens the journal in a folder, and the migration holds while it runs, for the tests that hold it to show what waits for it.</summary>
|
||||
internal static Lock SchemaLockFor(string directory) => JournalDatabase.LockFor(directory);
|
||||
}
|
||||
Reference in new issue
Block a user