README.md

@@ -100,7 +100,7 @@ config.withSearchableTable(new SearchableTableConfig("crmContact", List.of(
    .withRecordLabelFormat("%s %s", "firstName", "lastName"));
 ```
 
-Annotation and config-driven tables can be mixed; a table may appear in only one of them. Hidden and password fields are never indexed. A table without the change-detection field (`modifyDate` by default) is covered by reconcile instead of incremental basepull.
+Annotation and config-driven tables can be mixed; a table may appear in only one of them. Hidden and password fields are never indexed. A table without the change-detection field (`modifyDate` by default) is covered by reconcile instead of incremental basepull, and the QBit logs a warning for it at startup. Set `basepullTimestampField` to null on a config-driven table to opt out without the warning.
 
 ### Searching
 
@@ -146,6 +146,20 @@ QQQ core has its own record search (`RecordSearchAction`, served at `POST /qqq/v
 
 Without `schedulerName` the processes exist but nothing runs them; the QBit logs a warning at startup.
 
+### Scheduling
+
+The schedules run only when the host starts through QQQ's `QApplicationLauncher` (`com.kingsrook.qqq.middleware.javalin.QApplicationLauncher`, in `qqq-middleware-javalin`). `QApplicationLauncher.run(new MyApplication(), new QApplicationLauncherConfig())` starts the Javalin server, then the `QScheduleManager` (because the QBit's processes are scheduled), then the instance's runtime services. A host that starts only a server, or builds the `QInstance` itself, gets the processes but nothing runs them on a schedule, unless it calls `QScheduleManager.initInstance(qInstance, systemSessionSupplier).start()` itself. `qqq.scheduleManager.enabled=false` (or `QQQ_SCHEDULE_MANAGER_ENABLED=false`) turns schedules off on a node.
+
+Basepull's fixed interval runs on any QQQ scheduler. `reconcileCronExpression` needs a scheduler that supports cron, which in QQQ 4.1 means a `QuartzSchedulerMetaData`; a `SimpleSchedulerMetaData` runs fixed intervals only. The expression uses Quartz syntax, seconds first (`0 0 3 * * ?` is 03:00 every day), and needs `reconcileCronTimeZoneId`, a Java time zone ID such as `UTC` or `America/Chicago`; QQQ has no default zone. `produce()` rejects a cron without a zone, an invalid expression or zone, and a cron on a scheduler in the instance that does not support cron.
+
+```java
+qInstance.addScheduler(new QuartzSchedulerMetaData().withName("quartz").withProperties(quartzProperties));
+
+config.withSchedulerName("quartz")
+   .withReconcileCronExpression("0 0 3 * * ?")
+   .withReconcileCronTimeZoneId("America/Chicago");
+```
+
 ## Configuration reference
 
 | Field | Default | Description |
@@ -163,7 +177,7 @@ Without `schedulerName` the processes exist but nothing runs them; the QBit logs
 | `startupMode` | `FAIL_FAST` | `FAIL_FAST` fails `produce()` when the cluster is unreachable; `DEGRADED` boots and retries on first use |
 | `enableRealTimeIndexing` | `true` | Register the record-change listener |
 | `enableBasepullProcess`, `enableMaintenanceProcesses` | `true` | Register basepull; register full reindex and reconcile. `enableScheduledProcesses` is a deprecated alias for both |
-| `schedulerName`, `basepullRepeatSeconds`, `reconcileCronExpression`, `reconcileCronTimeZoneId` | none, 300, none | Scheduling |
+| `schedulerName`, `basepullRepeatSeconds`, `reconcileCronExpression`, `reconcileCronTimeZoneId` | none, 300, none, none | Scheduling; a cron needs the time zone ID and a Quartz scheduler, and nothing runs without `QApplicationLauncher` (see [Scheduling](#scheduling)) |
 | `basepullOverlapSeconds` | 300 | Re-read window before the previous run start |
 | `defaultBasepullIntervalMinutes` | 5 | Per-table due interval for config-driven tables |
 | `runHistoryRetentionDays` | 30 | Purge run records older than this; null keeps them |

docs/MIGRATION-1.0.md

@@ -12,7 +12,7 @@
 1. **Remove any manual `QBitMetaData` registration.** The producer now registers `com.kingsrook.qbits:quick-search` itself; a second `addQBit` with that name throws at startup. (qqq-all's `DemoQBits` did this in 0.x.)
 2. **Decide the startup mode.** `startupMode` defaults to `FAIL_FAST`: `produce()` fails when OpenSearch is unreachable. Demo profiles that may run without a cluster should set `withStartupMode(QuickSearchStartupMode.DEGRADED)`.
 3. **Run a full reindex once after upgrading.** The index mapping changed (version 2: search analyzer, text-typed `fieldValues`). The QBit logs a warning when it finds an index with the old mapping; "Quick Search Full Reindex" with no table name builds a new physical index and swaps the configured name to an alias. Search keeps working during the rebuild. The identity the QBit uses needs index create, delete and alias privileges for this (see the README role).
-4. **Set `schedulerName`** (and optionally `basepullRepeatSeconds`, `reconcileCronExpression`) so basepull and reconcile actually run. In 0.x they were never scheduled.
+4. **Set `schedulerName` and start the host through `QApplicationLauncher`** (and optionally `basepullRepeatSeconds`, or `reconcileCronExpression` with `reconcileCronTimeZoneId` on a Quartz scheduler) so basepull and reconcile actually run. In 0.x they were never scheduled. See [Scheduling](#scheduling).
 5. **Credentials over HTTP are refused.** Use `opensearchUrl` with `https://`, or set `withAllowPlaintextCredentials(true)` for local development. Prefer `${env.OPENSEARCH_PASSWORD}` references over literal secrets; never name an OpenSearch secret `QQQ_ENV_*` (those are sent to the frontend).
 6. **Grant the permissions.** Processes and the admin app now require `quickSearchAdmin.hasAccess`; the operational tables require read and write permissions. Grant the names listed under [Permissions](#permissions) to operators, or override `adminPermissionRules` and `tablePermissionRules`. Search now honours table read permissions and record security locks, so users see fewer results than in 0.x if they lacked access.
 7. **Update the operational tables before starting 1.0.** `quickSearchIndex` gains five columns, `searchableFieldsJson` changes from `STRING` to `TEXT`, and `tableName` becomes required with a unique key. `quickSearchIndexRun.errorMessage` changes from `STRING` to `TEXT`. There is a new `quickSearchFailedEvent` table. A host that checks its columns at startup rejects the 0.x tables. See [Schema changes](#schema-changes).
@@ -76,12 +76,20 @@ If your backend creates tables from metadata and you do not need the 0.x run his
 - Real-time indexing happens after commit. Code that searched for a record inside the same transaction that inserted it will not find it until commit.
 - Real-time failures no longer vanish: they land in `quickSearchFailedEvent` and are replayed by the next basepull. Review that table after an OpenSearch outage.
 - Inserted and updated records are re-read inside the write's transaction, with display values and possible-value labels, before they are indexed. That is one extra query per write batch on an indexed table, and it runs the host's possible-value providers. If the re-read fails, the write still succeeds and the records are replayed by basepull.
-- A table without the change-detection field (`modifyDate` by default) is indexed on the first basepull and thereafter only by reconcile.
+- A table without the change-detection field (`modifyDate` by default) is indexed on the first basepull and thereafter only by reconcile. The QBit logs a warning for each such table at startup, for annotated and config-driven tables alike; a config-driven table with `basepullTimestampField` set to null has opted out and gets no warning.
 - `limit` is capped (default 100), `offset + limit` at 10,000, and search terms must be 2 to 100 characters.
 - Query semantics changed from OR over n-grams to AND over whole terms with prefix matching: "blue widget" now requires both words.
 - `enableScheduledProcesses` still works and sets both `enableBasepullProcess` and `enableMaintenanceProcesses`.
 - `QuickSearchQBitContext` is deprecated and removed in 1.1; use `QuickSearchRuntime.get()`.
 
+## Scheduling
+
+Setting `schedulerName` puts schedules on the basepull and reconcile processes; QQQ runs them only once something starts its `QScheduleManager`.
+
+- **Start through `QApplicationLauncher`.** In QQQ 4.1 that is `com.kingsrook.qqq.middleware.javalin.QApplicationLauncher` (`qqq-middleware-javalin`). `QApplicationLauncher.run(new MyApplication(), new QApplicationLauncherConfig())` starts the Javalin server, then the `QScheduleManager` when any process is scheduled, then the instance's runtime services, including this QBit's. A host that starts only a `QApplicationJavalinServer`, or builds its `QInstance` without the launcher, gets the processes but no scheduled runs, unless it calls `QScheduleManager.initInstance(qInstance, systemSessionSupplier).start()` itself. `qqq.scheduleManager.enabled=false` (or `QQQ_SCHEDULE_MANAGER_ENABLED=false`) turns schedules off on a node.
+- **A reconcile cron needs a Quartz scheduler.** Basepull's fixed interval runs on any QQQ scheduler, but only `QuartzSchedulerMetaData` supports cron; QQQ's `SimpleSchedulerMetaData` runs fixed intervals only, and a cron schedule on it fails when the schedule manager starts.
+- **A reconcile cron needs a time zone.** Set `reconcileCronTimeZoneId` to a Java time zone ID (`UTC`, `America/Chicago`). QQQ has no default zone: its instance validation rejects a cron schedule without one at startup. `produce()` now catches this first, along with an invalid Quartz expression (seconds first, for example `0 0 3 * * ?`), an unknown zone ID, and a cron on a scheduler in the instance that does not support cron.
+
 ## Removed
 
 - `QuickSearchPostInsertCustomizer`, `QuickSearchPostUpdateCustomizer`, `QuickSearchPostDeleteCustomizer`. Hosts that referenced them (for example to compose with their own customizers) should remove those references; the record-change listener coexists with host customizers.

src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitConfig.java

@@ -18,10 +18,13 @@ package com.kingsrook.qbits.quicksearch;
 
 
 import java.net.URI;
+import java.text.ParseException;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
+import java.util.TimeZone;
 import java.util.regex.Pattern;
 import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
@@ -30,12 +33,14 @@ import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
+import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QSchedulerMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qbits.quicksearch.model.QuickSearchIndex;
 import com.kingsrook.qbits.quicksearch.model.QuickSearchIndexRun;
 import com.kingsrook.qbits.quicksearch.publisher.IndexEventPublisher;
+import org.quartz.CronExpression;
 
 
 /*******************************************************************************
@@ -421,18 +426,53 @@ public class QuickSearchQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    **
+    ** The scheduler must exist when the instance already has schedulers. A
+    ** reconcile cron gets the checks QQQ's instance validator applies to every
+    ** cron schedule (Quartz syntax, a required and known time zone, which QQQ
+    ** never defaults), plus a scheduler that supports cron, so a bad schedule
+    ** fails produce() naming these fields instead of failing at startup.
     *******************************************************************************/
    private void validateScheduling(QInstance qInstance, List<String> errors)
    {
-      if(StringUtils.hasContent(schedulerName) && qInstance != null && !CollectionUtils.nullSafeIsEmpty(qInstance.getSchedulers()) && qInstance.getScheduler(schedulerName) == null)
+      QSchedulerMetaData scheduler = null;
+      if(StringUtils.hasContent(schedulerName) && qInstance != null && !CollectionUtils.nullSafeIsEmpty(qInstance.getSchedulers()))
       {
-         errors.add("Scheduler not found: " + schedulerName);
+         scheduler = qInstance.getScheduler(schedulerName);
+         if(scheduler == null)
+         {
+            errors.add("Scheduler not found: " + schedulerName);
+         }
       }
 
-      if(StringUtils.hasContent(reconcileCronExpression) && !StringUtils.hasContent(schedulerName))
+      if(StringUtils.hasContent(reconcileCronExpression))
       {
-         errors.add("reconcileCronExpression requires schedulerName");
+         if(!StringUtils.hasContent(schedulerName))
+         {
+            errors.add("reconcileCronExpression requires schedulerName");
+         }
+         else if(scheduler != null && !scheduler.supportsCronSchedules())
+         {
+            errors.add("reconcileCronExpression requires a scheduler that supports cron schedules, such as a QuartzSchedulerMetaData; scheduler [" + schedulerName + "] is of type " + scheduler.getType());
+         }
+
+         try
+         {
+            CronExpression.validateExpression(reconcileCronExpression);
+         }
+         catch(ParseException e)
+         {
+            errors.add("reconcileCronExpression [" + reconcileCronExpression + "] is not a valid Quartz cron expression: " + e.getMessage());
+         }
+
+         if(!StringUtils.hasContent(reconcileCronTimeZoneId))
+         {
+            errors.add("reconcileCronExpression requires reconcileCronTimeZoneId (a time zone ID such as UTC or America/Chicago)");
+         }
+      }
+
+      if(StringUtils.hasContent(reconcileCronTimeZoneId) && !Arrays.asList(TimeZone.getAvailableIDs()).contains(reconcileCronTimeZoneId))
+      {
+         errors.add("reconcileCronTimeZoneId [" + reconcileCronTimeZoneId + "] is not a recognized time zone ID");
       }
    }
 
@@ -1588,7 +1628,9 @@ public class QuickSearchQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    ** Getter for reconcileCronExpression: optional cron schedule for reconcile (needs a scheduler that supports cron)
+    ** Getter for reconcileCronExpression: optional Quartz cron schedule for
+    ** reconcile; needs reconcileCronTimeZoneId and a scheduler that supports
+    ** cron (Quartz)
     *******************************************************************************/
    public String getReconcileCronExpression()
    {
@@ -1620,7 +1662,9 @@ public class QuickSearchQBitConfig implements QBitConfig
 
 
    /*******************************************************************************
-    ** Getter for reconcileCronTimeZoneId: time zone for reconcileCronExpression
+    ** Getter for reconcileCronTimeZoneId: time zone ID for
+    ** reconcileCronExpression (UTC, America/Chicago); required with it, since
+    ** QQQ has no default zone for cron schedules
     *******************************************************************************/
    public String getReconcileCronTimeZoneId()
    {

src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducer.java

@@ -474,8 +474,12 @@ public class QuickSearchQBitProducer implements QBitMetaDataProducer<QuickSearch
          QTableMetaData table                  = qInstance.getTable(stc.getTableName());
          if(table != null)
          {
-            primaryKeyField        = table.getPrimaryKeyField();
-            basepullTimestampField = (basepullTimestampField != null && !table.getFields().containsKey(basepullTimestampField)) ? null : basepullTimestampField;
+            primaryKeyField = table.getPrimaryKeyField();
+            if(basepullTimestampField != null && !table.getFields().containsKey(basepullTimestampField))
+            {
+               warnIncrementalBasepullDisabled(table, basepullTimestampField);
+               basepullTimestampField = null;
+            }
             removeHiddenFields(table, searchableFields, fieldWeights, fieldIncludeLabels);
          }
 
@@ -532,7 +536,7 @@ public class QuickSearchQBitProducer implements QBitMetaDataProducer<QuickSearch
       {
          if(QuickSearchQBitConfig.DEFAULT_BASEPULL_TIMESTAMP_FIELD.equals(basepullTimestampField))
          {
-            LOG.warn("Table has no modifyDate field; incremental basepull is disabled for it (reconcile still covers it)", logPair("tableName", table.getName()));
+            warnIncrementalBasepullDisabled(table, basepullTimestampField);
             return (null);
          }
          throw (new QException("basepullTimestampField [" + basepullTimestampField + "] does not exist on table [" + table.getName() + "]"));
@@ -542,6 +546,19 @@ public class QuickSearchQBitProducer implements QBitMetaDataProducer<QuickSearch
 
 
 
+   /*******************************************************************************
+    ** The startup warning, once per table, for annotated and config-driven
+    ** tables alike, when the basepull timestamp field is missing. A field
+    ** explicitly set to null is an opt-out and is not warned about.
+    *******************************************************************************/
+   private static void warnIncrementalBasepullDisabled(QTableMetaData table, String basepullTimestampField)
+   {
+      LOG.warn("Table has no basepull timestamp field; incremental basepull is disabled for it (reconcile still covers it)",
+         logPair("tableName", table.getName()), logPair("timestampField", basepullTimestampField));
+   }
+
+
+
    /*******************************************************************************
     ** Hidden and password fields are never indexed: their values would leak
     ** through searchable text and highlights.

src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitConfigValidationTest.java

@@ -23,6 +23,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.quartz.QuartzSchedulerMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
@@ -246,18 +247,18 @@ class QuickSearchQBitConfigValidationTest
    void testScheduling_schedulerMustExist_andCronNeedsScheduler()
    {
       QInstance qInstance = instanceWithBackendAndCustomer();
-      qInstance.addScheduler(new SimpleSchedulerMetaData().withName("sched"));
+      qInstance.addScheduler(new QuartzSchedulerMetaData().withName("sched"));
 
       List<String> errors = new ArrayList<>();
       baseConfig().withSchedulerName("missing").validate(qInstance, errors);
       assertThat(errors).anyMatch(e -> e.contains("Scheduler not found: missing"));
 
       errors = new ArrayList<>();
-      baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").validate(qInstance, errors);
+      baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("America/Chicago").validate(qInstance, errors);
       assertThat(errors).isEmpty();
 
       errors = new ArrayList<>();
-      baseConfig().withReconcileCronExpression("0 0 3 * * ?").validate(qInstance, errors);
+      baseConfig().withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("UTC").validate(qInstance, errors);
       assertThat(errors).anyMatch(e -> e.contains("requires schedulerName"));
 
       ///////////////////////////////////////////////////////////////////////
@@ -268,6 +269,72 @@ class QuickSearchQBitConfigValidationTest
 
 
 
+   /*******************************************************************************
+    ** QQQ's instance validator rejects a cron schedule without a time zone and
+    ** has no default, so the QBit rejects it at produce time, naming its fields.
+    *******************************************************************************/
+   @Test
+   void testScheduling_cronWithoutTimeZone_addsError()
+   {
+      QInstance qInstance = instanceWithBackendAndCustomer();
+      qInstance.addScheduler(new QuartzSchedulerMetaData().withName("sched"));
+
+      List<String> errors = new ArrayList<>();
+      baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").validate(qInstance, errors);
+      assertThat(errors).containsExactly("reconcileCronExpression requires reconcileCronTimeZoneId (a time zone ID such as UTC or America/Chicago)");
+
+      ///////////////////////////////////////////////////////////////////
+      // without a scheduler in the instance the rule still applies    //
+      ///////////////////////////////////////////////////////////////////
+      assertThat(validate(baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId(" ")))
+         .anyMatch(e -> e.startsWith("reconcileCronExpression requires reconcileCronTimeZoneId"));
+   }
+
+
+
+   /*******************************************************************************
+    ** The cron expression and the zone id are checked the way QQQ's instance
+    ** validator checks them: Quartz cron syntax and a known TimeZone id.
+    *******************************************************************************/
+   @Test
+   void testScheduling_invalidCronOrTimeZone_addsErrors()
+   {
+      List<String> errors = validate(baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * *").withReconcileCronTimeZoneId("UTC"));
+      assertThat(errors).singleElement().satisfies(e -> assertThat(e).startsWith("reconcileCronExpression [0 0 3 * *] is not a valid Quartz cron expression"));
+
+      errors = validate(baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("Mars/Olympus_Mons"));
+      assertThat(errors).containsExactly("reconcileCronTimeZoneId [Mars/Olympus_Mons] is not a recognized time zone ID");
+
+      errors = validate(baseConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("UTC"));
+      assertThat(errors).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** QQQ's simple scheduler only runs fixed intervals; a cron schedule on it
+    ** fails when the schedule manager starts.
+    *******************************************************************************/
+   @Test
+   void testScheduling_cronOnSchedulerWithoutCronSupport_addsError()
+   {
+      QInstance qInstance = instanceWithBackendAndCustomer();
+      qInstance.addScheduler(new SimpleSchedulerMetaData().withName("simple"));
+
+      List<String> errors = new ArrayList<>();
+      baseConfig().withSchedulerName("simple").withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("UTC").validate(qInstance, errors);
+      assertThat(errors).containsExactly("reconcileCronExpression requires a scheduler that supports cron schedules, such as a QuartzSchedulerMetaData; scheduler [simple] is of type simple");
+
+      ////////////////////////////////////////////////
+      // basepull's fixed interval runs on either   //
+      ////////////////////////////////////////////////
+      errors = new ArrayList<>();
+      baseConfig().withSchedulerName("simple").validate(qInstance, errors);
+      assertThat(errors).isEmpty();
+   }
+
+
+
    @Test
    void testNumbers_mustBePositive()
    {

src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducerTest.java

@@ -16,12 +16,16 @@
 package com.kingsrook.qbits.quicksearch;
 
 
+import java.util.ArrayList;
 import java.time.Instant;
 import java.util.List;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.actions.tables.RecordSearchProviderInterface;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.CollectedLogMessage;
+import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
@@ -38,9 +42,11 @@ import com.kingsrook.qbits.quicksearch.annotations.QuickSearchField;
 import com.kingsrook.qbits.quicksearch.annotations.QuickSearchable;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
-import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.quartz.QuartzSchedulerMetaData;
 import com.kingsrook.qbits.quicksearch.actions.QuickSearchRecordSearchProvider;
 import com.kingsrook.qbits.quicksearch.listeners.QuickSearchRecordChangeListener;
+import org.apache.logging.log4j.Level;
+import org.json.JSONObject;
 import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
@@ -150,6 +156,18 @@ class QuickSearchQBitProducerTest
 
 
 
+   /***************************************************************************
+    ** Annotated entity on a table without the default modifyDate field.
+    ***************************************************************************/
+   @QuickSearchable(tableName = "tag")
+   static class TagEntity
+   {
+      @QuickSearchField
+      private String body;
+   }
+
+
+
    @BeforeEach
    void setUp()
    {
@@ -706,18 +724,37 @@ class QuickSearchQBitProducerTest
    void testProduce_schedulesWhenSchedulerConfigured() throws QException
    {
       QInstance qInstance = buildQInstance();
-      qInstance.addScheduler(new SimpleSchedulerMetaData().withName("sched"));
+      qInstance.addScheduler(new QuartzSchedulerMetaData().withName("sched"));
       QContext.init(qInstance, new QSession());
-      new QuickSearchQBitProducer().withConfig(buildValidConfig().withSchedulerName("sched").withBasepullRepeatSeconds(120).withReconcileCronExpression("0 0 3 * * ?")).produce(qInstance);
+      new QuickSearchQBitProducer().withConfig(buildValidConfig().withSchedulerName("sched").withBasepullRepeatSeconds(120).withReconcileCronExpression("0 0 3 * * ?").withReconcileCronTimeZoneId("America/Chicago")).produce(qInstance);
 
       assertThat(qInstance.getProcess(QuickSearchQBitProducer.BASEPULL_PROCESS_NAME).getSchedule().getSchedulerName()).isEqualTo("sched");
       assertThat(qInstance.getProcess(QuickSearchQBitProducer.BASEPULL_PROCESS_NAME).getSchedule().getRepeatSeconds()).isEqualTo(120);
       assertThat(qInstance.getProcess(QuickSearchQBitProducer.RECONCILE_PROCESS_NAME).getSchedule().getCronExpression()).isEqualTo("0 0 3 * * ?");
+      assertThat(qInstance.getProcess(QuickSearchQBitProducer.RECONCILE_PROCESS_NAME).getSchedule().getCronTimeZoneId()).isEqualTo("America/Chicago");
       assertThat(qInstance.getProcess(QuickSearchQBitProducer.FULL_REINDEX_PROCESS_NAME).getSchedule()).isNull();
    }
 
 
 
+   /*******************************************************************************
+    ** A reconcile cron without a time zone used to pass produce() and then fail
+    ** QQQ's instance validation at startup; now produce() rejects it.
+    *******************************************************************************/
+   @Test
+   void testProduce_cronWithoutTimeZone_failsProduce()
+   {
+      QInstance qInstance = buildQInstance();
+      qInstance.addScheduler(new QuartzSchedulerMetaData().withName("sched"));
+      QContext.init(qInstance, new QSession());
+
+      assertThatThrownBy(() -> new QuickSearchQBitProducer().withConfig(buildValidConfig().withSchedulerName("sched").withReconcileCronExpression("0 0 3 * * ?")).produce(qInstance))
+         .isInstanceOf(QException.class)
+         .hasMessageContaining("reconcileCronExpression requires reconcileCronTimeZoneId");
+   }
+
+
+
    @Test
    void testProduce_noScheduler_processesUnscheduled() throws QException
    {
@@ -1030,6 +1067,67 @@ class QuickSearchQBitProducerTest
 
 
 
+   /***************************************************************************
+    ** A table without its basepull timestamp field gets the same startup
+    ** warning, once, whether it is annotated or config-driven; a config-driven
+    ** table that sets the field to null has opted out and is not warned about.
+    ***************************************************************************/
+   @Test
+   void testProduce_missingTimestampField_warnsOncePerTableForBothSources() throws QException
+   {
+      QInstance qInstance = buildQInstance();
+      addTableWithoutTimestamp(qInstance, "tag");
+      addTableWithoutTimestamp(qInstance, "note");
+      addTableWithoutTimestamp(qInstance, "memo");
+      QContext.init(qInstance, new QSession());
+
+      QuickSearchQBitConfig config = buildValidConfig()
+         .withSearchableEntityClasses(List.of(TagEntity.class))
+         .withSearchableTable("note", List.of(new SearchableFieldConfig("body")))
+         .withSearchableTable(new SearchableTableConfig("memo", List.of(new SearchableFieldConfig("body"))).withBasepullTimestampField(null));
+
+      QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(QuickSearchQBitProducer.class);
+      try
+      {
+         new QuickSearchQBitProducer().withConfig(config).produce(qInstance);
+      }
+      finally
+      {
+         QLogger.deactivateCollectingLoggerForClass(QuickSearchQBitProducer.class);
+      }
+
+      List<JSONObject> warnings = new ArrayList<>();
+      for(CollectedLogMessage message : collectingLogger.getCollectedMessages())
+      {
+         if(Level.WARN.equals(message.getLevel()) && message.getMessage().contains("incremental basepull is disabled"))
+         {
+            warnings.add(message.getMessageAsJSONObject());
+         }
+      }
+
+      assertThat(warnings).extracting(w -> w.getString("tableName")).containsExactly("tag", "note");
+      assertThat(warnings).extracting(w -> w.getString("timestampField")).containsOnly("modifyDate");
+      assertThat(warnings).extracting(w -> w.getString("message")).containsOnly(warnings.get(0).getString("message"));
+      assertThat(QuickSearchRuntime.get().getDiscoveredTables()).allSatisfy(t -> assertThat(t.getBasepullTimestampField()).isNull());
+   }
+
+
+
+   /***************************************************************************
+    ** Add a table with an id and a body field, but no modifyDate.
+    ***************************************************************************/
+   private void addTableWithoutTimestamp(QInstance qInstance, String tableName)
+   {
+      qInstance.addTable(new QTableMetaData()
+         .withName(tableName)
+         .withBackendName(BACKEND_NAME)
+         .withPrimaryKeyField("id")
+         .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
+         .withField(new QFieldMetaData("body", QFieldType.STRING)));
+   }
+
+
+
    /***************************************************************************
     ** Build a base QInstance with a memory backend and source tables.
     ***************************************************************************/