replace all relative program paths (e.g., cp) with absolute ones (e.g., /bin/cp), in constants (e.g., CP);

update some methods in here to take session as parameter;

docs/Introduction.adoc

@@ -91,7 +91,7 @@ And then having a bug in the check permission logic on the _Light Bulb Inventory
 No!
 
 All of the (really important, even though application developers hate doing it) aspects of security - you don't need to write ANY code for dealing with that.
-Just tell QQQ what Authentication provider you want to use (e.g., https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
+Just tell QQQ what Authentication provider you want to use (e.g., OAuth2 or https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
 QQQ just does it.
 
 ''''

docs/index.adoc

@@ -31,11 +31,9 @@ include::metaData/PermissionRules.adoc[leveloffset=+1]
 
 == Services
 
+include::misc/Javalin.adoc[leveloffset=+1]
 include::misc/ScheduledJobs.adoc[leveloffset=+1]
 
-=== Web server (Javalin)
-#todo#
-
 === API server (OpenAPI)
 #todo#
 

docs/misc/Javalin.adoc

@@ -0,0 +1,109 @@
+== QQQ Middleware:  Javalin web server
+include::../variables.adoc[]
+
+QQQ provides a standard implementation of a middleware layer - that is - code that exists between the
+QQQ backend and user interface.  This implementation is a web server built using the https://javalin.io/[Javalin framework],
+packaged and deployed in the `qqq-middleware-javalin` maven module
+
+The de facto way to create a QQQ application server is to write a class which uses an instance of one of the
+subclasses of `QApplicationJavalinServer`.
+
+For example, if your application metadata is defined in a directory of yaml files, your server class could be implemented as:
+
+[source,java]
+.ConfigFileBasedQQQApplication usage example
+----
+public static void main(String[] args)
+{
+   try
+   {
+      String path = "src/main/resources/metadata";
+      ConfigFilesBasedQQQApplication application = new ConfigFilesBasedQQQApplication(path);
+      QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(application);
+      javalinServer.start();
+   }
+   catch(Exception e)
+   {
+      LOG.error("Failed to start javalin server.  See stack trace for details.", e);
+   }
+}
+----
+
+A similar class exists if your metadata is produced by a package of Java MetaDataProducer objects:  `MetaDataProducerBasedQQQApplication`.
+
+=== QApplicationJavalinServer
+This class provides the bridge between your QQQ Application (e.g., your metadata) and the QQQ Middleware layer
+served by a Javalin web server.  It has several properties to control behaviors:
+
+* `Integer port` - (default `8000`) - port to use for serving HTTP.
+* `boolean serveFrontendMaterialDashboard` - (default `true`) whether to serve the javascript frontend provided
+in the maven artifact `qqq-frontend-material-dashboard`.
+* `boolean serveLegacyUnversionedMiddlewareAPI` - (default `true`) whether to serve a version the original implementation
+of the QQQ middleware, which current version of `qqq-frontend-material-dashboard` are compatible with.
+* `List<AbstractMiddlewareVersion> middlewareVersionList` - (default contains `MiddlewareVersionV1`) - list of
+newer, formally versioned implementations of the QQQ middleware interface to be served.
+* `Consumer<Javalin> javalinConfigurationCustomizer` - (default `null`) - optional hook to customize the
+javalin service object before it is started.
+* `List<QJavalinRouteProviderInterface> additionalRouteProviders` - (default `null`) - list of fully custom
+implementations of `QJavalinRouteProviderInterface`, to add additional endpoints to the javalin server.
+** _Note, you may first want to consider using JavalinRouteProviderMetaData instead - see below._
+* `QJavalinMetaData javalinMetaData` - (default `null`) - optional alternative place to define `JavalinMetaData` (vs.
+defining it in the `QInstance`). _Note that if it is set in both places, the one in the QApplicationJavalinServer
+is used._
+
+=== JavalinMetaData
+Certain behaviors of a QQQ Javalin server are configured in a declarative manner by adding a `QJavalinMetaData`
+object to the `supplementalMetaData` in your `QInstance` (or, as mentioned above, by setting it directly on the
+`QApplicationJavalinServer`):
+
+* `List<JavalinRouteProviderMetaData> routeProviders` - (default `null`) optional list of custom route providers to
+add to the Javalin server.  See below for details.
+* `String uploadedFileArchiveTableName` - (default `null`) - reference to a QQQ Table in your application instance,
+needed to support the Bulk Load process, as well as any other processes which need to accept an uploaded file
+as input.
+* `boolean loggerDisabled` - (default `false`)
+* `Function<QJavalinAccessLogger.LogEntry, Boolean> logFilter` - (default `null`)
+* `boolean queryWithoutLimitAllowed` - (default `false`)
+* `Integer queryWithoutLimitDefault` - (default `1000`)
+* `Level queryWithoutLimitLogLevel` - (default `INFO`)
+
+==== JavalinRouteProviderMetaData
+This type of metadata allows you to add additional http route providers to your Javalin instance, e.g., for
+serving static files or for running custom code from your application (in the form of QQQ Processes) to respond
+to HTTP requests.
+
+* `String hostedPath` - (required)
+* `String fileSystemPath` - (required for a static router)
+* `String processName` - required for a dynamic, process-based router.  Must be a process name within the QQQ Instance.
+See below for additional details
+* `List<String> methods` - required list of HTTP methods (verbs) that are served by the route provider
+* `QCodeReference routeAuthenticator - Optional reference to a class that implements `RouteAuthenticatorInterface`,
+to provide security authentication to all requests handled by the route provider.
+** A default implementation is provided as `SimpleRouteAuthenticator`, which requires that a user session be present
+to access paths served by the route provider.
+
+===== Process-based route provider processes
+If you define a `JavalinRouteProviderMetaData` with a `processName` (e.g., to serve dynamic HTTP responses from your javalin
+server), the process that you implement will be called to respond to any HTTP requests received by the javalin
+server which match the `hostedPath` and `methods` that are specified in the metadata.
+
+The QQQ javalin server will marshal request data from the javalin context into the process's payload, conforming to
+the shape of the `ProcessBasedRouterPayload` class.  Similarly, the http response will be built by taking values from
+the process's output/state conforming to the fields in that class.  As such, it is recommended to use a
+`ProcessBasedRouterPayload`  instance, as show in this example:
+
+[source,java]
+.Process-based router usage example (including ProcessBasedRouterPayload)
+----
+public class MyDynamicSiteProcessStep implements BackendStep
+{
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+   {
+      ProcessBasedRouterPayload payload = input.getProcessPayload(ProcessBasedRouterPayload.class);
+      String path = payload.getPath();
+      payload.setResponseString("You requested: " + path);
+      output.setProcessPayload(payload);
+   }
+}
+----

pom.xml

@@ -48,7 +48,7 @@
    </modules>
 
    <properties>
-      <revision>0.25.0-SNAPSHOT</revision>
+      <revision>0.26.0-SNAPSHOT</revision>
 
       <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
       <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

qqq-backend-core/pom.xml

@@ -129,10 +129,16 @@
          <version>2.16.0</version>
       </dependency>
 
+      <dependency>
+         <groupId>com.nimbusds</groupId>
+         <artifactId>oauth2-oidc-sdk</artifactId>
+         <version>11.23.1</version>
+      </dependency>
+
       <dependency>
          <groupId>com.auth0</groupId>
          <artifactId>auth0</artifactId>
-         <version>2.1.0</version>
+         <version>2.18.0</version>
       </dependency>
       <dependency>
          <groupId>com.auth0</groupId>
@@ -142,12 +148,12 @@
       <dependency>
          <groupId>com.auth0</groupId>
          <artifactId>jwks-rsa</artifactId>
-         <version>0.22.0</version>
+         <version>0.22.1</version>
       </dependency>
       <dependency>
          <groupId>io.github.cdimascio</groupId>
-         <artifactId>java-dotenv</artifactId>
-         <version>5.2.2</version>
+         <artifactId>dotenv-java</artifactId>
+         <version>3.2.0</version>
       </dependency>
       <dependency>
          <groupId>org.apache.velocity</groupId>

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/MultiCustomizer.java

@@ -0,0 +1,226 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.customizers;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.InitializableViaCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties;
+import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
+
+
+/*******************************************************************************
+ ** Implementation of TableCustomizerInterface that runs multiple other customizers
+ *******************************************************************************/
+public class MultiCustomizer implements InitializableViaCodeReference, TableCustomizerInterface
+{
+   private static final String KEY_CODE_REFERENCES = "codeReferences";
+
+   private List<TableCustomizerInterface> customizers = new ArrayList<>();
+
+
+   /***************************************************************************
+    * Factory method that builds a {@link QCodeReferenceWithProperties} that will
+    * allow this multi-customizer to be assigned to a table, and to track
+    * in that code ref's properties, the "sub" QCodeReferences to be used.
+    *
+    * Added to a table as in:
+    * <pre>
+    * table.withCustomizer(TableCustomizers.POST_INSERT_RECORD,
+    *    MultiCustomizer.of(QCodeReference(x), QCodeReference(y)));
+    * </pre>
+    *
+    * @param codeReferences
+    * one or more {@link QCodeReference objects} to run when this customizer
+    * runs.  note that they will run in the order provided in this list.
+    ***************************************************************************/
+   public static QCodeReferenceWithProperties of(QCodeReference... codeReferences)
+   {
+      ArrayList<QCodeReference> list = new ArrayList<>(Arrays.stream(codeReferences).toList());
+      return (new QCodeReferenceWithProperties(MultiCustomizer.class, MapBuilder.of(KEY_CODE_REFERENCES, list)));
+   }
+
+
+   /***************************************************************************
+    * Add an additional table customizer code reference to an existing
+    * codeReference, e.g., constructed by the `of` factory method.
+    *
+    * @see #of(QCodeReference...)
+    ***************************************************************************/
+   public static void addTableCustomizer(QCodeReferenceWithProperties existingMultiCustomizerCodeReference, QCodeReference codeReference)
+   {
+      ArrayList<QCodeReference> list = (ArrayList<QCodeReference>) existingMultiCustomizerCodeReference.getProperties().computeIfAbsent(KEY_CODE_REFERENCES, key -> new ArrayList<>());
+      list.add(codeReference);
+   }
+
+
+
+   /***************************************************************************
+    * When this class is instantiated by the QCodeLoader, initialize the
+    * sub-customizer objects.
+    ***************************************************************************/
+   @Override
+   public void initialize(QCodeReference codeReference)
+   {
+      if(codeReference instanceof QCodeReferenceWithProperties codeReferenceWithProperties)
+      {
+         Serializable codeReferencesPropertyValue = codeReferenceWithProperties.getProperties().get(KEY_CODE_REFERENCES);
+         if(codeReferencesPropertyValue instanceof List<?> list)
+         {
+            for(Object o : list)
+            {
+               if(o instanceof QCodeReference reference)
+               {
+                  TableCustomizerInterface customizer = QCodeLoader.getAdHoc(TableCustomizerInterface.class, reference);
+                  customizers.add(customizer);
+               }
+            }
+         }
+         else
+         {
+            LOG.warn("Property KEY_CODE_REFERENCES [" + KEY_CODE_REFERENCES + "] must be a List<QCodeReference>.");
+         }
+      }
+
+      if(customizers.isEmpty())
+      {
+         LOG.info("No TableCustomizers were specified for MultiCustomizer.");
+      }
+   }
+
+
+
+   /***************************************************************************
+    * run postQuery method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.postQuery(queryInput, records);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run preInsert method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.preInsert(insertInput, records, isPreview);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run postInsert method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.postInsert(insertInput, records);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run preUpdate method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.preUpdate(updateInput, records, isPreview, oldRecordList);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run postUpdate method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.postUpdate(updateInput, records, oldRecordList);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run preDelete method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.preDelete(deleteInput, records, isPreview);
+      }
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * run postDelete method over all sub-customizers
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
+   {
+      for(TableCustomizerInterface customizer : customizers)
+      {
+         records = customizer.postDelete(deleteInput, records);
+      }
+      return records;
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/OldRecordHelper.java

@@ -0,0 +1,88 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.customizers;
+
+
+import java.io.Serializable;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.utils.collections.TypeTolerantKeyMap;
+
+
+/*******************************************************************************
+ ** utility class to help table customizers working with the oldRecordList.
+ ** Usage is just 2 lines:
+ ** outside of loop-over-records:
+ **   - OldRecordHelper oldRecordHelper = new OldRecordHelper(updateInput.getTableName(), oldRecordList);
+ ** then inside the record loop:
+ **   - Optional<QRecord> oldRecord = oldRecordHelper.getOldRecord(record);
+ *******************************************************************************/
+public class OldRecordHelper
+{
+   private String     primaryKeyField;
+   private QFieldType primaryKeyType;
+
+   private Optional<List<QRecord>>    oldRecordList;
+   private Map<Serializable, QRecord> oldRecordMap;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public OldRecordHelper(String tableName, Optional<List<QRecord>> oldRecordList)
+   {
+      this.primaryKeyField = QContext.getQInstance().getTable(tableName).getPrimaryKeyField();
+      this.primaryKeyType = QContext.getQInstance().getTable(tableName).getField(primaryKeyField).getType();
+
+      this.oldRecordList = oldRecordList;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public Optional<QRecord> getOldRecord(QRecord record)
+   {
+      if(oldRecordMap == null)
+      {
+         if(oldRecordList.isPresent())
+         {
+            oldRecordMap = new TypeTolerantKeyMap<>(primaryKeyType);
+            oldRecordList.get().forEach(r -> oldRecordMap.put(r.getValue(primaryKeyField), r));
+         }
+         else
+         {
+            oldRecordMap = Collections.emptyMap();
+         }
+      }
+
+      return (Optional.ofNullable(oldRecordMap.get(record.getValue(primaryKeyField))));
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/DeleteAction.java

@@ -401,6 +401,7 @@ public class DeleteAction
          if(CollectionUtils.nullSafeHasContents(associatedKeys))
          {
             DeleteInput nextLevelDeleteInput = new DeleteInput();
+            nextLevelDeleteInput.setFlags(deleteInput.getFlags());
             nextLevelDeleteInput.setTransaction(deleteInput.getTransaction());
             nextLevelDeleteInput.setTableName(association.getAssociatedTableName());
             nextLevelDeleteInput.setPrimaryKeys(associatedKeys);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/InsertAction.java

@@ -34,7 +34,6 @@ import java.util.Set;
 import java.util.stream.Collectors;
 import com.kingsrook.qqq.backend.core.actions.AbstractQActionFunction;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
-import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.actions.audits.DMLAuditAction;
 import com.kingsrook.qqq.backend.core.actions.automation.AutomationStatus;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationStatusUpdater;
@@ -54,6 +53,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
@@ -157,7 +157,7 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
       //////////////////////////////////////////////////
       // insert any associations in the input records //
       //////////////////////////////////////////////////
-      manageAssociations(table, insertOutput.getRecords(), insertInput.getTransaction());
+      manageAssociations(table, insertOutput.getRecords(), insertInput);
 
       //////////////////
       // do the audit //
@@ -174,9 +174,21 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
             .withRecordList(insertOutput.getRecords()));
       }
 
-      //////////////////////////////////////////////////////////////
-      // finally, run the post-insert customizer, if there is one //
-      //////////////////////////////////////////////////////////////
+      ////////////////////////////////////////////////////////////////
+      // finally, run the post-insert customizers, if there are any //
+      ////////////////////////////////////////////////////////////////
+      runPostInsertCustomizers(insertInput, table, insertOutput);
+
+      return insertOutput;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static void runPostInsertCustomizers(InsertInput insertInput, QTableMetaData table, InsertOutput insertOutput)
+   {
       Optional<TableCustomizerInterface> postInsertCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_INSERT_RECORD.getRole());
       if(postInsertCustomizer.isPresent())
       {
@@ -193,7 +205,25 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
          }
       }
 
-      return insertOutput;
+      ///////////////////////////////////////////////
+      // run all of the instance-level customizers //
+      ///////////////////////////////////////////////
+      List<QCodeReference> tableCustomizerCodes = QContext.getQInstance().getTableCustomizers(TableCustomizers.POST_INSERT_RECORD);
+      for(QCodeReference tableCustomizerCode : tableCustomizerCodes)
+      {
+         try
+         {
+            TableCustomizerInterface tableCustomizer = QCodeLoader.getAdHoc(TableCustomizerInterface.class, tableCustomizerCode);
+            insertOutput.setRecords(tableCustomizer.postInsert(insertInput, insertOutput.getRecords()));
+         }
+         catch(Exception e)
+         {
+            for(QRecord record : insertOutput.getRecords())
+            {
+               record.addWarning(new QWarningMessage("An error occurred after the insert: " + e.getMessage()));
+            }
+         }
+      }
    }
 
 
@@ -308,6 +338,19 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
             insertInput.setRecords(preInsertCustomizer.get().preInsert(insertInput, insertInput.getRecords(), isPreview));
          }
       }
+
+      ///////////////////////////////////////////////
+      // run all of the instance-level customizers //
+      ///////////////////////////////////////////////
+      List<QCodeReference> tableCustomizerCodes = QContext.getQInstance().getTableCustomizers(TableCustomizers.PRE_INSERT_RECORD);
+      for(QCodeReference tableCustomizerCode : tableCustomizerCodes)
+      {
+         TableCustomizerInterface tableCustomizer = QCodeLoader.getAdHoc(TableCustomizerInterface.class, tableCustomizerCode);
+         if(whenToRun.equals(tableCustomizer.whenToRunPreInsert(insertInput, isPreview)))
+         {
+            insertInput.setRecords(tableCustomizer.preInsert(insertInput, insertInput.getRecords(), isPreview));
+         }
+      }
    }
 
 
@@ -342,7 +385,7 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
    /*******************************************************************************
     **
     *******************************************************************************/
-   private void manageAssociations(QTableMetaData table, List<QRecord> insertedRecords, QBackendTransaction transaction) throws QException
+   private void manageAssociations(QTableMetaData table, List<QRecord> insertedRecords, InsertInput insertInput) throws QException
    {
       for(Association association : CollectionUtils.nonNullList(table.getAssociations()))
       {
@@ -375,7 +418,8 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
          if(CollectionUtils.nullSafeHasContents(nextLevelInserts))
          {
             InsertInput nextLevelInsertInput = new InsertInput();
-            nextLevelInsertInput.setTransaction(transaction);
+            nextLevelInsertInput.withFlags(insertInput.getFlags());
+            nextLevelInsertInput.setTransaction(insertInput.getTransaction());
             nextLevelInsertInput.setTableName(association.getAssociatedTableName());
             nextLevelInsertInput.setRecords(nextLevelInserts);
             InsertOutput nextLevelInsertOutput = new InsertAction().execute(nextLevelInsertInput);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/ReplaceAction.java

@@ -126,6 +126,7 @@ public class ReplaceAction extends AbstractQActionFunction<ReplaceInput, Replace
          InsertInput insertInput = new InsertInput();
          insertInput.setTableName(table.getName());
          insertInput.setRecords(insertList);
+         insertInput.withFlags(input.getFlags());
          insertInput.setTransaction(transaction);
          insertInput.setOmitDmlAudit(input.getOmitDmlAudit());
          InsertOutput insertOutput = new InsertAction().execute(insertInput);
@@ -135,6 +136,7 @@ public class ReplaceAction extends AbstractQActionFunction<ReplaceInput, Replace
          UpdateInput updateInput = new UpdateInput();
          updateInput.setTableName(table.getName());
          updateInput.setRecords(updateList);
+         updateInput.withFlags(input.getFlags());
          updateInput.setTransaction(transaction);
          updateInput.setOmitDmlAudit(input.getOmitDmlAudit());
          UpdateOutput updateOutput = new UpdateAction().execute(updateInput);
@@ -151,6 +153,7 @@ public class ReplaceAction extends AbstractQActionFunction<ReplaceInput, Replace
             DeleteInput deleteInput = new DeleteInput();
             deleteInput.setTableName(table.getName());
             deleteInput.setQueryFilter(deleteFilter);
+            deleteInput.withFlags(input.getFlags());
             deleteInput.setTransaction(transaction);
             deleteInput.setOmitDmlAudit(input.getOmitDmlAudit());
             DeleteOutput deleteOutput = new DeleteAction().execute(deleteInput);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/UpdateAction.java

@@ -57,6 +57,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldBehavior;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
@@ -199,6 +200,18 @@ public class UpdateAction
       //////////////////////////////////////////////////////////////
       // finally, run the post-update customizer, if there is one //
       //////////////////////////////////////////////////////////////
+      runPostUpdateCustomizers(updateInput, table, updateOutput, oldRecordList);
+
+      return updateOutput;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static void runPostUpdateCustomizers(UpdateInput updateInput, QTableMetaData table, UpdateOutput updateOutput, Optional<List<QRecord>> oldRecordList)
+   {
       Optional<TableCustomizerInterface> postUpdateCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_UPDATE_RECORD.getRole());
       if(postUpdateCustomizer.isPresent())
       {
@@ -215,7 +228,49 @@ public class UpdateAction
          }
       }
 
-      return updateOutput;
+      ///////////////////////////////////////////////
+      // run all of the instance-level customizers //
+      ///////////////////////////////////////////////
+      List<QCodeReference> tableCustomizerCodes = QContext.getQInstance().getTableCustomizers(TableCustomizers.POST_UPDATE_RECORD);
+      for(QCodeReference tableCustomizerCode : tableCustomizerCodes)
+      {
+         try
+         {
+            TableCustomizerInterface tableCustomizer = QCodeLoader.getAdHoc(TableCustomizerInterface.class, tableCustomizerCode);
+            updateOutput.setRecords(tableCustomizer.postUpdate(updateInput, updateOutput.getRecords(), oldRecordList));
+         }
+         catch(Exception e)
+         {
+            for(QRecord record : updateOutput.getRecords())
+            {
+               record.addWarning(new QWarningMessage("An error occurred after the update: " + e.getMessage()));
+            }
+         }
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static void runPreUpdateCustomizers(UpdateInput updateInput, QTableMetaData table, Optional<List<QRecord>> oldRecordList, boolean isPreview) throws QException
+   {
+      Optional<TableCustomizerInterface> preUpdateCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_UPDATE_RECORD.getRole());
+      if(preUpdateCustomizer.isPresent())
+      {
+         updateInput.setRecords(preUpdateCustomizer.get().preUpdate(updateInput, updateInput.getRecords(), isPreview, oldRecordList));
+      }
+
+      ///////////////////////////////////////////////
+      // run all of the instance-level customizers //
+      ///////////////////////////////////////////////
+      List<QCodeReference> tableCustomizerCodes = QContext.getQInstance().getTableCustomizers(TableCustomizers.PRE_UPDATE_RECORD);
+      for(QCodeReference tableCustomizerCode : tableCustomizerCodes)
+      {
+         TableCustomizerInterface tableCustomizer = QCodeLoader.getAdHoc(TableCustomizerInterface.class, tableCustomizerCode);
+         updateInput.setRecords(tableCustomizer.preUpdate(updateInput, updateInput.getRecords(), isPreview, oldRecordList));
+      }
    }
 
 
@@ -278,11 +333,7 @@ public class UpdateAction
       ///////////////////////////////////////////////////////////////////////////
       // after all validations, run the pre-update customizer, if there is one //
       ///////////////////////////////////////////////////////////////////////////
-      Optional<TableCustomizerInterface> preUpdateCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_UPDATE_RECORD.getRole());
-      if(preUpdateCustomizer.isPresent())
-      {
-         updateInput.setRecords(preUpdateCustomizer.get().preUpdate(updateInput, updateInput.getRecords(), isPreview, oldRecordList));
-      }
+      runPreUpdateCustomizers(updateInput, table, oldRecordList, isPreview);
    }
 
 
@@ -405,7 +456,7 @@ public class UpdateAction
                   QFieldType   fieldType = table.getField(lock.getFieldName()).getType();
                   Serializable lockValue = ValueUtils.getValueAsFieldType(fieldType, oldRecord.getValue(lock.getFieldName()));
 
-                  List<QErrorMessage> errors = ValidateRecordSecurityLockHelper.validateRecordSecurityValue(table, lock, lockValue, fieldType, ValidateRecordSecurityLockHelper.Action.UPDATE, Collections.emptyMap());
+                  List<QErrorMessage> errors = ValidateRecordSecurityLockHelper.validateRecordSecurityValue(table, lock, lockValue, fieldType, ValidateRecordSecurityLockHelper.Action.UPDATE, Collections.emptyMap(), QContext.getQSession());
                   if(CollectionUtils.nullSafeHasContents(errors))
                   {
                      errors.forEach(e -> record.addError(e));
@@ -554,6 +605,7 @@ public class UpdateAction
                {
                   LOG.debug("Deleting associatedRecords", logPair("associatedTable", associatedTable.getName()), logPair("noOfRecords", queryOutput.getRecords().size()));
                   DeleteInput deleteInput = new DeleteInput();
+                  deleteInput.setFlags(updateInput.getFlags());
                   deleteInput.setTransaction(updateInput.getTransaction());
                   deleteInput.setTableName(association.getAssociatedTableName());
                   deleteInput.setPrimaryKeys(queryOutput.getRecords().stream().map(r -> r.getValue(associatedTable.getPrimaryKeyField())).collect(Collectors.toList()));
@@ -566,6 +618,7 @@ public class UpdateAction
                LOG.debug("Updating associatedRecords", logPair("associatedTable", associatedTable.getName()), logPair("noOfRecords", nextLevelUpdates.size()));
                UpdateInput nextLevelUpdateInput = new UpdateInput();
                nextLevelUpdateInput.setTransaction(updateInput.getTransaction());
+               nextLevelUpdateInput.setFlags(updateInput.getFlags());
                nextLevelUpdateInput.setTableName(association.getAssociatedTableName());
                nextLevelUpdateInput.setRecords(nextLevelUpdates);
                UpdateOutput nextLevelUpdateOutput = new UpdateAction().execute(nextLevelUpdateInput);
@@ -576,6 +629,7 @@ public class UpdateAction
                LOG.debug("Inserting associatedRecords", logPair("associatedTable", associatedTable.getName()), logPair("noOfRecords", nextLevelUpdates.size()));
                InsertInput nextLevelInsertInput = new InsertInput();
                nextLevelInsertInput.setTransaction(updateInput.getTransaction());
+               nextLevelInsertInput.setFlags(updateInput.getFlags());
                nextLevelInsertInput.setTableName(association.getAssociatedTableName());
                nextLevelInsertInput.setRecords(nextLevelInserts);
                InsertOutput nextLevelInsertOutput = new InsertAction().execute(nextLevelInsertInput);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/helpers/ValidateRecordSecurityLockHelper.java

@@ -50,6 +50,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLockFilters;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.model.statusmessages.PermissionDeniedMessage;
 import com.kingsrook.qqq.backend.core.model.statusmessages.QErrorMessage;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
@@ -102,7 +103,7 @@ public class ValidateRecordSecurityLockHelper
       // actually check lock values //
       ////////////////////////////////
       Map<Serializable, RecordWithErrors> errorRecords = new HashMap<>();
-      evaluateRecordLocks(table, records, action, locksToCheck, errorRecords, new ArrayList<>(), madeUpPrimaryKeys, transaction);
+      evaluateRecordLocks(table, records, action, locksToCheck, errorRecords, new ArrayList<>(), madeUpPrimaryKeys, transaction, QContext.getQSession());
 
       /////////////////////////////////
       // propagate errors to records //
@@ -124,6 +125,29 @@ public class ValidateRecordSecurityLockHelper
 
 
 
+   /***************************************************************************
+    ** return boolean if given session can read given record
+    ***************************************************************************/
+   public static boolean allowedToReadRecord(QTableMetaData table, QRecord record, QSession qSession, QBackendTransaction transaction) throws QException
+   {
+      MultiRecordSecurityLock locksToCheck = getRecordSecurityLocks(table, Action.SELECT);
+      if(locksToCheck == null || CollectionUtils.nullSafeIsEmpty(locksToCheck.getLocks()))
+      {
+         return (true);
+      }
+
+      Map<Serializable, RecordWithErrors> errorRecords = new HashMap<>();
+      evaluateRecordLocks(table, List.of(record), Action.SELECT, locksToCheck, errorRecords, new ArrayList<>(), Collections.emptyMap(), transaction, qSession);
+      if(errorRecords.containsKey(record.getValue(table.getPrimaryKeyField())))
+      {
+         return (false);
+      }
+
+      return (true);
+   }
+
+
+
    /*******************************************************************************
     ** For a list of `records` from a `table`, and a given `action`, evaluate a
     ** `recordSecurityLock` (which may be a multi-lock) - populating the input map
@@ -142,7 +166,7 @@ public class ValidateRecordSecurityLockHelper
     ** BUT - WRITE locks - in their case, we read the record no matter what, and in
     ** here we need to verify we have a key that allows us to WRITE the record.
     *******************************************************************************/
-   private static void evaluateRecordLocks(QTableMetaData table, List<QRecord> records, Action action, RecordSecurityLock recordSecurityLock, Map<Serializable, RecordWithErrors> errorRecords, List<Integer> treePosition, Map<Serializable, QRecord> madeUpPrimaryKeys, QBackendTransaction transaction) throws QException
+   private static void evaluateRecordLocks(QTableMetaData table, List<QRecord> records, Action action, RecordSecurityLock recordSecurityLock, Map<Serializable, RecordWithErrors> errorRecords, List<Integer> treePosition, Map<Serializable, QRecord> madeUpPrimaryKeys, QBackendTransaction transaction, QSession qSession) throws QException
    {
       if(recordSecurityLock instanceof MultiRecordSecurityLock multiRecordSecurityLock)
       {
@@ -153,7 +177,7 @@ public class ValidateRecordSecurityLockHelper
          for(RecordSecurityLock childLock : CollectionUtils.nonNullList(multiRecordSecurityLock.getLocks()))
          {
             treePosition.add(i);
-            evaluateRecordLocks(table, records, action, childLock, errorRecords, treePosition, madeUpPrimaryKeys, transaction);
+            evaluateRecordLocks(table, records, action, childLock, errorRecords, treePosition, madeUpPrimaryKeys, transaction, qSession);
             treePosition.remove(treePosition.size() - 1);
             i++;
          }
@@ -165,7 +189,7 @@ public class ValidateRecordSecurityLockHelper
       // if this lock has an all-access key, and the user has that key, then there can't be any errors here, so return early //
       /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
       QSecurityKeyType securityKeyType = QContext.getQInstance().getSecurityKeyType(recordSecurityLock.getSecurityKeyType());
-      if(StringUtils.hasContent(securityKeyType.getAllAccessKeyName()) && QContext.getQSession().hasSecurityKeyValue(securityKeyType.getAllAccessKeyName(), true, QFieldType.BOOLEAN))
+      if(StringUtils.hasContent(securityKeyType.getAllAccessKeyName()) && qSession.hasSecurityKeyValue(securityKeyType.getAllAccessKeyName(), true, QFieldType.BOOLEAN))
       {
          return;
       }
@@ -193,7 +217,7 @@ public class ValidateRecordSecurityLockHelper
             }
 
             Serializable        recordSecurityValue = record.getValue(field.getName());
-            List<QErrorMessage> recordErrors        = validateRecordSecurityValue(table, recordSecurityLock, recordSecurityValue, field.getType(), action, madeUpPrimaryKeys);
+            List<QErrorMessage> recordErrors        = validateRecordSecurityValue(table, recordSecurityLock, recordSecurityValue, field.getType(), action, madeUpPrimaryKeys, qSession);
             if(CollectionUtils.nullSafeHasContents(recordErrors))
             {
                errorRecords.computeIfAbsent(record.getValue(primaryKeyField), (k) -> new RecordWithErrors(record)).addAll(recordErrors, treePosition);
@@ -339,7 +363,7 @@ public class ValidateRecordSecurityLockHelper
 
                   for(QRecord inputRecord : inputRecords)
                   {
-                     List<QErrorMessage> recordErrors = validateRecordSecurityValue(table, recordSecurityLock, recordSecurityValue, field.getType(), action, madeUpPrimaryKeys);
+                     List<QErrorMessage> recordErrors = validateRecordSecurityValue(table, recordSecurityLock, recordSecurityValue, field.getType(), action, madeUpPrimaryKeys, qSession);
                      if(CollectionUtils.nullSafeHasContents(recordErrors))
                      {
                         errorRecords.computeIfAbsent(inputRecord.getValue(primaryKeyField), (k) -> new RecordWithErrors(inputRecord)).addAll(recordErrors, treePosition);
@@ -446,7 +470,7 @@ public class ValidateRecordSecurityLockHelper
    /*******************************************************************************
     **
     *******************************************************************************/
-   public static List<QErrorMessage> validateRecordSecurityValue(QTableMetaData table, RecordSecurityLock recordSecurityLock, Serializable recordSecurityValue, QFieldType fieldType, Action action, Map<Serializable, QRecord> madeUpPrimaryKeys)
+   public static List<QErrorMessage> validateRecordSecurityValue(QTableMetaData table, RecordSecurityLock recordSecurityLock, Serializable recordSecurityValue, QFieldType fieldType, Action action, Map<Serializable, QRecord> madeUpPrimaryKeys, QSession qSession)
    {
       if(recordSecurityValue == null || (madeUpPrimaryKeys != null && madeUpPrimaryKeys.containsKey(recordSecurityValue)))
       {
@@ -461,7 +485,7 @@ public class ValidateRecordSecurityLockHelper
       }
       else
       {
-         if(!QContext.getQSession().hasSecurityKeyValue(recordSecurityLock.getSecurityKeyType(), recordSecurityValue, fieldType))
+         if(!qSession.hasSecurityKeyValue(recordSecurityLock.getSecurityKeyType(), recordSecurityValue, fieldType))
          {
             if(CollectionUtils.nullSafeHasContents(recordSecurityLock.getJoinNameChain()))
             {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/BasicCustomPossibleValueProvider.java

@@ -47,12 +47,12 @@ public abstract class BasicCustomPossibleValueProvider<S, ID extends Serializabl
    /***************************************************************************
     **
     ***************************************************************************/
-   protected abstract S getSourceObject(Serializable id);
+   protected abstract S getSourceObject(Serializable id) throws QException;
 
    /***************************************************************************
     **
     ***************************************************************************/
-   protected abstract List<S> getAllSourceObjects();
+   protected abstract List<S> getAllSourceObjects() throws QException;
 
 
 
@@ -60,7 +60,7 @@ public abstract class BasicCustomPossibleValueProvider<S, ID extends Serializabl
     **
     ***************************************************************************/
    @Override
-   public QPossibleValue<ID> getPossibleValue(Serializable idValue)
+   public QPossibleValue<ID> getPossibleValue(Serializable idValue) throws QException
    {
       S sourceObject = getSourceObject(idValue);
       if(sourceObject == null)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/QCustomPossibleValueProvider.java

@@ -45,7 +45,7 @@ public interface QCustomPossibleValueProvider<T extends Serializable>
    /*******************************************************************************
     **
     *******************************************************************************/
-   QPossibleValue<T> getPossibleValue(Serializable idValue);
+   QPossibleValue<T> getPossibleValue(Serializable idValue) throws QException;
 
    /*******************************************************************************
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/AbstractMetaDataProducerBasedQQQApplication.java

@@ -29,7 +29,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 
 /*******************************************************************************
  ** Version of AbstractQQQApplication that assumes all meta-data is produced
- ** by MetaDataProducers in a single package.
+ ** by MetaDataProducers in (or under) a single package.
  *******************************************************************************/
 public abstract class AbstractMetaDataProducerBasedQQQApplication extends AbstractQQQApplication
 {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/ConfigFilesBasedQQQApplication.java

@@ -0,0 +1,61 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.instances.loaders.MetaDataLoaderHelper;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+
+
+/*******************************************************************************
+ ** Version of AbstractQQQApplication that assumes all meta-data is defined in
+ ** config files (yaml, json, etc) under a given directory path.
+ *******************************************************************************/
+public class ConfigFilesBasedQQQApplication extends AbstractQQQApplication
+{
+   private final String path;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public ConfigFilesBasedQQQApplication(String path)
+   {
+      this.path = path;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QInstance defineQInstance() throws QException
+   {
+      QInstance qInstance = new QInstance();
+      MetaDataLoaderHelper.processAllMetaDataFilesInDirectory(qInstance, path);
+      return (qInstance);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/MetaDataProducerBasedQQQApplication.java

@@ -0,0 +1,67 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances;
+
+
+/*******************************************************************************
+ ** Version of AbstractQQQApplication that assumes all meta-data is produced
+ ** by MetaDataProducers in (or under) a single package (where you can pass that
+ ** package into the constructor, vs. the abstract base class, where you extend
+ ** it and override the getMetaDataPackageName method.
+ *******************************************************************************/
+public class MetaDataProducerBasedQQQApplication extends AbstractMetaDataProducerBasedQQQApplication
+{
+   private final String metaDataPackageName;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public MetaDataProducerBasedQQQApplication(String metaDataPackageName)
+   {
+      this.metaDataPackageName = metaDataPackageName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public MetaDataProducerBasedQQQApplication(Class<?> aClassInMetaDataPackage)
+   {
+      this(aClassInMetaDataPackage.getPackageName());
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getMetaDataPackageName()
+   {
+      return (this.metaDataPackageName);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceEnricher.java

@@ -47,6 +47,7 @@ import com.kingsrook.qqq.backend.core.instances.enrichment.plugins.QInstanceEnri
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
@@ -210,6 +211,11 @@ public class QInstanceEnricher
     ***************************************************************************/
    private void enrichInstance()
    {
+      for(QSupplementalInstanceMetaData supplementalInstanceMetaData : qInstance.getSupplementalMetaData().values())
+      {
+         supplementalInstanceMetaData.enrich(qInstance);
+      }
+
       runPlugins(QInstance.class, qInstance, qInstance);
    }
 
@@ -330,7 +336,21 @@ public class QInstanceEnricher
 
       if(table.getFields() != null)
       {
-         table.getFields().values().forEach(this::enrichField);
+         for(Map.Entry<String, QFieldMetaData> entry : table.getFields().entrySet())
+         {
+            String         name  = entry.getKey();
+            QFieldMetaData field = entry.getValue();
+
+            ////////////////////////////////////////////////////////////////////////////
+            // in case the field wasn't given a name, use its key from the fields map //
+            ////////////////////////////////////////////////////////////////////////////
+            if(!StringUtils.hasContent(field.getName()))
+            {
+               field.setName(name);
+            }
+
+            enrichField(field);
+         }
 
          for(QSupplementalTableMetaData supplementalTableMetaData : CollectionUtils.nonNullMap(table.getSupplementalMetaData()).values())
          {
@@ -1390,7 +1410,7 @@ public class QInstanceEnricher
          if(possibleValueSource.getIdType() == null)
          {
             QTableMetaData table = qInstance.getTable(possibleValueSource.getTableName());
-            if(table != null)
+            if(table != null && table.getFields() != null)
             {
                String         primaryKeyField         = table.getPrimaryKeyField();
                QFieldMetaData primaryKeyFieldMetaData = table.getFields().get(primaryKeyField);
@@ -1463,7 +1483,18 @@ public class QInstanceEnricher
       if(enrichMethod.isPresent())
       {
          Class<?> parameterType = enrichMethod.get().getParameterTypes()[0];
-         enricherPlugins.add(parameterType, plugin);
+
+         Set<String> existingPluginIdentifiers = enricherPlugins.getOrDefault(parameterType, Collections.emptyList())
+            .stream().map(p -> p.getPluginIdentifier())
+            .collect(Collectors.toSet());
+         if(existingPluginIdentifiers.contains(plugin.getPluginIdentifier()))
+         {
+            LOG.debug("Enricher plugin is already registered - not re-adding it", logPair("pluginIdentifer", plugin.getPluginIdentifier()));
+         }
+         else
+         {
+            enricherPlugins.add(parameterType, plugin);
+         }
       }
       else
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceValidator.java

@@ -42,6 +42,7 @@ import java.util.function.Supplier;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationHandler;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
 import com.kingsrook.qqq.backend.core.actions.metadata.JoinGraph;
@@ -64,6 +65,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaDa
 import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.automation.QAutomationProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceLambda;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeType;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.ParentWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
@@ -251,6 +253,17 @@ public class QInstanceValidator
       {
          validateSimpleCodeReference("Instance metaDataActionCustomizer ", qInstance.getMetaDataActionCustomizer(), MetaDataActionCustomizerInterface.class);
       }
+
+      if(qInstance.getTableCustomizers() != null)
+      {
+         for(Map.Entry<String, List<QCodeReference>> entry : qInstance.getTableCustomizers().entrySet())
+         {
+            for(QCodeReference codeReference : CollectionUtils.nonNullList(entry.getValue()))
+            {
+               validateSimpleCodeReference("Instance tableCustomizer of type " + entry.getKey() + ": ", codeReference, TableCustomizerInterface.class);
+            }
+         }
+      }
    }
 
 
@@ -282,7 +295,18 @@ public class QInstanceValidator
       if(validateMethod.isPresent())
       {
          Class<?> parameterType = validateMethod.get().getParameterTypes()[0];
-         validatorPlugins.add(parameterType, plugin);
+
+         Set<String> existingPluginIdentifiers = validatorPlugins.getOrDefault(parameterType, Collections.emptyList())
+            .stream().map(p -> p.getPluginIdentifier())
+            .collect(Collectors.toSet());
+         if(existingPluginIdentifiers.contains(plugin.getPluginIdentifier()))
+         {
+            LOG.debug("Validator plugin is already registered - not re-adding it", logPair("pluginIdentifer", plugin.getPluginIdentifier()));
+         }
+         else
+         {
+            validatorPlugins.add(parameterType, plugin);
+         }
       }
       else
       {
@@ -302,6 +326,17 @@ public class QInstanceValidator
 
 
 
+   /*******************************************************************************
+    ** Getter for validatorPlugins
+    **
+    *******************************************************************************/
+   public static ListingHash<Class<?>, QInstanceValidatorPluginInterface<?>> getValidatorPlugins()
+   {
+      return validatorPlugins;
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -651,6 +686,8 @@ public class QInstanceValidator
             validateSimpleCodeReference("Instance Authentication meta data customizer ", authentication.getCustomizer(), QAuthenticationModuleCustomizerInterface.class);
          }
 
+         authentication.validate(qInstance, this);
+
          runPlugins(QAuthenticationMetaData.class, authentication, qInstance);
       }
    }
@@ -1406,7 +1443,7 @@ public class QInstanceValidator
             //////////////////////////////////////////////////
             // make sure the customizer can be instantiated //
             //////////////////////////////////////////////////
-            Object customizerInstance = getInstanceOfCodeReference(prefix, customizerClass);
+            Object customizerInstance = getInstanceOfCodeReference(prefix, customizerClass, codeReference);
 
             TableCustomizers tableCustomizer = TableCustomizers.forRole(roleName);
             if(tableCustomizer == null)
@@ -1467,8 +1504,13 @@ public class QInstanceValidator
    /*******************************************************************************
     **
     *******************************************************************************/
-   private Object getInstanceOfCodeReference(String prefix, Class<?> clazz)
+   private Object getInstanceOfCodeReference(String prefix, Class<?> clazz, QCodeReference codeReference)
    {
+      if(codeReference instanceof QCodeReferenceLambda<?> lambdaCodeReference)
+      {
+         return (lambdaCodeReference.getLambda());
+      }
+
       Object instance = null;
       try
       {
@@ -1647,21 +1689,26 @@ public class QInstanceValidator
             Set<String> usedStepNames = new HashSet<>();
             if(assertCondition(CollectionUtils.nullSafeHasContents(process.getStepList()), "At least 1 step must be defined in process " + processName + "."))
             {
-               int index = 0;
+               int index = -1;
                for(QStepMetaData step : process.getStepList())
                {
+                  index++;
                   if(assertCondition(StringUtils.hasContent(step.getName()), "Missing name for a step at index " + index + " in process " + processName))
                   {
                      assertCondition(!usedStepNames.contains(step.getName()), "Duplicate step name [" + step.getName() + "] in process " + processName);
                      usedStepNames.add(step.getName());
                   }
-                  index++;
 
                   ////////////////////////////////////////////
                   // validate instantiation of step classes //
                   ////////////////////////////////////////////
                   if(step instanceof QBackendStepMetaData backendStepMetaData)
                   {
+                     if(assertCondition(backendStepMetaData.getCode() != null, "Missing code for a backend step at index " + index + " in process " + processName))
+                     {
+                        validateSimpleCodeReference("Process " + processName + ", backend step at index " + index + ", code reference: ", backendStepMetaData.getCode(), BackendStep.class);
+                     }
+
                      if(backendStepMetaData.getInputMetaData() != null && CollectionUtils.nullSafeHasContents(backendStepMetaData.getInputMetaData().getFieldList()))
                      {
                         for(QFieldMetaData fieldMetaData : backendStepMetaData.getInputMetaData().getFieldList())
@@ -2225,8 +2272,7 @@ public class QInstanceValidator
    /*******************************************************************************
     **
     *******************************************************************************/
-   @SafeVarargs
-   private void validateSimpleCodeReference(String prefix, QCodeReference codeReference, Class<?>... anyOfExpectedClasses)
+   public void validateSimpleCodeReference(String prefix, QCodeReference codeReference, Class<?>... anyOfExpectedClasses)
    {
       if(!preAssertionsForCodeReference(codeReference, prefix))
       {
@@ -2247,7 +2293,7 @@ public class QInstanceValidator
             //////////////////////////////////////////////////
             // make sure the customizer can be instantiated //
             //////////////////////////////////////////////////
-            Object classInstance = getInstanceOfCodeReference(prefix, clazz);
+            Object classInstance = getInstanceOfCodeReference(prefix, clazz, codeReference);
 
             ////////////////////////////////////////////////////////////////////////
             // make sure the customizer instance can be cast to the expected type //
@@ -2270,6 +2316,11 @@ public class QInstanceValidator
       Class<?> clazz = null;
       try
       {
+         if(codeReference instanceof QCodeReferenceLambda<?> lambdaCodeReference)
+         {
+            return (lambdaCodeReference.getLambda().getClass());
+         }
+
          clazz = Class.forName(codeReference.getName());
       }
       catch(ClassNotFoundException e)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/SecretsManagerUtils.java

@@ -72,6 +72,18 @@ public class SecretsManagerUtils
     ** and write them to a .env file (backing up any pre-existing .env files first).
     *******************************************************************************/
    public static void writeEnvFromSecretsWithNamePrefix(String prefix) throws IOException
+   {
+      writeEnvFromSecretsWithNamePrefix(prefix, true);
+   }
+
+
+
+   /*******************************************************************************
+    ** IF secret manager ENV vars are set,
+    ** THEN lookup all secrets starting with the given prefix,
+    ** and write them to a .env file (backing up any pre-existing .env files first).
+    *******************************************************************************/
+   public static void writeEnvFromSecretsWithNamePrefix(String prefix, boolean quoteValues) throws IOException
    {
       Optional<AWSSecretsManager> optionalSecretsManagerClient = getSecretsManagerClient();
       if(optionalSecretsManagerClient.isPresent())
@@ -91,7 +103,9 @@ public class SecretsManagerUtils
                Optional<String> secretValue       = getSecret(prefix, nameWithoutPrefix);
                if(secretValue.isPresent())
                {
-                  String envLine = nameWithoutPrefix + "=" + secretValue.get();
+                  String envLine = quoteValues
+                     ? nameWithoutPrefix + "=\"" + secretValue.get() + "\""
+                     : nameWithoutPrefix + "=" + secretValue.get();
                   fullEnv.append(envLine).append('\n');
                }
             }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/assessment/Assessable.java

@@ -0,0 +1,37 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.assessment;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+
+
+/*******************************************************************************
+ ** marker for an object which can be processed by the QInstanceAssessor.
+ *******************************************************************************/
+public interface Assessable
+{
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   void assess(QInstanceAssessor qInstanceAssessor, QInstance qInstance);
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/assessment/QInstanceAssessor.java

@@ -0,0 +1,219 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.assessment;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ ** POC of a class that is meant to review meta-data for accuracy vs. real backends.
+ *******************************************************************************/
+public class QInstanceAssessor
+{
+   private static final QLogger LOG = QLogger.getLogger(QInstanceAssessor.class);
+
+   private final QInstance qInstance;
+
+   private List<String> errors      = new ArrayList<>();
+   private List<String> warnings    = new ArrayList<>();
+   private List<String> suggestions = new ArrayList<>();
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QInstanceAssessor(QInstance qInstance)
+   {
+      this.qInstance = qInstance;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void assess()
+   {
+      for(QBackendMetaData backend : qInstance.getBackends().values())
+      {
+         if(backend instanceof Assessable assessable)
+         {
+            assessable.assess(this, qInstance);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @SuppressWarnings("checkstyle:AvoidEscapedUnicodeCharacters")
+   public String getSummary()
+   {
+      StringBuilder rs = new StringBuilder();
+
+      ///////////////////////////
+      // print header & errors //
+      ///////////////////////////
+      if(CollectionUtils.nullSafeIsEmpty(errors))
+      {
+         rs.append("Assessment passed with no errors! \uD83D\uDE0E\n");
+      }
+      else
+      {
+         rs.append("Assessment found the following ").append(StringUtils.plural(errors, "error", "errors")).append(": \uD83D\uDE32\n");
+
+         for(String error : errors)
+         {
+            rs.append(" - ").append(error).append("\n");
+         }
+      }
+
+      /////////////////////////////////////
+      // print warnings if there are any //
+      /////////////////////////////////////
+      if(CollectionUtils.nullSafeHasContents(warnings))
+      {
+         rs.append("\nAssessment found the following ").append(StringUtils.plural(warnings, "warning", "warnings")).append(": \uD83E\uDD28\n");
+
+         for(String warning : warnings)
+         {
+            rs.append(" - ").append(warning).append("\n");
+         }
+      }
+
+      //////////////////////////////////////////
+      // print suggestions, if there were any //
+      //////////////////////////////////////////
+      if(CollectionUtils.nullSafeHasContents(suggestions))
+      {
+         rs.append("\nThe following ").append(StringUtils.plural(suggestions, "fix is", "fixes are")).append(" suggested: \uD83E\uDD13\n");
+
+         for(String suggestion : suggestions)
+         {
+            rs.append("\n").append(suggestion).append("\n\n");
+         }
+      }
+
+      return (rs.toString());
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for qInstance
+    **
+    *******************************************************************************/
+   public QInstance getInstance()
+   {
+      return qInstance;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for errors
+    **
+    *******************************************************************************/
+   public List<String> getErrors()
+   {
+      return errors;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for warnings
+    **
+    *******************************************************************************/
+   public List<String> getWarnings()
+   {
+      return warnings;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addError(String errorMessage)
+   {
+      errors.add(errorMessage);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addWarning(String warningMessage)
+   {
+      warnings.add(warningMessage);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addError(String errorMessage, Exception e)
+   {
+      addError(errorMessage + " : " + e.getMessage());
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addSuggestion(String message)
+   {
+      suggestions.add(message);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public int getExitCode()
+   {
+      if(CollectionUtils.nullSafeHasContents(errors))
+      {
+         return (1);
+      }
+      else
+      {
+         return (0);
+      }
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/enrichment/plugins/QInstanceEnricherPluginInterface.java

@@ -37,4 +37,13 @@ public interface QInstanceEnricherPluginInterface<T>
     *******************************************************************************/
    void enrich(T object, QInstance qInstance);
 
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   default String getPluginIdentifier()
+   {
+      return getClass().getName();
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/AbstractMetaDataLoader.java

@@ -0,0 +1,510 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.Serializable;
+import java.lang.reflect.Method;
+import java.lang.reflect.ParameterizedType;
+import java.lang.reflect.Type;
+import java.math.BigDecimal;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.YamlUtils;
+import org.apache.commons.io.IOUtils;
+import static com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsInteger;
+import static com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsString;
+
+
+/*******************************************************************************
+ ** Abstract base class in hierarchy of classes that know how to construct &
+ ** populate QMetaDataObject instances, based on input streams (e.g., from files).
+ *******************************************************************************/
+public abstract class AbstractMetaDataLoader<T extends QMetaDataObject>
+{
+   private static final QLogger LOG = QLogger.getLogger(AbstractMetaDataLoader.class);
+
+   private String fileName;
+
+   private List<LoadingProblem> problems = new ArrayList<>();
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public T fileToMetaDataObject(QInstance qInstance, InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      this.fileName = fileName;
+      Map<String, Object> map = fileToMap(inputStream, fileName);
+      LoadingContext loadingContext = new LoadingContext(fileName, "/");
+      return (mapToMetaDataObject(qInstance, map, loadingContext));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public abstract T mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException;
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected Map<String, Object> fileToMap(InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      try
+      {
+         String string = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
+         string = StringUtils.ltrim(string);
+         if(fileName.toLowerCase().endsWith(".json"))
+         {
+            return JsonUtils.toObject(string, new TypeReference<>() {});
+         }
+         else if(fileName.toLowerCase().endsWith(".yaml") || fileName.toLowerCase().endsWith(".yml"))
+         {
+            return YamlUtils.toMap(string);
+         }
+
+         throw (new QMetaDataLoaderException("Unsupported file format (based on file name: " + fileName + ")"));
+      }
+      catch(IOException e)
+      {
+         throw new QMetaDataLoaderException("Error building map from file: " + fileName, e);
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   protected void reflectivelyMap(QInstance qInstance, QMetaDataObject targetObject, Map<String, Object> map, LoadingContext context)
+   {
+      Class<? extends QMetaDataObject> targetClass    = targetObject.getClass();
+      Set<String>                      usedFieldNames = new HashSet<>();
+
+      for(Method method : targetClass.getMethods())
+      {
+         try
+         {
+            if(method.getName().startsWith("set") && method.getParameterTypes().length == 1)
+            {
+               String propertyName = StringUtils.lcFirst(method.getName().substring(3));
+
+               if(map.containsKey(propertyName))
+               {
+                  usedFieldNames.add(propertyName);
+                  Class<?> parameterType = method.getParameterTypes()[0];
+                  Object   rawValue      = map.get(propertyName);
+
+                  try
+                  {
+                     Object mappedValue = reflectivelyMapValue(qInstance, method, parameterType, rawValue, context.descendToProperty(propertyName));
+                     method.invoke(targetObject, mappedValue);
+                  }
+                  catch(NoValueException nve)
+                  {
+                     ///////////////////////
+                     // don't call setter //
+                     ///////////////////////
+                     LOG.debug("at " + context + ": No value was mapped for property [" + propertyName + "] on " + targetClass.getSimpleName() + "." + method.getName() + ", raw value: [" + rawValue + "]");
+                  }
+               }
+            }
+         }
+         catch(Exception e)
+         {
+            addProblem(new LoadingProblem(context, "Error reflectively mapping on " + targetClass.getName() + "." + method.getName(), e));
+         }
+      }
+
+      //////////////////////////
+      // mmm, slightly sus... //
+      //////////////////////////
+      map.remove("class");
+      map.remove("version");
+
+      Set<String> unrecognizedKeys = new HashSet<>(map.keySet());
+      unrecognizedKeys.removeAll(usedFieldNames);
+
+      if(!unrecognizedKeys.isEmpty())
+      {
+         addProblem(new LoadingProblem(context, unrecognizedKeys.size() + " Unrecognized " + StringUtils.plural(unrecognizedKeys, "property", "properties") + ": " + unrecognizedKeys));
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public Object reflectivelyMapValue(QInstance qInstance, Method method, Class<?> parameterType, Object rawValue, LoadingContext context) throws Exception
+   {
+      if(rawValue instanceof String s && s.matches("^\\$\\{.+\\..+}"))
+      {
+         rawValue = new QMetaDataVariableInterpreter().interpret(s);
+         LOG.debug("Interpreted raw value [" + s + "] as [" + StringUtils.maskAndTruncate(ValueUtils.getValueAsString(rawValue) + "]"));
+      }
+
+      if(parameterType.equals(String.class))
+      {
+         return (getValueAsString(rawValue));
+      }
+      else if(parameterType.equals(Integer.class))
+      {
+         try
+         {
+            return (getValueAsInteger(rawValue));
+         }
+         catch(Exception e)
+         {
+            addProblem(new LoadingProblem(context, "[" + rawValue + "] is not an Integer value."));
+         }
+      }
+      else if(parameterType.equals(Boolean.class))
+      {
+         if("true".equals(rawValue) || Boolean.TRUE.equals(rawValue))
+         {
+            return (true);
+         }
+         else if("false".equals(rawValue) || Boolean.FALSE.equals(rawValue))
+         {
+            return (false);
+         }
+         else if(rawValue == null)
+         {
+            return (null);
+         }
+         else
+         {
+            addProblem(new LoadingProblem(context, "[" + rawValue + "] is not a boolean value (must be 'true' or 'false')."));
+            return (null);
+         }
+      }
+      else if(parameterType.equals(boolean.class))
+      {
+         if("true".equals(rawValue) || Boolean.TRUE.equals(rawValue))
+         {
+            return (true);
+         }
+         else if("false".equals(rawValue) || Boolean.FALSE.equals(rawValue))
+         {
+            return (false);
+         }
+         else
+         {
+            addProblem(new LoadingProblem(context, rawValue + " is not a boolean value (must be 'true' or 'false')."));
+            throw (new NoValueException());
+         }
+      }
+      else if(parameterType.equals(List.class))
+      {
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")List valueList)
+         {
+            List<Object> mappedValueList = new ArrayList<>();
+            for(Object o : valueList)
+            {
+               try
+               {
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, o, context);
+                  mappedValueList.add(mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave off list
+               }
+            }
+            return (mappedValueList);
+         }
+      }
+      else if(parameterType.equals(Set.class))
+      {
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")List valueList)
+         {
+            Set<Object> mappedValueSet = new LinkedHashSet<>();
+            for(Object o : valueList)
+            {
+               try
+               {
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, o, context);
+                  mappedValueSet.add(mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave off list
+               }
+            }
+            return (mappedValueSet);
+         }
+      }
+      else if(parameterType.equals(Map.class))
+      {
+         Type keyType = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         if(!keyType.equals(String.class))
+         {
+            addProblem(new LoadingProblem(context, "Unsupported key type for " + method + " got [" + keyType + "], expected [String]"));
+            throw new NoValueException();
+         }
+         // todo make sure string
+
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[1];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            Map<String, Object> mappedValueMap = new LinkedHashMap<>();
+            for(Object o : valueMap.entrySet())
+            {
+               try
+               {
+                  @SuppressWarnings("unchecked")
+                  Map.Entry<String, Object> entry = (Map.Entry<String, Object>) o;
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, entry.getValue(), context);
+                  mappedValueMap.put(entry.getKey(), mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave out of map
+               }
+            }
+            return (mappedValueMap);
+         }
+      }
+      else if(parameterType.isEnum())
+      {
+         String value = getValueAsString(rawValue);
+         for(Object enumConstant : parameterType.getEnumConstants())
+         {
+            if(((Enum<?>) enumConstant).name().equals(value))
+            {
+               return (enumConstant);
+            }
+         }
+
+         addProblem(new LoadingProblem(context, "Unrecognized value [" + rawValue + "].  Expected one of: " + Arrays.toString(parameterType.getEnumConstants())));
+      }
+      else if(MetaDataLoaderRegistry.hasLoaderForClass(parameterType))
+      {
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            Class<? extends AbstractMetaDataLoader<?>> loaderClass = MetaDataLoaderRegistry.getLoaderForClass(parameterType);
+            AbstractMetaDataLoader<?>                  loader      = loaderClass.getConstructor().newInstance();
+            //noinspection unchecked
+            return (loader.mapToMetaDataObject(qInstance, valueMap, context));
+         }
+      }
+      else if(QMetaDataObject.class.isAssignableFrom(parameterType))
+      {
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            QMetaDataObject childObject = (QMetaDataObject) parameterType.getConstructor().newInstance();
+            //noinspection unchecked
+            reflectivelyMap(qInstance, childObject, valueMap, context);
+            return (childObject);
+         }
+      }
+      else if(parameterType.equals(Serializable.class))
+      {
+         if(rawValue instanceof String
+            || rawValue instanceof Integer
+            || rawValue instanceof BigDecimal
+            || rawValue instanceof Boolean
+         )
+         {
+            return rawValue;
+         }
+      }
+      else
+      {
+         // todo clean up this message/level
+         addProblem(new LoadingProblem(context, "No case for " + parameterType + " (arg to: " + method + ")"));
+      }
+
+      throw new NoValueException();
+   }
+
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   // unclear if the below is needed.  if so, useful to not re-write, but is hurting test coverage, so zombie until used //
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected ListOfMapOrMapOfMap getListOfMapOrMapOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof List)
+   //      {
+   //         return (new ListOfMapOrMapOfMap((List<Map<String, Object>>) map.get(key)));
+   //      }
+   //      else if(map.get(key) instanceof Map)
+   //      {
+   //         return (new ListOfMapOrMapOfMap((Map<String, Map<String, Object>>) map.get(key)));
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected list or map under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected List<Map<String, Object>> getListOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof List)
+   //      {
+   //         return (List<Map<String, Object>>) map.get(key);
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected list under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected Map<String, Map<String, Object>> getMapOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof Map)
+   //      {
+   //         return (Map<String, Map<String, Object>>) map.get(key);
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected map under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // **
+   // ***************************************************************************/
+   //protected record ListOfMapOrMapOfMap(List<Map<String, Object>> listOf, Map<String, Map<String, Object>> mapOf)
+   //{
+   //   /*******************************************************************************
+   //    ** Constructor
+   //    **
+   //    *******************************************************************************/
+   //   public ListOfMapOrMapOfMap(List<Map<String, Object>> listOf)
+   //   {
+   //      this(listOf, null);
+   //   }
+
+   //   /*******************************************************************************
+   //    ** Constructor
+   //    **
+   //    *******************************************************************************/
+   //   public ListOfMapOrMapOfMap(Map<String, Map<String, Object>> mapOf)
+   //   {
+   //      this(null, mapOf);
+   //   }
+   //}
+
+
+
+   /*******************************************************************************
+    ** Getter for fileName
+    **
+    *******************************************************************************/
+   public String getFileName()
+   {
+      return fileName;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static class NoValueException extends Exception
+   {
+      /*******************************************************************************
+       ** Constructor
+       **
+       *******************************************************************************/
+      public NoValueException()
+      {
+         super("No value");
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public void addProblem(LoadingProblem problem)
+   {
+      problems.add(problem);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for problems
+    **
+    *******************************************************************************/
+   public List<LoadingProblem> getProblems()
+   {
+      return (problems);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/ClassDetectingMetaDataLoader.java

@@ -0,0 +1,120 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.InputStream;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.instances.loaders.implementations.GenericMetaDataLoader;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.utils.ClassPathUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.AnyKey;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
+
+
+/*******************************************************************************
+ ** Generic implementation of AbstractMetaDataLoader, who "detects" the class
+ ** of meta data object to be created, then defers to an appropriate subclass
+ ** to do the work.
+ *******************************************************************************/
+public class ClassDetectingMetaDataLoader extends AbstractMetaDataLoader<QMetaDataObject>
+{
+   private static final Memoization<AnyKey, List<Class<?>>> memoizedMetaDataObjectClasses = new Memoization<>();
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public AbstractMetaDataLoader<?> getLoaderForFile(InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      Map<String, Object> map = fileToMap(inputStream, fileName);
+      return (getLoaderForMap(map));
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public AbstractMetaDataLoader<?> getLoaderForMap(Map<String, Object> map) throws QMetaDataLoaderException
+   {
+      if(map.containsKey("class"))
+      {
+         String classProperty = ValueUtils.getValueAsString(map.get("class"));
+         try
+         {
+            if(MetaDataLoaderRegistry.hasLoaderForSimpleName(classProperty))
+            {
+               Class<? extends AbstractMetaDataLoader<?>> loaderClass = MetaDataLoaderRegistry.getLoaderForSimpleName(classProperty);
+               return (loaderClass.getConstructor().newInstance());
+            }
+            else
+            {
+               Optional<List<Class<?>>> metaDataClasses = memoizedMetaDataObjectClasses.getResult(AnyKey.getInstance(), k -> ClassPathUtils.getClassesContainingNameAndOfType("MetaData", QMetaDataObject.class));
+               if(metaDataClasses.isEmpty())
+               {
+                  throw (new QMetaDataLoaderException("Could not get list of metaDataObjects from class loader"));
+               }
+
+               for(Class<?> c : metaDataClasses.get())
+               {
+                  if(c.getSimpleName().equals(classProperty) && QMetaDataObject.class.isAssignableFrom(c))
+                  {
+                     @SuppressWarnings("unchecked")
+                     Class<? extends QMetaDataObject> metaDataClass = (Class<? extends QMetaDataObject>) c;
+                     return new GenericMetaDataLoader<>(metaDataClass);
+                  }
+               }
+            }
+            throw new QMetaDataLoaderException("Unexpected class [" + classProperty + "] (not a QMetaDataObject; doesn't have a registered MetaDataLoader) specified in " + getFileName());
+         }
+         catch(QMetaDataLoaderException qmdle)
+         {
+            throw (qmdle);
+         }
+         catch(Exception e)
+         {
+            throw new QMetaDataLoaderException("Error handling class [" + classProperty + "] specified in " + getFileName(), e);
+         }
+      }
+      else
+      {
+         throw new QMetaDataLoaderException("Cannot detect meta-data type, because [class] attribute was not specified in file: " + getFileName());
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QMetaDataObject mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      AbstractMetaDataLoader<?> loaderForMap = getLoaderForMap(map);
+      return loaderForMap.mapToMetaDataObject(qInstance, map, context);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/LoadingContext.java

@@ -0,0 +1,38 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ ** Record to track where loader objects are - e.g., what file they're on,
+ ** and at what property path within the file (e.g., helps report problems).
+ *******************************************************************************/
+public record LoadingContext(String fileName, String propertyPath)
+{
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public LoadingContext descendToProperty(String propertyName)
+   {
+      return new LoadingContext(fileName, propertyPath + propertyName + "/");
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/LoadingProblem.java

@@ -0,0 +1,49 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ ** record that tracks a problem that was encountered when loading files.
+ *******************************************************************************/
+public record LoadingProblem(LoadingContext context, String message, Exception exception) // todo Level if useful
+{
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public LoadingProblem(LoadingContext context, String message)
+   {
+      this(context, message, null);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String toString()
+   {
+      return "at[" + context.fileName() + "][" + context.propertyPath() + "]: " + message;
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/MetaDataLoaderHelper.java

@@ -0,0 +1,118 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.Pair;
+
+
+/*******************************************************************************
+ ** class that loads a directory full of meta data files into meta data objects,
+ ** and then sets all of them in a QInstance.
+ *******************************************************************************/
+public class MetaDataLoaderHelper
+{
+   private static final QLogger LOG = QLogger.getLogger(MetaDataLoaderHelper.class);
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public static void processAllMetaDataFilesInDirectory(QInstance qInstance, String path) throws QException
+   {
+      List<Pair<File, AbstractMetaDataLoader<?>>> loaders = new ArrayList<>();
+
+      File directory = new File(path);
+      processAllMetaDataFilesInDirectory(loaders, directory);
+
+      // todo - some version of sorting the loaders by type or possibly a sort field within the files (or file names)
+
+      for(Pair<File, AbstractMetaDataLoader<?>> pair : loaders)
+      {
+         File                      file   = pair.getA();
+         AbstractMetaDataLoader<?> loader = pair.getB();
+         try(FileInputStream fileInputStream = new FileInputStream(file))
+         {
+            QMetaDataObject qMetaDataObject = loader.fileToMetaDataObject(qInstance, fileInputStream, file.getName());
+
+            if(CollectionUtils.nullSafeHasContents(loader.getProblems()))
+            {
+               loader.getProblems().forEach(System.out::println);
+            }
+
+            if(qMetaDataObject instanceof TopLevelMetaDataInterface topLevelMetaData)
+            {
+               topLevelMetaData.addSelfToInstance(qInstance);
+            }
+            else
+            {
+               LOG.warn("Received a non-topLevelMetaDataObject from file: " + file.getAbsolutePath());
+            }
+         }
+         catch(Exception e)
+         {
+            LOG.error("Error processing file: " + file.getAbsolutePath(), e);
+         }
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   private static void processAllMetaDataFilesInDirectory(List<Pair<File, AbstractMetaDataLoader<?>>> loaders, File directory) throws QException
+   {
+      for(File file : Objects.requireNonNullElse(directory.listFiles(), new File[0]))
+      {
+         if(file.isDirectory())
+         {
+            processAllMetaDataFilesInDirectory(loaders, file);
+         }
+         else
+         {
+            try(FileInputStream fileInputStream = new FileInputStream(file))
+            {
+               AbstractMetaDataLoader<?> loader = new ClassDetectingMetaDataLoader().getLoaderForFile(fileInputStream, file.getName());
+               loaders.add(Pair.of(file, loader));
+            }
+            catch(Exception e)
+            {
+               LOG.error("Error processing file: " + file.getAbsolutePath(), e);
+            }
+         }
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/MetaDataLoaderRegistry.java

@@ -0,0 +1,120 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.lang.reflect.ParameterizedType;
+import java.lang.reflect.Type;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.implementations.QTableMetaDataLoader;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.utils.ClassPathUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class MetaDataLoaderRegistry
+{
+   private static final QLogger LOG = QLogger.getLogger(AbstractMetaDataLoader.class);
+
+   private static final Map<Class<?>, Class<? extends AbstractMetaDataLoader<?>>> registeredLoaders                   = new HashMap<>();
+   private static final Map<String, Class<? extends AbstractMetaDataLoader<?>>>   registeredLoadersByTargetSimpleName = new HashMap<>();
+
+   static
+   {
+      try
+      {
+         List<Class<?>> classesInPackage = ClassPathUtils.getClassesInPackage(QTableMetaDataLoader.class.getPackageName());
+         for(Class<?> possibleLoaderClass : classesInPackage)
+         {
+            try
+            {
+               Type superClass = possibleLoaderClass.getGenericSuperclass();
+               if(superClass.getTypeName().startsWith(AbstractMetaDataLoader.class.getName() + "<"))
+               {
+                  Type actualTypeArgument = ((ParameterizedType) superClass).getActualTypeArguments()[0];
+                  if(actualTypeArgument instanceof Class)
+                  {
+                     //noinspection unchecked
+                     Class<? extends AbstractMetaDataLoader<?>> loaderClass = (Class<? extends AbstractMetaDataLoader<?>>) possibleLoaderClass;
+
+                     Class<?> metaDataObjectType = Class.forName(actualTypeArgument.getTypeName());
+                     registeredLoaders.put(metaDataObjectType, loaderClass);
+                     registeredLoadersByTargetSimpleName.put(metaDataObjectType.getSimpleName(), loaderClass);
+                  }
+               }
+            }
+            catch(Exception e)
+            {
+               LOG.info("Error on class: " + possibleLoaderClass, e);
+            }
+         }
+
+         System.out.println("Registered loaders: " + registeredLoadersByTargetSimpleName);
+      }
+      catch(Exception e)
+      {
+         LOG.error("Error in static init block for MetaDataLoaderRegistry", e);
+      }
+   }
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static boolean hasLoaderForClass(Class<?> metaDataClass)
+   {
+      return registeredLoaders.containsKey(metaDataClass);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static Class<? extends AbstractMetaDataLoader<?>> getLoaderForClass(Class<?> metaDataClass)
+   {
+      return registeredLoaders.get(metaDataClass);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static boolean hasLoaderForSimpleName(String targetSimpleName)
+   {
+      return registeredLoadersByTargetSimpleName.containsKey(targetSimpleName);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static Class<? extends AbstractMetaDataLoader<?>> getLoaderForSimpleName(String targetSimpleName)
+   {
+      return registeredLoadersByTargetSimpleName.get(targetSimpleName);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/QMetaDataLoaderException.java

@@ -0,0 +1,50 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QMetaDataLoaderException extends Exception
+{
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QMetaDataLoaderException(String message)
+   {
+      super(message);
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QMetaDataLoaderException(String message, Throwable cause)
+   {
+      super(message, cause);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/GenericMetaDataLoader.java

@@ -0,0 +1,71 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class GenericMetaDataLoader<T extends QMetaDataObject> extends AbstractMetaDataLoader<T>
+{
+   private final Class<T> metaDataClass;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public GenericMetaDataLoader(Class<T> metaDataClass)
+   {
+      this.metaDataClass = metaDataClass;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public T mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      try
+      {
+         T object = metaDataClass.getConstructor().newInstance();
+         reflectivelyMap(qInstance, object, map, context);
+         return (object);
+      }
+      catch(Exception e)
+      {
+         throw (new QMetaDataLoaderException("Error loading metaData object of type " + metaDataClass.getSimpleName(), e));
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/QStepDataLoader.java

@@ -0,0 +1,85 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QStepMetaData;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QStepDataLoader extends AbstractMetaDataLoader<QStepMetaData>
+{
+   private static final QLogger LOG = QLogger.getLogger(QStepDataLoader.class);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QStepMetaData mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      String stepType = ValueUtils.getValueAsString(map.get("stepType"));
+
+      if(!StringUtils.hasContent(stepType))
+      {
+         throw (new QMetaDataLoaderException("stepType was not specified for process step"));
+      }
+
+      QStepMetaData step;
+      if("backend".equalsIgnoreCase(stepType))
+      {
+         step = new QBackendStepMetaData();
+         reflectivelyMap(qInstance, step, map, context);
+      }
+      else if("frontend".equalsIgnoreCase(stepType))
+      {
+         step = new QFrontendStepMetaData();
+         reflectivelyMap(qInstance, step, map, context);
+      }
+      // todo - we have custom factory methods for this, so, maybe needs all custom loader?
+      // else if("stateMachine".equalsIgnoreCase(stepType))
+      // {
+      //    step = new QStateMachineStep();
+      //    reflectivelyMap(qInstance, step, map, context);
+      // }
+      else
+      {
+         throw (new QMetaDataLoaderException("Unsupported step stepType: " + stepType));
+      }
+
+      return (step);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/QTableMetaDataLoader.java

@@ -0,0 +1,58 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QTableMetaDataLoader extends AbstractMetaDataLoader<QTableMetaData>
+{
+   private static final QLogger LOG = QLogger.getLogger(QTableMetaDataLoader.class);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QTableMetaData mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      QTableMetaData table = new QTableMetaData();
+
+      reflectivelyMap(qInstance, table, map, context);
+
+      // todo - handle QTableBackendDetails, based on backend's type
+
+      return (table);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/validation/plugins/QInstanceValidatorPluginInterface.java

@@ -38,4 +38,13 @@ public interface QInstanceValidatorPluginInterface<T>
     *******************************************************************************/
    void validate(T object, QInstance qInstance, QInstanceValidator qInstanceValidator);
 
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   default String getPluginIdentifier()
+   {
+      return getClass().getName();
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/ProcessSummaryLineInterface.java

@@ -23,7 +23,11 @@ package com.kingsrook.qqq.backend.core.model.actions.processes;
 
 
 import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.List;
 import com.kingsrook.qqq.backend.core.logging.LogPair;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -31,6 +35,45 @@ import com.kingsrook.qqq.backend.core.logging.LogPair;
  *******************************************************************************/
 public interface ProcessSummaryLineInterface extends Serializable
 {
+   QLogger LOG = QLogger.getLogger(ProcessSummaryLineInterface.class);
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   static void log(String message, Serializable summaryLines, List<LogPair> additionalLogPairs)
+   {
+      try
+      {
+         if(summaryLines instanceof List)
+         {
+            List<ProcessSummaryLineInterface> list = (List<ProcessSummaryLineInterface>) summaryLines;
+
+            List<LogPair> logPairs = new ArrayList<>();
+            for(ProcessSummaryLineInterface processSummaryLineInterface : list)
+            {
+               LogPair logPair = processSummaryLineInterface.toLogPair();
+               logPair.setKey(logPair.getKey() + logPairs.size());
+               logPairs.add(logPair);
+            }
+
+            if(additionalLogPairs != null)
+            {
+               logPairs.addAll(0, additionalLogPairs);
+            }
+            logPairs.add(0, logPair("message", message));
+
+            LOG.info(logPairs);
+         }
+         else
+         {
+            LOG.info("Unrecognized type for summaryLines (expected List)", logPair("processSummary", summaryLines));
+         }
+      }
+      catch(Exception e)
+      {
+         LOG.info("Error logging a process summary", e, logPair("processSummary", summaryLines));
+      }
+   }
 
    /*******************************************************************************
     ** Getter for status

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/QProcessPayload.java

@@ -0,0 +1,162 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.processes;
+
+
+import java.io.Serializable;
+import java.lang.reflect.Method;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QRuntimeException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntityField;
+import com.kingsrook.qqq.backend.core.utils.ListingHash;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
+
+
+/*******************************************************************************
+ ** base-class for bean-like classes to represent the fields of a process.
+ ** similar in spirit to QRecordEntity, but for processes.
+ *******************************************************************************/
+public class QProcessPayload
+{
+   private static final QLogger LOG = QLogger.getLogger(QProcessPayload.class);
+
+   private static final ListingHash<Class<? extends QProcessPayload>, QRecordEntityField> fieldMapping = new ListingHash<>();
+
+
+
+   /*******************************************************************************
+    ** Build an entity of this QRecord type from a QRecord
+    **
+    *******************************************************************************/
+   public static <T extends QProcessPayload> T fromProcessState(Class<T> c, ProcessState processState) throws QException
+   {
+      try
+      {
+         T entity = c.getConstructor().newInstance();
+         entity.populateFromProcessState(processState);
+         return (entity);
+      }
+      catch(Exception e)
+      {
+         throw (new QException("Error building process payload from state.", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected void populateFromProcessState(ProcessState processState)
+   {
+      try
+      {
+         List<QRecordEntityField> fieldList = getFieldList(this.getClass());
+
+         for(QRecordEntityField qRecordEntityField : fieldList)
+         {
+            Serializable value      = processState.getValues().get(qRecordEntityField.getFieldName());
+            Object       typedValue = qRecordEntityField.convertValueType(value);
+            qRecordEntityField.getSetter().invoke(this, typedValue);
+         }
+      }
+      catch(Exception e)
+      {
+         throw (new QRuntimeException("Error building process payload from process state.", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Copy the values from this payload into the given process state.
+    ** ALL fields in the entity will be set in the process state.
+    **
+    *******************************************************************************/
+   public void toProcessState(ProcessState processState) throws QRuntimeException
+   {
+      try
+      {
+         for(QRecordEntityField qRecordEntityField : getFieldList(this.getClass()))
+         {
+            processState.getValues().put(qRecordEntityField.getFieldName(), (Serializable) qRecordEntityField.getGetter().invoke(this));
+         }
+      }
+      catch(Exception e)
+      {
+         throw (new QRuntimeException("Error populating process state from process payload.", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public static Set<Class<?>> allowedFieldTypes()
+   {
+      HashSet<Class<?>> classes = new HashSet<>(ReflectiveBeanLikeClassUtils.defaultAllowedTypes());
+      classes.add(Map.class);
+      classes.add(List.class);
+      return (classes);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static List<QRecordEntityField> getFieldList(Class<? extends QProcessPayload> c)
+   {
+      if(!fieldMapping.containsKey(c))
+      {
+         List<QRecordEntityField> fieldList = new ArrayList<>();
+         for(Method possibleGetter : c.getMethods())
+         {
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, false, allowedFieldTypes()))
+            {
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
+
+               if(setter.isPresent())
+               {
+                  String fieldName = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
+                  fieldList.add(new QRecordEntityField(fieldName, possibleGetter, setter.get(), possibleGetter.getReturnType(), null));
+               }
+               else
+               {
+                  LOG.debug("Getter method [" + possibleGetter.getName() + "] does not have a corresponding setter.");
+               }
+            }
+         }
+         fieldMapping.put(c, fieldList);
+      }
+      return (fieldMapping.get(c));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunBackendStepInput.java

@@ -628,4 +628,15 @@ public class RunBackendStepInput extends AbstractActionInput
    {
       return (QContext.getQInstance().getProcess(getProcessName()));
    }
+
+
+
+   /***************************************************************************
+    ** return a QProcessPayload subclass instance, with values populated from
+    ** the current process state.
+    ***************************************************************************/
+   public <T extends QProcessPayload> T getProcessPayload(Class<T> payloadClass) throws QException
+   {
+      return QProcessPayload.fromProcessState(payloadClass, getProcessState());
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunBackendStepOutput.java

@@ -445,4 +445,14 @@ public class RunBackendStepOutput extends AbstractActionOutput implements Serial
       this.processState.setProcessMetaDataAdjustment(processMetaDataAdjustment);
    }
 
+
+
+   /***************************************************************************
+    ** Update the process state with values from the input processPayload
+    ** subclass instance.
+    ***************************************************************************/
+   public void setProcessPayload(QProcessPayload processPayload)
+   {
+      processPayload.toProcessState(getProcessState());
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/ActionFlag.java

@@ -0,0 +1,35 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.tables;
+
+
+import java.io.Serializable;
+
+
+/*******************************************************************************
+ ** interface to mark enums (presumably classes too, but the original intent is
+ ** enums) that can be added to insert/update/delete action inputs to flag behaviors
+ *******************************************************************************/
+public interface ActionFlag extends Serializable
+{
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/delete/DeleteInput.java

@@ -24,9 +24,12 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.delete;
 
 import java.io.Serializable;
 import java.util.ArrayList;
+import java.util.HashSet;
 import java.util.List;
+import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
 import com.kingsrook.qqq.backend.core.model.actions.tables.InputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
@@ -47,6 +50,8 @@ public class DeleteInput extends AbstractTableActionInput
    private boolean omitDmlAudit = false;
    private String  auditContext = null;
 
+   private Set<ActionFlag> flags;
+
 
 
    /*******************************************************************************
@@ -295,4 +300,65 @@ public class DeleteInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for flags
+    *******************************************************************************/
+   public Set<ActionFlag> getFlags()
+   {
+      return (this.flags);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for flags
+    *******************************************************************************/
+   public void setFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for flags
+    *******************************************************************************/
+   public DeleteInput withFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public DeleteInput withFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         this.flags = new HashSet<>();
+      }
+      this.flags.add(flag);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public boolean hasFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         return (false);
+      }
+
+      return (this.flags.contains(flag));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/insert/InsertInput.java

@@ -23,9 +23,12 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.insert;
 
 
 import java.util.ArrayList;
+import java.util.HashSet;
 import java.util.List;
+import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
 import com.kingsrook.qqq.backend.core.model.actions.tables.InputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -48,6 +51,8 @@ public class InsertInput extends AbstractTableActionInput
    private boolean omitDmlAudit = false;
    private String  auditContext = null;
 
+   private Set<ActionFlag> flags;
+
 
 
    /*******************************************************************************
@@ -316,4 +321,65 @@ public class InsertInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for flags
+    *******************************************************************************/
+   public Set<ActionFlag> getFlags()
+   {
+      return (this.flags);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for flags
+    *******************************************************************************/
+   public void setFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for flags
+    *******************************************************************************/
+   public InsertInput withFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public InsertInput withFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         this.flags = new HashSet<>();
+      }
+      this.flags.add(flag);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public boolean hasFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         return (false);
+      }
+
+      return (this.flags.contains(flag));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QFilterCriteria.java

@@ -33,6 +33,7 @@ import java.util.Set;
 import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.serialization.QFilterCriteriaDeserializer;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
@@ -42,7 +43,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  *
  *******************************************************************************/
 @JsonDeserialize(using = QFilterCriteriaDeserializer.class)
-public class QFilterCriteria implements Serializable, Cloneable
+public class QFilterCriteria implements Serializable, Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QFilterCriteria.class);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QFilterOrderBy.java

@@ -23,13 +23,14 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query;
 
 
 import java.io.Serializable;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Bean representing an element of a query order-by clause.
  **
  *******************************************************************************/
-public class QFilterOrderBy implements Serializable, Cloneable
+public class QFilterOrderBy implements Serializable, Cloneable, QMetaDataObject
 {
    private String  fieldName;
    private boolean isAscending = true;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QQueryFilter.java

@@ -36,6 +36,7 @@ import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.AbstractFilterExpression;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.FilterVariableExpression;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
@@ -45,7 +46,7 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  * Full "filter" for a query - a list of criteria and order-bys
  *
  *******************************************************************************/
-public class QQueryFilter implements Serializable, Cloneable
+public class QQueryFilter implements Serializable, Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QQueryFilter.class);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/replace/ReplaceInput.java

@@ -22,9 +22,12 @@
 package com.kingsrook.qqq.backend.core.model.actions.tables.replace;
 
 
+import java.util.HashSet;
 import java.util.List;
+import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
@@ -39,12 +42,14 @@ public class ReplaceInput extends AbstractTableActionInput
    private UniqueKey           key;
    private List<QRecord>       records;
    private QQueryFilter        filter;
-   private boolean performDeletes                 = true;
-   private boolean allowNullKeyValuesToEqual      = false;
-   private boolean setPrimaryKeyInInsertedRecords = false;
+   private boolean             performDeletes                 = true;
+   private boolean             allowNullKeyValuesToEqual      = false;
+   private boolean             setPrimaryKeyInInsertedRecords = false;
 
    private boolean omitDmlAudit = false;
 
+   private Set<ActionFlag> flags;
+
 
 
    /*******************************************************************************
@@ -303,4 +308,65 @@ public class ReplaceInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for flags
+    *******************************************************************************/
+   public Set<ActionFlag> getFlags()
+   {
+      return (this.flags);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for flags
+    *******************************************************************************/
+   public void setFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for flags
+    *******************************************************************************/
+   public ReplaceInput withFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public ReplaceInput withFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         this.flags = new HashSet<>();
+      }
+      this.flags.add(flag);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public boolean hasFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         return (false);
+      }
+
+      return (this.flags.contains(flag));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/update/UpdateInput.java

@@ -23,9 +23,12 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.update;
 
 
 import java.util.ArrayList;
+import java.util.HashSet;
 import java.util.List;
+import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
 import com.kingsrook.qqq.backend.core.model.actions.tables.InputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -56,6 +59,8 @@ public class UpdateInput extends AbstractTableActionInput
    private boolean omitModifyDateUpdate      = false;
    private String  auditContext              = null;
 
+   private Set<ActionFlag> flags;
+
 
 
    /*******************************************************************************
@@ -385,4 +390,65 @@ public class UpdateInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for flags
+    *******************************************************************************/
+   public Set<ActionFlag> getFlags()
+   {
+      return (this.flags);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for flags
+    *******************************************************************************/
+   public void setFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for flags
+    *******************************************************************************/
+   public UpdateInput withFlags(Set<ActionFlag> flags)
+   {
+      this.flags = flags;
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public UpdateInput withFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         this.flags = new HashSet<>();
+      }
+      this.flags.add(flag);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public boolean hasFlag(ActionFlag flag)
+   {
+      if(this.flags == null)
+      {
+         return (false);
+      }
+
+      return (this.flags.contains(flag));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/common/TimeZonePossibleValueSourceMetaDataProvider.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.model.common;
 
 
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.List;
 import java.util.TimeZone;
 import java.util.function.Function;
@@ -47,7 +48,7 @@ public class TimeZonePossibleValueSourceMetaDataProvider
     *******************************************************************************/
    public QPossibleValueSource produce()
    {
-      return (produce(null, null));
+      return (produce(null, null, null));
    }
 
 
@@ -56,6 +57,16 @@ public class TimeZonePossibleValueSourceMetaDataProvider
     **
     *******************************************************************************/
    public QPossibleValueSource produce(Predicate<String> filter, Function<String, String> labelMapper)
+   {
+      return (produce(filter, labelMapper, null));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QPossibleValueSource produce(Predicate<String> filter, Function<String, String> labelMapper, Comparator<QPossibleValue<?>> comparator)
    {
       QPossibleValueSource possibleValueSource = new QPossibleValueSource()
          .withName("timeZones")
@@ -72,6 +83,11 @@ public class TimeZonePossibleValueSourceMetaDataProvider
          }
       }
 
+      if(comparator != null)
+      {
+         enumValues.sort(comparator);
+      }
+
       possibleValueSource.withEnumValues(enumValues);
       return (possibleValueSource);
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEntity.java

@@ -49,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.ObjectUtils;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -325,13 +326,13 @@ public abstract class QRecordEntity
          List<QRecordEntityField> fieldList = new ArrayList<>();
          for(Method possibleGetter : c.getMethods())
          {
-            if(isGetter(possibleGetter))
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, true))
             {
-               Optional<Method> setter = getSetterForGetter(c, possibleGetter);
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
 
                if(setter.isPresent())
                {
-                  String           fieldName       = getFieldNameFromGetter(possibleGetter);
+                  String           fieldName       = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
                   Optional<QField> fieldAnnotation = getQFieldAnnotation(c, fieldName);
 
                   if(fieldAnnotation.isPresent())
@@ -378,19 +379,19 @@ public abstract class QRecordEntity
          List<QRecordEntityAssociation> associationList = new ArrayList<>();
          for(Method possibleGetter : c.getMethods())
          {
-            if(isGetter(possibleGetter))
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, true))
             {
-               Optional<Method> setter = getSetterForGetter(c, possibleGetter);
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
 
                if(setter.isPresent())
                {
-                  String                 fieldName             = getFieldNameFromGetter(possibleGetter);
+                  String                 fieldName             = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
                   Optional<QAssociation> associationAnnotation = getQAssociationAnnotation(c, fieldName);
 
                   if(associationAnnotation.isPresent())
                   {
                      @SuppressWarnings("unchecked")
-                     Class<? extends QRecordEntity> listTypeParam = (Class<? extends QRecordEntity>) getListTypeParam(possibleGetter.getReturnType(), possibleGetter.getAnnotatedReturnType());
+                     Class<? extends QRecordEntity> listTypeParam = (Class<? extends QRecordEntity>) ReflectiveBeanLikeClassUtils.getListTypeParam(possibleGetter.getReturnType(), possibleGetter.getAnnotatedReturnType());
                      associationList.add(new QRecordEntityAssociation(fieldName, possibleGetter, setter.get(), listTypeParam, associationAnnotation.orElse(null)));
                   }
                }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEntityField.java

@@ -28,6 +28,7 @@ import java.math.BigDecimal;
 import java.time.Instant;
 import java.time.LocalDate;
 import java.time.LocalTime;
+import java.util.Map;
 import com.kingsrook.qqq.backend.core.exceptions.QValueException;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
@@ -170,6 +171,11 @@ public class QRecordEntityField
          {
             return (ValueUtils.getValueAsByteArray(value));
          }
+
+         if(type.equals(Map.class))
+         {
+            return (ValueUtils.getValueAsMap(value));
+         }
       }
       catch(Exception e)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducerHelper.java

@@ -177,6 +177,18 @@ public class MetaDataProducerHelper
       /////////////////////////////////////////////////////////////////////////////////////////////
       // sort them by sort order, then by the type that they return, as set up in the static map //
       /////////////////////////////////////////////////////////////////////////////////////////////
+      sortMetaDataProducers(producers);
+
+      return (producers);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static void sortMetaDataProducers(List<MetaDataProducerInterface<?>> producers)
+   {
       producers.sort(Comparator
          .comparing((MetaDataProducerInterface<?> p) -> p.getSortOrder())
          .thenComparing((MetaDataProducerInterface<?> p) ->
@@ -191,11 +203,10 @@ public class MetaDataProducerHelper
                return (0);
             }
          }));
-
-      return (producers);
    }
 
 
+
    /*******************************************************************************
     ** Recursively find all classes in the given package, that implement MetaDataProducerInterface
     ** run them, and add their output to the given qInstance.
@@ -417,7 +428,7 @@ public class MetaDataProducerHelper
          return (null);
       }
 
-      ChildJoinFromRecordEntityGenericMetaDataProducer producer = new ChildJoinFromRecordEntityGenericMetaDataProducer(childTableName, parentTableName, possibleValueFieldName, childTable.childJoin().orderBy());
+      ChildJoinFromRecordEntityGenericMetaDataProducer producer = new ChildJoinFromRecordEntityGenericMetaDataProducer(childTableName, parentTableName, possibleValueFieldName, childTable.childJoin().orderBy(), childTable.childJoin().isOneToOne());
       producer.setSourceClass(entityClass);
       return producer;
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducerMultiOutput.java

@@ -86,7 +86,7 @@ public class MetaDataProducerMultiOutput implements MetaDataProducerOutput, Sour
    {
       List<T> rs = new ArrayList<>();
 
-      for(MetaDataProducerOutput content : contents)
+      for(MetaDataProducerOutput content : CollectionUtils.nonNullList(contents))
       {
          if(content instanceof MetaDataProducerMultiOutput multiOutput)
          {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QAuthenticationType.java

@@ -28,6 +28,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
  *******************************************************************************/
 public enum QAuthenticationType
 {
+   OAUTH2("OAuth2"),
    AUTH_0("auth0"),
    TABLE_BASED("tableBased"),
    FULLY_ANONYMOUS("fullyAnonymous"),

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QInstance.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
 
 
 import java.util.ArrayList;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.LinkedHashMap;
 import java.util.LinkedHashSet;
@@ -30,6 +31,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Set;
 import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.metadata.JoinGraph;
 import com.kingsrook.qqq.backend.core.actions.metadata.MetaDataAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -65,6 +67,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.scheduler.schedulable.SchedulableType;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import io.github.cdimascio.dotenv.Dotenv;
 import io.github.cdimascio.dotenv.DotenvEntry;
@@ -116,6 +119,8 @@ public class QInstance
    private QPermissionRules defaultPermissionRules = QPermissionRules.defaultInstance();
    private QAuditRules      defaultAuditRules      = QAuditRules.defaultInstanceLevelNone();
 
+   private ListingHash<String, QCodeReference> tableCustomizers;
+
    @Deprecated(since = "migrated to metaDataCustomizer")
    private QCodeReference metaDataFilter = null;
 
@@ -1250,7 +1255,7 @@ public class QInstance
       {
          this.supplementalMetaData = new HashMap<>();
       }
-      this.supplementalMetaData.put(supplementalMetaData.getType(), supplementalMetaData);
+      this.supplementalMetaData.put(supplementalMetaData.getName(), supplementalMetaData);
       return (this);
    }
 
@@ -1623,4 +1628,76 @@ public class QInstance
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for tableCustomizers
+    *******************************************************************************/
+   public ListingHash<String, QCodeReference> getTableCustomizers()
+   {
+      return (this.tableCustomizers);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tableCustomizers
+    *******************************************************************************/
+   public void setTableCustomizers(ListingHash<String, QCodeReference> tableCustomizers)
+   {
+      this.tableCustomizers = tableCustomizers;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tableCustomizers
+    *******************************************************************************/
+   public QInstance withTableCustomizers(ListingHash<String, QCodeReference> tableCustomizers)
+   {
+      this.tableCustomizers = tableCustomizers;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QInstance withTableCustomizer(String role, QCodeReference customizer)
+   {
+      if(this.tableCustomizers == null)
+      {
+         this.tableCustomizers = new ListingHash<>();
+      }
+
+      this.tableCustomizers.add(role, customizer);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QInstance withTableCustomizer(TableCustomizers tableCustomizer, QCodeReference customizer)
+   {
+      return (withTableCustomizer(tableCustomizer.getRole(), customizer));
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tableCustomizers
+    *******************************************************************************/
+   public List<QCodeReference> getTableCustomizers(TableCustomizers tableCustomizer)
+   {
+      if(this.tableCustomizers == null)
+      {
+         return (Collections.emptyList());
+      }
+
+      return (this.tableCustomizers.getOrDefault(tableCustomizer.getRole(), Collections.emptyList()));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QMetaDataObject.java

@@ -0,0 +1,34 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata;
+
+
+import java.io.Serializable;
+
+
+/*******************************************************************************
+ ** interface common among all objects that can be considered qqq meta data -
+ ** e.g., stored in a QInstance.
+ *******************************************************************************/
+public interface QMetaDataObject extends Serializable
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QSupplementalInstanceMetaData.java

@@ -22,28 +22,21 @@
 package com.kingsrook.qqq.backend.core.model.metadata;
 
 
+import java.util.function.Supplier;
 import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
-import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 
 
 /*******************************************************************************
  ** Base-class for instance-level meta-data defined by some supplemental module, etc,
  ** outside of qqq core
  *******************************************************************************/
-public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataInterface
+public interface QSupplementalInstanceMetaData extends TopLevelMetaDataInterface
 {
 
-   /*******************************************************************************
-    ** Getter for type
-    *******************************************************************************/
-   public abstract String getType();
-
-
-
    /*******************************************************************************
     **
     *******************************************************************************/
-   public void enrich(QTableMetaData table)
+   default void enrich(QInstance qInstance)
    {
       ////////////////////////
       // noop in base class //
@@ -55,7 +48,7 @@ public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataI
    /*******************************************************************************
     **
     *******************************************************************************/
-   public void validate(QInstance qInstance, QInstanceValidator validator)
+   default void validate(QInstance qInstance, QInstanceValidator validator)
    {
       ////////////////////////
       // noop in base class //
@@ -68,9 +61,33 @@ public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataI
     **
     *******************************************************************************/
    @Override
-   public void addSelfToInstance(QInstance qInstance)
+   default void addSelfToInstance(QInstance qInstance)
    {
       qInstance.withSupplementalMetaData(this);
    }
 
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   static <S extends QSupplementalInstanceMetaData> S of(QInstance qInstance, String name)
+   {
+      return ((S) qInstance.getSupplementalMetaData(name));
+   }
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   static <S extends QSupplementalInstanceMetaData> S ofOrWithNew(QInstance qInstance, String name, Supplier<S> supplier)
+   {
+      S s = (S) qInstance.getSupplementalMetaData(name);
+      if(s == null)
+      {
+         s = supplier.get();
+         s.addSelfToInstance(qInstance);
+      }
+      return (s);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/TopLevelMetaDataInterface.java

@@ -26,7 +26,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
  ** Interface for meta-data classes that can be added directly (e.g, at the top
  ** level) to a QInstance (such as a QTableMetaData - not a QFieldMetaData).
  *******************************************************************************/
-public interface TopLevelMetaDataInterface extends MetaDataProducerOutput
+public interface TopLevelMetaDataInterface extends MetaDataProducerOutput, QMetaDataObject
 {
 
    /*******************************************************************************

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/audits/QAuditRules.java

@@ -22,10 +22,13 @@
 package com.kingsrook.qqq.backend.core.model.metadata.audits;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QAuditRules
+public class QAuditRules implements QMetaDataObject
 {
    private AuditLevel auditLevel;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/authentication/OAuth2AuthenticationMetaData.java

@@ -0,0 +1,320 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2022.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.authentication;
+
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.backend.core.modules.authentication.implementations.OAuth2AuthenticationModule;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ ** Meta-data to provide details of an OAuth2 Authentication module
+ *******************************************************************************/
+public class OAuth2AuthenticationMetaData extends QAuthenticationMetaData
+{
+   private String baseUrl;
+   private String tokenUrl;
+   private String clientId;
+   private String scopes;
+
+   private String userSessionTableName;
+   private String redirectStateTableName;
+
+   ////////////////////////////////////////////////////////////////////////////////////////
+   // keep this secret, on the server - don't let it be serialized and sent to a client! //
+   ////////////////////////////////////////////////////////////////////////////////////////
+   @JsonIgnore
+   private String clientSecret;
+
+
+
+   /*******************************************************************************
+    ** Default Constructor.
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData()
+   {
+      super();
+      setType(QAuthenticationType.OAUTH2);
+
+      //////////////////////////////////////////////////////////
+      // ensure this module is registered with the dispatcher //
+      //////////////////////////////////////////////////////////
+      QAuthenticationModuleDispatcher.registerModule(QAuthenticationType.OAUTH2.getName(), OAuth2AuthenticationModule.class.getName());
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public void validate(QInstance qInstance, QInstanceValidator qInstanceValidator)
+   {
+      super.validate(qInstance, qInstanceValidator);
+
+      String prefix = "OAuth2AuthenticationMetaData (named '" + getName() + "'): ";
+
+      qInstanceValidator.assertCondition(StringUtils.hasContent(baseUrl), prefix + "baseUrl must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(clientId), prefix + "clientId must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(clientSecret), prefix + "clientSecret must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(scopes), prefix + "scopes must be set");
+
+      if(qInstanceValidator.assertCondition(StringUtils.hasContent(userSessionTableName), prefix + "userSessionTableName must be set"))
+      {
+         qInstanceValidator.assertCondition(qInstance.getTable(userSessionTableName) != null, prefix + "userSessionTableName ('" + userSessionTableName + "') was not found in the instance");
+      }
+
+      if(qInstanceValidator.assertCondition(StringUtils.hasContent(redirectStateTableName), prefix + "redirectStateTableName must be set"))
+      {
+         qInstanceValidator.assertCondition(qInstance.getTable(redirectStateTableName) != null, prefix + "redirectStateTableName ('" + redirectStateTableName + "') was not found in the instance");
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withBaseUrl(String baseUrl)
+   {
+      setBaseUrl(baseUrl);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for baseUrl
+    **
+    *******************************************************************************/
+   public String getBaseUrl()
+   {
+      return baseUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for baseUrl
+    **
+    *******************************************************************************/
+   public void setBaseUrl(String baseUrl)
+   {
+      this.baseUrl = baseUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withClientId(String clientId)
+   {
+      setClientId(clientId);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientId
+    **
+    *******************************************************************************/
+   public String getClientId()
+   {
+      return clientId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for clientId
+    **
+    *******************************************************************************/
+   public void setClientId(String clientId)
+   {
+      this.clientId = clientId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withClientSecret(String clientSecret)
+   {
+      setClientSecret(clientSecret);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientSecret
+    **
+    *******************************************************************************/
+   public String getClientSecret()
+   {
+      return clientSecret;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for clientSecret
+    **
+    *******************************************************************************/
+   public void setClientSecret(String clientSecret)
+   {
+      this.clientSecret = clientSecret;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tokenUrl
+    *******************************************************************************/
+   public String getTokenUrl()
+   {
+      return (this.tokenUrl);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tokenUrl
+    *******************************************************************************/
+   public void setTokenUrl(String tokenUrl)
+   {
+      this.tokenUrl = tokenUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tokenUrl
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withTokenUrl(String tokenUrl)
+   {
+      this.tokenUrl = tokenUrl;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for userSessionTableName
+    *******************************************************************************/
+   public String getUserSessionTableName()
+   {
+      return (this.userSessionTableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for userSessionTableName
+    *******************************************************************************/
+   public void setUserSessionTableName(String userSessionTableName)
+   {
+      this.userSessionTableName = userSessionTableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for userSessionTableName
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withUserSessionTableName(String userSessionTableName)
+   {
+      this.userSessionTableName = userSessionTableName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for redirectStateTableName
+    *******************************************************************************/
+   public String getRedirectStateTableName()
+   {
+      return (this.redirectStateTableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for redirectStateTableName
+    *******************************************************************************/
+   public void setRedirectStateTableName(String redirectStateTableName)
+   {
+      this.redirectStateTableName = redirectStateTableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for redirectStateTableName
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withRedirectStateTableName(String redirectStateTableName)
+   {
+      this.redirectStateTableName = redirectStateTableName;
+      return (this);
+   }
+
+
+   /*******************************************************************************
+    ** Getter for scopes
+    *******************************************************************************/
+   public String getScopes()
+   {
+      return (this.scopes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for scopes
+    *******************************************************************************/
+   public void setScopes(String scopes)
+   {
+      this.scopes = scopes;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for scopes
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withScopes(String scopes)
+   {
+      this.scopes = scopes;
+      return (this);
+   }
+
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/authentication/QAuthenticationMetaData.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.authentication;
 import java.util.HashMap;
 import java.util.Map;
 import com.fasterxml.jackson.annotation.JsonFilter;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
 import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
@@ -225,4 +226,15 @@ public class QAuthenticationMetaData implements TopLevelMetaDataInterface
       return (this);
    }
 
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public void validate(QInstance qInstance, QInstanceValidator qInstanceValidator)
+   {
+      //////////////////
+      // noop at base //
+      //////////////////
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/code/QCodeReference.java

@@ -23,13 +23,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.code;
 
 
 import java.io.Serializable;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Pointer to code to be ran by the qqq framework, e.g., for custom behavior -
  ** maybe process steps, maybe customization to a table, etc.
  *******************************************************************************/
-public class QCodeReference implements Serializable, Cloneable
+public class QCodeReference implements Serializable, Cloneable, QMetaDataObject
 {
    private String    name;
    private QCodeType codeType;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/fields/QFieldMetaData.java

@@ -40,11 +40,13 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.data.QField;
 import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -54,7 +56,7 @@ import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
  ** Meta-data to represent a single field in a table.
  **
  *******************************************************************************/
-public class QFieldMetaData implements Cloneable
+public class QFieldMetaData implements Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QFieldMetaData.class);
 
@@ -187,7 +189,7 @@ public class QFieldMetaData implements Cloneable
    {
       try
       {
-         this.name = QRecordEntity.getFieldNameFromGetter(getter);
+         this.name = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(getter);
          this.type = QFieldType.fromClass(getter.getReturnType());
 
          @SuppressWarnings("unchecked")

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/help/QHelpContent.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.help;
 import java.util.Collections;
 import java.util.HashSet;
 import java.util.Set;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
@@ -41,7 +42,7 @@ import java.util.Set;
  ** May be dynamically added to meta-data via (non-meta-) data - see
  ** HelpContentMetaDataProvider and QInstanceHelpContentManager
  *******************************************************************************/
-public class QHelpContent
+public class QHelpContent implements QMetaDataObject
 {
    private String        content;
    private HelpFormat    format;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QAppChildMetaData.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Interface shared by meta-data objects which can be placed into an App.
  ** e.g., Tables, Processes, and Apps themselves (since they can be nested)
  *******************************************************************************/
-public interface QAppChildMetaData
+public interface QAppChildMetaData extends QMetaDataObject
 {
    /*******************************************************************************
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QAppSection.java

@@ -24,12 +24,13 @@ package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 import java.util.ArrayList;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** A section of apps/tables/processes - a logical grouping.
  *******************************************************************************/
-public class QAppSection implements Cloneable
+public class QAppSection implements Cloneable, QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QIcon.java

@@ -22,6 +22,9 @@
 package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Icon to show associated with an App, Table, Process, etc.
  **
@@ -31,7 +34,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.layout;
  ** Future may allow something like a "namespace", and/or multiple icons for
  ** use in different frontends, etc.
  *******************************************************************************/
-public class QIcon implements Cloneable
+public class QIcon implements Cloneable, QMetaDataObject
 {
    private String name;
    private String path;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/permissions/QPermissionRules.java

@@ -22,13 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.permissions;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 
 
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QPermissionRules implements Cloneable
+public class QPermissionRules implements Cloneable, QMetaDataObject
 {
    private PermissionLevel level;
    private DenyBehavior    denyBehavior;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QFrontendComponentMetaData.java

@@ -25,12 +25,13 @@ package com.kingsrook.qqq.backend.core.model.metadata.processes;
 import java.io.Serializable;
 import java.util.HashMap;
 import java.util.Map;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Definition of a UI component in a frontend process steps.
  *******************************************************************************/
-public class QFrontendComponentMetaData
+public class QFrontendComponentMetaData implements QMetaDataObject
 {
    private QComponentType type;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QProcessMetaData.java

@@ -327,12 +327,23 @@ public class QProcessMetaData implements QAppChildMetaData, MetaDataWithPermissi
 
 
    /*******************************************************************************
-    ** Setter for stepList
+    ** Setter for stepList - note - calling this method ALSO overwrites the steps map!
     **
     *******************************************************************************/
    public void setStepList(List<QStepMetaData> stepList)
    {
-      this.stepList = stepList;
+      if(stepList == null)
+      {
+         this.stepList = null;
+         this.steps = null;
+      }
+      else
+      {
+         this.stepList = new ArrayList<>();
+         this.steps = new HashMap<>();
+      }
+
+      withStepList(stepList);
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QStateMachineStep.java

@@ -185,4 +185,25 @@ public class QStateMachineStep extends QStepMetaData
       return (rs);
    }
 
+
+   /*******************************************************************************
+    ** Setter for subSteps
+    *******************************************************************************/
+   public void setSubSteps(List<QStepMetaData> subSteps)
+   {
+      this.subSteps = subSteps;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for subSteps
+    *******************************************************************************/
+   public QStateMachineStep withSubSteps(List<QStepMetaData> subSteps)
+   {
+      this.subSteps = subSteps;
+      return (this);
+   }
+
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QStepMetaData.java

@@ -26,6 +26,7 @@ import java.util.ArrayList;
 import java.util.List;
 import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.serialization.QStepMetaDataDeserializer;
 
@@ -37,7 +38,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.serialization.QStepMetaData
  **
  *******************************************************************************/
 @JsonDeserialize(using = QStepMetaDataDeserializer.class)
-public abstract class QStepMetaData
+public abstract class QStepMetaData implements QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/producers/ChildJoinFromRecordEntityGenericMetaDataProducer.java

@@ -26,12 +26,15 @@ import java.util.Objects;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
 import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildJoin;
+import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 
 
 /*******************************************************************************
@@ -62,6 +65,7 @@ public class ChildJoinFromRecordEntityGenericMetaDataProducer implements MetaDat
    private String childTableName; // e.g., lineItem
    private String parentTableName; // e.g., order
    private String foreignKeyFieldName; // e.g., orderId
+   private boolean isOneToOne;
 
    private ChildJoin.OrderBy[] orderBys;
 
@@ -72,7 +76,7 @@ public class ChildJoinFromRecordEntityGenericMetaDataProducer implements MetaDat
    /***************************************************************************
     **
     ***************************************************************************/
-   public ChildJoinFromRecordEntityGenericMetaDataProducer(String childTableName, String parentTableName, String foreignKeyFieldName, ChildJoin.OrderBy[] orderBys)
+   public ChildJoinFromRecordEntityGenericMetaDataProducer(String childTableName, String parentTableName, String foreignKeyFieldName, ChildJoin.OrderBy[] orderBys, boolean isOneToOne)
    {
       Objects.requireNonNull(childTableName, "childTableName cannot be null");
       Objects.requireNonNull(parentTableName, "parentTableName cannot be null");
@@ -82,6 +86,7 @@ public class ChildJoinFromRecordEntityGenericMetaDataProducer implements MetaDat
       this.parentTableName = parentTableName;
       this.foreignKeyFieldName = foreignKeyFieldName;
       this.orderBys = orderBys;
+      this.isOneToOne = isOneToOne;
    }
 
 
@@ -92,23 +97,14 @@ public class ChildJoinFromRecordEntityGenericMetaDataProducer implements MetaDat
    @Override
    public QJoinMetaData produce(QInstance qInstance) throws QException
    {
-      QTableMetaData parentTable = qInstance.getTable(parentTableName);
-      if(parentTable == null)
-      {
-         throw (new QException("Could not find tableMetaData " + parentTableName));
-      }
-
-      QTableMetaData childTable = qInstance.getTable(childTableName);
-      if(childTable == null)
-      {
-         throw (new QException("Could not find tableMetaData " + childTable));
-      }
+      QTableMetaData parentTable = getTable(qInstance, parentTableName);
+      QTableMetaData childTable  = getTable(qInstance, childTableName);
 
       QJoinMetaData join = new QJoinMetaData()
          .withLeftTable(parentTableName)
          .withRightTable(childTableName)
          .withInferredName()
-         .withType(JoinType.ONE_TO_MANY)
+         .withType(isOneToOne ? JoinType.ONE_TO_ONE : JoinType.ONE_TO_MANY)
          .withJoinOn(new JoinOn(parentTable.getPrimaryKeyField(), foreignKeyFieldName));
 
       if(orderBys != null && orderBys.length > 0)
@@ -131,6 +127,41 @@ public class ChildJoinFromRecordEntityGenericMetaDataProducer implements MetaDat
 
 
 
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   private QTableMetaData getTable(QInstance qInstance, String tableName) throws QException
+   {
+      QTableMetaData table = qInstance.getTable(tableName);
+      if(table == null)
+      {
+         ///////////////////////////////////////////////////////////////////////////////
+         // in case we're producing a QBit, and it's added a table to a multi-output, //
+         // but not yet the instance, see if we can get table from there              //
+         ///////////////////////////////////////////////////////////////////////////////
+         for(MetaDataProducerMultiOutput metaDataProducerMultiOutput : QBitProductionContext.getReadOnlyViewOfMetaDataProducerMultiOutputStack())
+         {
+            table = CollectionUtils.nonNullList(metaDataProducerMultiOutput.getEach(QTableMetaData.class)).stream()
+               .filter(t -> t.getName().equals(tableName))
+               .findFirst().orElse(null);
+
+            if(table != null)
+            {
+               break;
+            }
+         }
+      }
+
+      if(table == null)
+      {
+         throw (new QException("Could not find tableMetaData: " + table));
+      }
+
+      return table;
+   }
+
+
+
    /*******************************************************************************
     ** Getter for sourceClass
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/producers/ChildRecordListWidgetFromRecordEntityGenericMetaDataProducer.java

@@ -25,10 +25,13 @@ package com.kingsrook.qqq.backend.core.model.metadata.producers;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildRecordListWidget;
+import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -89,6 +92,26 @@ public class ChildRecordListWidgetFromRecordEntityGenericMetaDataProducer implem
       String        name = QJoinMetaData.makeInferredJoinName(parentTableName, childTableName);
       QJoinMetaData join = qInstance.getJoin(name);
 
+      if(join == null)
+      {
+         for(MetaDataProducerMultiOutput metaDataProducerMultiOutput : QBitProductionContext.getReadOnlyViewOfMetaDataProducerMultiOutputStack())
+         {
+            join = CollectionUtils.nonNullList(metaDataProducerMultiOutput.getEach(QJoinMetaData.class)).stream()
+               .filter(t -> t.getName().equals(name))
+               .findFirst().orElse(null);
+
+            if(join != null)
+            {
+               break;
+            }
+         }
+      }
+
+      if(join == null)
+      {
+         throw (new QException("Could not find joinMetaData: " + name));
+      }
+
       QWidgetMetaData widget = ChildRecordListRenderer.widgetMetaDataBuilder(join)
          .withName(name)
          .withLabel(childRecordListWidget.label())

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/producers/annotations/ChildJoin.java

@@ -38,6 +38,8 @@ public @interface ChildJoin
 
    OrderBy[] orderBy() default { };
 
+   boolean isOneToOne() default false;
+
    /***************************************************************************
     **
     ***************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/qbits/QBitConfig.java

@@ -107,4 +107,14 @@ public interface QBitConfig extends Serializable
    {
       return (null);
    }
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   default String getDefaultBackendNameForTables()
+   {
+      return (null);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/qbits/QBitMetaDataProducer.java

@@ -0,0 +1,192 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.qbits;
+
+
+import java.util.List;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QRuntimeException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerHelper;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** 2nd generation interface for top-level meta-data production classes that make
+ ** a qbit (evolution over QBitProducer).
+ **
+ *******************************************************************************/
+public interface QBitMetaDataProducer<C extends QBitConfig> extends MetaDataProducerInterface<MetaDataProducerMultiOutput>
+{
+   QLogger LOG = QLogger.getLogger(QBitMetaDataProducer.class);
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   C getQBitConfig();
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   QBitMetaData getQBitMetaData();
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   default String getNamespace()
+   {
+      return (null);
+   }
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   default void postProduceActions(MetaDataProducerMultiOutput metaDataProducerMultiOutput, QInstance qinstance)
+   {
+      /////////////////////
+      // noop by default //
+      /////////////////////
+   }
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   default String getPackageNameForFindingMetaDataProducers()
+   {
+      Class<?> clazz = getClass();
+
+      ////////////////////////////////////////////////////////////////
+      // Walk up the hierarchy until we find the direct implementer //
+      ////////////////////////////////////////////////////////////////
+      while(clazz != null)
+      {
+         Class<?>[] interfaces = clazz.getInterfaces();
+         for(Class<?> interfaze : interfaces)
+         {
+            if(interfaze == QBitMetaDataProducer.class)
+            {
+               return clazz.getPackageName();
+            }
+         }
+         clazz = clazz.getSuperclass();
+      }
+
+      throw (new QRuntimeException("Unable to find packageName for QBitMetaDataProducer.  You may need to implement getPackageName yourself..."));
+   }
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   default MetaDataProducerMultiOutput produce(QInstance qInstance) throws QException
+   {
+      MetaDataProducerMultiOutput rs = new MetaDataProducerMultiOutput();
+
+      QBitMetaData qBitMetaData = getQBitMetaData();
+      C            qBitConfig   = getQBitConfig();
+
+      qInstance.addQBit(qBitMetaData);
+
+      QBitProductionContext.pushQBitConfig(qBitConfig);
+      QBitProductionContext.pushMetaDataProducerMultiOutput(rs);
+
+      try
+      {
+         qBitConfig.validate(qInstance);
+
+         List<MetaDataProducerInterface<?>> producers = MetaDataProducerHelper.findProducers(getPackageNameForFindingMetaDataProducers());
+         MetaDataProducerHelper.sortMetaDataProducers(producers);
+         for(MetaDataProducerInterface<?> producer : producers)
+         {
+            if(producer.getClass().equals(this.getClass()))
+            {
+               /////////////////////////////////////////////
+               // avoid recursive processing of ourselves //
+               /////////////////////////////////////////////
+               continue;
+            }
+
+            ////////////////////////////////////////////////////////////////////////////
+            // todo is this deprecated in favor of QBitProductionContext's stack... ? //
+            ////////////////////////////////////////////////////////////////////////////
+            if(producer instanceof QBitComponentMetaDataProducer<?, ?>)
+            {
+               QBitComponentMetaDataProducer<?, C> qBitComponentMetaDataProducer = (QBitComponentMetaDataProducer<?, C>) producer;
+               qBitComponentMetaDataProducer.setQBitConfig(qBitConfig);
+            }
+
+            if(!producer.isEnabled())
+            {
+               LOG.debug("Not using producer which is not enabled", logPair("producer", producer.getClass().getSimpleName()));
+               continue;
+            }
+
+            MetaDataProducerOutput subProducerOutput = producer.produce(qInstance);
+
+            /////////////////////////////////////////////////
+            // apply some things from the config to tables //
+            /////////////////////////////////////////////////
+            if(subProducerOutput instanceof QTableMetaData table)
+            {
+               if(qBitConfig.getTableMetaDataCustomizer() != null)
+               {
+                  subProducerOutput = qBitConfig.getTableMetaDataCustomizer().customizeMetaData(qInstance, table);
+               }
+
+               if(!StringUtils.hasContent(table.getBackendName()) && StringUtils.hasContent(qBitConfig.getDefaultBackendNameForTables()))
+               {
+                  table.setBackendName(qBitConfig.getDefaultBackendNameForTables());
+               }
+            }
+
+            ////////////////////////////////////////////////////////////
+            // set source qbit, if subProducerOutput is aware of such //
+            ////////////////////////////////////////////////////////////
+            if(subProducerOutput instanceof SourceQBitAware sourceQBitAware)
+            {
+               sourceQBitAware.setSourceQBitName(qBitMetaData.getName());
+            }
+
+            rs.add(subProducerOutput);
+         }
+
+         postProduceActions(rs, qInstance);
+
+         return (rs);
+      }
+      finally
+      {
+         QBitProductionContext.popQBitConfig();
+         QBitProductionContext.popMetaDataProducerMultiOutput();
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/qbits/QBitProducer.java

@@ -76,9 +76,6 @@ public interface QBitProducer
    {
       qBitConfig.validate(qInstance);
 
-      ///////////////////////////////
-      // todo - move to base class //
-      ///////////////////////////////
       for(MetaDataProducerInterface<?> producer : producers)
       {
          if(producer instanceof QBitComponentMetaDataProducer<?, ?>)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/qbits/QBitProductionContext.java

@@ -0,0 +1,136 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.qbits;
+
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Stack;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+
+
+/*******************************************************************************
+ ** While a qbit is being produced, track the context of the current config
+ ** and metaDataProducerMultiOutput that is being used.  also, in case one
+ ** qbit produces another, push these contextual objects on a stack.
+ *******************************************************************************/
+public class QBitProductionContext
+{
+   private static final QLogger LOG = QLogger.getLogger(QBitProductionContext.class);
+
+   private static Stack<QBitConfig>                  qbitConfigStack                  = new Stack<>();
+   private static Stack<MetaDataProducerMultiOutput> metaDataProducerMultiOutputStack = new Stack<>();
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static void pushQBitConfig(QBitConfig qBitConfig)
+   {
+      qbitConfigStack.push(qBitConfig);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static QBitConfig peekQBitConfig()
+   {
+      if(qbitConfigStack.isEmpty())
+      {
+         LOG.warn("Request to peek at empty QBitProductionContext configStack - returning null");
+         return (null);
+      }
+      return qbitConfigStack.peek();
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static void popQBitConfig()
+   {
+      if(qbitConfigStack.isEmpty())
+      {
+         LOG.warn("Request to pop empty QBitProductionContext configStack - returning with noop");
+         return;
+      }
+
+      qbitConfigStack.pop();
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static void pushMetaDataProducerMultiOutput(MetaDataProducerMultiOutput metaDataProducerMultiOutput)
+   {
+      metaDataProducerMultiOutputStack.push(metaDataProducerMultiOutput);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static MetaDataProducerMultiOutput peekMetaDataProducerMultiOutput()
+   {
+      if(metaDataProducerMultiOutputStack.isEmpty())
+      {
+         LOG.warn("Request to peek at empty QBitProductionContext configStack - returning null");
+         return (null);
+      }
+      return metaDataProducerMultiOutputStack.peek();
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static List<MetaDataProducerMultiOutput> getReadOnlyViewOfMetaDataProducerMultiOutputStack()
+   {
+      return Collections.unmodifiableList(metaDataProducerMultiOutputStack);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static void popMetaDataProducerMultiOutput()
+   {
+      if(metaDataProducerMultiOutputStack.isEmpty())
+      {
+         LOG.warn("Request to pop empty QBitProductionContext metaDataProducerMultiOutput - returning with noop");
+         return;
+      }
+
+      metaDataProducerMultiOutputStack.pop();
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/scheduleing/QScheduleMetaData.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.model.metadata.scheduleing;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -35,7 +36,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  ** same moment.
  **
  *******************************************************************************/
-public class QScheduleMetaData
+public class QScheduleMetaData implements QMetaDataObject
 {
    private String schedulerName;
    private String description;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/security/MultiRecordSecurityLock.java

@@ -44,7 +44,7 @@ public class MultiRecordSecurityLock extends RecordSecurityLock implements Clone
     **
     *******************************************************************************/
    @Override
-   protected MultiRecordSecurityLock clone() throws CloneNotSupportedException
+   public MultiRecordSecurityLock clone()
    {
       MultiRecordSecurityLock clone = (MultiRecordSecurityLock) super.clone();
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/security/RecordSecurityLock.java

@@ -57,20 +57,27 @@ public class RecordSecurityLock implements Cloneable
     **
     *******************************************************************************/
    @Override
-   protected RecordSecurityLock clone() throws CloneNotSupportedException
+   public RecordSecurityLock clone()
    {
-      RecordSecurityLock clone = (RecordSecurityLock) super.clone();
-
-      /////////////////////////
-      // deep-clone the list //
-      /////////////////////////
-      if(joinNameChain != null)
+      try
       {
-         clone.joinNameChain = new ArrayList<>();
-         clone.joinNameChain.addAll(joinNameChain);
-      }
+         RecordSecurityLock clone = (RecordSecurityLock) super.clone();
 
-      return (clone);
+         /////////////////////////
+         // deep-clone the list //
+         /////////////////////////
+         if(joinNameChain != null)
+         {
+            clone.joinNameChain = new ArrayList<>();
+            clone.joinNameChain.addAll(joinNameChain);
+         }
+
+         return (clone);
+      }
+      catch(CloneNotSupportedException e)
+      {
+         throw (new RuntimeException("Could not clone", e));
+      }
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/Association.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.tables;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** definition of a qqq table that is "associated" with another table, e.g.,
  ** managed along with it - such as child-records under a parent record.
  *******************************************************************************/
-public class Association
+public class Association implements QMetaDataObject
 {
    private String name;
    private String associatedTableName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/QFieldSection.java

@@ -26,6 +26,7 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.instances.QInstanceHelpContentManager;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
@@ -36,7 +37,7 @@ import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
  ** A section of fields - a logical grouping.
  ** TODO - this class should be named QTableSection!
  *******************************************************************************/
-public class QFieldSection
+public class QFieldSection implements QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/UniqueKey.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -32,7 +33,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  ** Definition of a Unique Key (or "Constraint", if you wanna use fancy words)
  ** on a QTable.
  *******************************************************************************/
-public class UniqueKey
+public class UniqueKey implements QMetaDataObject
 {
    private List<String> fieldNames;
    private String       label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/AutomationStatusTracking.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Table-automation meta-data to define how this table's per-record automation
  ** status is tracked.
  *******************************************************************************/
-public class AutomationStatusTracking
+public class AutomationStatusTracking implements QMetaDataObject
 {
    private AutomationStatusTrackingType type;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/QTableAutomationDetails.java

@@ -24,13 +24,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 
 import java.util.ArrayList;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
 
 
 /*******************************************************************************
  ** Details about how this table's record automations are set up.
  *******************************************************************************/
-public class QTableAutomationDetails
+public class QTableAutomationDetails implements QMetaDataObject
 {
    private AutomationStatusTracking    statusTracking;
    private String                      providerName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/TableAutomationAction.java

@@ -25,13 +25,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 import java.io.Serializable;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 
 
 /*******************************************************************************
  ** Definition of a specific action to run against a table
  *******************************************************************************/
-public class TableAutomationAction
+public class TableAutomationAction implements QMetaDataObject
 {
    private String       name;
    private TriggerEvent triggerEvent;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/session/QUser.java

@@ -22,10 +22,13 @@
 package com.kingsrook.qqq.backend.core.model.session;
 
 
+import java.io.Serializable;
+
+
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QUser implements Cloneable
+public class QUser implements Cloneable, Serializable
 {
    private String idReference;
    private String fullName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/tables/QQQTableCustomPossibleValueProvider.java

@@ -0,0 +1,123 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.tables;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.actions.permissions.PermissionCheckResult;
+import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.values.BasicCustomPossibleValueProvider;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ ** possible-value source provider for the `QQQ Table` PVS - a list of all tables
+ ** in an application/qInstance (that you have permission to see)
+ *******************************************************************************/
+public class QQQTableCustomPossibleValueProvider extends BasicCustomPossibleValueProvider<QRecord, Integer>
+{
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   protected QPossibleValue<Integer> makePossibleValue(QRecord sourceObject)
+   {
+      return (new QPossibleValue<>(sourceObject.getValueInteger("id"), sourceObject.getValueString("label")));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   protected QRecord getSourceObject(Serializable id) throws QException
+   {
+      QRecord qqqTableRecord = GetAction.execute(QQQTable.TABLE_NAME, id);
+      if(qqqTableRecord == null)
+      {
+         return (null);
+      }
+
+      QTableMetaData table = QContext.getQInstance().getTable(qqqTableRecord.getValueString("name"));
+      return isTableAllowed(table) ? qqqTableRecord : null;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   protected List<QRecord> getAllSourceObjects() throws QException
+   {
+      List<QRecord>      records = QueryAction.execute(QQQTable.TABLE_NAME, null);
+      ArrayList<QRecord> rs      = new ArrayList<>();
+      for(QRecord record : records)
+      {
+         QTableMetaData table = QContext.getQInstance().getTable(record.getValueString("name"));
+         if(isTableAllowed(table))
+         {
+            rs.add(record);
+         }
+      }
+
+      return rs;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private boolean isTableAllowed(QTableMetaData table)
+   {
+      if(table == null)
+      {
+         return (false);
+      }
+
+      if(table.getIsHidden())
+      {
+         return (false);
+      }
+
+      PermissionCheckResult permissionCheckResult = PermissionsHelper.getPermissionCheckResult(new QueryInput(table.getName()), table);
+      if(!PermissionCheckResult.ALLOW.equals(permissionCheckResult))
+      {
+         return (false);
+      }
+
+      return (true);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/tables/QQQTableTableManager.java

@@ -22,18 +22,29 @@
 package com.kingsrook.qqq.backend.core.model.tables;
 
 
+import java.io.Serializable;
 import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
 import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.processes.utils.GeneralProcessUtils;
 import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
@@ -89,4 +100,41 @@ public class QQQTableTableManager
 
       return getOutput.getRecord().getValueInteger("id");
    }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static List<QRecord> setRecordLinksToRecordsFromTableDynamicForPostQuery(QueryOrGetInputInterface queryInput, List<QRecord> records, String tableIdField, String recordIdField) throws QException
+   {
+      /////////////////////////////////////////////////////////////////////////////////////////
+      // note, this is a second copy of this logic (first being in standard process traces). //
+      // let the rule of 3 apply if we find ourselves copying it again                       //
+      /////////////////////////////////////////////////////////////////////////////////////////
+      if(queryInput.getShouldGenerateDisplayValues())
+      {
+         ///////////////////////////////////////////////////////////////////////////////////////////
+         // for records with a table id value - look up that table name, then set a display-value //
+         // for the Link type adornment, to the inputRecordId record within that table.           //
+         ///////////////////////////////////////////////////////////////////////////////////////////
+         Set<Serializable> tableIds = records.stream().map(r -> r.getValue(tableIdField)).filter(Objects::nonNull).collect(Collectors.toSet());
+         if(!tableIds.isEmpty())
+         {
+            Map<Serializable, QRecord> tableMap = GeneralProcessUtils.loadTableToMap(QQQTable.TABLE_NAME, "id", new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, tableIds)));
+
+            for(QRecord record : records)
+            {
+               QRecord qqqTableRecord = tableMap.get(record.getValue(tableIdField));
+               if(qqqTableRecord != null && record.getValue(recordIdField) != null)
+               {
+                  record.setDisplayValue(recordIdField + ":" + AdornmentType.LinkValues.TO_RECORD_FROM_TABLE_DYNAMIC, qqqTableRecord.getValueString("name"));
+               }
+            }
+         }
+      }
+
+      return (records);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/tables/QQQTablesMetaDataProvider.java

@@ -27,6 +27,9 @@ import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PVSValueFormatAndFields;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
@@ -125,10 +128,11 @@ public class QQQTablesMetaDataProvider
    public QPossibleValueSource defineQQQTablePossibleValueSource()
    {
       return (new QPossibleValueSource()
-         .withType(QPossibleValueSourceType.TABLE)
          .withName(QQQTable.TABLE_NAME)
-         .withTableName(QQQTable.TABLE_NAME))
-         .withOrderByField("label");
+         .withType(QPossibleValueSourceType.CUSTOM)
+         .withIdType(QFieldType.INTEGER)
+         .withCustomCodeReference(new QCodeReference(QQQTableCustomPossibleValueProvider.class))
+         .withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY));
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/QAuthenticationModuleInterface.java

@@ -25,15 +25,14 @@ package com.kingsrook.qqq.backend.core.modules.authentication;
 import java.io.Serializable;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.exceptions.AccessTokenException;
 import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang.NotImplementedException;
 
 
+
 /*******************************************************************************
  ** Interface that a QAuthenticationModule must implement.
  **
@@ -82,12 +81,12 @@ public interface QAuthenticationModuleInterface
    }
 
 
-   /*******************************************************************************
+   /***************************************************************************
     **
-    *******************************************************************************/
-   default String createAccessToken(QAuthenticationMetaData metaData, String clientId, String clientSecret) throws AccessTokenException
+    ***************************************************************************/
+   default String getLoginRedirectUrl(String originalUrl) throws QAuthenticationException
    {
-      throw (new NotImplementedException("The method createAccessToken() is not implemented in the class: " + this.getClass().getSimpleName()));
+      throw (new NotImplementedException("The method getLoginRedirectUrl() is not implemented in the authentication module: " + this.getClass().getSimpleName()));
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/Auth0AuthenticationModule.java

@@ -1020,7 +1020,7 @@ public class Auth0AuthenticationModule implements QAuthenticationModuleInterface
             // decode the accessToken and make sure it is not expired //
             ////////////////////////////////////////////////////////////
             boolean needNewToken = true;
-            if(accessToken != null)
+            if(StringUtils.hasContent(accessToken))
             {
                DecodedJWT jwt     = JWT.decode(accessToken);
                String     payload = jwt.getPayload();

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/FullyAnonymousAuthenticationModule.java

@@ -24,9 +24,7 @@ package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
 
 import java.util.Map;
 import java.util.UUID;
-import com.kingsrook.qqq.backend.core.exceptions.AccessTokenException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.model.session.QUser;
 import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleInterface;
@@ -77,15 +75,4 @@ public class FullyAnonymousAuthenticationModule implements QAuthenticationModule
       return session != null;
    }
 
-
-
-   /*******************************************************************************
-    ** Load an instance of the appropriate state provider
-    **
-    *******************************************************************************/
-   public String createAccessToken(QAuthenticationMetaData metaData, String clientId, String clientSecret) throws AccessTokenException
-   {
-      return (TEST_ACCESS_TOKEN);
-   }
-
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/MockAuthenticationModule.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
 
 import java.util.Map;
 import java.util.UUID;
+import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
@@ -45,8 +46,13 @@ public class MockAuthenticationModule implements QAuthenticationModuleInterface
     **
     *******************************************************************************/
    @Override
-   public QSession createSession(QInstance qInstance, Map<String, String> context)
+   public QSession createSession(QInstance qInstance, Map<String, String> context) throws QAuthenticationException
    {
+      if("Deny".equalsIgnoreCase(context.get("accessToken")))
+      {
+         throw (new QAuthenticationException("Access denied (per accessToken requesting as such)"));
+      }
+
       QUser qUser = new QUser();
       qUser.setIdReference("User:" + (System.currentTimeMillis() % USER_ID_MODULO));
       qUser.setFullName("John Smith");
@@ -80,4 +86,16 @@ public class MockAuthenticationModule implements QAuthenticationModuleInterface
 
       return (true);
    }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getLoginRedirectUrl(String originalUrl)
+   {
+      return originalUrl + "?createMockSession=true";
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/OAuth2AuthenticationModule.java

@@ -0,0 +1,486 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
+
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.net.URI;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.temporal.ChronoUnit;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.atomic.AtomicReference;
+import com.auth0.jwt.JWT;
+import com.auth0.jwt.interfaces.DecodedJWT;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.context.CapturedContext;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.OAuth2AuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
+import com.kingsrook.qqq.backend.core.model.session.QUser;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleInterface;
+import com.kingsrook.qqq.backend.core.modules.authentication.implementations.model.UserSession;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
+import com.nimbusds.oauth2.sdk.AuthorizationCode;
+import com.nimbusds.oauth2.sdk.AuthorizationCodeGrant;
+import com.nimbusds.oauth2.sdk.AuthorizationGrant;
+import com.nimbusds.oauth2.sdk.ErrorObject;
+import com.nimbusds.oauth2.sdk.GeneralException;
+import com.nimbusds.oauth2.sdk.ParseException;
+import com.nimbusds.oauth2.sdk.Scope;
+import com.nimbusds.oauth2.sdk.TokenRequest;
+import com.nimbusds.oauth2.sdk.TokenResponse;
+import com.nimbusds.oauth2.sdk.auth.ClientAuthentication;
+import com.nimbusds.oauth2.sdk.auth.ClientSecretBasic;
+import com.nimbusds.oauth2.sdk.auth.Secret;
+import com.nimbusds.oauth2.sdk.id.ClientID;
+import com.nimbusds.oauth2.sdk.id.Issuer;
+import com.nimbusds.oauth2.sdk.id.State;
+import com.nimbusds.oauth2.sdk.pkce.CodeVerifier;
+import com.nimbusds.oauth2.sdk.token.AccessToken;
+import com.nimbusds.openid.connect.sdk.op.OIDCProviderMetadata;
+import org.json.JSONObject;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Implementation of OAuth2 authentication.
+ *******************************************************************************/
+public class OAuth2AuthenticationModule implements QAuthenticationModuleInterface
+{
+   private static final QLogger LOG = QLogger.getLogger(OAuth2AuthenticationModule.class);
+
+   private static boolean mayMemoize = true;
+
+   private static final Memoization<String, String> getAccessTokenFromSessionUUIDMemoization = new Memoization<String, String>()
+      .withTimeout(Duration.of(1, ChronoUnit.MINUTES))
+      .withMaxSize(1000);
+
+   private static final Memoization<String, OIDCProviderMetadata> oidcProviderMetadataMemoization = new Memoization<String, OIDCProviderMetadata>()
+      .withMayStoreNullValues(false);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QSession createSession(QInstance qInstance, Map<String, String> context) throws QAuthenticationException
+   {
+      try
+      {
+         OAuth2AuthenticationMetaData oauth2MetaData = (OAuth2AuthenticationMetaData) qInstance.getAuthentication();
+
+         if(context.containsKey("code") && context.containsKey("state"))
+         {
+            ///////////////////////////////////////////////////////////////////////
+            // handle a callback to initially auth a user for a traditional      //
+            // (non-js) site - where the code & state params come to the backend //
+            ///////////////////////////////////////////////////////////////////////
+            AuthorizationCode code = new AuthorizationCode(context.get("code"));
+
+            /////////////////////////////////////////
+            // verify the state in our state table //
+            /////////////////////////////////////////
+            AtomicReference<String> redirectUri = new AtomicReference<>(null);
+            QContext.withTemporaryContext(new CapturedContext(qInstance, new QSystemUserSession()), () ->
+            {
+               QRecord redirectStateRecord = GetAction.execute(oauth2MetaData.getRedirectStateTableName(), Map.of("state", context.get("state")));
+               if(redirectStateRecord == null)
+               {
+                  throw (new QAuthenticationException("State not found"));
+               }
+               redirectUri.set(redirectStateRecord.getValueString("redirectUri"));
+            });
+
+            URI                    redirectURI       = new URI(redirectUri.get());
+            ClientSecretBasic      clientSecretBasic = new ClientSecretBasic(new ClientID(oauth2MetaData.getClientId()), new Secret(oauth2MetaData.getClientSecret()));
+            AuthorizationCodeGrant codeGrant         = new AuthorizationCodeGrant(code, redirectURI);
+
+            URI          tokenEndpoint = getOIDCProviderMetadata(oauth2MetaData).getTokenEndpointURI();
+            Scope        scope         = new Scope(oauth2MetaData.getScopes());
+            TokenRequest tokenRequest  = new TokenRequest(tokenEndpoint, clientSecretBasic, codeGrant, scope);
+
+            return createSessionFromTokenRequest(tokenRequest);
+         }
+         else if(context.containsKey("code") && context.containsKey("redirectUri") && context.containsKey("codeVerifier"))
+         {
+            ////////////////////////////////////////////////////////////////////////////////
+            // handle a call down to this backend code to initially auth a user for an    //
+            // SPA that received a code (where the javascript generated the codeVerifier) //
+            ////////////////////////////////////////////////////////////////////////////////
+            AuthorizationCode  code         = new AuthorizationCode(context.get("code"));
+            URI                callback     = new URI(context.get("redirectUri"));
+            CodeVerifier       codeVerifier = new CodeVerifier(context.get("codeVerifier"));
+            AuthorizationGrant codeGrant    = new AuthorizationCodeGrant(code, callback, codeVerifier);
+
+            ClientID             clientID     = new ClientID(oauth2MetaData.getClientId());
+            Secret               clientSecret = new Secret(oauth2MetaData.getClientSecret());
+            ClientAuthentication clientAuth   = new ClientSecretBasic(clientID, clientSecret);
+
+            URI          tokenEndpoint = getOIDCProviderMetadata(oauth2MetaData).getTokenEndpointURI();
+            Scope        scope         = new Scope(oauth2MetaData.getScopes());
+            TokenRequest tokenRequest  = new TokenRequest(tokenEndpoint, clientAuth, codeGrant, scope);
+
+            return createSessionFromTokenRequest(tokenRequest);
+         }
+         else if(context.containsKey("sessionUUID") || context.containsKey("sessionId") || context.containsKey("uuid"))
+         {
+            //////////////////////////////////////////////////////////////////////
+            // handle a "normal" request, where we aren't opening a new session //
+            // per-se, but instead are looking for one in our userSession table //
+            //////////////////////////////////////////////////////////////////////
+            String uuid = Objects.requireNonNullElseGet(context.get("sessionUUID"), () ->
+               Objects.requireNonNullElseGet(context.get("sessionId"), () ->
+                  context.get("uuid")));
+
+            String   accessToken = getAccessTokenFromSessionUUID(uuid);
+            QSession session     = createSessionFromToken(accessToken);
+            session.setUuid(uuid);
+
+            //////////////////////////////////////////////////////////////////
+            // todo - do we need to validate its age or ping the provider?? //
+            //////////////////////////////////////////////////////////////////
+
+            return (session);
+         }
+         else
+         {
+            String message = "Did not receive recognized values in context for creating session";
+            LOG.warn(message, logPair("contextKeys", context.keySet()));
+            throw (new QAuthenticationException(message));
+         }
+      }
+      catch(QAuthenticationException qae)
+      {
+         throw (qae);
+      }
+      catch(Exception e)
+      {
+         throw (new QAuthenticationException("Failed to create session (token)", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private QSession createSessionFromTokenRequest(TokenRequest tokenRequest) throws ParseException, IOException, QException
+   {
+      TokenResponse tokenResponse = TokenResponse.parse(tokenRequest.toHTTPRequest().send());
+
+      if(tokenResponse.indicatesSuccess())
+      {
+         AccessToken accessToken = tokenResponse.toSuccessResponse().getTokens().getAccessToken();
+
+         ////////////////////////////////////////////////////////////////////
+         // todo - do we want to try to do anything with a refresh token?? //
+         ////////////////////////////////////////////////////////////////////
+         // RefreshToken refreshToken = tokenResponse.toSuccessResponse().getTokens().getRefreshToken();
+
+         QSession session = createSessionFromToken(accessToken.getValue());
+         insertUserSession(accessToken.getValue(), session);
+         return (session);
+      }
+      else
+      {
+         ErrorObject errorObject = tokenResponse.toErrorResponse().getErrorObject();
+         LOG.info("Token request failed", logPair("code", errorObject.getCode()), logPair("description", errorObject.getDescription()));
+         throw (new QAuthenticationException(errorObject.getDescription()));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public boolean isSessionValid(QInstance instance, QSession session)
+   {
+      if(session instanceof QSystemUserSession)
+      {
+         return (true);
+      }
+
+      try
+      {
+         String     accessToken = getAccessTokenFromSessionUUID(session.getUuid());
+         DecodedJWT jwt         = JWT.decode(accessToken);
+         if(jwt.getExpiresAtAsInstant().isBefore(Instant.now()))
+         {
+            LOG.debug("accessToken is expired", logPair("sessionUUID", session.getUuid()));
+            return (false);
+         }
+
+         return true;
+      }
+      catch(QAuthenticationException e)
+      {
+         return (false);
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getLoginRedirectUrl(String originalUrl) throws QAuthenticationException
+   {
+      try
+      {
+         QInstance                    qInstance      = QContext.getQInstance();
+         OAuth2AuthenticationMetaData oauth2MetaData = (OAuth2AuthenticationMetaData) qInstance.getAuthentication();
+         String                       authUrl        = getOIDCProviderMetadata(oauth2MetaData).getAuthorizationEndpointURI().toString();
+
+         QTableMetaData stateTable = QContext.getQInstance().getTable(oauth2MetaData.getRedirectStateTableName());
+         if(stateTable == null)
+         {
+            throw (new QAuthenticationException("The table specified as the oauthRedirectStateTableName [" + oauth2MetaData.getRedirectStateTableName() + "] is not defined in the QInstance"));
+         }
+
+         ///////////////////////////////////////////////////////////////////
+         // generate a secure state, of either default length (32 bytes), //
+         // or at a size (base64 encoded) that fits in the state table    //
+         ///////////////////////////////////////////////////////////////////
+         Integer stateStringLength = stateTable.getField("state").getMaxLength();
+         State   state             = stateStringLength == null ? new State(32) : new State((stateStringLength / 4) * 3);
+         String  stateValue        = state.getValue();
+
+         /////////////////////////////
+         // insert the state record //
+         /////////////////////////////
+         QContext.withTemporaryContext(new CapturedContext(qInstance, new QSystemUserSession()), () ->
+         {
+            QRecord insertedState = new InsertAction().execute(new InsertInput(oauth2MetaData.getRedirectStateTableName()).withRecord(new QRecord()
+               .withValue("state", stateValue)
+               .withValue("redirectUri", originalUrl))).getRecords().get(0);
+            if(CollectionUtils.nullSafeHasContents(insertedState.getErrors()))
+            {
+               throw (new QAuthenticationException("Error storing redirect state: " + insertedState.getErrorsAsString()));
+            }
+         });
+
+         return authUrl
+            + "?client_id=" + URLEncoder.encode(oauth2MetaData.getClientId(), StandardCharsets.UTF_8)
+            + "&redirect_uri=" + URLEncoder.encode(originalUrl, StandardCharsets.UTF_8)
+            + "&response_type=code"
+            + "&scope=" + URLEncoder.encode(oauth2MetaData.getScopes(), StandardCharsets.UTF_8)
+            + "&state=" + URLEncoder.encode(state.getValue(), StandardCharsets.UTF_8);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error getting login redirect url", e);
+         throw (new QAuthenticationException("Error getting login redirect url", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private QSession createSessionFromToken(String accessToken) throws QException
+   {
+      DecodedJWT     jwt           = JWT.decode(accessToken);
+      Base64.Decoder decoder       = Base64.getUrlDecoder();
+      String         payloadString = new String(decoder.decode(jwt.getPayload()));
+      JSONObject     payload       = new JSONObject(payloadString);
+
+      QSession session = new QSession();
+      QUser    user    = new QUser();
+      session.setUser(user);
+
+      user.setFullName("Unknown");
+      String email = Objects.requireNonNullElseGet(payload.optString("email", null), () -> payload.optString("sub", null));
+      String name  = payload.optString("name", email);
+
+      user.setIdReference(email);
+      user.setFullName(name);
+
+      ////////////////////////////////////////////////////////////
+      // todo wip - this needs to be much better standardized w/ fe //
+      ////////////////////////////////////////////////////////////
+      session.withValueForFrontend("user", new HashMap<>(Map.of("name", name, "email", email)));
+
+      return session;
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert a session as a new record into userSession table
+    *******************************************************************************/
+   private void insertUserSession(String accessToken, QSession qSession) throws QException
+   {
+      CapturedContext capturedContext = QContext.capture();
+      try
+      {
+         QContext.init(capturedContext.qInstance(), new QSystemUserSession());
+
+         UserSession userSession = new UserSession()
+            .withUuid(qSession.getUuid())
+            .withUserId(qSession.getUser().getIdReference())
+            .withAccessToken(accessToken);
+
+         new InsertAction().execute(new InsertInput(UserSession.TABLE_NAME).withRecordEntity(userSession));
+      }
+      finally
+      {
+         QContext.init(capturedContext);
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QSession createAutomatedSessionForUser(QInstance qInstance, Serializable userId) throws QAuthenticationException
+   {
+      return QAuthenticationModuleInterface.super.createAutomatedSessionForUser(qInstance, userId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Look up access_token from session UUID
+    **
+    *******************************************************************************/
+   private String getAccessTokenFromSessionUUID(String sessionUUID) throws QAuthenticationException
+   {
+      if(mayMemoize)
+      {
+         return getAccessTokenFromSessionUUIDMemoization.getResultThrowing(sessionUUID, (String x) ->
+            doGetAccessTokenFromSessionUUID(sessionUUID)
+         ).orElse(null);
+      }
+      else
+      {
+         return (doGetAccessTokenFromSessionUUID(sessionUUID));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private String doGetAccessTokenFromSessionUUID(String sessionUUID) throws QAuthenticationException
+   {
+      String   accessToken   = null;
+      QSession beforeSession = QContext.getQSession();
+
+      try
+      {
+         QContext.setQSession(new QSystemUserSession());
+
+         ///////////////////////////////////////
+         // query for the user session record //
+         ///////////////////////////////////////
+         QRecord userSessionRecord = new GetAction().executeForRecord(new GetInput(UserSession.TABLE_NAME)
+            .withUniqueKey(Map.of("uuid", sessionUUID))
+            .withShouldMaskPasswords(false)
+            .withShouldOmitHiddenFields(false));
+
+         if(userSessionRecord != null)
+         {
+            accessToken = userSessionRecord.getValueString("accessToken");
+
+            ////////////////////////////////////////////////////////////
+            // decode the accessToken and make sure it is not expired //
+            ////////////////////////////////////////////////////////////
+            if(accessToken != null)
+            {
+               DecodedJWT jwt = JWT.decode(accessToken);
+               if(jwt.getExpiresAtAsInstant().isBefore(Instant.now()))
+               {
+                  throw (new QAuthenticationException("accessToken is expired"));
+               }
+            }
+         }
+      }
+      catch(QAuthenticationException qae)
+      {
+         throw (qae);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error looking up userSession by sessionUUID", e);
+         throw (new QAuthenticationException("Error looking up userSession by sessionUUID", e));
+      }
+      finally
+      {
+         QContext.setQSession(beforeSession);
+      }
+
+      return (accessToken);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public boolean usesSessionIdCookie()
+   {
+      return (true);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private OIDCProviderMetadata getOIDCProviderMetadata(OAuth2AuthenticationMetaData oAuth2AuthenticationMetaData) throws GeneralException, IOException
+   {
+      return oidcProviderMetadataMemoization.getResult(oAuth2AuthenticationMetaData.getName(), (name ->
+      {
+         Issuer               issuer   = new Issuer(oAuth2AuthenticationMetaData.getBaseUrl());
+         OIDCProviderMetadata metadata = OIDCProviderMetadata.resolve(issuer);
+         return (metadata);
+      })).orElseThrow(() -> new GeneralException("Could not resolve OIDCProviderMetadata for " + oAuth2AuthenticationMetaData.getName()));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/TableBasedAuthenticationModule.java

@@ -406,6 +406,7 @@ public class TableBasedAuthenticationModule implements QAuthenticationModuleInte
          qUser.setIdReference(userRecord.getValueString(metaData.getUserTableUsernameField()));
 
          QSession qSession = new QSession();
+         qSession.setUuid(sessionUuid);
          qSession.setIdReference(sessionUuid);
          qSession.setUser(qUser);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/metadata/RedirectStateMetaDataProducer.java

@@ -0,0 +1,82 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2023.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.authentication.implementations.metadata;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
+import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
+
+
+/*******************************************************************************
+ ** Meta Data Producer for RedirectState table
+ *******************************************************************************/
+public class RedirectStateMetaDataProducer extends MetaDataProducer<QTableMetaData>
+{
+   public static final String TABLE_NAME = "redirectState";
+
+   private final String backendName;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public RedirectStateMetaDataProducer(String backendName)
+   {
+      this.backendName = backendName;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QTableMetaData produce(QInstance qInstance) throws QException
+   {
+      QTableMetaData tableMetaData = new QTableMetaData()
+         .withName(TABLE_NAME)
+         .withBackendName(backendName)
+         .withRecordLabelFormat("%s")
+         .withRecordLabelFields("state")
+         .withPrimaryKeyField("id")
+         .withUniqueKey(new UniqueKey("state"))
+         .withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.NONE))
+
+         .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
+         .withField(new QFieldMetaData("state", QFieldType.STRING).withIsEditable(false).withMaxLength(45).withBehavior(ValueTooLongBehavior.ERROR))
+         .withField(new QFieldMetaData("redirectUri", QFieldType.STRING).withIsEditable(false).withMaxLength(4096).withBehavior(ValueTooLongBehavior.ERROR))
+         .withField(new QFieldMetaData("createDate", QFieldType.DATE_TIME).withIsEditable(false));
+
+      return tableMetaData;
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/backend/implementations/memory/MemoryRecordStore.java

@@ -74,6 +74,7 @@ import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -364,6 +365,9 @@ public class MemoryRecordStore
          // differently from other backends, because of having the same record variable in the backend store and in the user-code. //
          ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
          QRecord recordToInsert = new QRecord(record);
+
+         makeValueTypesMatchFieldTypes(table, recordToInsert);
+
          if(CollectionUtils.nullSafeHasContents(recordToInsert.getErrors()))
          {
             outputRecords.add(recordToInsert);
@@ -414,6 +418,30 @@ public class MemoryRecordStore
 
 
 
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static void makeValueTypesMatchFieldTypes(QTableMetaData table, QRecord recordToInsert)
+   {
+      for(QFieldMetaData field : table.getFields().values())
+      {
+         Serializable value = recordToInsert.getValue(field.getName());
+         if(value != null)
+         {
+            try
+            {
+               recordToInsert.setValue(field.getName(), ValueUtils.getValueAsFieldType(field.getType(), value));
+            }
+            catch(Exception e)
+            {
+               LOG.info("Error converting value to field's type", e, logPair("fieldName", field.getName()), logPair("value", value));
+            }
+         }
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -444,7 +472,19 @@ public class MemoryRecordStore
             QRecord recordToUpdate = tableData.get(primaryKeyValue);
             for(Map.Entry<String, Serializable> valueEntry : record.getValues().entrySet())
             {
-               recordToUpdate.setValue(valueEntry.getKey(), valueEntry.getValue());
+               String fieldName = valueEntry.getKey();
+               try
+               {
+                  ///////////////////////////////////////////////
+                  // try to make field values match field type //
+                  ///////////////////////////////////////////////
+                  recordToUpdate.setValue(fieldName, ValueUtils.getValueAsFieldType(table.getField(fieldName).getType(), valueEntry.getValue()));
+               }
+               catch(Exception e)
+               {
+                  LOG.info("Error converting value to field's type", e, logPair("fieldName", fieldName), logPair("value", valueEntry.getValue()));
+                  recordToUpdate.setValue(fieldName, valueEntry.getValue());
+               }
             }
 
             if(returnUpdatedRecords)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/scheduler/processes/AbstractRecordSyncToScheduledJobProcess.java

@@ -0,0 +1,276 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.scheduler.processes;
+
+
+import java.io.Serializable;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QSchedulerMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobParameter;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobType;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.processes.implementations.tablesync.AbstractTableSyncTransformStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.tablesync.TableSyncProcess;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.collections.ListBuilder;
+
+
+/*******************************************************************************
+ * Base class to manage creating scheduled jobs based on records in another table
+ *
+ * Expected to be used via BaseSyncToScheduledJobTableCustomizer - see its javadoc.
+ * @see BaseSyncToScheduledJobTableCustomizer
+ *******************************************************************************/
+public abstract class AbstractRecordSyncToScheduledJobProcess extends AbstractTableSyncTransformStep implements MetaDataProducerInterface<QProcessMetaData>
+{
+   private static final QLogger LOG = QLogger.getLogger(AbstractRecordSyncToScheduledJobProcess.class);
+
+   public static final String SCHEDULER_NAME_FIELD_NAME = "schedulerName";
+
+
+
+   /***************************************************************************
+    * action flags that can be put in an insert/update/delete input to control
+    * behavior of this process.
+    ***************************************************************************/
+   public enum ActionFlags implements ActionFlag
+   {
+      /***************************************************************************
+       * tell this process not to run upon such an action taken on the source table.
+       ***************************************************************************/
+      DO_NOT_SYNC
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QProcessMetaData produce(QInstance qInstance) throws QException
+   {
+      QProcessMetaData processMetaData = TableSyncProcess.processMetaDataBuilder(false)
+         .withName(getClass().getSimpleName())
+         .withSyncTransformStepClass(getClass())
+         .withReviewStepRecordFields(List.of(
+            new QFieldMetaData(getRecordForeignKeyFieldName(), QFieldType.INTEGER).withPossibleValueSourceName(getRecordForeignKeyPossibleValueSourceName()),
+            new QFieldMetaData("cronExpression", QFieldType.STRING),
+            new QFieldMetaData("isActive", QFieldType.BOOLEAN)
+         ))
+         .getProcessMetaData();
+
+      processMetaData.getBackendStep(StreamedETLWithFrontendProcess.STEP_NAME_PREVIEW).getInputMetaData()
+         .withField(new QFieldMetaData(SCHEDULER_NAME_FIELD_NAME, QFieldType.STRING));
+
+      return (processMetaData);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QRecord populateRecordToStore(RunBackendStepInput runBackendStepInput, QRecord destinationRecord, QRecord sourceRecord) throws QException
+   {
+      ScheduledJob scheduledJob;
+      if(destinationRecord == null || destinationRecord.getValue("id") == null)
+      {
+         QInstance qInstance = QContext.getQInstance();
+
+         ////////////////////////////////////////////////////////////////
+         // this is the table at which the scheduled job will point to //
+         ////////////////////////////////////////////////////////////////
+         QTableMetaData sourceTableMetaData = qInstance.getTable(getSourceTableName());
+         String         sourceTableId       = String.valueOf(sourceRecord.getValueString(sourceTableMetaData.getPrimaryKeyField()));
+         String         sourceTableJobKey   = getSourceTableName() + "Id";
+
+         ///////////////////////////////////////////////////////////
+         // this is the table that the scheduled record points to //
+         ///////////////////////////////////////////////////////////
+         QTableMetaData recordForeignTableMetaData = qInstance.getTable(getRecordForeignKeyPossibleValueSourceName());
+         String         sourceRecordForeignKeyId   = sourceRecord.getValueString(getRecordForeignKeyFieldName());
+
+         ////////////////////////////////////////////////////////////////////////
+         // need to do an insert - set lots of key values in the scheduled job //
+         ////////////////////////////////////////////////////////////////////////
+         scheduledJob = new ScheduledJob();
+         scheduledJob.setSchedulerName(runBackendStepInput.getValueString(SCHEDULER_NAME_FIELD_NAME));
+         scheduledJob.setType(ScheduledJobType.PROCESS.name());
+         scheduledJob.setForeignKeyType(getSourceTableName());
+         scheduledJob.setForeignKeyValue(sourceTableId);
+         scheduledJob.setJobParameters(ListBuilder.of(
+            new ScheduledJobParameter().withKey("isScheduledJob").withValue("true"),
+            new ScheduledJobParameter().withKey("processName").withValue(getProcessNameScheduledJobParameter()),
+            new ScheduledJobParameter().withKey(sourceTableJobKey).withValue(sourceTableId),
+            new ScheduledJobParameter().withKey("recordId").withValue(ValueUtils.getValueAsString(sourceRecordForeignKeyId))
+         ));
+
+         //////////////////////////////////////////////////////////////////////////
+         // make a call to allow subclasses to customize parts of the job record //
+         //////////////////////////////////////////////////////////////////////////
+         scheduledJob.setLabel(recordForeignTableMetaData.getLabel() + " " + sourceRecordForeignKeyId);
+         scheduledJob.setDescription("Job to run " + sourceTableMetaData.getLabel() + " Id " + sourceTableId
+            + " (which runs for " + recordForeignTableMetaData.getLabel() + " Id " + sourceRecordForeignKeyId + ")");
+      }
+      else
+      {
+         //////////////////////////////////////////////////////////////////////////////////
+         // else doing an update - populate scheduled job entity from destination record //
+         //////////////////////////////////////////////////////////////////////////////////
+         scheduledJob = new ScheduledJob(destinationRecord);
+      }
+
+      //////////////////////////////////////////////////////////////////////////////////
+      // these fields sync on insert and update                                       //
+      // todo - if no diffs, should we return null (to avoid changing quartz at all?) //
+      //////////////////////////////////////////////////////////////////////////////////
+      scheduledJob.setCronExpression(sourceRecord.getValueString("cronExpression"));
+      scheduledJob.setCronTimeZoneId(sourceRecord.getValueString("cronTimeZoneId"));
+      scheduledJob.setIsActive(true);
+
+      scheduledJob = customizeScheduledJob(scheduledJob, sourceRecord);
+
+      ////////////////////////////////////////////////////////////////////
+      // try to make sure scheduler name is set (and fail if it isn't!) //
+      ////////////////////////////////////////////////////////////////////
+      makeSureSchedulerNameIsSet(scheduledJob);
+
+      return scheduledJob.toQRecord();
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected void makeSureSchedulerNameIsSet(ScheduledJob scheduledJob) throws QException
+   {
+      if(!StringUtils.hasContent(scheduledJob.getSchedulerName()))
+      {
+         Map<String, QSchedulerMetaData> schedulers = QContext.getQInstance().getSchedulers();
+         if(schedulers.size() == 1)
+         {
+            scheduledJob.setSchedulerName(schedulers.keySet().iterator().next());
+         }
+      }
+
+      if(!StringUtils.hasContent(scheduledJob.getSchedulerName()))
+      {
+         String message = "Could not determine scheduler name for webhook scheduled job.";
+         LOG.warn(message);
+         throw (new QException(message));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected ScheduledJob customizeScheduledJob(ScheduledJob scheduledJob, QRecord sourceRecord) throws QException
+   {
+      return (scheduledJob);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   protected QQueryFilter getExistingRecordQueryFilter(RunBackendStepInput runBackendStepInput, List<Serializable> sourceKeyList)
+   {
+      return super.getExistingRecordQueryFilter(runBackendStepInput, sourceKeyList)
+         .withCriteria(new QFilterCriteria("foreignKeyType", QCriteriaOperator.EQUALS, getScheduledJobForeignKeyType()));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   protected SyncProcessConfig getSyncProcessConfig()
+   {
+      return new SyncProcessConfig(getSourceTableName(), getSourceTableKeyField(), ScheduledJob.TABLE_NAME, "foreignKeyValue", true, true);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected abstract String getScheduledJobForeignKeyType();
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected abstract String getRecordForeignKeyFieldName();
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected abstract String getRecordForeignKeyPossibleValueSourceName();
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected abstract String getSourceTableName();
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected abstract String getProcessNameScheduledJobParameter();
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   protected String getSourceTableKeyField()
+   {
+      return ("id");
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/scheduler/processes/BaseSyncToScheduledJobTableCustomizer.java

@@ -0,0 +1,387 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.scheduler.processes;
+
+
+import java.io.Serializable;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
+import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.InitializableViaCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** an implementation of a TableCustomizer that runs a subclass of
+ ** AbstractRecordSyncToScheduledJobProcess - to manage scheduledJob records that
+ ** correspond to records in another table (e.g., a job for each Client)
+ **
+ ** Easiest way to use is:
+ ** - BaseSyncToScheduledJobTableCustomizer.setTableCustomizers(tableMetaData, new YourSyncScheduledJobProcessSubclass());
+ ** which adds post-insert, -update, and -delete customizers to your table.
+ **
+ ** If you need additional table customizer code in those slots, I suppose you could
+ ** simply make your customizer create an instance of this class, set its
+ ** properties, and run its appropriate postInsertOrUpdate/postDelete methods.
+ *******************************************************************************/
+public class BaseSyncToScheduledJobTableCustomizer implements TableCustomizerInterface, InitializableViaCodeReference
+{
+   private static final QLogger LOG = QLogger.getLogger(BaseSyncToScheduledJobTableCustomizer.class);
+
+   public static final String KEY_TABLE_NAME                     = "tableName";
+   public static final String KEY_SYNC_PROCESS_NAME              = "syncProcessName";
+   public static final String KEY_SCHEDULED_JOB_FOREIGN_KEY_TYPE = "scheduledJobForeignKeyType";
+
+   private String tableName;
+   private String syncProcessName;
+   private String scheduledJobForeignKeyType;
+
+
+
+   /***************************************************************************
+    * Create a {@link QCodeReferenceWithProperties} that can be used to add this
+    * class to a table.
+    *
+    * If this is the only customizer for the post insert/update/delete events
+    * on your table, you can instead call setTableCustomizers.  But if you want,
+    * for example, a sync-scheduled-job (what this customizer does) plus some other
+    * customizers, then you can call this method to get a code reference that you
+    * can add, for example, to {@link com.kingsrook.qqq.backend.core.actions.customizers.MultiCustomizer}
+    *
+    * @param tableMetaData the table that the customizer will be used on.
+    * @param syncProcess instance of the subclass of AbstractRecordSyncToScheduledJobProcess
+    *                    that should run in the table's post insert/update/delete
+    *                    events.
+    * @see #setTableCustomizers(QTableMetaData, AbstractRecordSyncToScheduledJobProcess)
+    ***************************************************************************/
+   public static QCodeReferenceWithProperties makeCodeReference(QTableMetaData tableMetaData, AbstractRecordSyncToScheduledJobProcess syncProcess)
+   {
+      return new QCodeReferenceWithProperties(BaseSyncToScheduledJobTableCustomizer.class, Map.of(
+         KEY_TABLE_NAME, tableMetaData.getName(),
+         KEY_SYNC_PROCESS_NAME, syncProcess.getClass().getSimpleName(),
+         KEY_SCHEDULED_JOB_FOREIGN_KEY_TYPE, syncProcess.getScheduledJobForeignKeyType()
+      ));
+   }
+
+
+
+   /***************************************************************************
+    * Add post insert/update/delete customizers to a table, that will run a
+    * sync-scheduled-job process.
+    *
+    * @param tableMetaData the table that the customizer will be used on.
+    * @param syncProcess instance of the subclass of AbstractRecordSyncToScheduledJobProcess
+    *                    that should run in the table's post insert/update/delete
+    *                    events.
+    ***************************************************************************/
+   public static void setTableCustomizers(QTableMetaData tableMetaData, AbstractRecordSyncToScheduledJobProcess syncProcess)
+   {
+      QCodeReference codeReference = makeCodeReference(tableMetaData, syncProcess);
+      tableMetaData.withCustomizer(TableCustomizers.POST_INSERT_RECORD, codeReference);
+      tableMetaData.withCustomizer(TableCustomizers.POST_UPDATE_RECORD, codeReference);
+      tableMetaData.withCustomizer(TableCustomizers.POST_DELETE_RECORD, codeReference);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public void initialize(QCodeReference codeReference)
+   {
+      if(codeReference instanceof QCodeReferenceWithProperties codeReferenceWithProperties)
+      {
+         tableName = ValueUtils.getValueAsString(codeReferenceWithProperties.getProperties().get(KEY_TABLE_NAME));
+         syncProcessName = ValueUtils.getValueAsString(codeReferenceWithProperties.getProperties().get(KEY_SYNC_PROCESS_NAME));
+         scheduledJobForeignKeyType = ValueUtils.getValueAsString(codeReferenceWithProperties.getProperties().get(KEY_SCHEDULED_JOB_FOREIGN_KEY_TYPE));
+
+         if(!StringUtils.hasContent(tableName))
+         {
+            LOG.warn("Missing property under KEY_TABLE_NAME [" + KEY_TABLE_NAME + "] in codeReference for BaseSyncToScheduledJobTableCustomizer");
+         }
+
+         if(!StringUtils.hasContent(syncProcessName))
+         {
+            LOG.warn("Missing property under KEY_SYNC_PROCESS_NAME [" + KEY_SYNC_PROCESS_NAME + "] in codeReference for BaseSyncToScheduledJobTableCustomizer");
+         }
+
+         if(!StringUtils.hasContent(scheduledJobForeignKeyType))
+         {
+            LOG.warn("Missing property under KEY_SCHEDULED_JOB_FOREIGN_KEY_TYPE [" + KEY_SCHEDULED_JOB_FOREIGN_KEY_TYPE + "] in codeReference for BaseSyncToScheduledJobTableCustomizer");
+         }
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postInsertOrUpdate(AbstractActionInput input, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      if(input instanceof UpdateInput updateInput && updateInput.hasFlag(AbstractRecordSyncToScheduledJobProcess.ActionFlags.DO_NOT_SYNC))
+      {
+         return records;
+      }
+
+      if(input instanceof InsertInput insertInput && insertInput.hasFlag(AbstractRecordSyncToScheduledJobProcess.ActionFlags.DO_NOT_SYNC))
+      {
+         return records;
+      }
+
+      runSyncProcessForRecordList(records, syncProcessName);
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
+   {
+      deleteScheduledJobsForRecordList(records);
+      return records;
+   }
+
+
+
+   /***************************************************************************
+    * Run the named process over a set of records (e.g., that were inserted or
+    * updated).
+    *
+    * This method is normally called from within this class, in postInsertOrUpdate.
+    *
+    * Note that if the {@link ScheduledJob} table isn't defined in the QInstance,
+    * that the process will not be called.
+    *
+    * @param records list of records to use as source records in the table-sync
+    *                to the scheduledJob table.
+    * @param processName name of the sync-process to run.
+    ***************************************************************************/
+   public void runSyncProcessForRecordList(List<QRecord> records, String processName)
+   {
+      if(QContext.getQInstance().getTable(ScheduledJob.TABLE_NAME) == null)
+      {
+         LOG.info("ScheduledJob table not found, skipping scheduled job sync.");
+         return;
+      }
+
+      String primaryKeyField = QContext.getQInstance().getTable(tableName).getPrimaryKeyField();
+
+      List<Serializable> sourceRecordIds = records.stream()
+         .filter(r -> CollectionUtils.nullSafeIsEmpty(r.getErrors()))
+         .map(r -> r.getValue(primaryKeyField))
+         .filter(Objects::nonNull).toList();
+
+      if(CollectionUtils.nullSafeIsEmpty(sourceRecordIds))
+      {
+         return;
+      }
+
+      try
+      {
+         RunProcessInput runProcessInput = new RunProcessInput();
+         runProcessInput.setProcessName(processName);
+         runProcessInput.setCallback(QProcessCallbackFactory.forPrimaryKeys("id", sourceRecordIds));
+         runProcessInput.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
+         RunProcessOutput runProcessOutput = new RunProcessAction().execute(runProcessInput);
+
+         Serializable processSummary = runProcessOutput.getValue(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY);
+         ProcessSummaryLineInterface.log("Sync to ScheduledJob Process Summary", processSummary, List.of(logPair("sourceTable", tableName)));
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error syncing records to scheduled jobs", e, logPair("sourceTable", tableName), logPair("sourceRecordIds", sourceRecordIds));
+      }
+   }
+
+
+
+   /***************************************************************************
+    * Delete scheduled job records for source-table records that have been deleted.
+    *
+    * This method is normally called from within this class, in postDelete.
+    *
+    * Note that if the {@link ScheduledJob} table isn't defined in the QInstance,
+    * that the process will not be called.
+    *
+    * @param records list of records to use as foreign-key sources to identify
+    *                scheduledJob records to delete
+    ***************************************************************************/
+   public void deleteScheduledJobsForRecordList(List<QRecord> records)
+   {
+      if(QContext.getQInstance().getTable(ScheduledJob.TABLE_NAME) == null)
+      {
+         LOG.info("ScheduledJob table not found, skipping scheduled job delete.");
+         return;
+      }
+
+      List<String> sourceRecordIds = records.stream()
+         .filter(r -> CollectionUtils.nullSafeIsEmpty(r.getErrors()))
+         .map(r -> r.getValueString("id")).toList();
+
+      if(sourceRecordIds.isEmpty())
+      {
+         return;
+      }
+
+      ///////////////////////////////////////////////////
+      // delete any corresponding scheduledJob records //
+      ///////////////////////////////////////////////////
+      try
+      {
+         new DeleteAction().execute(new DeleteInput(ScheduledJob.TABLE_NAME).withQueryFilter(new QQueryFilter()
+            .withCriteria(new QFilterCriteria("foreignKeyType", QCriteriaOperator.EQUALS, getScheduledJobForeignKeyType()))
+            .withCriteria(new QFilterCriteria("foreignKeyValue", QCriteriaOperator.IN, sourceRecordIds))));
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error deleting scheduled jobs for scheduled records", e, logPair("sourceTable", tableName), logPair("sourceRecordIds", sourceRecordIds));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tableName
+    *******************************************************************************/
+   public String getTableName()
+   {
+      return (this.tableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tableName
+    *******************************************************************************/
+   public void setTableName(String tableName)
+   {
+      this.tableName = tableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tableName
+    *******************************************************************************/
+   public BaseSyncToScheduledJobTableCustomizer withTableName(String tableName)
+   {
+      this.tableName = tableName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for syncProcessName
+    *******************************************************************************/
+   public String getSyncProcessName()
+   {
+      return (this.syncProcessName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for syncProcessName
+    *******************************************************************************/
+   public void setSyncProcessName(String syncProcessName)
+   {
+      this.syncProcessName = syncProcessName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for syncProcessName
+    *******************************************************************************/
+   public BaseSyncToScheduledJobTableCustomizer withSyncProcessName(String syncProcessName)
+   {
+      this.syncProcessName = syncProcessName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for scheduledJobForeignKeyType
+    *******************************************************************************/
+   public String getScheduledJobForeignKeyType()
+   {
+      return (this.scheduledJobForeignKeyType);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for scheduledJobForeignKeyType
+    *******************************************************************************/
+   public void setScheduledJobForeignKeyType(String scheduledJobForeignKeyType)
+   {
+      this.scheduledJobForeignKeyType = scheduledJobForeignKeyType;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for scheduledJobForeignKeyType
+    *******************************************************************************/
+   public BaseSyncToScheduledJobTableCustomizer withScheduledJobForeignKeyType(String scheduledJobForeignKeyType)
+   {
+      this.scheduledJobForeignKeyType = scheduledJobForeignKeyType;
+      return (this);
+   }
+
+}