added tests

added labels as a param when searching for PVSs

docs/Introduction.adoc

@@ -91,7 +91,7 @@ And then having a bug in the check permission logic on the _Light Bulb Inventory
 No!
 
 All of the (really important, even though application developers hate doing it) aspects of security - you don't need to write ANY code for dealing with that.
-Just tell QQQ what Authentication provider you want to use (e.g., https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
+Just tell QQQ what Authentication provider you want to use (e.g., OAuth2 or https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
 QQQ just does it.
 
 ''''

docs/index.adoc

@@ -31,11 +31,9 @@ include::metaData/PermissionRules.adoc[leveloffset=+1]
 
 == Services
 
+include::misc/Javalin.adoc[leveloffset=+1]
 include::misc/ScheduledJobs.adoc[leveloffset=+1]
 
-=== Web server (Javalin)
-#todo#
-
 === API server (OpenAPI)
 #todo#
 

docs/misc/Javalin.adoc

@@ -0,0 +1,109 @@
+== QQQ Middleware:  Javalin web server
+include::../variables.adoc[]
+
+QQQ provides a standard implementation of a middleware layer - that is - code that exists between the
+QQQ backend and user interface.  This implementation is a web server built using the https://javalin.io/[Javalin framework],
+packaged and deployed in the `qqq-middleware-javalin` maven module
+
+The de facto way to create a QQQ application server is to write a class which uses an instance of one of the
+subclasses of `QApplicationJavalinServer`.
+
+For example, if your application metadata is defined in a directory of yaml files, your server class could be implemented as:
+
+[source,java]
+.ConfigFileBasedQQQApplication usage example
+----
+public static void main(String[] args)
+{
+   try
+   {
+      String path = "src/main/resources/metadata";
+      ConfigFilesBasedQQQApplication application = new ConfigFilesBasedQQQApplication(path);
+      QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(application);
+      javalinServer.start();
+   }
+   catch(Exception e)
+   {
+      LOG.error("Failed to start javalin server.  See stack trace for details.", e);
+   }
+}
+----
+
+A similar class exists if your metadata is produced by a package of Java MetaDataProducer objects:  `MetaDataProducerBasedQQQApplication`.
+
+=== QApplicationJavalinServer
+This class provides the bridge between your QQQ Application (e.g., your metadata) and the QQQ Middleware layer
+served by a Javalin web server.  It has several properties to control behaviors:
+
+* `Integer port` - (default `8000`) - port to use for serving HTTP.
+* `boolean serveFrontendMaterialDashboard` - (default `true`) whether to serve the javascript frontend provided
+in the maven artifact `qqq-frontend-material-dashboard`.
+* `boolean serveLegacyUnversionedMiddlewareAPI` - (default `true`) whether to serve a version the original implementation
+of the QQQ middleware, which current version of `qqq-frontend-material-dashboard` are compatible with.
+* `List<AbstractMiddlewareVersion> middlewareVersionList` - (default contains `MiddlewareVersionV1`) - list of
+newer, formally versioned implementations of the QQQ middleware interface to be served.
+* `Consumer<Javalin> javalinConfigurationCustomizer` - (default `null`) - optional hook to customize the
+javalin service object before it is started.
+* `List<QJavalinRouteProviderInterface> additionalRouteProviders` - (default `null`) - list of fully custom
+implementations of `QJavalinRouteProviderInterface`, to add additional endpoints to the javalin server.
+** _Note, you may first want to consider using JavalinRouteProviderMetaData instead - see below._
+* `QJavalinMetaData javalinMetaData` - (default `null`) - optional alternative place to define `JavalinMetaData` (vs.
+defining it in the `QInstance`). _Note that if it is set in both places, the one in the QApplicationJavalinServer
+is used._
+
+=== JavalinMetaData
+Certain behaviors of a QQQ Javalin server are configured in a declarative manner by adding a `QJavalinMetaData`
+object to the `supplementalMetaData` in your `QInstance` (or, as mentioned above, by setting it directly on the
+`QApplicationJavalinServer`):
+
+* `List<JavalinRouteProviderMetaData> routeProviders` - (default `null`) optional list of custom route providers to
+add to the Javalin server.  See below for details.
+* `String uploadedFileArchiveTableName` - (default `null`) - reference to a QQQ Table in your application instance,
+needed to support the Bulk Load process, as well as any other processes which need to accept an uploaded file
+as input.
+* `boolean loggerDisabled` - (default `false`)
+* `Function<QJavalinAccessLogger.LogEntry, Boolean> logFilter` - (default `null`)
+* `boolean queryWithoutLimitAllowed` - (default `false`)
+* `Integer queryWithoutLimitDefault` - (default `1000`)
+* `Level queryWithoutLimitLogLevel` - (default `INFO`)
+
+==== JavalinRouteProviderMetaData
+This type of metadata allows you to add additional http route providers to your Javalin instance, e.g., for
+serving static files or for running custom code from your application (in the form of QQQ Processes) to respond
+to HTTP requests.
+
+* `String hostedPath` - (required)
+* `String fileSystemPath` - (required for a static router)
+* `String processName` - required for a dynamic, process-based router.  Must be a process name within the QQQ Instance.
+See below for additional details
+* `List<String> methods` - required list of HTTP methods (verbs) that are served by the route provider
+* `QCodeReference routeAuthenticator - Optional reference to a class that implements `RouteAuthenticatorInterface`,
+to provide security authentication to all requests handled by the route provider.
+** A default implementation is provided as `SimpleRouteAuthenticator`, which requires that a user session be present
+to access paths served by the route provider.
+
+===== Process-based route provider processes
+If you define a `JavalinRouteProviderMetaData` with a `processName` (e.g., to serve dynamic HTTP responses from your javalin
+server), the process that you implement will be called to respond to any HTTP requests received by the javalin
+server which match the `hostedPath` and `methods` that are specified in the metadata.
+
+The QQQ javalin server will marshal request data from the javalin context into the process's payload, conforming to
+the shape of the `ProcessBasedRouterPayload` class.  Similarly, the http response will be built by taking values from
+the process's output/state conforming to the fields in that class.  As such, it is recommended to use a
+`ProcessBasedRouterPayload`  instance, as show in this example:
+
+[source,java]
+.Process-based router usage example (including ProcessBasedRouterPayload)
+----
+public class MyDynamicSiteProcessStep implements BackendStep
+{
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+   {
+      ProcessBasedRouterPayload payload = input.getProcessPayload(ProcessBasedRouterPayload.class);
+      String path = payload.getPath();
+      payload.setResponseString("You requested: " + path);
+      output.setProcessPayload(payload);
+   }
+}
+----

pom.xml

@@ -48,7 +48,7 @@
    </modules>
 
    <properties>
-      <revision>0.25.0-SNAPSHOT</revision>
+      <revision>0.26.0-SNAPSHOT</revision>
 
       <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
       <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

qqq-backend-core/pom.xml

@@ -65,7 +65,11 @@
          <artifactId>aws-java-sdk-secretsmanager</artifactId>
          <version>1.12.385</version>
       </dependency>
-
+      <dependency>
+         <groupId>com.ibm.icu</groupId>
+         <artifactId>icu4j</artifactId>
+         <version>77.1</version>
+      </dependency>
       <dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-databind</artifactId>
@@ -125,10 +129,16 @@
          <version>2.16.0</version>
       </dependency>
 
+      <dependency>
+         <groupId>com.nimbusds</groupId>
+         <artifactId>oauth2-oidc-sdk</artifactId>
+         <version>11.23.1</version>
+      </dependency>
+
       <dependency>
          <groupId>com.auth0</groupId>
          <artifactId>auth0</artifactId>
-         <version>2.1.0</version>
+         <version>2.18.0</version>
       </dependency>
       <dependency>
          <groupId>com.auth0</groupId>
@@ -138,12 +148,12 @@
       <dependency>
          <groupId>com.auth0</groupId>
          <artifactId>jwks-rsa</artifactId>
-         <version>0.22.0</version>
+         <version>0.22.1</version>
       </dependency>
       <dependency>
          <groupId>io.github.cdimascio</groupId>
-         <artifactId>java-dotenv</artifactId>
-         <version>5.2.2</version>
+         <artifactId>dotenv-java</artifactId>
+         <version>3.2.0</version>
       </dependency>
       <dependency>
          <groupId>org.apache.velocity</groupId>
@@ -151,16 +161,21 @@
          <version>2.3</version>
       </dependency>
 
-      <!-- the next 2 deps are for html to pdf - per https://www.baeldung.com/java-html-to-pdf -->
+      <!-- the next 3 deps are for html to pdf -->
       <dependency>
          <groupId>org.jsoup</groupId>
          <artifactId>jsoup</artifactId>
          <version>1.15.3</version>
       </dependency>
       <dependency>
-         <groupId>org.xhtmlrenderer</groupId>
-         <artifactId>flying-saucer-pdf-openpdf</artifactId>
-         <version>9.1.22</version>
+         <groupId>com.openhtmltopdf</groupId>
+         <artifactId>openhtmltopdf-core</artifactId>
+         <version>1.0.10</version>
+      </dependency>
+      <dependency>
+         <groupId>com.openhtmltopdf</groupId>
+         <artifactId>openhtmltopdf-pdfbox</artifactId>
+         <version>1.0.10</version>
       </dependency>
 
       <!-- the next 3 deps are being added for google drive support -->

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/permissions/BulkTableActionProcessPermissionChecker.java

@@ -57,7 +57,7 @@ public class BulkTableActionProcessPermissionChecker implements CustomPermission
          switch(bulkActionName)
          {
             case "bulkInsert" -> PermissionsHelper.checkTablePermissionThrowing(tableActionInput, TablePermissionSubType.INSERT);
-            case "bulkEdit" -> PermissionsHelper.checkTablePermissionThrowing(tableActionInput, TablePermissionSubType.EDIT);
+            case "bulkEdit", "bulkEditWithFile" -> PermissionsHelper.checkTablePermissionThrowing(tableActionInput, TablePermissionSubType.EDIT);
             case "bulkDelete" -> PermissionsHelper.checkTablePermissionThrowing(tableActionInput, TablePermissionSubType.DELETE);
             default -> LOG.warn("Unexpected bulk action name when checking permissions for process: " + processName);
          }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/templates/ConvertHtmlToPdfAction.java

@@ -26,22 +26,32 @@ import java.nio.file.Path;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.actions.AbstractQActionFunction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.templates.ConvertHtmlToPdfInput;
 import com.kingsrook.qqq.backend.core.model.actions.templates.ConvertHtmlToPdfOutput;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.openhtmltopdf.css.constants.IdentValue;
+import com.openhtmltopdf.pdfboxout.PdfBoxFontResolver;
+import com.openhtmltopdf.pdfboxout.PdfBoxRenderer;
+import com.openhtmltopdf.pdfboxout.PdfRendererBuilder;
 import org.jsoup.Jsoup;
+import org.jsoup.helper.W3CDom;
 import org.jsoup.nodes.Document;
-import org.xhtmlrenderer.layout.SharedContext;
-import org.xhtmlrenderer.pdf.ITextRenderer;
 
 
 /*******************************************************************************
  ** Action to convert a string of HTML to a PDF!
  **
  ** Much credit to https://www.baeldung.com/java-html-to-pdf
- *******************************************************************************/
+ **
+ ** Updated in March 2025 to go from flying-saucer-pdf-openpdf lib to openhtmltopdf,
+ ** mostly to get support for max-height on images...
+ ********************************************************************************/
 public class ConvertHtmlToPdfAction extends AbstractQActionFunction<ConvertHtmlToPdfInput, ConvertHtmlToPdfOutput>
 {
+   private static final QLogger LOG = QLogger.getLogger(ConvertHtmlToPdfAction.class);
+
+
 
    /*******************************************************************************
     **
@@ -58,35 +68,37 @@ public class ConvertHtmlToPdfAction extends AbstractQActionFunction<ConvertHtmlT
          //////////////////////////////////////////////////////////////////
          Document document = Jsoup.parse(input.getHtml());
          document.outputSettings().syntax(Document.OutputSettings.Syntax.xml);
+         org.w3c.dom.Document w3cDoc = new W3CDom().fromJsoup(document);
 
          //////////////////////////////
          // convert the XHTML to PDF //
          //////////////////////////////
-         ITextRenderer renderer      = new ITextRenderer();
-         SharedContext sharedContext = renderer.getSharedContext();
-         sharedContext.setPrint(true);
-         sharedContext.setInteractive(false);
+         PdfRendererBuilder builder = new PdfRendererBuilder();
+         builder.toStream(input.getOutputStream());
+         builder.useFastMode();
+         builder.withW3cDocument(w3cDoc, input.getBasePath() == null ? "./" : input.getBasePath().toUri().toString());
 
-         if(input.getBasePath() != null)
+         try(PdfBoxRenderer pdfBoxRenderer = builder.buildPdfRenderer())
          {
-            String baseUrl = input.getBasePath().toUri().toURL().toString();
-            renderer.setDocumentFromString(document.html(), baseUrl);
-         }
-         else
-         {
-            renderer.setDocumentFromString(document.html());
-         }
+            pdfBoxRenderer.layout();
+            pdfBoxRenderer.getSharedContext().setPrint(true);
+            pdfBoxRenderer.getSharedContext().setInteractive(false);
 
-         //////////////////////////////////////////////////
-         // register any custom fonts the input supplied //
-         //////////////////////////////////////////////////
-         for(Map.Entry<String, Path> entry : CollectionUtils.nonNullMap(input.getCustomFonts()).entrySet())
-         {
-            renderer.getFontResolver().addFont(entry.getValue().toAbsolutePath().toString(), entry.getKey(), "UTF-8", true, null);
-         }
+            for(Map.Entry<String, Path> entry : CollectionUtils.nonNullMap(input.getCustomFonts()).entrySet())
+            {
+               LOG.warn("Note:  Custom fonts appear to not be working in this class at this time...");
+               pdfBoxRenderer.getFontResolver().addFont(
+                  entry.getValue().toAbsolutePath().toFile(), // Path to the TrueType font file
+                  entry.getKey(),                             // Font family name to use in CSS
+                  400,                                        // Font weight (e.g., 400 for normal, 700 for bold)
+                  IdentValue.NORMAL,                          // Font style (e.g., NORMAL, ITALIC)
+                  true,                                       // Whether to subset the font
+                  PdfBoxFontResolver.FontGroup.MAIN           // ??
+               );
+            }
 
-         renderer.layout();
-         renderer.createPDF(input.getOutputStream());
+            pdfBoxRenderer.createPDF();
+         }
 
          return (output);
       }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/BasicCustomPossibleValueProvider.java

@@ -0,0 +1,91 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.values;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
+
+
+/*******************************************************************************
+ ** Basic implementation of a possible value provider, for where there's a limited
+ ** set of possible source objects - so you just have to define how to make one
+ ** PV from a source object, how to list all of the source objects, and how to
+ ** look up a PV from an id.
+ *******************************************************************************/
+public abstract class BasicCustomPossibleValueProvider<S, ID extends Serializable> implements QCustomPossibleValueProvider<ID>
+{
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected abstract QPossibleValue<ID> makePossibleValue(S sourceObject);
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected abstract S getSourceObject(Serializable id);
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected abstract List<S> getAllSourceObjects();
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QPossibleValue<ID> getPossibleValue(Serializable idValue)
+   {
+      S sourceObject = getSourceObject(idValue);
+      if(sourceObject == null)
+      {
+         return (null);
+      }
+
+      return makePossibleValue(sourceObject);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public List<QPossibleValue<ID>> search(SearchPossibleValueSourceInput input) throws QException
+   {
+      List<QPossibleValue<ID>> allPossibleValues = new ArrayList<>();
+      List<S>                  allSourceObjects  = getAllSourceObjects();
+      for(S sourceObject : allSourceObjects)
+      {
+         allPossibleValues.add(makePossibleValue(sourceObject));
+      }
+
+      return completeCustomPVSSearch(input, allPossibleValues);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/AbstractMetaDataProducerBasedQQQApplication.java

@@ -29,7 +29,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 
 /*******************************************************************************
  ** Version of AbstractQQQApplication that assumes all meta-data is produced
- ** by MetaDataProducers in a single package.
+ ** by MetaDataProducers in (or under) a single package.
  *******************************************************************************/
 public abstract class AbstractMetaDataProducerBasedQQQApplication extends AbstractQQQApplication
 {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/ConfigFilesBasedQQQApplication.java

@@ -0,0 +1,61 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.instances.loaders.MetaDataLoaderHelper;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+
+
+/*******************************************************************************
+ ** Version of AbstractQQQApplication that assumes all meta-data is defined in
+ ** config files (yaml, json, etc) under a given directory path.
+ *******************************************************************************/
+public class ConfigFilesBasedQQQApplication extends AbstractQQQApplication
+{
+   private final String path;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public ConfigFilesBasedQQQApplication(String path)
+   {
+      this.path = path;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QInstance defineQInstance() throws QException
+   {
+      QInstance qInstance = new QInstance();
+      MetaDataLoaderHelper.processAllMetaDataFilesInDirectory(qInstance, path);
+      return (qInstance);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/MetaDataProducerBasedQQQApplication.java

@@ -0,0 +1,67 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances;
+
+
+/*******************************************************************************
+ ** Version of AbstractQQQApplication that assumes all meta-data is produced
+ ** by MetaDataProducers in (or under) a single package (where you can pass that
+ ** package into the constructor, vs. the abstract base class, where you extend
+ ** it and override the getMetaDataPackageName method.
+ *******************************************************************************/
+public class MetaDataProducerBasedQQQApplication extends AbstractMetaDataProducerBasedQQQApplication
+{
+   private final String metaDataPackageName;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public MetaDataProducerBasedQQQApplication(String metaDataPackageName)
+   {
+      this.metaDataPackageName = metaDataPackageName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public MetaDataProducerBasedQQQApplication(Class<?> aClassInMetaDataPackage)
+   {
+      this(aClassInMetaDataPackage.getPackageName());
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getMetaDataPackageName()
+   {
+      return (this.metaDataPackageName);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceEnricher.java

@@ -45,6 +45,7 @@ import com.kingsrook.qqq.backend.core.actions.values.QCustomPossibleValueProvide
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.instances.enrichment.plugins.QInstanceEnricherPluginInterface;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.bulk.TableKeyFieldsPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
@@ -330,7 +331,21 @@ public class QInstanceEnricher
 
       if(table.getFields() != null)
       {
-         table.getFields().values().forEach(this::enrichField);
+         for(Map.Entry<String, QFieldMetaData> entry : table.getFields().entrySet())
+         {
+            String         name  = entry.getKey();
+            QFieldMetaData field = entry.getValue();
+
+            ////////////////////////////////////////////////////////////////////////////
+            // in case the field wasn't given a name, use its key from the fields map //
+            ////////////////////////////////////////////////////////////////////////////
+            if(!StringUtils.hasContent(field.getName()))
+            {
+               field.setName(name);
+            }
+
+            enrichField(field);
+         }
 
          for(QSupplementalTableMetaData supplementalTableMetaData : CollectionUtils.nonNullMap(table.getSupplementalMetaData()).values())
          {
@@ -843,6 +858,11 @@ public class QInstanceEnricher
     *******************************************************************************/
    private void defineTableBulkProcesses(QInstance qInstance)
    {
+      if(qInstance.getPossibleValueSource(TableKeyFieldsPossibleValueSource.NAME) == null)
+      {
+         qInstance.addPossibleValueSource(defineTableKeyFieldsPossibleValueSource());
+      }
+
       for(QTableMetaData table : qInstance.getTables().values())
       {
          if(table.getFields() == null)
@@ -866,6 +886,12 @@ public class QInstanceEnricher
             defineTableBulkEdit(qInstance, table, bulkEditProcessName);
          }
 
+         String bulkEditWithFileProcessName = table.getName() + ".bulkEditWithFile";
+         if(qInstance.getProcess(bulkEditWithFileProcessName) == null)
+         {
+            defineTableBulkEditWithFile(qInstance, table, bulkEditWithFileProcessName);
+         }
+
          String bulkDeleteProcessName = table.getName() + ".bulkDelete";
          if(qInstance.getProcess(bulkDeleteProcessName) == null)
          {
@@ -970,16 +996,16 @@ public class QInstanceEnricher
          .withCode(new QCodeReference(BulkInsertReceiveValueMappingStep.class));
 
       int i = 0;
-      process.addStep(i++, prepareFileUploadStep);
-      process.addStep(i++, uploadScreen);
+      process.withStep(i++, prepareFileUploadStep);
+      process.withStep(i++, uploadScreen);
 
-      process.addStep(i++, prepareFileMappingStep);
-      process.addStep(i++, fileMappingScreen);
-      process.addStep(i++, receiveFileMappingStep);
+      process.withStep(i++, prepareFileMappingStep);
+      process.withStep(i++, fileMappingScreen);
+      process.withStep(i++, receiveFileMappingStep);
 
-      process.addStep(i++, prepareValueMappingStep);
-      process.addStep(i++, valueMappingScreen);
-      process.addStep(i++, receiveValueMappingStep);
+      process.withStep(i++, prepareValueMappingStep);
+      process.withStep(i++, valueMappingScreen);
+      process.withStep(i++, receiveValueMappingStep);
 
       process.getFrontendStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW).setRecordListFields(editableFields);
 
@@ -1046,6 +1072,122 @@ public class QInstanceEnricher
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void defineTableBulkEditWithFile(QInstance qInstance, QTableMetaData table, String processName)
+   {
+      Map<String, Serializable> values = new HashMap<>();
+      values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table.getName());
+      values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, "This is a preview of the records that will be updated.");
+
+      QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(
+            BulkInsertExtractStep.class,
+            BulkInsertTransformStep.class,
+            BulkEditLoadStep.class,
+            values
+         )
+         .withName(processName)
+         .withLabel(table.getLabel() + " Bulk Edit With File")
+         .withTableName(table.getName())
+         .withIsHidden(true)
+         .withPermissionRules(qInstance.getDefaultPermissionRules().clone()
+            .withCustomPermissionChecker(new QCodeReference(BulkTableActionProcessPermissionChecker.class)));
+
+      List<QFieldMetaData> editableFields = table.getFields().values().stream()
+         .filter(QFieldMetaData::getIsEditable)
+         .filter(f -> !f.getType().equals(QFieldType.BLOB))
+         .toList();
+
+      QBackendStepMetaData prepareFileUploadStep = new QBackendStepMetaData()
+         .withName("prepareFileUpload")
+         .withCode(new QCodeReference(BulkInsertPrepareFileUploadStep.class));
+
+      QFrontendStepMetaData uploadScreen = new QFrontendStepMetaData()
+         .withName("upload")
+         .withLabel("Upload File")
+         .withFormField(new QFieldMetaData("theFile", QFieldType.BLOB)
+            .withFieldAdornment(FileUploadAdornment.newFieldAdornment()
+               .withValue(FileUploadAdornment.formatDragAndDrop())
+               .withValue(FileUploadAdornment.widthFull()))
+            .withLabel(table.getLabel() + " File")
+            .withIsRequired(true))
+         .withComponent(new QFrontendComponentMetaData().withType(QComponentType.HTML))
+         .withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM));
+
+      QBackendStepMetaData prepareFileMappingStep = new QBackendStepMetaData()
+         .withName("prepareFileMapping")
+         .withCode(new QCodeReference(BulkInsertPrepareFileMappingStep.class));
+
+      QFrontendStepMetaData fileMappingScreen = new QFrontendStepMetaData()
+         .withName("fileMapping")
+         .withLabel("File Mapping")
+         .withBackStepName("prepareFileUpload")
+         .withComponent(new QFrontendComponentMetaData().withType(QComponentType.BULK_LOAD_FILE_MAPPING_FORM))
+         .withFormField(new QFieldMetaData("hasHeaderRow", QFieldType.BOOLEAN))
+         .withFormField(new QFieldMetaData("layout", QFieldType.STRING)) // is actually PVS, but, this field is only added to help support helpContent, so :shrug:
+         .withFormField(new QFieldMetaData("tableKeyFields", QFieldType.STRING).withPossibleValueSourceName(TableKeyFieldsPossibleValueSource.NAME));
+
+      QBackendStepMetaData receiveFileMappingStep = new QBackendStepMetaData()
+         .withName("receiveFileMapping")
+         .withCode(new QCodeReference(BulkInsertReceiveFileMappingStep.class));
+
+      QBackendStepMetaData prepareValueMappingStep = new QBackendStepMetaData()
+         .withName("prepareValueMapping")
+         .withCode(new QCodeReference(BulkInsertPrepareValueMappingStep.class));
+
+      QFrontendStepMetaData valueMappingScreen = new QFrontendStepMetaData()
+         .withName("valueMapping")
+         .withLabel("Value Mapping")
+         .withBackStepName("prepareFileMapping")
+         .withComponent(new QFrontendComponentMetaData().withType(QComponentType.BULK_LOAD_VALUE_MAPPING_FORM));
+
+      QBackendStepMetaData receiveValueMappingStep = new QBackendStepMetaData()
+         .withName("receiveValueMapping")
+         .withCode(new QCodeReference(BulkInsertReceiveValueMappingStep.class));
+
+      int i = 0;
+      process.withStep(i++, prepareFileUploadStep);
+      process.withStep(i++, uploadScreen);
+
+      process.withStep(i++, prepareFileMappingStep);
+      process.withStep(i++, fileMappingScreen);
+      process.withStep(i++, receiveFileMappingStep);
+
+      process.withStep(i++, prepareValueMappingStep);
+      process.withStep(i++, valueMappingScreen);
+      process.withStep(i++, receiveValueMappingStep);
+
+      process.getFrontendStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW).setRecordListFields(editableFields);
+
+      //////////////////////////////////////////////////////////////////////////////////////////
+      // put the bulk-load profile form (e.g., for saving it) on the review & result screens) //
+      //////////////////////////////////////////////////////////////////////////////////////////
+      process.getFrontendStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW)
+         .withBackStepName("prepareFileMapping")
+         .getComponents().add(0, new QFrontendComponentMetaData().withType(QComponentType.BULK_LOAD_PROFILE_FORM));
+
+      process.getFrontendStep(StreamedETLWithFrontendProcess.STEP_NAME_RESULT)
+         .getComponents().add(0, new QFrontendComponentMetaData().withType(QComponentType.BULK_LOAD_PROFILE_FORM));
+
+      qInstance.addProcess(process);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QPossibleValueSource defineTableKeyFieldsPossibleValueSource()
+   {
+      return (new QPossibleValueSource()
+         .withName(TableKeyFieldsPossibleValueSource.NAME)
+         .withType(QPossibleValueSourceType.CUSTOM)
+         .withCustomCodeReference(new QCodeReference(TableKeyFieldsPossibleValueSource.class)));
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceValidator.java

@@ -64,6 +64,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaDa
 import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.automation.QAutomationProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceLambda;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeType;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.ParentWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
@@ -651,6 +652,8 @@ public class QInstanceValidator
             validateSimpleCodeReference("Instance Authentication meta data customizer ", authentication.getCustomizer(), QAuthenticationModuleCustomizerInterface.class);
          }
 
+         authentication.validate(qInstance, this);
+
          runPlugins(QAuthenticationMetaData.class, authentication, qInstance);
       }
    }
@@ -1406,7 +1409,7 @@ public class QInstanceValidator
             //////////////////////////////////////////////////
             // make sure the customizer can be instantiated //
             //////////////////////////////////////////////////
-            Object customizerInstance = getInstanceOfCodeReference(prefix, customizerClass);
+            Object customizerInstance = getInstanceOfCodeReference(prefix, customizerClass, codeReference);
 
             TableCustomizers tableCustomizer = TableCustomizers.forRole(roleName);
             if(tableCustomizer == null)
@@ -1467,8 +1470,13 @@ public class QInstanceValidator
    /*******************************************************************************
     **
     *******************************************************************************/
-   private Object getInstanceOfCodeReference(String prefix, Class<?> clazz)
+   private Object getInstanceOfCodeReference(String prefix, Class<?> clazz, QCodeReference codeReference)
    {
+      if(codeReference instanceof QCodeReferenceLambda<?> lambdaCodeReference)
+      {
+         return (lambdaCodeReference.getLambda());
+      }
+
       Object instance = null;
       try
       {
@@ -1647,21 +1655,26 @@ public class QInstanceValidator
             Set<String> usedStepNames = new HashSet<>();
             if(assertCondition(CollectionUtils.nullSafeHasContents(process.getStepList()), "At least 1 step must be defined in process " + processName + "."))
             {
-               int index = 0;
+               int index = -1;
                for(QStepMetaData step : process.getStepList())
                {
+                  index++;
                   if(assertCondition(StringUtils.hasContent(step.getName()), "Missing name for a step at index " + index + " in process " + processName))
                   {
                      assertCondition(!usedStepNames.contains(step.getName()), "Duplicate step name [" + step.getName() + "] in process " + processName);
                      usedStepNames.add(step.getName());
                   }
-                  index++;
 
                   ////////////////////////////////////////////
                   // validate instantiation of step classes //
                   ////////////////////////////////////////////
                   if(step instanceof QBackendStepMetaData backendStepMetaData)
                   {
+                     if(assertCondition(backendStepMetaData.getCode() != null, "Missing code for a backend step at index " + index + " in process " + processName))
+                     {
+                        validateSimpleCodeReference("Process " + processName + ", backend step at index " + index + ", code reference: ", backendStepMetaData.getCode(), BackendStep.class);
+                     }
+
                      if(backendStepMetaData.getInputMetaData() != null && CollectionUtils.nullSafeHasContents(backendStepMetaData.getInputMetaData().getFieldList()))
                      {
                         for(QFieldMetaData fieldMetaData : backendStepMetaData.getInputMetaData().getFieldList())
@@ -2247,7 +2260,7 @@ public class QInstanceValidator
             //////////////////////////////////////////////////
             // make sure the customizer can be instantiated //
             //////////////////////////////////////////////////
-            Object classInstance = getInstanceOfCodeReference(prefix, clazz);
+            Object classInstance = getInstanceOfCodeReference(prefix, clazz, codeReference);
 
             ////////////////////////////////////////////////////////////////////////
             // make sure the customizer instance can be cast to the expected type //
@@ -2270,6 +2283,11 @@ public class QInstanceValidator
       Class<?> clazz = null;
       try
       {
+         if(codeReference instanceof QCodeReferenceLambda<?> lambdaCodeReference)
+         {
+            return (lambdaCodeReference.getLambda().getClass());
+         }
+
          clazz = Class.forName(codeReference.getName());
       }
       catch(ClassNotFoundException e)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/SecretsManagerUtils.java

@@ -72,6 +72,18 @@ public class SecretsManagerUtils
     ** and write them to a .env file (backing up any pre-existing .env files first).
     *******************************************************************************/
    public static void writeEnvFromSecretsWithNamePrefix(String prefix) throws IOException
+   {
+      writeEnvFromSecretsWithNamePrefix(prefix, true);
+   }
+
+
+
+   /*******************************************************************************
+    ** IF secret manager ENV vars are set,
+    ** THEN lookup all secrets starting with the given prefix,
+    ** and write them to a .env file (backing up any pre-existing .env files first).
+    *******************************************************************************/
+   public static void writeEnvFromSecretsWithNamePrefix(String prefix, boolean quoteValues) throws IOException
    {
       Optional<AWSSecretsManager> optionalSecretsManagerClient = getSecretsManagerClient();
       if(optionalSecretsManagerClient.isPresent())
@@ -91,7 +103,9 @@ public class SecretsManagerUtils
                Optional<String> secretValue       = getSecret(prefix, nameWithoutPrefix);
                if(secretValue.isPresent())
                {
-                  String envLine = nameWithoutPrefix + "=" + secretValue.get();
+                  String envLine = quoteValues
+                     ? nameWithoutPrefix + "=\"" + secretValue.get() + "\""
+                     : nameWithoutPrefix + "=" + secretValue.get();
                   fullEnv.append(envLine).append('\n');
                }
             }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/AbstractMetaDataLoader.java

@@ -0,0 +1,510 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.Serializable;
+import java.lang.reflect.Method;
+import java.lang.reflect.ParameterizedType;
+import java.lang.reflect.Type;
+import java.math.BigDecimal;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.YamlUtils;
+import org.apache.commons.io.IOUtils;
+import static com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsInteger;
+import static com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsString;
+
+
+/*******************************************************************************
+ ** Abstract base class in hierarchy of classes that know how to construct &
+ ** populate QMetaDataObject instances, based on input streams (e.g., from files).
+ *******************************************************************************/
+public abstract class AbstractMetaDataLoader<T extends QMetaDataObject>
+{
+   private static final QLogger LOG = QLogger.getLogger(AbstractMetaDataLoader.class);
+
+   private String fileName;
+
+   private List<LoadingProblem> problems = new ArrayList<>();
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public T fileToMetaDataObject(QInstance qInstance, InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      this.fileName = fileName;
+      Map<String, Object> map = fileToMap(inputStream, fileName);
+      LoadingContext loadingContext = new LoadingContext(fileName, "/");
+      return (mapToMetaDataObject(qInstance, map, loadingContext));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public abstract T mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException;
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected Map<String, Object> fileToMap(InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      try
+      {
+         String string = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
+         string = StringUtils.ltrim(string);
+         if(fileName.toLowerCase().endsWith(".json"))
+         {
+            return JsonUtils.toObject(string, new TypeReference<>() {});
+         }
+         else if(fileName.toLowerCase().endsWith(".yaml") || fileName.toLowerCase().endsWith(".yml"))
+         {
+            return YamlUtils.toMap(string);
+         }
+
+         throw (new QMetaDataLoaderException("Unsupported file format (based on file name: " + fileName + ")"));
+      }
+      catch(IOException e)
+      {
+         throw new QMetaDataLoaderException("Error building map from file: " + fileName, e);
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   protected void reflectivelyMap(QInstance qInstance, QMetaDataObject targetObject, Map<String, Object> map, LoadingContext context)
+   {
+      Class<? extends QMetaDataObject> targetClass    = targetObject.getClass();
+      Set<String>                      usedFieldNames = new HashSet<>();
+
+      for(Method method : targetClass.getMethods())
+      {
+         try
+         {
+            if(method.getName().startsWith("set") && method.getParameterTypes().length == 1)
+            {
+               String propertyName = StringUtils.lcFirst(method.getName().substring(3));
+
+               if(map.containsKey(propertyName))
+               {
+                  usedFieldNames.add(propertyName);
+                  Class<?> parameterType = method.getParameterTypes()[0];
+                  Object   rawValue      = map.get(propertyName);
+
+                  try
+                  {
+                     Object mappedValue = reflectivelyMapValue(qInstance, method, parameterType, rawValue, context.descendToProperty(propertyName));
+                     method.invoke(targetObject, mappedValue);
+                  }
+                  catch(NoValueException nve)
+                  {
+                     ///////////////////////
+                     // don't call setter //
+                     ///////////////////////
+                     LOG.debug("at " + context + ": No value was mapped for property [" + propertyName + "] on " + targetClass.getSimpleName() + "." + method.getName() + ", raw value: [" + rawValue + "]");
+                  }
+               }
+            }
+         }
+         catch(Exception e)
+         {
+            addProblem(new LoadingProblem(context, "Error reflectively mapping on " + targetClass.getName() + "." + method.getName(), e));
+         }
+      }
+
+      //////////////////////////
+      // mmm, slightly sus... //
+      //////////////////////////
+      map.remove("class");
+      map.remove("version");
+
+      Set<String> unrecognizedKeys = new HashSet<>(map.keySet());
+      unrecognizedKeys.removeAll(usedFieldNames);
+
+      if(!unrecognizedKeys.isEmpty())
+      {
+         addProblem(new LoadingProblem(context, unrecognizedKeys.size() + " Unrecognized " + StringUtils.plural(unrecognizedKeys, "property", "properties") + ": " + unrecognizedKeys));
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public Object reflectivelyMapValue(QInstance qInstance, Method method, Class<?> parameterType, Object rawValue, LoadingContext context) throws Exception
+   {
+      if(rawValue instanceof String s && s.matches("^\\$\\{.+\\..+}"))
+      {
+         rawValue = new QMetaDataVariableInterpreter().interpret(s);
+         LOG.debug("Interpreted raw value [" + s + "] as [" + StringUtils.maskAndTruncate(ValueUtils.getValueAsString(rawValue) + "]"));
+      }
+
+      if(parameterType.equals(String.class))
+      {
+         return (getValueAsString(rawValue));
+      }
+      else if(parameterType.equals(Integer.class))
+      {
+         try
+         {
+            return (getValueAsInteger(rawValue));
+         }
+         catch(Exception e)
+         {
+            addProblem(new LoadingProblem(context, "[" + rawValue + "] is not an Integer value."));
+         }
+      }
+      else if(parameterType.equals(Boolean.class))
+      {
+         if("true".equals(rawValue) || Boolean.TRUE.equals(rawValue))
+         {
+            return (true);
+         }
+         else if("false".equals(rawValue) || Boolean.FALSE.equals(rawValue))
+         {
+            return (false);
+         }
+         else if(rawValue == null)
+         {
+            return (null);
+         }
+         else
+         {
+            addProblem(new LoadingProblem(context, "[" + rawValue + "] is not a boolean value (must be 'true' or 'false')."));
+            return (null);
+         }
+      }
+      else if(parameterType.equals(boolean.class))
+      {
+         if("true".equals(rawValue) || Boolean.TRUE.equals(rawValue))
+         {
+            return (true);
+         }
+         else if("false".equals(rawValue) || Boolean.FALSE.equals(rawValue))
+         {
+            return (false);
+         }
+         else
+         {
+            addProblem(new LoadingProblem(context, rawValue + " is not a boolean value (must be 'true' or 'false')."));
+            throw (new NoValueException());
+         }
+      }
+      else if(parameterType.equals(List.class))
+      {
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")List valueList)
+         {
+            List<Object> mappedValueList = new ArrayList<>();
+            for(Object o : valueList)
+            {
+               try
+               {
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, o, context);
+                  mappedValueList.add(mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave off list
+               }
+            }
+            return (mappedValueList);
+         }
+      }
+      else if(parameterType.equals(Set.class))
+      {
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")List valueList)
+         {
+            Set<Object> mappedValueSet = new LinkedHashSet<>();
+            for(Object o : valueList)
+            {
+               try
+               {
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, o, context);
+                  mappedValueSet.add(mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave off list
+               }
+            }
+            return (mappedValueSet);
+         }
+      }
+      else if(parameterType.equals(Map.class))
+      {
+         Type keyType = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[0];
+         if(!keyType.equals(String.class))
+         {
+            addProblem(new LoadingProblem(context, "Unsupported key type for " + method + " got [" + keyType + "], expected [String]"));
+            throw new NoValueException();
+         }
+         // todo make sure string
+
+         Type     actualTypeArgument = ((ParameterizedType) method.getGenericParameterTypes()[0]).getActualTypeArguments()[1];
+         Class<?> actualTypeClass    = Class.forName(actualTypeArgument.getTypeName());
+
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            Map<String, Object> mappedValueMap = new LinkedHashMap<>();
+            for(Object o : valueMap.entrySet())
+            {
+               try
+               {
+                  @SuppressWarnings("unchecked")
+                  Map.Entry<String, Object> entry = (Map.Entry<String, Object>) o;
+                  Object mappedValue = reflectivelyMapValue(qInstance, null, actualTypeClass, entry.getValue(), context);
+                  mappedValueMap.put(entry.getKey(), mappedValue);
+               }
+               catch(NoValueException nve)
+               {
+                  // leave out of map
+               }
+            }
+            return (mappedValueMap);
+         }
+      }
+      else if(parameterType.isEnum())
+      {
+         String value = getValueAsString(rawValue);
+         for(Object enumConstant : parameterType.getEnumConstants())
+         {
+            if(((Enum<?>) enumConstant).name().equals(value))
+            {
+               return (enumConstant);
+            }
+         }
+
+         addProblem(new LoadingProblem(context, "Unrecognized value [" + rawValue + "].  Expected one of: " + Arrays.toString(parameterType.getEnumConstants())));
+      }
+      else if(MetaDataLoaderRegistry.hasLoaderForClass(parameterType))
+      {
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            Class<? extends AbstractMetaDataLoader<?>> loaderClass = MetaDataLoaderRegistry.getLoaderForClass(parameterType);
+            AbstractMetaDataLoader<?>                  loader      = loaderClass.getConstructor().newInstance();
+            //noinspection unchecked
+            return (loader.mapToMetaDataObject(qInstance, valueMap, context));
+         }
+      }
+      else if(QMetaDataObject.class.isAssignableFrom(parameterType))
+      {
+         if(rawValue instanceof @SuppressWarnings("rawtypes")Map valueMap)
+         {
+            QMetaDataObject childObject = (QMetaDataObject) parameterType.getConstructor().newInstance();
+            //noinspection unchecked
+            reflectivelyMap(qInstance, childObject, valueMap, context);
+            return (childObject);
+         }
+      }
+      else if(parameterType.equals(Serializable.class))
+      {
+         if(rawValue instanceof String
+            || rawValue instanceof Integer
+            || rawValue instanceof BigDecimal
+            || rawValue instanceof Boolean
+         )
+         {
+            return rawValue;
+         }
+      }
+      else
+      {
+         // todo clean up this message/level
+         addProblem(new LoadingProblem(context, "No case for " + parameterType + " (arg to: " + method + ")"));
+      }
+
+      throw new NoValueException();
+   }
+
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   // unclear if the below is needed.  if so, useful to not re-write, but is hurting test coverage, so zombie until used //
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected ListOfMapOrMapOfMap getListOfMapOrMapOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof List)
+   //      {
+   //         return (new ListOfMapOrMapOfMap((List<Map<String, Object>>) map.get(key)));
+   //      }
+   //      else if(map.get(key) instanceof Map)
+   //      {
+   //         return (new ListOfMapOrMapOfMap((Map<String, Map<String, Object>>) map.get(key)));
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected list or map under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected List<Map<String, Object>> getListOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof List)
+   //      {
+   //         return (List<Map<String, Object>>) map.get(key);
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected list under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // *
+   // ***************************************************************************/
+   //protected Map<String, Map<String, Object>> getMapOfMap(Map<String, Object> map, String key)
+   //{
+   //   if(map.containsKey(key))
+   //   {
+   //      if(map.get(key) instanceof Map)
+   //      {
+   //         return (Map<String, Map<String, Object>>) map.get(key);
+   //      }
+   //      else
+   //      {
+   //         LOG.warn("Expected map under key [" + key + "] while processing [" + getClass().getSimpleName() + "] from [" + fileName + "], but found: " + (map.get(key) == null ? "null" : map.get(key).getClass().getSimpleName()));
+   //      }
+   //   }
+
+   //   return (null);
+   //}
+
+   ///***************************************************************************
+   // **
+   // ***************************************************************************/
+   //protected record ListOfMapOrMapOfMap(List<Map<String, Object>> listOf, Map<String, Map<String, Object>> mapOf)
+   //{
+   //   /*******************************************************************************
+   //    ** Constructor
+   //    **
+   //    *******************************************************************************/
+   //   public ListOfMapOrMapOfMap(List<Map<String, Object>> listOf)
+   //   {
+   //      this(listOf, null);
+   //   }
+
+   //   /*******************************************************************************
+   //    ** Constructor
+   //    **
+   //    *******************************************************************************/
+   //   public ListOfMapOrMapOfMap(Map<String, Map<String, Object>> mapOf)
+   //   {
+   //      this(null, mapOf);
+   //   }
+   //}
+
+
+
+   /*******************************************************************************
+    ** Getter for fileName
+    **
+    *******************************************************************************/
+   public String getFileName()
+   {
+      return fileName;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static class NoValueException extends Exception
+   {
+      /*******************************************************************************
+       ** Constructor
+       **
+       *******************************************************************************/
+      public NoValueException()
+      {
+         super("No value");
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public void addProblem(LoadingProblem problem)
+   {
+      problems.add(problem);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for problems
+    **
+    *******************************************************************************/
+   public List<LoadingProblem> getProblems()
+   {
+      return (problems);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/ClassDetectingMetaDataLoader.java

@@ -0,0 +1,120 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.InputStream;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.instances.loaders.implementations.GenericMetaDataLoader;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.utils.ClassPathUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.AnyKey;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
+
+
+/*******************************************************************************
+ ** Generic implementation of AbstractMetaDataLoader, who "detects" the class
+ ** of meta data object to be created, then defers to an appropriate subclass
+ ** to do the work.
+ *******************************************************************************/
+public class ClassDetectingMetaDataLoader extends AbstractMetaDataLoader<QMetaDataObject>
+{
+   private static final Memoization<AnyKey, List<Class<?>>> memoizedMetaDataObjectClasses = new Memoization<>();
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public AbstractMetaDataLoader<?> getLoaderForFile(InputStream inputStream, String fileName) throws QMetaDataLoaderException
+   {
+      Map<String, Object> map = fileToMap(inputStream, fileName);
+      return (getLoaderForMap(map));
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public AbstractMetaDataLoader<?> getLoaderForMap(Map<String, Object> map) throws QMetaDataLoaderException
+   {
+      if(map.containsKey("class"))
+      {
+         String classProperty = ValueUtils.getValueAsString(map.get("class"));
+         try
+         {
+            if(MetaDataLoaderRegistry.hasLoaderForSimpleName(classProperty))
+            {
+               Class<? extends AbstractMetaDataLoader<?>> loaderClass = MetaDataLoaderRegistry.getLoaderForSimpleName(classProperty);
+               return (loaderClass.getConstructor().newInstance());
+            }
+            else
+            {
+               Optional<List<Class<?>>> metaDataClasses = memoizedMetaDataObjectClasses.getResult(AnyKey.getInstance(), k -> ClassPathUtils.getClassesContainingNameAndOfType("MetaData", QMetaDataObject.class));
+               if(metaDataClasses.isEmpty())
+               {
+                  throw (new QMetaDataLoaderException("Could not get list of metaDataObjects from class loader"));
+               }
+
+               for(Class<?> c : metaDataClasses.get())
+               {
+                  if(c.getSimpleName().equals(classProperty) && QMetaDataObject.class.isAssignableFrom(c))
+                  {
+                     @SuppressWarnings("unchecked")
+                     Class<? extends QMetaDataObject> metaDataClass = (Class<? extends QMetaDataObject>) c;
+                     return new GenericMetaDataLoader<>(metaDataClass);
+                  }
+               }
+            }
+            throw new QMetaDataLoaderException("Unexpected class [" + classProperty + "] (not a QMetaDataObject; doesn't have a registered MetaDataLoader) specified in " + getFileName());
+         }
+         catch(QMetaDataLoaderException qmdle)
+         {
+            throw (qmdle);
+         }
+         catch(Exception e)
+         {
+            throw new QMetaDataLoaderException("Error handling class [" + classProperty + "] specified in " + getFileName(), e);
+         }
+      }
+      else
+      {
+         throw new QMetaDataLoaderException("Cannot detect meta-data type, because [class] attribute was not specified in file: " + getFileName());
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QMetaDataObject mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      AbstractMetaDataLoader<?> loaderForMap = getLoaderForMap(map);
+      return loaderForMap.mapToMetaDataObject(qInstance, map, context);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/LoadingContext.java

@@ -0,0 +1,38 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ ** Record to track where loader objects are - e.g., what file they're on,
+ ** and at what property path within the file (e.g., helps report problems).
+ *******************************************************************************/
+public record LoadingContext(String fileName, String propertyPath)
+{
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public LoadingContext descendToProperty(String propertyName)
+   {
+      return new LoadingContext(fileName, propertyPath + propertyName + "/");
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/LoadingProblem.java

@@ -0,0 +1,49 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ ** record that tracks a problem that was encountered when loading files.
+ *******************************************************************************/
+public record LoadingProblem(LoadingContext context, String message, Exception exception) // todo Level if useful
+{
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public LoadingProblem(LoadingContext context, String message)
+   {
+      this(context, message, null);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String toString()
+   {
+      return "at[" + context.fileName() + "][" + context.propertyPath() + "]: " + message;
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/MetaDataLoaderHelper.java

@@ -0,0 +1,118 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.Pair;
+
+
+/*******************************************************************************
+ ** class that loads a directory full of meta data files into meta data objects,
+ ** and then sets all of them in a QInstance.
+ *******************************************************************************/
+public class MetaDataLoaderHelper
+{
+   private static final QLogger LOG = QLogger.getLogger(MetaDataLoaderHelper.class);
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public static void processAllMetaDataFilesInDirectory(QInstance qInstance, String path) throws QException
+   {
+      List<Pair<File, AbstractMetaDataLoader<?>>> loaders = new ArrayList<>();
+
+      File directory = new File(path);
+      processAllMetaDataFilesInDirectory(loaders, directory);
+
+      // todo - some version of sorting the loaders by type or possibly a sort field within the files (or file names)
+
+      for(Pair<File, AbstractMetaDataLoader<?>> pair : loaders)
+      {
+         File                      file   = pair.getA();
+         AbstractMetaDataLoader<?> loader = pair.getB();
+         try(FileInputStream fileInputStream = new FileInputStream(file))
+         {
+            QMetaDataObject qMetaDataObject = loader.fileToMetaDataObject(qInstance, fileInputStream, file.getName());
+
+            if(CollectionUtils.nullSafeHasContents(loader.getProblems()))
+            {
+               loader.getProblems().forEach(System.out::println);
+            }
+
+            if(qMetaDataObject instanceof TopLevelMetaDataInterface topLevelMetaData)
+            {
+               topLevelMetaData.addSelfToInstance(qInstance);
+            }
+            else
+            {
+               LOG.warn("Received a non-topLevelMetaDataObject from file: " + file.getAbsolutePath());
+            }
+         }
+         catch(Exception e)
+         {
+            LOG.error("Error processing file: " + file.getAbsolutePath(), e);
+         }
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   private static void processAllMetaDataFilesInDirectory(List<Pair<File, AbstractMetaDataLoader<?>>> loaders, File directory) throws QException
+   {
+      for(File file : Objects.requireNonNullElse(directory.listFiles(), new File[0]))
+      {
+         if(file.isDirectory())
+         {
+            processAllMetaDataFilesInDirectory(loaders, file);
+         }
+         else
+         {
+            try(FileInputStream fileInputStream = new FileInputStream(file))
+            {
+               AbstractMetaDataLoader<?> loader = new ClassDetectingMetaDataLoader().getLoaderForFile(fileInputStream, file.getName());
+               loaders.add(Pair.of(file, loader));
+            }
+            catch(Exception e)
+            {
+               LOG.error("Error processing file: " + file.getAbsolutePath(), e);
+            }
+         }
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/MetaDataLoaderRegistry.java

@@ -0,0 +1,120 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+import java.lang.reflect.ParameterizedType;
+import java.lang.reflect.Type;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.implementations.QTableMetaDataLoader;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.utils.ClassPathUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class MetaDataLoaderRegistry
+{
+   private static final QLogger LOG = QLogger.getLogger(AbstractMetaDataLoader.class);
+
+   private static final Map<Class<?>, Class<? extends AbstractMetaDataLoader<?>>> registeredLoaders                   = new HashMap<>();
+   private static final Map<String, Class<? extends AbstractMetaDataLoader<?>>>   registeredLoadersByTargetSimpleName = new HashMap<>();
+
+   static
+   {
+      try
+      {
+         List<Class<?>> classesInPackage = ClassPathUtils.getClassesInPackage(QTableMetaDataLoader.class.getPackageName());
+         for(Class<?> possibleLoaderClass : classesInPackage)
+         {
+            try
+            {
+               Type superClass = possibleLoaderClass.getGenericSuperclass();
+               if(superClass.getTypeName().startsWith(AbstractMetaDataLoader.class.getName() + "<"))
+               {
+                  Type actualTypeArgument = ((ParameterizedType) superClass).getActualTypeArguments()[0];
+                  if(actualTypeArgument instanceof Class)
+                  {
+                     //noinspection unchecked
+                     Class<? extends AbstractMetaDataLoader<?>> loaderClass = (Class<? extends AbstractMetaDataLoader<?>>) possibleLoaderClass;
+
+                     Class<?> metaDataObjectType = Class.forName(actualTypeArgument.getTypeName());
+                     registeredLoaders.put(metaDataObjectType, loaderClass);
+                     registeredLoadersByTargetSimpleName.put(metaDataObjectType.getSimpleName(), loaderClass);
+                  }
+               }
+            }
+            catch(Exception e)
+            {
+               LOG.info("Error on class: " + possibleLoaderClass, e);
+            }
+         }
+
+         System.out.println("Registered loaders: " + registeredLoadersByTargetSimpleName);
+      }
+      catch(Exception e)
+      {
+         LOG.error("Error in static init block for MetaDataLoaderRegistry", e);
+      }
+   }
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static boolean hasLoaderForClass(Class<?> metaDataClass)
+   {
+      return registeredLoaders.containsKey(metaDataClass);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static Class<? extends AbstractMetaDataLoader<?>> getLoaderForClass(Class<?> metaDataClass)
+   {
+      return registeredLoaders.get(metaDataClass);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static boolean hasLoaderForSimpleName(String targetSimpleName)
+   {
+      return registeredLoadersByTargetSimpleName.containsKey(targetSimpleName);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static Class<? extends AbstractMetaDataLoader<?>> getLoaderForSimpleName(String targetSimpleName)
+   {
+      return registeredLoadersByTargetSimpleName.get(targetSimpleName);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/QMetaDataLoaderException.java

@@ -0,0 +1,50 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QMetaDataLoaderException extends Exception
+{
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QMetaDataLoaderException(String message)
+   {
+      super(message);
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QMetaDataLoaderException(String message, Throwable cause)
+   {
+      super(message, cause);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/GenericMetaDataLoader.java

@@ -0,0 +1,71 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class GenericMetaDataLoader<T extends QMetaDataObject> extends AbstractMetaDataLoader<T>
+{
+   private final Class<T> metaDataClass;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public GenericMetaDataLoader(Class<T> metaDataClass)
+   {
+      this.metaDataClass = metaDataClass;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public T mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      try
+      {
+         T object = metaDataClass.getConstructor().newInstance();
+         reflectivelyMap(qInstance, object, map, context);
+         return (object);
+      }
+      catch(Exception e)
+      {
+         throw (new QMetaDataLoaderException("Error loading metaData object of type " + metaDataClass.getSimpleName(), e));
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/QStepDataLoader.java

@@ -0,0 +1,85 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QStepMetaData;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QStepDataLoader extends AbstractMetaDataLoader<QStepMetaData>
+{
+   private static final QLogger LOG = QLogger.getLogger(QStepDataLoader.class);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QStepMetaData mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      String stepType = ValueUtils.getValueAsString(map.get("stepType"));
+
+      if(!StringUtils.hasContent(stepType))
+      {
+         throw (new QMetaDataLoaderException("stepType was not specified for process step"));
+      }
+
+      QStepMetaData step;
+      if("backend".equalsIgnoreCase(stepType))
+      {
+         step = new QBackendStepMetaData();
+         reflectivelyMap(qInstance, step, map, context);
+      }
+      else if("frontend".equalsIgnoreCase(stepType))
+      {
+         step = new QFrontendStepMetaData();
+         reflectivelyMap(qInstance, step, map, context);
+      }
+      // todo - we have custom factory methods for this, so, maybe needs all custom loader?
+      // else if("stateMachine".equalsIgnoreCase(stepType))
+      // {
+      //    step = new QStateMachineStep();
+      //    reflectivelyMap(qInstance, step, map, context);
+      // }
+      else
+      {
+         throw (new QMetaDataLoaderException("Unsupported step stepType: " + stepType));
+      }
+
+      return (step);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/loaders/implementations/QTableMetaDataLoader.java

@@ -0,0 +1,58 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.loaders.implementations;
+
+
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.instances.loaders.AbstractMetaDataLoader;
+import com.kingsrook.qqq.backend.core.instances.loaders.LoadingContext;
+import com.kingsrook.qqq.backend.core.instances.loaders.QMetaDataLoaderException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class QTableMetaDataLoader extends AbstractMetaDataLoader<QTableMetaData>
+{
+   private static final QLogger LOG = QLogger.getLogger(QTableMetaDataLoader.class);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QTableMetaData mapToMetaDataObject(QInstance qInstance, Map<String, Object> map, LoadingContext context) throws QMetaDataLoaderException
+   {
+      QTableMetaData table = new QTableMetaData();
+
+      reflectivelyMap(qInstance, table, map, context);
+
+      // todo - handle QTableBackendDetails, based on backend's type
+
+      return (table);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/QProcessPayload.java

@@ -0,0 +1,162 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.processes;
+
+
+import java.io.Serializable;
+import java.lang.reflect.Method;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QRuntimeException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntityField;
+import com.kingsrook.qqq.backend.core.utils.ListingHash;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
+
+
+/*******************************************************************************
+ ** base-class for bean-like classes to represent the fields of a process.
+ ** similar in spirit to QRecordEntity, but for processes.
+ *******************************************************************************/
+public class QProcessPayload
+{
+   private static final QLogger LOG = QLogger.getLogger(QProcessPayload.class);
+
+   private static final ListingHash<Class<? extends QProcessPayload>, QRecordEntityField> fieldMapping = new ListingHash<>();
+
+
+
+   /*******************************************************************************
+    ** Build an entity of this QRecord type from a QRecord
+    **
+    *******************************************************************************/
+   public static <T extends QProcessPayload> T fromProcessState(Class<T> c, ProcessState processState) throws QException
+   {
+      try
+      {
+         T entity = c.getConstructor().newInstance();
+         entity.populateFromProcessState(processState);
+         return (entity);
+      }
+      catch(Exception e)
+      {
+         throw (new QException("Error building process payload from state.", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   protected void populateFromProcessState(ProcessState processState)
+   {
+      try
+      {
+         List<QRecordEntityField> fieldList = getFieldList(this.getClass());
+
+         for(QRecordEntityField qRecordEntityField : fieldList)
+         {
+            Serializable value      = processState.getValues().get(qRecordEntityField.getFieldName());
+            Object       typedValue = qRecordEntityField.convertValueType(value);
+            qRecordEntityField.getSetter().invoke(this, typedValue);
+         }
+      }
+      catch(Exception e)
+      {
+         throw (new QRuntimeException("Error building process payload from process state.", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Copy the values from this payload into the given process state.
+    ** ALL fields in the entity will be set in the process state.
+    **
+    *******************************************************************************/
+   public void toProcessState(ProcessState processState) throws QRuntimeException
+   {
+      try
+      {
+         for(QRecordEntityField qRecordEntityField : getFieldList(this.getClass()))
+         {
+            processState.getValues().put(qRecordEntityField.getFieldName(), (Serializable) qRecordEntityField.getGetter().invoke(this));
+         }
+      }
+      catch(Exception e)
+      {
+         throw (new QRuntimeException("Error populating process state from process payload.", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    *
+    ***************************************************************************/
+   public static Set<Class<?>> allowedFieldTypes()
+   {
+      HashSet<Class<?>> classes = new HashSet<>(ReflectiveBeanLikeClassUtils.defaultAllowedTypes());
+      classes.add(Map.class);
+      classes.add(List.class);
+      return (classes);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static List<QRecordEntityField> getFieldList(Class<? extends QProcessPayload> c)
+   {
+      if(!fieldMapping.containsKey(c))
+      {
+         List<QRecordEntityField> fieldList = new ArrayList<>();
+         for(Method possibleGetter : c.getMethods())
+         {
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, false, allowedFieldTypes()))
+            {
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
+
+               if(setter.isPresent())
+               {
+                  String fieldName = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
+                  fieldList.add(new QRecordEntityField(fieldName, possibleGetter, setter.get(), possibleGetter.getReturnType(), null));
+               }
+               else
+               {
+                  LOG.debug("Getter method [" + possibleGetter.getName() + "] does not have a corresponding setter.");
+               }
+            }
+         }
+         fieldMapping.put(c, fieldList);
+      }
+      return (fieldMapping.get(c));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunBackendStepInput.java

@@ -628,4 +628,15 @@ public class RunBackendStepInput extends AbstractActionInput
    {
       return (QContext.getQInstance().getProcess(getProcessName()));
    }
+
+
+
+   /***************************************************************************
+    ** return a QProcessPayload subclass instance, with values populated from
+    ** the current process state.
+    ***************************************************************************/
+   public <T extends QProcessPayload> T getProcessPayload(Class<T> payloadClass) throws QException
+   {
+      return QProcessPayload.fromProcessState(payloadClass, getProcessState());
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunBackendStepOutput.java

@@ -445,4 +445,14 @@ public class RunBackendStepOutput extends AbstractActionOutput implements Serial
       this.processState.setProcessMetaDataAdjustment(processMetaDataAdjustment);
    }
 
+
+
+   /***************************************************************************
+    ** Update the process state with values from the input processPayload
+    ** subclass instance.
+    ***************************************************************************/
+   public void setProcessPayload(QProcessPayload processPayload)
+   {
+      processPayload.toProcessState(getProcessState());
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QFilterCriteria.java

@@ -33,6 +33,7 @@ import java.util.Set;
 import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.serialization.QFilterCriteriaDeserializer;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
@@ -42,7 +43,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  *
  *******************************************************************************/
 @JsonDeserialize(using = QFilterCriteriaDeserializer.class)
-public class QFilterCriteria implements Serializable, Cloneable
+public class QFilterCriteria implements Serializable, Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QFilterCriteria.class);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QFilterOrderBy.java

@@ -23,13 +23,14 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query;
 
 
 import java.io.Serializable;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Bean representing an element of a query order-by clause.
  **
  *******************************************************************************/
-public class QFilterOrderBy implements Serializable, Cloneable
+public class QFilterOrderBy implements Serializable, Cloneable, QMetaDataObject
 {
    private String  fieldName;
    private boolean isAscending = true;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QQueryFilter.java

@@ -36,6 +36,7 @@ import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.AbstractFilterExpression;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.FilterVariableExpression;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
@@ -45,7 +46,7 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  * Full "filter" for a query - a list of criteria and order-bys
  *
  *******************************************************************************/
-public class QQueryFilter implements Serializable, Cloneable
+public class QQueryFilter implements Serializable, Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QQueryFilter.class);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/values/SearchPossibleValueSourceInput.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.actions.values;
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Map;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 
@@ -34,11 +35,13 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
  *******************************************************************************/
 public class SearchPossibleValueSourceInput extends AbstractActionInput implements Cloneable
 {
-   private String             possibleValueSourceName;
-   private QQueryFilter       defaultQueryFilter;
-   private String             searchTerm;
-   private List<Serializable> idList;
-   private List<String>       labelList;
+   private String                    possibleValueSourceName;
+   private QQueryFilter              defaultQueryFilter;
+   private String                    searchTerm;
+   private List<Serializable>        idList;
+   private List<String>              labelList;
+   private Map<String, String>       pathParamMap;
+   private Map<String, List<String>> queryParamMap;
 
    private Integer skip  = 0;
    private Integer limit = 250;
@@ -284,6 +287,7 @@ public class SearchPossibleValueSourceInput extends AbstractActionInput implemen
    }
 
 
+
    /*******************************************************************************
     ** Getter for labelList
     *******************************************************************************/
@@ -313,4 +317,66 @@ public class SearchPossibleValueSourceInput extends AbstractActionInput implemen
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for pathParamMap
+    *******************************************************************************/
+   public Map<String, String> getPathParamMap()
+   {
+      return (this.pathParamMap);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for pathParamMap
+    *******************************************************************************/
+   public void setPathParamMap(Map<String, String> pathParamMap)
+   {
+      this.pathParamMap = pathParamMap;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for pathParamMap
+    *******************************************************************************/
+   public SearchPossibleValueSourceInput withPathParamMap(Map<String, String> pathParamMap)
+   {
+      this.pathParamMap = pathParamMap;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for queryParamMap
+    *******************************************************************************/
+   public Map<String, List<String>> getQueryParamMap()
+   {
+      return (this.queryParamMap);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for queryParamMap
+    *******************************************************************************/
+   public void setQueryParamMap(Map<String, List<String>> queryParamMap)
+   {
+      this.queryParamMap = queryParamMap;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for queryParamMap
+    *******************************************************************************/
+   public SearchPossibleValueSourceInput withQueryParamMap(Map<String, List<String>> queryParamMap)
+   {
+      this.queryParamMap = queryParamMap;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/bulk/TableKeyFieldsPossibleValueSource.java

@@ -0,0 +1,153 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.bulk;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.actions.values.QCustomPossibleValueProvider;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class TableKeyFieldsPossibleValueSource implements QCustomPossibleValueProvider<String>
+{
+   public static final String NAME = "tableKeyFields";
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QPossibleValue<String> getPossibleValue(Serializable tableAndKey)
+   {
+      QPossibleValue<String> possibleValue = null;
+
+      /////////////////////////////////////////////////////////////
+      // keys are in the format <tableName>-<key1>|<key2>|<key3> //
+      /////////////////////////////////////////////////////////////
+      String[] keyParts  = tableAndKey.toString().split("-");
+      String   tableName = keyParts[0];
+      String   key       = keyParts[1];
+
+      QTableMetaData table = QContext.getQInstance().getTable(tableName);
+      if(table.getPrimaryKeyField().equals(key))
+      {
+         String id    = table.getPrimaryKeyField();
+         String label = table.getField(table.getPrimaryKeyField()).getLabel();
+         possibleValue = new QPossibleValue<>(id, label);
+      }
+      else
+      {
+         for(UniqueKey uniqueKey : table.getUniqueKeys())
+         {
+            String potentialMatch = getIdFromUniqueKey(uniqueKey);
+            if(potentialMatch.equals(key))
+            {
+               String id    = potentialMatch;
+               String label = getLabelFromUniqueKey(table, uniqueKey);
+               possibleValue = new QPossibleValue<>(id, label);
+               break;
+            }
+         }
+      }
+
+      return (possibleValue);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QPossibleValue<String>> search(SearchPossibleValueSourceInput input) throws QException
+   {
+      List<QPossibleValue<String>> rs = new ArrayList<>();
+      if(!CollectionUtils.nonNullMap(input.getPathParamMap()).containsKey("processName") || input.getPathParamMap().get("processName") == null || input.getPathParamMap().get("processName").isEmpty())
+      {
+         throw (new QException("Path Param of processName was not found."));
+      }
+
+      ////////////////////////////////////////////////////
+      // process name will be like tnt.bulkEditWithFile //
+      ////////////////////////////////////////////////////
+      String processName = input.getPathParamMap().get("processName");
+      String tableName   = processName.split("\\.")[0];
+
+      QTableMetaData table = QContext.getQInstance().getTable(tableName);
+      for(UniqueKey uniqueKey : CollectionUtils.nonNullList(table.getUniqueKeys()))
+      {
+         String id    = getIdFromUniqueKey(uniqueKey);
+         String label = getLabelFromUniqueKey(table, uniqueKey);
+         if(!StringUtils.hasContent(input.getSearchTerm()) || input.getSearchTerm().equals(id))
+         {
+            rs.add(new QPossibleValue<>(id, label));
+         }
+      }
+      rs.sort(Comparator.comparing(QPossibleValue::getLabel));
+
+      ///////////////////////////////
+      // put the primary key first //
+      ///////////////////////////////
+      if(!StringUtils.hasContent(input.getSearchTerm()) || input.getSearchTerm().equals(table.getPrimaryKeyField()))
+      {
+         rs.add(0, new QPossibleValue<>(table.getPrimaryKeyField(), table.getField(table.getPrimaryKeyField()).getLabel()));
+      }
+
+      return rs;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private String getIdFromUniqueKey(UniqueKey uniqueKey)
+   {
+      return (StringUtils.join("|", uniqueKey.getFieldNames()));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private String getLabelFromUniqueKey(QTableMetaData tableMetaData, UniqueKey uniqueKey)
+   {
+      List<String> fieldLabels = new ArrayList<>(uniqueKey.getFieldNames().stream().map(f -> tableMetaData.getField(f).getLabel()).toList());
+      fieldLabels.sort(Comparator.naturalOrder());
+      return (StringUtils.joinWithCommasAndAnd(fieldLabels));
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/common/TimeZonePossibleValueSourceMetaDataProvider.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.model.common;
 
 
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.List;
 import java.util.TimeZone;
 import java.util.function.Function;
@@ -47,7 +48,7 @@ public class TimeZonePossibleValueSourceMetaDataProvider
     *******************************************************************************/
    public QPossibleValueSource produce()
    {
-      return (produce(null, null));
+      return (produce(null, null, null));
    }
 
 
@@ -56,6 +57,16 @@ public class TimeZonePossibleValueSourceMetaDataProvider
     **
     *******************************************************************************/
    public QPossibleValueSource produce(Predicate<String> filter, Function<String, String> labelMapper)
+   {
+      return (produce(filter, labelMapper, null));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QPossibleValueSource produce(Predicate<String> filter, Function<String, String> labelMapper, Comparator<QPossibleValue<?>> comparator)
    {
       QPossibleValueSource possibleValueSource = new QPossibleValueSource()
          .withName("timeZones")
@@ -72,6 +83,11 @@ public class TimeZonePossibleValueSourceMetaDataProvider
          }
       }
 
+      if(comparator != null)
+      {
+         enumValues.sort(comparator);
+      }
+
       possibleValueSource.withEnumValues(enumValues);
       return (possibleValueSource);
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/WidgetType.java

@@ -66,6 +66,7 @@ public enum WidgetType
    // record view/edit widgets //
    //////////////////////////////
    CHILD_RECORD_LIST("childRecordList"),
+   CUSTOM_COMPONENT("customComponent"),
    DYNAMIC_FORM("dynamicForm"),
    DATA_BAG_VIEWER("dataBagViewer"),
    PIVOT_TABLE_SETUP("pivotTableSetup"),

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecord.java

@@ -468,7 +468,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public String getValueString(String fieldName)
    {
-      return (ValueUtils.getValueAsString(values.get(fieldName)));
+      return (ValueUtils.getValueAsString(getValue(fieldName)));
    }
 
 
@@ -479,7 +479,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public Integer getValueInteger(String fieldName)
    {
-      return (ValueUtils.getValueAsInteger(values.get(fieldName)));
+      return (ValueUtils.getValueAsInteger(getValue(fieldName)));
    }
 
 
@@ -490,7 +490,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public Long getValueLong(String fieldName)
    {
-      return (ValueUtils.getValueAsLong(values.get(fieldName)));
+      return (ValueUtils.getValueAsLong(getValue(fieldName)));
    }
 
 
@@ -500,7 +500,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public BigDecimal getValueBigDecimal(String fieldName)
    {
-      return (ValueUtils.getValueAsBigDecimal(values.get(fieldName)));
+      return (ValueUtils.getValueAsBigDecimal(getValue(fieldName)));
    }
 
 
@@ -510,7 +510,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public Boolean getValueBoolean(String fieldName)
    {
-      return (ValueUtils.getValueAsBoolean(values.get(fieldName)));
+      return (ValueUtils.getValueAsBoolean(getValue(fieldName)));
    }
 
 
@@ -520,7 +520,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public LocalTime getValueLocalTime(String fieldName)
    {
-      return (ValueUtils.getValueAsLocalTime(values.get(fieldName)));
+      return (ValueUtils.getValueAsLocalTime(getValue(fieldName)));
    }
 
 
@@ -530,7 +530,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public LocalDate getValueLocalDate(String fieldName)
    {
-      return (ValueUtils.getValueAsLocalDate(values.get(fieldName)));
+      return (ValueUtils.getValueAsLocalDate(getValue(fieldName)));
    }
 
 
@@ -540,7 +540,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public byte[] getValueByteArray(String fieldName)
    {
-      return (ValueUtils.getValueAsByteArray(values.get(fieldName)));
+      return (ValueUtils.getValueAsByteArray(getValue(fieldName)));
    }
 
 
@@ -550,7 +550,7 @@ public class QRecord implements Serializable
     *******************************************************************************/
    public Instant getValueInstant(String fieldName)
    {
-      return (ValueUtils.getValueAsInstant(values.get(fieldName)));
+      return (ValueUtils.getValueAsInstant(getValue(fieldName)));
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEntity.java

@@ -49,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.ObjectUtils;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -325,13 +326,13 @@ public abstract class QRecordEntity
          List<QRecordEntityField> fieldList = new ArrayList<>();
          for(Method possibleGetter : c.getMethods())
          {
-            if(isGetter(possibleGetter))
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, true))
             {
-               Optional<Method> setter = getSetterForGetter(c, possibleGetter);
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
 
                if(setter.isPresent())
                {
-                  String           fieldName       = getFieldNameFromGetter(possibleGetter);
+                  String           fieldName       = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
                   Optional<QField> fieldAnnotation = getQFieldAnnotation(c, fieldName);
 
                   if(fieldAnnotation.isPresent())
@@ -378,19 +379,19 @@ public abstract class QRecordEntity
          List<QRecordEntityAssociation> associationList = new ArrayList<>();
          for(Method possibleGetter : c.getMethods())
          {
-            if(isGetter(possibleGetter))
+            if(ReflectiveBeanLikeClassUtils.isGetter(possibleGetter, true))
             {
-               Optional<Method> setter = getSetterForGetter(c, possibleGetter);
+               Optional<Method> setter = ReflectiveBeanLikeClassUtils.getSetterForGetter(c, possibleGetter);
 
                if(setter.isPresent())
                {
-                  String                 fieldName             = getFieldNameFromGetter(possibleGetter);
+                  String                 fieldName             = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(possibleGetter);
                   Optional<QAssociation> associationAnnotation = getQAssociationAnnotation(c, fieldName);
 
                   if(associationAnnotation.isPresent())
                   {
                      @SuppressWarnings("unchecked")
-                     Class<? extends QRecordEntity> listTypeParam = (Class<? extends QRecordEntity>) getListTypeParam(possibleGetter.getReturnType(), possibleGetter.getAnnotatedReturnType());
+                     Class<? extends QRecordEntity> listTypeParam = (Class<? extends QRecordEntity>) ReflectiveBeanLikeClassUtils.getListTypeParam(possibleGetter.getReturnType(), possibleGetter.getAnnotatedReturnType());
                      associationList.add(new QRecordEntityAssociation(fieldName, possibleGetter, setter.get(), listTypeParam, associationAnnotation.orElse(null)));
                   }
                }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEntityField.java

@@ -28,6 +28,7 @@ import java.math.BigDecimal;
 import java.time.Instant;
 import java.time.LocalDate;
 import java.time.LocalTime;
+import java.util.Map;
 import com.kingsrook.qqq.backend.core.exceptions.QValueException;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
@@ -170,6 +171,11 @@ public class QRecordEntityField
          {
             return (ValueUtils.getValueAsByteArray(value));
          }
+
+         if(type.equals(Map.class))
+         {
+            return (ValueUtils.getValueAsMap(value));
+         }
       }
       catch(Exception e)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordWithJoinedRecords.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.model.data;
 
 import java.io.Serializable;
 import java.util.LinkedHashMap;
+import java.util.List;
 import java.util.Map;
 import java.util.function.BiFunction;
 
@@ -198,4 +199,62 @@ public class QRecordWithJoinedRecords extends QRecord
       return (rs);
    }
 
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public Map<String, List<QRecord>> getAssociatedRecords()
+   {
+      return mainRecord.getAssociatedRecords();
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QRecord withAssociatedRecord(String name, QRecord associatedRecord)
+   {
+      mainRecord.withAssociatedRecord(name, associatedRecord);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QRecord withAssociatedRecords(Map<String, List<QRecord>> associatedRecords)
+   {
+      mainRecord.withAssociatedRecords(associatedRecords);
+      return (this);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public void setAssociatedRecords(Map<String, List<QRecord>> associatedRecords)
+   {
+      mainRecord.setAssociatedRecords(associatedRecords);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QRecord withAssociatedRecords(String name, List<QRecord> associatedRecords)
+   {
+      mainRecord.withAssociatedRecords(name, associatedRecords);
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/helpcontent/HelpContentMetaDataProvider.java

@@ -82,6 +82,7 @@ public class HelpContentMetaDataProvider
       table.getField("key").withFieldAdornment(AdornmentType.Size.LARGE.toAdornment());
       table.getField("content").withFieldAdornment(AdornmentType.Size.LARGE.toAdornment());
       table.getField("content").withFieldAdornment(new FieldAdornment(AdornmentType.CODE_EDITOR).withValue(AdornmentType.CodeEditorValues.languageMode("html")));
+      table.getField("content").withGridColumns(12);
 
       if(backendDetailEnricher != null)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QAuthenticationType.java

@@ -28,6 +28,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
  *******************************************************************************/
 public enum QAuthenticationType
 {
+   OAUTH2("OAuth2"),
    AUTH_0("auth0"),
    TABLE_BASED("tableBased"),
    FULLY_ANONYMOUS("fullyAnonymous"),

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QInstance.java

@@ -1250,7 +1250,7 @@ public class QInstance
       {
          this.supplementalMetaData = new HashMap<>();
       }
-      this.supplementalMetaData.put(supplementalMetaData.getType(), supplementalMetaData);
+      this.supplementalMetaData.put(supplementalMetaData.getName(), supplementalMetaData);
       return (this);
    }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QMetaDataObject.java

@@ -0,0 +1,34 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata;
+
+
+import java.io.Serializable;
+
+
+/*******************************************************************************
+ ** interface common among all objects that can be considered qqq meta data -
+ ** e.g., stored in a QInstance.
+ *******************************************************************************/
+public interface QMetaDataObject extends Serializable
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QSupplementalInstanceMetaData.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.model.metadata;
 
 
+import java.util.function.Supplier;
 import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 
@@ -30,20 +31,13 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
  ** Base-class for instance-level meta-data defined by some supplemental module, etc,
  ** outside of qqq core
  *******************************************************************************/
-public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataInterface
+public interface QSupplementalInstanceMetaData extends TopLevelMetaDataInterface
 {
 
-   /*******************************************************************************
-    ** Getter for type
-    *******************************************************************************/
-   public abstract String getType();
-
-
-
    /*******************************************************************************
     **
     *******************************************************************************/
-   public void enrich(QTableMetaData table)
+   default void enrich(QTableMetaData table)
    {
       ////////////////////////
       // noop in base class //
@@ -55,7 +49,7 @@ public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataI
    /*******************************************************************************
     **
     *******************************************************************************/
-   public void validate(QInstance qInstance, QInstanceValidator validator)
+   default void validate(QInstance qInstance, QInstanceValidator validator)
    {
       ////////////////////////
       // noop in base class //
@@ -68,9 +62,33 @@ public abstract class QSupplementalInstanceMetaData implements TopLevelMetaDataI
     **
     *******************************************************************************/
    @Override
-   public void addSelfToInstance(QInstance qInstance)
+   default void addSelfToInstance(QInstance qInstance)
    {
       qInstance.withSupplementalMetaData(this);
    }
 
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   static <S extends QSupplementalInstanceMetaData> S of(QInstance qInstance, String name)
+   {
+      return ((S) qInstance.getSupplementalMetaData(name));
+   }
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   static <S extends QSupplementalInstanceMetaData> S ofOrWithNew(QInstance qInstance, String name, Supplier<S> supplier)
+   {
+      S s = (S) qInstance.getSupplementalMetaData(name);
+      if(s == null)
+      {
+         s = supplier.get();
+         s.addSelfToInstance(qInstance);
+      }
+      return (s);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/TopLevelMetaDataInterface.java

@@ -26,7 +26,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
  ** Interface for meta-data classes that can be added directly (e.g, at the top
  ** level) to a QInstance (such as a QTableMetaData - not a QFieldMetaData).
  *******************************************************************************/
-public interface TopLevelMetaDataInterface extends MetaDataProducerOutput
+public interface TopLevelMetaDataInterface extends MetaDataProducerOutput, QMetaDataObject
 {
 
    /*******************************************************************************

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/audits/QAuditRules.java

@@ -22,10 +22,13 @@
 package com.kingsrook.qqq.backend.core.model.metadata.audits;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QAuditRules
+public class QAuditRules implements QMetaDataObject
 {
    private AuditLevel auditLevel;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/authentication/OAuth2AuthenticationMetaData.java

@@ -0,0 +1,320 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2022.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.authentication;
+
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.backend.core.modules.authentication.implementations.OAuth2AuthenticationModule;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ ** Meta-data to provide details of an OAuth2 Authentication module
+ *******************************************************************************/
+public class OAuth2AuthenticationMetaData extends QAuthenticationMetaData
+{
+   private String baseUrl;
+   private String tokenUrl;
+   private String clientId;
+   private String scopes;
+
+   private String userSessionTableName;
+   private String redirectStateTableName;
+
+   ////////////////////////////////////////////////////////////////////////////////////////
+   // keep this secret, on the server - don't let it be serialized and sent to a client! //
+   ////////////////////////////////////////////////////////////////////////////////////////
+   @JsonIgnore
+   private String clientSecret;
+
+
+
+   /*******************************************************************************
+    ** Default Constructor.
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData()
+   {
+      super();
+      setType(QAuthenticationType.OAUTH2);
+
+      //////////////////////////////////////////////////////////
+      // ensure this module is registered with the dispatcher //
+      //////////////////////////////////////////////////////////
+      QAuthenticationModuleDispatcher.registerModule(QAuthenticationType.OAUTH2.getName(), OAuth2AuthenticationModule.class.getName());
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public void validate(QInstance qInstance, QInstanceValidator qInstanceValidator)
+   {
+      super.validate(qInstance, qInstanceValidator);
+
+      String prefix = "OAuth2AuthenticationMetaData (named '" + getName() + "'): ";
+
+      qInstanceValidator.assertCondition(StringUtils.hasContent(baseUrl), prefix + "baseUrl must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(clientId), prefix + "clientId must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(clientSecret), prefix + "clientSecret must be set");
+      qInstanceValidator.assertCondition(StringUtils.hasContent(scopes), prefix + "scopes must be set");
+
+      if(qInstanceValidator.assertCondition(StringUtils.hasContent(userSessionTableName), prefix + "userSessionTableName must be set"))
+      {
+         qInstanceValidator.assertCondition(qInstance.getTable(userSessionTableName) != null, prefix + "userSessionTableName ('" + userSessionTableName + "') was not found in the instance");
+      }
+
+      if(qInstanceValidator.assertCondition(StringUtils.hasContent(redirectStateTableName), prefix + "redirectStateTableName must be set"))
+      {
+         qInstanceValidator.assertCondition(qInstance.getTable(redirectStateTableName) != null, prefix + "redirectStateTableName ('" + redirectStateTableName + "') was not found in the instance");
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withBaseUrl(String baseUrl)
+   {
+      setBaseUrl(baseUrl);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for baseUrl
+    **
+    *******************************************************************************/
+   public String getBaseUrl()
+   {
+      return baseUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for baseUrl
+    **
+    *******************************************************************************/
+   public void setBaseUrl(String baseUrl)
+   {
+      this.baseUrl = baseUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withClientId(String clientId)
+   {
+      setClientId(clientId);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientId
+    **
+    *******************************************************************************/
+   public String getClientId()
+   {
+      return clientId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for clientId
+    **
+    *******************************************************************************/
+   public void setClientId(String clientId)
+   {
+      this.clientId = clientId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter, override to help fluent flows
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withClientSecret(String clientSecret)
+   {
+      setClientSecret(clientSecret);
+      return this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for clientSecret
+    **
+    *******************************************************************************/
+   public String getClientSecret()
+   {
+      return clientSecret;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for clientSecret
+    **
+    *******************************************************************************/
+   public void setClientSecret(String clientSecret)
+   {
+      this.clientSecret = clientSecret;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tokenUrl
+    *******************************************************************************/
+   public String getTokenUrl()
+   {
+      return (this.tokenUrl);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tokenUrl
+    *******************************************************************************/
+   public void setTokenUrl(String tokenUrl)
+   {
+      this.tokenUrl = tokenUrl;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tokenUrl
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withTokenUrl(String tokenUrl)
+   {
+      this.tokenUrl = tokenUrl;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for userSessionTableName
+    *******************************************************************************/
+   public String getUserSessionTableName()
+   {
+      return (this.userSessionTableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for userSessionTableName
+    *******************************************************************************/
+   public void setUserSessionTableName(String userSessionTableName)
+   {
+      this.userSessionTableName = userSessionTableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for userSessionTableName
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withUserSessionTableName(String userSessionTableName)
+   {
+      this.userSessionTableName = userSessionTableName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for redirectStateTableName
+    *******************************************************************************/
+   public String getRedirectStateTableName()
+   {
+      return (this.redirectStateTableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for redirectStateTableName
+    *******************************************************************************/
+   public void setRedirectStateTableName(String redirectStateTableName)
+   {
+      this.redirectStateTableName = redirectStateTableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for redirectStateTableName
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withRedirectStateTableName(String redirectStateTableName)
+   {
+      this.redirectStateTableName = redirectStateTableName;
+      return (this);
+   }
+
+
+   /*******************************************************************************
+    ** Getter for scopes
+    *******************************************************************************/
+   public String getScopes()
+   {
+      return (this.scopes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for scopes
+    *******************************************************************************/
+   public void setScopes(String scopes)
+   {
+      this.scopes = scopes;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for scopes
+    *******************************************************************************/
+   public OAuth2AuthenticationMetaData withScopes(String scopes)
+   {
+      this.scopes = scopes;
+      return (this);
+   }
+
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/authentication/QAuthenticationMetaData.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.authentication;
 import java.util.HashMap;
 import java.util.Map;
 import com.fasterxml.jackson.annotation.JsonFilter;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
 import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
@@ -225,4 +226,15 @@ public class QAuthenticationMetaData implements TopLevelMetaDataInterface
       return (this);
    }
 
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public void validate(QInstance qInstance, QInstanceValidator qInstanceValidator)
+   {
+      //////////////////
+      // noop at base //
+      //////////////////
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/code/QCodeReference.java

@@ -23,13 +23,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.code;
 
 
 import java.io.Serializable;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Pointer to code to be ran by the qqq framework, e.g., for custom behavior -
  ** maybe process steps, maybe customization to a table, etc.
  *******************************************************************************/
-public class QCodeReference implements Serializable, Cloneable
+public class QCodeReference implements Serializable, Cloneable, QMetaDataObject
 {
    private String    name;
    private QCodeType codeType;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/fields/QFieldMetaData.java

@@ -40,11 +40,13 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.data.QField;
 import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.ReflectiveBeanLikeClassUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -54,7 +56,7 @@ import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
  ** Meta-data to represent a single field in a table.
  **
  *******************************************************************************/
-public class QFieldMetaData implements Cloneable
+public class QFieldMetaData implements Cloneable, QMetaDataObject
 {
    private static final QLogger LOG = QLogger.getLogger(QFieldMetaData.class);
 
@@ -187,7 +189,7 @@ public class QFieldMetaData implements Cloneable
    {
       try
       {
-         this.name = QRecordEntity.getFieldNameFromGetter(getter);
+         this.name = ReflectiveBeanLikeClassUtils.getFieldNameFromGetter(getter);
          this.type = QFieldType.fromClass(getter.getReturnType());
 
          @SuppressWarnings("unchecked")

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/fields/WhiteSpaceBehavior.java

@@ -0,0 +1,174 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.fields;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.function.Function;
+import com.kingsrook.qqq.backend.core.actions.values.ValueBehaviorApplier;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+
+
+/*******************************************************************************
+ ** Field behavior that changes the whitespace of string values.
+ *******************************************************************************/
+public enum WhiteSpaceBehavior implements FieldBehavior<WhiteSpaceBehavior>, FieldBehaviorForFrontend, FieldFilterBehavior<WhiteSpaceBehavior>
+{
+   NONE(null),
+   REMOVE_ALL_WHITESPACE((String s) -> s.chars().filter(c -> !Character.isWhitespace(c)).collect(StringBuilder::new, StringBuilder::appendCodePoint, StringBuilder::append).toString()),
+   TRIM((String s) -> s.trim()),
+   TRIM_LEFT((String s) -> s.stripLeading()),
+   TRIM_RIGHT((String s) -> s.stripTrailing());
+
+
+   private final Function<String, String> function;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   WhiteSpaceBehavior(Function<String, String> function)
+   {
+      this.function = function;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public WhiteSpaceBehavior getDefault()
+   {
+      return (NONE);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void apply(ValueBehaviorApplier.Action action, List<QRecord> recordList, QInstance instance, QTableMetaData table, QFieldMetaData field)
+   {
+      if(this.equals(NONE))
+      {
+         return;
+      }
+
+      switch(this)
+      {
+         case REMOVE_ALL_WHITESPACE, TRIM, TRIM_LEFT, TRIM_RIGHT -> applyFunction(recordList, table, field);
+         default -> throw new IllegalStateException("Unexpected enum value: " + this);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void applyFunction(List<QRecord> recordList, QTableMetaData table, QFieldMetaData field)
+   {
+      String fieldName = field.getName();
+      for(QRecord record : CollectionUtils.nonNullList(recordList))
+      {
+         String value = record.getValueString(fieldName);
+         if(value != null && function != null)
+         {
+            record.setValue(fieldName, function.apply(value));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public Serializable applyToFilterCriteriaValue(Serializable value, QInstance instance, QTableMetaData table, QFieldMetaData field)
+   {
+      if(this.equals(NONE) || function == null)
+      {
+         return (value);
+      }
+
+      if(value instanceof String s)
+      {
+         String newValue = function.apply(s);
+         if(!Objects.equals(value, newValue))
+         {
+            return (newValue);
+         }
+      }
+
+      return (value);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public boolean allowMultipleBehaviorsOfThisType()
+   {
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<String> validateBehaviorConfiguration(QTableMetaData tableMetaData, QFieldMetaData fieldMetaData)
+   {
+      if(this == NONE)
+      {
+         return Collections.emptyList();
+      }
+
+      List<String> errors      = new ArrayList<>();
+      String       errorSuffix = " field [" + fieldMetaData.getName() + "] in table [" + tableMetaData.getName() + "]";
+
+      if(fieldMetaData.getType() != null)
+      {
+         if(!fieldMetaData.getType().isStringLike())
+         {
+            errors.add("A WhiteSpaceBehavior was a applied to a non-String-like field:" + errorSuffix);
+         }
+      }
+
+      return (errors);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/help/QHelpContent.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.help;
 import java.util.Collections;
 import java.util.HashSet;
 import java.util.Set;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
@@ -41,7 +42,7 @@ import java.util.Set;
  ** May be dynamically added to meta-data via (non-meta-) data - see
  ** HelpContentMetaDataProvider and QInstanceHelpContentManager
  *******************************************************************************/
-public class QHelpContent
+public class QHelpContent implements QMetaDataObject
 {
    private String        content;
    private HelpFormat    format;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QAppChildMetaData.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Interface shared by meta-data objects which can be placed into an App.
  ** e.g., Tables, Processes, and Apps themselves (since they can be nested)
  *******************************************************************************/
-public interface QAppChildMetaData
+public interface QAppChildMetaData extends QMetaDataObject
 {
    /*******************************************************************************
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QAppSection.java

@@ -24,12 +24,13 @@ package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 import java.util.ArrayList;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** A section of apps/tables/processes - a logical grouping.
  *******************************************************************************/
-public class QAppSection implements Cloneable
+public class QAppSection implements Cloneable, QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/layout/QIcon.java

@@ -22,6 +22,9 @@
 package com.kingsrook.qqq.backend.core.model.metadata.layout;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Icon to show associated with an App, Table, Process, etc.
  **
@@ -31,7 +34,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.layout;
  ** Future may allow something like a "namespace", and/or multiple icons for
  ** use in different frontends, etc.
  *******************************************************************************/
-public class QIcon implements Cloneable
+public class QIcon implements Cloneable, QMetaDataObject
 {
    private String name;
    private String path;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/permissions/QPermissionRules.java

@@ -22,13 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.permissions;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 
 
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QPermissionRules implements Cloneable
+public class QPermissionRules implements Cloneable, QMetaDataObject
 {
    private PermissionLevel level;
    private DenyBehavior    denyBehavior;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QFrontendComponentMetaData.java

@@ -25,12 +25,13 @@ package com.kingsrook.qqq.backend.core.model.metadata.processes;
 import java.io.Serializable;
 import java.util.HashMap;
 import java.util.Map;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 
 
 /*******************************************************************************
  ** Definition of a UI component in a frontend process steps.
  *******************************************************************************/
-public class QFrontendComponentMetaData
+public class QFrontendComponentMetaData implements QMetaDataObject
 {
    private QComponentType type;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QProcessMetaData.java

@@ -327,12 +327,23 @@ public class QProcessMetaData implements QAppChildMetaData, MetaDataWithPermissi
 
 
    /*******************************************************************************
-    ** Setter for stepList
+    ** Setter for stepList - note - calling this method ALSO overwrites the steps map!
     **
     *******************************************************************************/
    public void setStepList(List<QStepMetaData> stepList)
    {
-      this.stepList = stepList;
+      if(stepList == null)
+      {
+         this.stepList = null;
+         this.steps = null;
+      }
+      else
+      {
+         this.stepList = new ArrayList<>();
+         this.steps = new HashMap<>();
+      }
+
+      withStepList(stepList);
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QStateMachineStep.java

@@ -185,4 +185,25 @@ public class QStateMachineStep extends QStepMetaData
       return (rs);
    }
 
+
+   /*******************************************************************************
+    ** Setter for subSteps
+    *******************************************************************************/
+   public void setSubSteps(List<QStepMetaData> subSteps)
+   {
+      this.subSteps = subSteps;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for subSteps
+    *******************************************************************************/
+   public QStateMachineStep withSubSteps(List<QStepMetaData> subSteps)
+   {
+      this.subSteps = subSteps;
+      return (this);
+   }
+
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QStepMetaData.java

@@ -26,6 +26,7 @@ import java.util.ArrayList;
 import java.util.List;
 import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.serialization.QStepMetaDataDeserializer;
 
@@ -37,7 +38,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.serialization.QStepMetaData
  **
  *******************************************************************************/
 @JsonDeserialize(using = QStepMetaDataDeserializer.class)
-public abstract class QStepMetaData
+public abstract class QStepMetaData implements QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/scheduleing/QScheduleMetaData.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.model.metadata.scheduleing;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -35,7 +36,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  ** same moment.
  **
  *******************************************************************************/
-public class QScheduleMetaData
+public class QScheduleMetaData implements QMetaDataObject
 {
    private String schedulerName;
    private String description;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/Association.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.tables;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** definition of a qqq table that is "associated" with another table, e.g.,
  ** managed along with it - such as child-records under a parent record.
  *******************************************************************************/
-public class Association
+public class Association implements QMetaDataObject
 {
    private String name;
    private String associatedTableName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/QFieldSection.java

@@ -26,6 +26,7 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.instances.QInstanceHelpContentManager;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
@@ -36,7 +37,7 @@ import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
  ** A section of fields - a logical grouping.
  ** TODO - this class should be named QTableSection!
  *******************************************************************************/
-public class QFieldSection
+public class QFieldSection implements QMetaDataObject
 {
    private String name;
    private String label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/TablesCustomPossibleValueProvider.java

@@ -27,11 +27,9 @@ import java.util.ArrayList;
 import java.util.List;
 import com.kingsrook.qqq.backend.core.actions.permissions.PermissionCheckResult;
 import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
-import com.kingsrook.qqq.backend.core.actions.values.QCustomPossibleValueProvider;
+import com.kingsrook.qqq.backend.core.actions.values.BasicCustomPossibleValueProvider;
 import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
@@ -40,26 +38,16 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  ** possible-value source provider for the `Tables` PVS - a list of all tables
  ** in an application/qInstance.
  *******************************************************************************/
-public class TablesCustomPossibleValueProvider implements QCustomPossibleValueProvider<String>
+public class TablesCustomPossibleValueProvider extends BasicCustomPossibleValueProvider<QTableMetaData, String>
 {
 
    /***************************************************************************
     **
     ***************************************************************************/
    @Override
-   public QPossibleValue<String> getPossibleValue(Serializable idValue)
+   protected QPossibleValue<String> makePossibleValue(QTableMetaData sourceObject)
    {
-      QTableMetaData table = QContext.getQInstance().getTable(ValueUtils.getValueAsString(idValue));
-      if(table != null && !table.getIsHidden())
-      {
-         PermissionCheckResult permissionCheckResult = PermissionsHelper.getPermissionCheckResult(new QueryInput(table.getName()), table);
-         if(PermissionCheckResult.ALLOW.equals(permissionCheckResult))
-         {
-            return (new QPossibleValue<>(table.getName(), table.getLabel()));
-         }
-      }
-
-      return null;
+      return (new QPossibleValue<>(sourceObject.getName(), sourceObject.getLabel()));
    }
 
 
@@ -68,22 +56,54 @@ public class TablesCustomPossibleValueProvider implements QCustomPossibleValuePr
     **
     ***************************************************************************/
    @Override
-   public List<QPossibleValue<String>> search(SearchPossibleValueSourceInput input) throws QException
+   protected QTableMetaData getSourceObject(Serializable id)
    {
-      /////////////////////////////////////////////////////////////////////////////////////
-      // build all of the possible values (note, will be filtered by user's permissions) //
-      /////////////////////////////////////////////////////////////////////////////////////
-      List<QPossibleValue<String>> allPossibleValues = new ArrayList<>();
+      QTableMetaData table = QContext.getQInstance().getTable(ValueUtils.getValueAsString(id));
+      return isTableAllowed(table) ? table : null;
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   protected List<QTableMetaData> getAllSourceObjects()
+   {
+      ArrayList<QTableMetaData> rs = new ArrayList<>();
       for(QTableMetaData table : QContext.getQInstance().getTables().values())
       {
-         QPossibleValue<String> possibleValue = getPossibleValue(table.getName());
-         if(possibleValue != null)
+         if(isTableAllowed(table))
          {
-            allPossibleValues.add(possibleValue);
+            rs.add(table);
          }
       }
+      return rs;
+   }
 
-      return completeCustomPVSSearch(input, allPossibleValues);
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private boolean isTableAllowed(QTableMetaData table)
+   {
+      if(table == null)
+      {
+         return (false);
+      }
+
+      if(table.getIsHidden())
+      {
+         return (false);
+      }
+
+      PermissionCheckResult permissionCheckResult = PermissionsHelper.getPermissionCheckResult(new QueryInput(table.getName()), table);
+      if(!PermissionCheckResult.ALLOW.equals(permissionCheckResult))
+      {
+         return (false);
+      }
+
+      return (true);
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/TablesPossibleValueSourceMetaDataProvider.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables;
 
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PVSValueFormatAndFields;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
@@ -45,6 +46,7 @@ public class TablesPossibleValueSourceMetaDataProvider
    {
       QPossibleValueSource possibleValueSource = new QPossibleValueSource()
          .withName(NAME)
+         .withIdType(QFieldType.STRING)
          .withType(QPossibleValueSourceType.CUSTOM)
          .withCustomCodeReference(new QCodeReference(TablesCustomPossibleValueProvider.class))
          .withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/UniqueKey.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -32,7 +33,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  ** Definition of a Unique Key (or "Constraint", if you wanna use fancy words)
  ** on a QTable.
  *******************************************************************************/
-public class UniqueKey
+public class UniqueKey implements QMetaDataObject
 {
    private List<String> fieldNames;
    private String       label;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/AutomationStatusTracking.java

@@ -22,11 +22,14 @@
 package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 
 
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
+
+
 /*******************************************************************************
  ** Table-automation meta-data to define how this table's per-record automation
  ** status is tracked.
  *******************************************************************************/
-public class AutomationStatusTracking
+public class AutomationStatusTracking implements QMetaDataObject
 {
    private AutomationStatusTrackingType type;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/QTableAutomationDetails.java

@@ -24,13 +24,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 
 import java.util.ArrayList;
 import java.util.List;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
 
 
 /*******************************************************************************
  ** Details about how this table's record automations are set up.
  *******************************************************************************/
-public class QTableAutomationDetails
+public class QTableAutomationDetails implements QMetaDataObject
 {
    private AutomationStatusTracking    statusTracking;
    private String                      providerName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/automation/TableAutomationAction.java

@@ -25,13 +25,14 @@ package com.kingsrook.qqq.backend.core.model.metadata.tables.automation;
 import java.io.Serializable;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.QMetaDataObject;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 
 
 /*******************************************************************************
  ** Definition of a specific action to run against a table
  *******************************************************************************/
-public class TableAutomationAction
+public class TableAutomationAction implements QMetaDataObject
 {
    private String       name;
    private TriggerEvent triggerEvent;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedbulkloadprofiles/SavedBulkLoadProfile.java

@@ -60,6 +60,9 @@ public class SavedBulkLoadProfile extends QRecordEntity
    @QField(label = "Mapping JSON")
    private String mappingJson;
 
+   @QField()
+   private Boolean isBulkEdit;
+
 
 
    /*******************************************************************************
@@ -251,7 +254,6 @@ public class SavedBulkLoadProfile extends QRecordEntity
 
 
 
-
    /*******************************************************************************
     ** Getter for mappingJson
     *******************************************************************************/
@@ -282,4 +284,34 @@ public class SavedBulkLoadProfile extends QRecordEntity
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for isBulkEdit
+    *******************************************************************************/
+   public Boolean getIsBulkEdit()
+   {
+      return (this.isBulkEdit);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for isBulkEdit
+    *******************************************************************************/
+   public void setIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isBulkEdit
+    *******************************************************************************/
+   public SavedBulkLoadProfile withIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedbulkloadprofiles/SavedBulkLoadProfileMetaDataProvider.java

@@ -113,7 +113,7 @@ public class SavedBulkLoadProfileMetaDataProvider
          .withFieldsFromEntity(SavedBulkLoadProfile.class)
          .withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD))
          .withSection(new QFieldSection("identity", new QIcon().withName("badge"), Tier.T1, List.of("id", "label", "tableName")))
-         .withSection(new QFieldSection("details", new QIcon().withName("text_snippet"), Tier.T2, List.of("userId", "mappingJson")))
+         .withSection(new QFieldSection("details", new QIcon().withName("text_snippet"), Tier.T2, List.of("userId", "mappingJson", "isBulkEdit")))
          .withSection(new QFieldSection("dates", new QIcon().withName("calendar_month"), Tier.T3, List.of("createDate", "modifyDate")));
 
       table.getField("mappingJson").withBehavior(SavedBulkLoadProfileJsonFieldDisplayValueFormatter.getInstance());

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/session/QUser.java

@@ -22,10 +22,13 @@
 package com.kingsrook.qqq.backend.core.model.session;
 
 
+import java.io.Serializable;
+
+
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QUser implements Cloneable
+public class QUser implements Cloneable, Serializable
 {
    private String idReference;
    private String fullName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/QAuthenticationModuleInterface.java

@@ -25,15 +25,14 @@ package com.kingsrook.qqq.backend.core.modules.authentication;
 import java.io.Serializable;
 import java.util.Map;
 import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.exceptions.AccessTokenException;
 import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang.NotImplementedException;
 
 
+
 /*******************************************************************************
  ** Interface that a QAuthenticationModule must implement.
  **
@@ -82,12 +81,12 @@ public interface QAuthenticationModuleInterface
    }
 
 
-   /*******************************************************************************
+   /***************************************************************************
     **
-    *******************************************************************************/
-   default String createAccessToken(QAuthenticationMetaData metaData, String clientId, String clientSecret) throws AccessTokenException
+    ***************************************************************************/
+   default String getLoginRedirectUrl(String originalUrl) throws QAuthenticationException
    {
-      throw (new NotImplementedException("The method createAccessToken() is not implemented in the class: " + this.getClass().getSimpleName()));
+      throw (new NotImplementedException("The method getLoginRedirectUrl() is not implemented in the authentication module: " + this.getClass().getSimpleName()));
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/Auth0AuthenticationModule.java

@@ -1020,7 +1020,7 @@ public class Auth0AuthenticationModule implements QAuthenticationModuleInterface
             // decode the accessToken and make sure it is not expired //
             ////////////////////////////////////////////////////////////
             boolean needNewToken = true;
-            if(accessToken != null)
+            if(StringUtils.hasContent(accessToken))
             {
                DecodedJWT jwt     = JWT.decode(accessToken);
                String     payload = jwt.getPayload();

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/FullyAnonymousAuthenticationModule.java

@@ -24,9 +24,7 @@ package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
 
 import java.util.Map;
 import java.util.UUID;
-import com.kingsrook.qqq.backend.core.exceptions.AccessTokenException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.model.session.QUser;
 import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleInterface;
@@ -77,15 +75,4 @@ public class FullyAnonymousAuthenticationModule implements QAuthenticationModule
       return session != null;
    }
 
-
-
-   /*******************************************************************************
-    ** Load an instance of the appropriate state provider
-    **
-    *******************************************************************************/
-   public String createAccessToken(QAuthenticationMetaData metaData, String clientId, String clientSecret) throws AccessTokenException
-   {
-      return (TEST_ACCESS_TOKEN);
-   }
-
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/MockAuthenticationModule.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
 
 import java.util.Map;
 import java.util.UUID;
+import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
@@ -45,8 +46,13 @@ public class MockAuthenticationModule implements QAuthenticationModuleInterface
     **
     *******************************************************************************/
    @Override
-   public QSession createSession(QInstance qInstance, Map<String, String> context)
+   public QSession createSession(QInstance qInstance, Map<String, String> context) throws QAuthenticationException
    {
+      if("Deny".equalsIgnoreCase(context.get("accessToken")))
+      {
+         throw (new QAuthenticationException("Access denied (per accessToken requesting as such)"));
+      }
+
       QUser qUser = new QUser();
       qUser.setIdReference("User:" + (System.currentTimeMillis() % USER_ID_MODULO));
       qUser.setFullName("John Smith");
@@ -80,4 +86,16 @@ public class MockAuthenticationModule implements QAuthenticationModuleInterface
 
       return (true);
    }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getLoginRedirectUrl(String originalUrl)
+   {
+      return originalUrl + "?createMockSession=true";
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/OAuth2AuthenticationModule.java

@@ -0,0 +1,486 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
+
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.net.URI;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.temporal.ChronoUnit;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.atomic.AtomicReference;
+import com.auth0.jwt.JWT;
+import com.auth0.jwt.interfaces.DecodedJWT;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.context.CapturedContext;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.OAuth2AuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
+import com.kingsrook.qqq.backend.core.model.session.QUser;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleInterface;
+import com.kingsrook.qqq.backend.core.modules.authentication.implementations.model.UserSession;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
+import com.nimbusds.oauth2.sdk.AuthorizationCode;
+import com.nimbusds.oauth2.sdk.AuthorizationCodeGrant;
+import com.nimbusds.oauth2.sdk.AuthorizationGrant;
+import com.nimbusds.oauth2.sdk.ErrorObject;
+import com.nimbusds.oauth2.sdk.GeneralException;
+import com.nimbusds.oauth2.sdk.ParseException;
+import com.nimbusds.oauth2.sdk.Scope;
+import com.nimbusds.oauth2.sdk.TokenRequest;
+import com.nimbusds.oauth2.sdk.TokenResponse;
+import com.nimbusds.oauth2.sdk.auth.ClientAuthentication;
+import com.nimbusds.oauth2.sdk.auth.ClientSecretBasic;
+import com.nimbusds.oauth2.sdk.auth.Secret;
+import com.nimbusds.oauth2.sdk.id.ClientID;
+import com.nimbusds.oauth2.sdk.id.Issuer;
+import com.nimbusds.oauth2.sdk.id.State;
+import com.nimbusds.oauth2.sdk.pkce.CodeVerifier;
+import com.nimbusds.oauth2.sdk.token.AccessToken;
+import com.nimbusds.openid.connect.sdk.op.OIDCProviderMetadata;
+import org.json.JSONObject;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Implementation of OAuth2 authentication.
+ *******************************************************************************/
+public class OAuth2AuthenticationModule implements QAuthenticationModuleInterface
+{
+   private static final QLogger LOG = QLogger.getLogger(OAuth2AuthenticationModule.class);
+
+   private static boolean mayMemoize = true;
+
+   private static final Memoization<String, String> getAccessTokenFromSessionUUIDMemoization = new Memoization<String, String>()
+      .withTimeout(Duration.of(1, ChronoUnit.MINUTES))
+      .withMaxSize(1000);
+
+   private static final Memoization<String, OIDCProviderMetadata> oidcProviderMetadataMemoization = new Memoization<String, OIDCProviderMetadata>()
+      .withMayStoreNullValues(false);
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QSession createSession(QInstance qInstance, Map<String, String> context) throws QAuthenticationException
+   {
+      try
+      {
+         OAuth2AuthenticationMetaData oauth2MetaData = (OAuth2AuthenticationMetaData) qInstance.getAuthentication();
+
+         if(context.containsKey("code") && context.containsKey("state"))
+         {
+            ///////////////////////////////////////////////////////////////////////
+            // handle a callback to initially auth a user for a traditional      //
+            // (non-js) site - where the code & state params come to the backend //
+            ///////////////////////////////////////////////////////////////////////
+            AuthorizationCode code = new AuthorizationCode(context.get("code"));
+
+            /////////////////////////////////////////
+            // verify the state in our state table //
+            /////////////////////////////////////////
+            AtomicReference<String> redirectUri = new AtomicReference<>(null);
+            QContext.withTemporaryContext(new CapturedContext(qInstance, new QSystemUserSession()), () ->
+            {
+               QRecord redirectStateRecord = GetAction.execute(oauth2MetaData.getRedirectStateTableName(), Map.of("state", context.get("state")));
+               if(redirectStateRecord == null)
+               {
+                  throw (new QAuthenticationException("State not found"));
+               }
+               redirectUri.set(redirectStateRecord.getValueString("redirectUri"));
+            });
+
+            URI                    redirectURI       = new URI(redirectUri.get());
+            ClientSecretBasic      clientSecretBasic = new ClientSecretBasic(new ClientID(oauth2MetaData.getClientId()), new Secret(oauth2MetaData.getClientSecret()));
+            AuthorizationCodeGrant codeGrant         = new AuthorizationCodeGrant(code, redirectURI);
+
+            URI          tokenEndpoint = getOIDCProviderMetadata(oauth2MetaData).getTokenEndpointURI();
+            Scope        scope         = new Scope(oauth2MetaData.getScopes());
+            TokenRequest tokenRequest  = new TokenRequest(tokenEndpoint, clientSecretBasic, codeGrant, scope);
+
+            return createSessionFromTokenRequest(tokenRequest);
+         }
+         else if(context.containsKey("code") && context.containsKey("redirectUri") && context.containsKey("codeVerifier"))
+         {
+            ////////////////////////////////////////////////////////////////////////////////
+            // handle a call down to this backend code to initially auth a user for an    //
+            // SPA that received a code (where the javascript generated the codeVerifier) //
+            ////////////////////////////////////////////////////////////////////////////////
+            AuthorizationCode  code         = new AuthorizationCode(context.get("code"));
+            URI                callback     = new URI(context.get("redirectUri"));
+            CodeVerifier       codeVerifier = new CodeVerifier(context.get("codeVerifier"));
+            AuthorizationGrant codeGrant    = new AuthorizationCodeGrant(code, callback, codeVerifier);
+
+            ClientID             clientID     = new ClientID(oauth2MetaData.getClientId());
+            Secret               clientSecret = new Secret(oauth2MetaData.getClientSecret());
+            ClientAuthentication clientAuth   = new ClientSecretBasic(clientID, clientSecret);
+
+            URI          tokenEndpoint = getOIDCProviderMetadata(oauth2MetaData).getTokenEndpointURI();
+            Scope        scope         = new Scope(oauth2MetaData.getScopes());
+            TokenRequest tokenRequest  = new TokenRequest(tokenEndpoint, clientAuth, codeGrant, scope);
+
+            return createSessionFromTokenRequest(tokenRequest);
+         }
+         else if(context.containsKey("sessionUUID") || context.containsKey("sessionId") || context.containsKey("uuid"))
+         {
+            //////////////////////////////////////////////////////////////////////
+            // handle a "normal" request, where we aren't opening a new session //
+            // per-se, but instead are looking for one in our userSession table //
+            //////////////////////////////////////////////////////////////////////
+            String uuid = Objects.requireNonNullElseGet(context.get("sessionUUID"), () ->
+               Objects.requireNonNullElseGet(context.get("sessionId"), () ->
+                  context.get("uuid")));
+
+            String   accessToken = getAccessTokenFromSessionUUID(uuid);
+            QSession session     = createSessionFromToken(accessToken);
+            session.setUuid(uuid);
+
+            //////////////////////////////////////////////////////////////////
+            // todo - do we need to validate its age or ping the provider?? //
+            //////////////////////////////////////////////////////////////////
+
+            return (session);
+         }
+         else
+         {
+            String message = "Did not receive recognized values in context for creating session";
+            LOG.warn(message, logPair("contextKeys", context.keySet()));
+            throw (new QAuthenticationException(message));
+         }
+      }
+      catch(QAuthenticationException qae)
+      {
+         throw (qae);
+      }
+      catch(Exception e)
+      {
+         throw (new QAuthenticationException("Failed to create session (token)", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private QSession createSessionFromTokenRequest(TokenRequest tokenRequest) throws ParseException, IOException, QException
+   {
+      TokenResponse tokenResponse = TokenResponse.parse(tokenRequest.toHTTPRequest().send());
+
+      if(tokenResponse.indicatesSuccess())
+      {
+         AccessToken accessToken = tokenResponse.toSuccessResponse().getTokens().getAccessToken();
+
+         ////////////////////////////////////////////////////////////////////
+         // todo - do we want to try to do anything with a refresh token?? //
+         ////////////////////////////////////////////////////////////////////
+         // RefreshToken refreshToken = tokenResponse.toSuccessResponse().getTokens().getRefreshToken();
+
+         QSession session = createSessionFromToken(accessToken.getValue());
+         insertUserSession(accessToken.getValue(), session);
+         return (session);
+      }
+      else
+      {
+         ErrorObject errorObject = tokenResponse.toErrorResponse().getErrorObject();
+         LOG.info("Token request failed", logPair("code", errorObject.getCode()), logPair("description", errorObject.getDescription()));
+         throw (new QAuthenticationException(errorObject.getDescription()));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public boolean isSessionValid(QInstance instance, QSession session)
+   {
+      if(session instanceof QSystemUserSession)
+      {
+         return (true);
+      }
+
+      try
+      {
+         String     accessToken = getAccessTokenFromSessionUUID(session.getUuid());
+         DecodedJWT jwt         = JWT.decode(accessToken);
+         if(jwt.getExpiresAtAsInstant().isBefore(Instant.now()))
+         {
+            LOG.debug("accessToken is expired", logPair("sessionUUID", session.getUuid()));
+            return (false);
+         }
+
+         return true;
+      }
+      catch(QAuthenticationException e)
+      {
+         return (false);
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public String getLoginRedirectUrl(String originalUrl) throws QAuthenticationException
+   {
+      try
+      {
+         QInstance                    qInstance      = QContext.getQInstance();
+         OAuth2AuthenticationMetaData oauth2MetaData = (OAuth2AuthenticationMetaData) qInstance.getAuthentication();
+         String                       authUrl        = getOIDCProviderMetadata(oauth2MetaData).getAuthorizationEndpointURI().toString();
+
+         QTableMetaData stateTable = QContext.getQInstance().getTable(oauth2MetaData.getRedirectStateTableName());
+         if(stateTable == null)
+         {
+            throw (new QAuthenticationException("The table specified as the oauthRedirectStateTableName [" + oauth2MetaData.getRedirectStateTableName() + "] is not defined in the QInstance"));
+         }
+
+         ///////////////////////////////////////////////////////////////////
+         // generate a secure state, of either default length (32 bytes), //
+         // or at a size (base64 encoded) that fits in the state table    //
+         ///////////////////////////////////////////////////////////////////
+         Integer stateStringLength = stateTable.getField("state").getMaxLength();
+         State   state             = stateStringLength == null ? new State(32) : new State((stateStringLength / 4) * 3);
+         String  stateValue        = state.getValue();
+
+         /////////////////////////////
+         // insert the state record //
+         /////////////////////////////
+         QContext.withTemporaryContext(new CapturedContext(qInstance, new QSystemUserSession()), () ->
+         {
+            QRecord insertedState = new InsertAction().execute(new InsertInput(oauth2MetaData.getRedirectStateTableName()).withRecord(new QRecord()
+               .withValue("state", stateValue)
+               .withValue("redirectUri", originalUrl))).getRecords().get(0);
+            if(CollectionUtils.nullSafeHasContents(insertedState.getErrors()))
+            {
+               throw (new QAuthenticationException("Error storing redirect state: " + insertedState.getErrorsAsString()));
+            }
+         });
+
+         return authUrl
+            + "?client_id=" + URLEncoder.encode(oauth2MetaData.getClientId(), StandardCharsets.UTF_8)
+            + "&redirect_uri=" + URLEncoder.encode(originalUrl, StandardCharsets.UTF_8)
+            + "&response_type=code"
+            + "&scope=" + URLEncoder.encode(oauth2MetaData.getScopes(), StandardCharsets.UTF_8)
+            + "&state=" + URLEncoder.encode(state.getValue(), StandardCharsets.UTF_8);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error getting login redirect url", e);
+         throw (new QAuthenticationException("Error getting login redirect url", e));
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private QSession createSessionFromToken(String accessToken) throws QException
+   {
+      DecodedJWT     jwt           = JWT.decode(accessToken);
+      Base64.Decoder decoder       = Base64.getUrlDecoder();
+      String         payloadString = new String(decoder.decode(jwt.getPayload()));
+      JSONObject     payload       = new JSONObject(payloadString);
+
+      QSession session = new QSession();
+      QUser    user    = new QUser();
+      session.setUser(user);
+
+      user.setFullName("Unknown");
+      String email = Objects.requireNonNullElseGet(payload.optString("email", null), () -> payload.optString("sub", null));
+      String name  = payload.optString("name", email);
+
+      user.setIdReference(email);
+      user.setFullName(name);
+
+      ////////////////////////////////////////////////////////////
+      // todo wip - this needs to be much better standardized w/ fe //
+      ////////////////////////////////////////////////////////////
+      session.withValueForFrontend("user", new HashMap<>(Map.of("name", name, "email", email)));
+
+      return session;
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert a session as a new record into userSession table
+    *******************************************************************************/
+   private void insertUserSession(String accessToken, QSession qSession) throws QException
+   {
+      CapturedContext capturedContext = QContext.capture();
+      try
+      {
+         QContext.init(capturedContext.qInstance(), new QSystemUserSession());
+
+         UserSession userSession = new UserSession()
+            .withUuid(qSession.getUuid())
+            .withUserId(qSession.getUser().getIdReference())
+            .withAccessToken(accessToken);
+
+         new InsertAction().execute(new InsertInput(UserSession.TABLE_NAME).withRecordEntity(userSession));
+      }
+      finally
+      {
+         QContext.init(capturedContext);
+      }
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public QSession createAutomatedSessionForUser(QInstance qInstance, Serializable userId) throws QAuthenticationException
+   {
+      return QAuthenticationModuleInterface.super.createAutomatedSessionForUser(qInstance, userId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Look up access_token from session UUID
+    **
+    *******************************************************************************/
+   private String getAccessTokenFromSessionUUID(String sessionUUID) throws QAuthenticationException
+   {
+      if(mayMemoize)
+      {
+         return getAccessTokenFromSessionUUIDMemoization.getResultThrowing(sessionUUID, (String x) ->
+            doGetAccessTokenFromSessionUUID(sessionUUID)
+         ).orElse(null);
+      }
+      else
+      {
+         return (doGetAccessTokenFromSessionUUID(sessionUUID));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private String doGetAccessTokenFromSessionUUID(String sessionUUID) throws QAuthenticationException
+   {
+      String   accessToken   = null;
+      QSession beforeSession = QContext.getQSession();
+
+      try
+      {
+         QContext.setQSession(new QSystemUserSession());
+
+         ///////////////////////////////////////
+         // query for the user session record //
+         ///////////////////////////////////////
+         QRecord userSessionRecord = new GetAction().executeForRecord(new GetInput(UserSession.TABLE_NAME)
+            .withUniqueKey(Map.of("uuid", sessionUUID))
+            .withShouldMaskPasswords(false)
+            .withShouldOmitHiddenFields(false));
+
+         if(userSessionRecord != null)
+         {
+            accessToken = userSessionRecord.getValueString("accessToken");
+
+            ////////////////////////////////////////////////////////////
+            // decode the accessToken and make sure it is not expired //
+            ////////////////////////////////////////////////////////////
+            if(accessToken != null)
+            {
+               DecodedJWT jwt = JWT.decode(accessToken);
+               if(jwt.getExpiresAtAsInstant().isBefore(Instant.now()))
+               {
+                  throw (new QAuthenticationException("accessToken is expired"));
+               }
+            }
+         }
+      }
+      catch(QAuthenticationException qae)
+      {
+         throw (qae);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error looking up userSession by sessionUUID", e);
+         throw (new QAuthenticationException("Error looking up userSession by sessionUUID", e));
+      }
+      finally
+      {
+         QContext.setQSession(beforeSession);
+      }
+
+      return (accessToken);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   @Override
+   public boolean usesSessionIdCookie()
+   {
+      return (true);
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private OIDCProviderMetadata getOIDCProviderMetadata(OAuth2AuthenticationMetaData oAuth2AuthenticationMetaData) throws GeneralException, IOException
+   {
+      return oidcProviderMetadataMemoization.getResult(oAuth2AuthenticationMetaData.getName(), (name ->
+      {
+         Issuer               issuer   = new Issuer(oAuth2AuthenticationMetaData.getBaseUrl());
+         OIDCProviderMetadata metadata = OIDCProviderMetadata.resolve(issuer);
+         return (metadata);
+      })).orElseThrow(() -> new GeneralException("Could not resolve OIDCProviderMetadata for " + oAuth2AuthenticationMetaData.getName()));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/TableBasedAuthenticationModule.java

@@ -406,6 +406,7 @@ public class TableBasedAuthenticationModule implements QAuthenticationModuleInte
          qUser.setIdReference(userRecord.getValueString(metaData.getUserTableUsernameField()));
 
          QSession qSession = new QSession();
+         qSession.setUuid(sessionUuid);
          qSession.setIdReference(sessionUuid);
          qSession.setUser(qUser);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/metadata/RedirectStateMetaDataProducer.java

@@ -0,0 +1,82 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2023.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.authentication.implementations.metadata;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
+import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
+
+
+/*******************************************************************************
+ ** Meta Data Producer for RedirectState table
+ *******************************************************************************/
+public class RedirectStateMetaDataProducer extends MetaDataProducer<QTableMetaData>
+{
+   public static final String TABLE_NAME = "redirectState";
+
+   private final String backendName;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public RedirectStateMetaDataProducer(String backendName)
+   {
+      this.backendName = backendName;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QTableMetaData produce(QInstance qInstance) throws QException
+   {
+      QTableMetaData tableMetaData = new QTableMetaData()
+         .withName(TABLE_NAME)
+         .withBackendName(backendName)
+         .withRecordLabelFormat("%s")
+         .withRecordLabelFields("state")
+         .withPrimaryKeyField("id")
+         .withUniqueKey(new UniqueKey("state"))
+         .withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.NONE))
+
+         .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
+         .withField(new QFieldMetaData("state", QFieldType.STRING).withIsEditable(false).withMaxLength(45).withBehavior(ValueTooLongBehavior.ERROR))
+         .withField(new QFieldMetaData("redirectUri", QFieldType.STRING).withIsEditable(false).withMaxLength(4096).withBehavior(ValueTooLongBehavior.ERROR))
+         .withField(new QFieldMetaData("createDate", QFieldType.DATE_TIME).withIsEditable(false));
+
+      return tableMetaData;
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/edit/BulkEditLoadStep.java

@@ -36,10 +36,12 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.InputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.BulkInsertTransformStep;
 import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.LoadViaUpdateStep;
 import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
 import com.kingsrook.qqq.backend.core.processes.implementations.general.ProcessSummaryWarningsAndErrorsRollup;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import org.apache.commons.lang.BooleanUtils;
 import static com.kingsrook.qqq.backend.core.processes.implementations.bulk.edit.BulkEditTransformStep.buildInfoSummaryLines;
 
 
@@ -53,6 +55,9 @@ public class BulkEditLoadStep extends LoadViaUpdateStep implements ProcessSummar
    private ProcessSummaryLine       okSummary     = new ProcessSummaryLine(Status.OK);
    private List<ProcessSummaryLine> infoSummaries = new ArrayList<>();
 
+   private Serializable firstInsertedPrimaryKey = null;
+   private Serializable lastInsertedPrimaryKey  = null;
+
    private ProcessSummaryWarningsAndErrorsRollup processSummaryWarningsAndErrorsRollup = ProcessSummaryWarningsAndErrorsRollup.build("edited");
 
    private String tableLabel;
@@ -106,7 +111,15 @@ public class BulkEditLoadStep extends LoadViaUpdateStep implements ProcessSummar
          tableLabel = table.getLabel();
       }
 
-      buildInfoSummaryLines(runBackendStepInput, table, infoSummaries, true);
+      boolean isBulkEdit = BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("isBulkEdit"));
+      if(isBulkEdit)
+      {
+         buildBulkUpdateWithFileInfoSummaryLines(runBackendStepOutput, table);
+      }
+      else
+      {
+         buildInfoSummaryLines(runBackendStepInput, table, infoSummaries, true);
+      }
    }
 
 
@@ -146,4 +159,83 @@ public class BulkEditLoadStep extends LoadViaUpdateStep implements ProcessSummar
       }
    }
 
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private void buildBulkUpdateWithFileInfoSummaryLines(RunBackendStepOutput runBackendStepOutput, QTableMetaData table)
+   {
+      /////////////////////////////////////////////////////////////////////////////////////////////
+      // the transform step builds summary lines that it predicts will update successfully.      //
+      // but those lines don't have ids, which we'd like to have (e.g., for a process trace that //
+      // might link to the built record).  also, it's possible that there was a fail that only   //
+      // happened in the actual update, so, basically, re-do the summary here                    //
+      /////////////////////////////////////////////////////////////////////////////////////////////
+      BulkInsertTransformStep transformStep = (BulkInsertTransformStep) getTransformStep();
+      ProcessSummaryLine      okSummary     = transformStep.okSummary;
+      okSummary.setCount(0);
+      okSummary.setPrimaryKeys(new ArrayList<>());
+
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////
+      // but - since errors from the transform step don't even make it through to us here in the load step,   //
+      // do re-use the ProcessSummaryWarningsAndErrorsRollup from transform step as follows:                  //
+      // clear out its warnings - we'll completely rebuild them here (with primary keys)                      //
+      // and add new error lines, e.g., in case of errors that only happened past the validation if possible. //
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////
+      ProcessSummaryWarningsAndErrorsRollup processSummaryWarningsAndErrorsRollup = transformStep.processSummaryWarningsAndErrorsRollup;
+      processSummaryWarningsAndErrorsRollup.resetWarnings();
+
+      List<QRecord> updatedRecords = runBackendStepOutput.getRecords();
+      for(QRecord updatedRecord : updatedRecords)
+      {
+         Serializable primaryKey = updatedRecord.getValue(table.getPrimaryKeyField());
+         if(CollectionUtils.nullSafeIsEmpty(updatedRecord.getErrors()) && primaryKey != null)
+         {
+            /////////////////////////////////////////////////////////////////////////
+            // if the record had no errors, and we have a primary key for it, then //
+            // keep track of the range of primary keys (first and last)            //
+            /////////////////////////////////////////////////////////////////////////
+            if(firstInsertedPrimaryKey == null)
+            {
+               firstInsertedPrimaryKey = primaryKey;
+            }
+
+            lastInsertedPrimaryKey = primaryKey;
+
+            if(!CollectionUtils.nullSafeIsEmpty(updatedRecord.getWarnings()))
+            {
+               ////////////////////////////////////////////////////////////////////////////
+               // if there were warnings on the updated record, put it in a warning line //
+               ////////////////////////////////////////////////////////////////////////////
+               String message = updatedRecord.getWarnings().get(0).getMessage();
+               processSummaryWarningsAndErrorsRollup.addWarning(message, primaryKey);
+            }
+            else
+            {
+               ///////////////////////////////////////////////////////////////////////
+               // if no warnings for the updated record, then put it in the OK line //
+               ///////////////////////////////////////////////////////////////////////
+               okSummary.incrementCountAndAddPrimaryKey(primaryKey);
+            }
+         }
+         else
+         {
+            //////////////////////////////////////////////////////////////////////
+            // else if there were errors or no primary key, build an error line //
+            //////////////////////////////////////////////////////////////////////
+            String message = "Failed to update";
+            if(!CollectionUtils.nullSafeIsEmpty(updatedRecord.getErrors()))
+            {
+               //////////////////////////////////////////////////////////
+               // use the error message from the record if we have one //
+               //////////////////////////////////////////////////////////
+               message = updatedRecord.getErrors().get(0).getMessage();
+            }
+            processSummaryWarningsAndErrorsRollup.addError(message, primaryKey);
+         }
+      }
+
+      okSummary.pickMessage(true);
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertExtractStep.java

@@ -55,8 +55,6 @@ public class BulkInsertExtractStep extends AbstractExtractStep
    @Override
    public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
    {
-      runBackendStepInput.traceMessage(BulkInsertStepUtils.getProcessTracerKeyRecordMessage(runBackendStepInput));
-
       int rowsAdded     = 0;
       int originalLimit = Objects.requireNonNullElse(getLimit(), Integer.MAX_VALUE);
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertLoadStep.java

@@ -41,6 +41,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.LoadViaInsertStep;
 import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
+import com.kingsrook.qqq.backend.core.processes.implementations.general.ProcessSummaryWarningsAndErrorsRollup;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 
 
@@ -77,19 +78,77 @@ public class BulkInsertLoadStep extends LoadViaInsertStep implements ProcessSumm
 
       QTableMetaData table = QContext.getQInstance().getTable(runBackendStepInput.getValueString("tableName"));
 
+      /////////////////////////////////////////////////////////////////////////////////////////////
+      // the transform step builds summary lines that it predicts will insert successfully.      //
+      // but those lines don't have ids, which we'd like to have (e.g., for a process trace that //
+      // might link to the built record).  also, it's possible that there was a fail that only   //
+      // happened in the actual insert, so, basically, re-do the summary here                    //
+      /////////////////////////////////////////////////////////////////////////////////////////////
+      BulkInsertTransformStep transformStep = (BulkInsertTransformStep) getTransformStep();
+      ProcessSummaryLine      okSummary     = transformStep.okSummary;
+      okSummary.setCount(0);
+      okSummary.setPrimaryKeys(new ArrayList<>());
+
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////
+      // but - since errors from the transform step don't even make it through to us here in the load step,   //
+      // do re-use the ProcessSummaryWarningsAndErrorsRollup from transform step as follows:                  //
+      // clear out its warnings - we'll completely rebuild them here (with primary keys)                      //
+      // and add new error lines, e.g., in case of errors that only happened past the validation if possible. //
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////
+      ProcessSummaryWarningsAndErrorsRollup processSummaryWarningsAndErrorsRollup = transformStep.processSummaryWarningsAndErrorsRollup;
+      processSummaryWarningsAndErrorsRollup.resetWarnings();
+
       List<QRecord> insertedRecords = runBackendStepOutput.getRecords();
       for(QRecord insertedRecord : insertedRecords)
       {
-         if(CollectionUtils.nullSafeIsEmpty(insertedRecord.getErrors()))
+         Serializable primaryKey = insertedRecord.getValue(table.getPrimaryKeyField());
+         if(CollectionUtils.nullSafeIsEmpty(insertedRecord.getErrors()) && primaryKey != null)
          {
+            /////////////////////////////////////////////////////////////////////////
+            // if the record had no errors, and we have a primary key for it, then //
+            // keep track of the range of primary keys (first and last)            //
+            /////////////////////////////////////////////////////////////////////////
             if(firstInsertedPrimaryKey == null)
             {
-               firstInsertedPrimaryKey = insertedRecord.getValue(table.getPrimaryKeyField());
+               firstInsertedPrimaryKey = primaryKey;
             }
 
-            lastInsertedPrimaryKey = insertedRecord.getValue(table.getPrimaryKeyField());
+            lastInsertedPrimaryKey = primaryKey;
+
+            if(!CollectionUtils.nullSafeIsEmpty(insertedRecord.getWarnings()))
+            {
+               /////////////////////////////////////////////////////////////////////////////
+               // if there were warnings on the inserted record, put it in a warning line //
+               /////////////////////////////////////////////////////////////////////////////
+               String message = insertedRecord.getWarnings().get(0).getMessage();
+               processSummaryWarningsAndErrorsRollup.addWarning(message, primaryKey);
+            }
+            else
+            {
+               ////////////////////////////////////////////////////////////////////////
+               // if no warnings for the inserted record, then put it in the OK line //
+               ////////////////////////////////////////////////////////////////////////
+               okSummary.incrementCountAndAddPrimaryKey(primaryKey);
+            }
+         }
+         else
+         {
+            //////////////////////////////////////////////////////////////////////
+            // else if there were errors or no primary key, build an error line //
+            //////////////////////////////////////////////////////////////////////
+            String message = "Failed to insert";
+            if(!CollectionUtils.nullSafeIsEmpty(insertedRecord.getErrors()))
+            {
+               //////////////////////////////////////////////////////////
+               // use the error message from the record if we have one //
+               //////////////////////////////////////////////////////////
+               message = insertedRecord.getErrors().get(0).getMessage();
+            }
+            processSummaryWarningsAndErrorsRollup.addError(message, primaryKey);
          }
       }
+
+      okSummary.pickMessage(true);
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertPrepareFileMappingStep.java

@@ -48,6 +48,7 @@ import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.JsonUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import org.apache.commons.lang.BooleanUtils;
 import org.json.JSONObject;
 
 
@@ -65,9 +66,11 @@ public class BulkInsertPrepareFileMappingStep implements BackendStep
    {
       buildFileDetailsForMappingStep(runBackendStepInput, runBackendStepOutput);
 
+      boolean                isBulkEdit     = BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("isBulkEdit"));
       String                 tableName      = runBackendStepInput.getValueString("tableName");
-      BulkLoadTableStructure tableStructure = BulkLoadTableStructureBuilder.buildTableStructure(tableName);
+      BulkLoadTableStructure tableStructure = BulkLoadTableStructureBuilder.buildTableStructure(tableName, isBulkEdit);
       runBackendStepOutput.addValue("tableStructure", tableStructure);
+      runBackendStepOutput.addValue("isBulkEdit", isBulkEdit);
 
       boolean needSuggestedMapping = true;
       if(runBackendStepOutput.getProcessState().getIsStepBack())
@@ -81,7 +84,7 @@ public class BulkInsertPrepareFileMappingStep implements BackendStep
       {
          @SuppressWarnings("unchecked")
          List<String> headerValues = (List<String>) runBackendStepOutput.getValue("headerValues");
-         buildSuggestedMapping(headerValues, getPrepopulatedValues(runBackendStepInput), tableStructure, runBackendStepOutput);
+         buildSuggestedMapping(isBulkEdit, headerValues, getPrepopulatedValues(runBackendStepInput), tableStructure, runBackendStepOutput);
       }
    }
 
@@ -95,8 +98,8 @@ public class BulkInsertPrepareFileMappingStep implements BackendStep
       String prepopulatedValuesJson = runBackendStepInput.getValueString("prepopulatedValues");
       if(StringUtils.hasContent(prepopulatedValuesJson))
       {
-         Map<String, Serializable> rs = new LinkedHashMap<>();
-         JSONObject jsonObject = JsonUtils.toJSONObject(prepopulatedValuesJson);
+         Map<String, Serializable> rs         = new LinkedHashMap<>();
+         JSONObject                jsonObject = JsonUtils.toJSONObject(prepopulatedValuesJson);
          for(String key : jsonObject.keySet())
          {
             rs.put(key, jsonObject.optString(key, null));
@@ -112,16 +115,16 @@ public class BulkInsertPrepareFileMappingStep implements BackendStep
    /***************************************************************************
     **
     ***************************************************************************/
-   private void buildSuggestedMapping(List<String> headerValues, Map<String, Serializable> prepopulatedValues, BulkLoadTableStructure tableStructure, RunBackendStepOutput runBackendStepOutput)
+   private void buildSuggestedMapping(boolean isBulkEdit, List<String> headerValues, Map<String, Serializable> prepopulatedValues, BulkLoadTableStructure tableStructure, RunBackendStepOutput runBackendStepOutput)
    {
       BulkLoadMappingSuggester bulkLoadMappingSuggester = new BulkLoadMappingSuggester();
-      BulkLoadProfile          bulkLoadProfile          = bulkLoadMappingSuggester.suggestBulkLoadMappingProfile(tableStructure, headerValues);
+      BulkLoadProfile          bulkLoadProfile          = bulkLoadMappingSuggester.suggestBulkLoadMappingProfile(tableStructure, headerValues, isBulkEdit);
 
       if(CollectionUtils.nullSafeHasContents(prepopulatedValues))
       {
          for(Map.Entry<String, Serializable> entry : prepopulatedValues.entrySet())
          {
-            String fieldName = entry.getKey();
+            String  fieldName           = entry.getKey();
             boolean foundFieldInProfile = false;
 
             for(BulkLoadProfileField bulkLoadProfileField : bulkLoadProfile.getFieldList())

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertPrepareFileUploadStep.java

@@ -52,6 +52,11 @@ public class BulkInsertPrepareFileUploadStep implements BackendStep
    @Override
    public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
    {
+      ////////////////////////////////////////////////////////////////////////////////////////
+      // for headless-bulk load (e.g., sftp import), set up the process tracer's key record //
+      ////////////////////////////////////////////////////////////////////////////////////////
+      runBackendStepInput.traceMessage(BulkInsertStepUtils.getProcessTracerKeyRecordMessage(runBackendStepInput));
+
       ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
       // if user has come back here, clear out file (else the storageInput object that it is comes to the frontend, which isn't what we want!) //
       ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@@ -60,10 +65,12 @@ public class BulkInsertPrepareFileUploadStep implements BackendStep
          runBackendStepOutput.addValue("theFile", null);
       }
 
+      boolean                isBulkEdit     = runBackendStepInput.getProcessName().endsWith("EditWithFile");
       String                 tableName      = runBackendStepInput.getValueString("tableName");
       QTableMetaData         table          = QContext.getQInstance().getTable(tableName);
       BulkLoadTableStructure tableStructure = BulkLoadTableStructureBuilder.buildTableStructure(tableName);
       runBackendStepOutput.addValue("tableStructure", tableStructure);
+      runBackendStepOutput.addValue("isBulkEdit", isBulkEdit);
 
       List<QFieldMetaData> requiredFields   = new ArrayList<>();
       List<QFieldMetaData> additionalFields = new ArrayList<>();
@@ -79,6 +86,14 @@ public class BulkInsertPrepareFileUploadStep implements BackendStep
          }
       }
 
+      /////////////////////////////////////////////
+      // bulk edit allows primary key as a field //
+      /////////////////////////////////////////////
+      if(isBulkEdit)
+      {
+         requiredFields.add(0, table.getField(table.getPrimaryKeyField()));
+      }
+
       StringBuilder html;
       String        childTableLabels = "";
 
@@ -91,11 +106,11 @@ public class BulkInsertPrepareFileUploadStep implements BackendStep
       //////////////////////////////////////////////////////////////////////////////////////////////////////////////
       boolean listFieldsInHelpText = false;
 
-      if(!CollectionUtils.nullSafeHasContents(tableStructure.getAssociations()))
+      if(isBulkEdit || !CollectionUtils.nullSafeHasContents(tableStructure.getAssociations()))
       {
          html = new StringBuilder("""
             <p>Upload either a CSV or Excel (.xlsx) file, with one row for each record you want to
-            insert in the ${tableLabel} table.</p><br />
+            ${action} in the ${tableLabel} table.</p><br />
             
             <p>Your file can contain any number of columns.  You will be prompted to map fields from
             the ${tableLabel} table to columns from your file or default values for all records that
@@ -199,6 +214,7 @@ public class BulkInsertPrepareFileUploadStep implements BackendStep
       finishCSV(flatCSV);
 
       String htmlString = html.toString()
+         .replace("${action}", (isBulkEdit ? "edit" : "insert"))
          .replace("${tableLabel}", table.getLabel())
          .replace("${childTableLabels}", childTableLabels)
          .replace("${flatCSV}", Base64.getEncoder().encodeToString(flatCSV.toString().getBytes(StandardCharsets.UTF_8)))

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertReceiveFileMappingStep.java

@@ -46,6 +46,7 @@ import com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.mode
 import com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.model.BulkLoadProfileField;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang3.BooleanUtils;
 
@@ -77,10 +78,14 @@ public class BulkInsertReceiveFileMappingStep implements BackendStep
             //////////////////////////////////////////////////////////////////////////////
             if(savedBulkLoadProfileRecord == null)
             {
-               throw (new QUserFacingException("Did not receive a saved bulk load profile record as input - unable to perform headless bulk load"));
+               throw (new QUserFacingException("Did not receive a Bulk Load Profile record as input.  Unable to perform headless bulk load"));
             }
 
             SavedBulkLoadProfile savedBulkLoadProfile = new SavedBulkLoadProfile(savedBulkLoadProfileRecord);
+            if(!StringUtils.hasContent(savedBulkLoadProfile.getMappingJson()))
+            {
+               throw (new QUserFacingException("Bulk Load Profile record's Mapping is empty.  Unable to perform headless bulk load"));
+            }
 
             try
             {
@@ -88,7 +93,7 @@ public class BulkInsertReceiveFileMappingStep implements BackendStep
             }
             catch(Exception e)
             {
-               throw (new QUserFacingException("Error processing saved bulk load profile record - unable to perform headless bulk load", e));
+               throw (new QUserFacingException("Error processing Bulk Load Profile record.  Unable to perform headless bulk load", e));
             }
          }
          else
@@ -240,6 +245,11 @@ public class BulkInsertReceiveFileMappingStep implements BackendStep
             }
          }
       }
+      catch(QUserFacingException ufe)
+      {
+         LOG.warn("User-facing error in bulk insert receive mapping", ufe);
+         throw ufe;
+      }
       catch(Exception e)
       {
          LOG.warn("Error in bulk insert receive mapping", e);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertStepUtils.java

@@ -113,6 +113,8 @@ public class BulkInsertStepUtils
       {
          String  layout       = runBackendStepInput.getValueString("layout");
          Boolean hasHeaderRow = runBackendStepInput.getValueBoolean("hasHeaderRow");
+         String  keyFields    = runBackendStepInput.getValueString("keyFields");
+         Boolean isBulkEdit   = runBackendStepInput.getValueBoolean("isBulkEdit");
 
          ArrayList<BulkLoadProfileField> fieldList = new ArrayList<>();
 
@@ -127,6 +129,7 @@ public class BulkInsertStepUtils
             bulkLoadProfileField.setColumnIndex(jsonObject.has("columnIndex") ? jsonObject.getInt("columnIndex") : null);
             bulkLoadProfileField.setDefaultValue((Serializable) jsonObject.opt("defaultValue"));
             bulkLoadProfileField.setDoValueMapping(jsonObject.optBoolean("doValueMapping"));
+            bulkLoadProfileField.setClearIfEmpty(jsonObject.optBoolean("clearIfEmpty"));
 
             if(BooleanUtils.isTrue(bulkLoadProfileField.getDoValueMapping()) && jsonObject.has("valueMappings"))
             {
@@ -140,6 +143,8 @@ public class BulkInsertStepUtils
          }
 
          BulkLoadProfile bulkLoadProfile = new BulkLoadProfile()
+            .withIsBulkEdit(isBulkEdit)
+            .withKeyFields(keyFields)
             .withVersion(version)
             .withFieldList(fieldList)
             .withHasHeaderRow(hasHeaderRow)
@@ -213,7 +218,7 @@ public class BulkInsertStepUtils
       {
          return (processTracerKeyRecordMessage);
       }
-      
+
       return (null);
    }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/BulkInsertTransformStep.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert;
 
 import java.io.Serializable;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.LinkedHashSet;
@@ -32,12 +33,13 @@ import java.util.Map;
 import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreInsertCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreInsertCustomizer.WhenToRun;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
 import com.kingsrook.qqq.backend.core.actions.tables.helpers.UniqueKeyHelper;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -48,6 +50,11 @@ import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutp
 import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
 import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
@@ -68,6 +75,9 @@ import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import org.apache.commons.lang.BooleanUtils;
+import org.json.JSONArray;
+import org.json.JSONObject;
 
 
 /*******************************************************************************
@@ -75,9 +85,9 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  *******************************************************************************/
 public class BulkInsertTransformStep extends AbstractTransformStep
 {
-   private ProcessSummaryLine okSummary = new ProcessSummaryLine(Status.OK);
+   public ProcessSummaryLine okSummary = new ProcessSummaryLine(Status.OK);
 
-   private ProcessSummaryWarningsAndErrorsRollup processSummaryWarningsAndErrorsRollup = ProcessSummaryWarningsAndErrorsRollup.build("inserted")
+   public ProcessSummaryWarningsAndErrorsRollup processSummaryWarningsAndErrorsRollup = ProcessSummaryWarningsAndErrorsRollup.build("inserted")
       .withDoReplaceSingletonCountLinesWithSuffixOnly(false);
 
    private ListingHash<String, RowValue> errorToExampleRowValueMap = new ListingHash<>();
@@ -190,6 +200,252 @@ public class BulkInsertTransformStep extends AbstractTransformStep
       QTableMetaData table   = QContext.getQInstance().getTable(runBackendStepInput.getTableName());
       List<QRecord>  records = runBackendStepInput.getRecords();
 
+      if(BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("isBulkEdit")))
+      {
+         handleBulkEdit(runBackendStepInput, runBackendStepOutput, records, table);
+         runBackendStepOutput.addValue("isBulkEdit", true);
+      }
+      else
+      {
+         handleBulkLoad(runBackendStepInput, runBackendStepOutput, records, table);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void handleBulkEdit(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput, List<QRecord> records, QTableMetaData table) throws QException
+   {
+      ///////////////////////////////////////////
+      // get the key fields for this bulk edit //
+      ///////////////////////////////////////////
+      String       keyFieldsString = runBackendStepInput.getValueString("keyFields");
+      List<String> keyFields       = Arrays.asList(keyFieldsString.split("\\|"));
+
+      //////////////////////////////////////////////////////////////////////////
+      // if the key field is the primary key, then just look up those records //
+      //////////////////////////////////////////////////////////////////////////
+      List<QRecord> nonMatchingRecords = new ArrayList<>();
+      List<QRecord> oldRecords         = new ArrayList<>();
+      List<QRecord> recordsToUpdate    = new ArrayList<>();
+      if(keyFields.size() == 1 && table.getPrimaryKeyField().equals(keyFields.get(0)))
+      {
+         recordsToUpdate = records;
+         String             primaryKeyName = table.getPrimaryKeyField();
+         List<Serializable> primaryKeys    = records.stream().map(record -> record.getValue(primaryKeyName)).toList();
+         oldRecords = new QueryAction().execute(new QueryInput(table.getName()).withFilter(new QQueryFilter(new QFilterCriteria(table.getPrimaryKeyField(), QCriteriaOperator.IN, primaryKeys)))).getRecords();
+
+         ///////////////////////////////////////////
+         // get a set of old records primary keys //
+         ///////////////////////////////////////////
+         Set<Serializable> matchedPrimaryKeys = oldRecords.stream()
+            .map(r -> r.getValue(table.getPrimaryKeyField()))
+            .collect(java.util.stream.Collectors.toSet());
+
+         ////////////////////////////////////////////////////////////////////////////////////////////////////
+         // iterate over file records and if primary keys dont match, add to the non matching records list //
+         ////////////////////////////////////////////////////////////////////////////////////////////////////
+         for(QRecord record : records)
+         {
+            Serializable recordKey = record.getValue(table.getPrimaryKeyField());
+            if(!matchedPrimaryKeys.contains(recordKey))
+            {
+               nonMatchingRecords.add(record);
+            }
+         }
+      }
+      else
+      {
+         Set<Serializable> uniqueIds        = new HashSet<>();
+         List<QRecord>     potentialRecords = new ArrayList<>();
+
+         ////////////////////////////////////////////////////////////////////////////////////////////////////
+         // if not using the primary key, then we will look up all records for each part of the unique key //
+         // and for each found, if all unique parts match we will add to our list of database records      //
+         ////////////////////////////////////////////////////////////////////////////////////////////////////
+         for(String uniqueKeyPart : keyFields)
+         {
+            List<Serializable> values = records.stream().map(record -> record.getValue(uniqueKeyPart)).toList();
+            for(QRecord databaseRecord : new QueryAction().execute(new QueryInput(table.getName()).withFilter(new QQueryFilter(new QFilterCriteria(uniqueKeyPart, QCriteriaOperator.IN, values)))).getRecords())
+            {
+               if(!uniqueIds.contains(databaseRecord.getValue(table.getPrimaryKeyField())))
+               {
+                  potentialRecords.add(databaseRecord);
+                  uniqueIds.add(databaseRecord.getValue(table.getPrimaryKeyField()));
+               }
+            }
+         }
+
+         ///////////////////////////////////////////////////////////////////////////////
+         // now iterate over all of the potential records checking each unique fields //
+         ///////////////////////////////////////////////////////////////////////////////
+         fileRecordLoop:
+         for(QRecord fileRecord : records)
+         {
+            for(QRecord databaseRecord : potentialRecords)
+            {
+               boolean allMatch = true;
+
+               for(String uniqueKeyPart : keyFields)
+               {
+                  if(!Objects.equals(fileRecord.getValue(uniqueKeyPart), databaseRecord.getValue(uniqueKeyPart)))
+                  {
+                     allMatch = false;
+                  }
+               }
+
+               //////////////////////////////////////////////////////////////////////////////////////
+               // if we get here with all matching, update the record from the file's primary key, //
+               // add it to the list to update, and continue looping over file records             //
+               //////////////////////////////////////////////////////////////////////////////////////
+               if(allMatch)
+               {
+                  oldRecords.add(databaseRecord);
+                  fileRecord.setValue(table.getPrimaryKeyField(), databaseRecord.getValue(table.getPrimaryKeyField()));
+
+                  //////////////////////////////////////////////////////////////////////////////////////////////////////////
+                  // iterate over the fields in the bulk load profile, if the value for that field is empty and the value //
+                  // of 'clear if empty' is set to true, then update the record to update with the old record's value     //
+                  //////////////////////////////////////////////////////////////////////////////////////////////////////////
+                  JSONArray array = new JSONArray(runBackendStepInput.getValueString("fieldListJSON"));
+                  for(int i = 0; i < array.length(); i++)
+                  {
+                     JSONObject jsonObject   = array.getJSONObject(i);
+                     String     fieldName    = jsonObject.optString("fieldName");
+                     boolean    clearIfEmpty = jsonObject.optBoolean("clearIfEmpty");
+
+                     if(fileRecord.getValue(fieldName) == null)
+                     {
+                        if(clearIfEmpty)
+                        {
+                           fileRecord.setValue(fieldName, null);
+                        }
+                        else
+                        {
+                           fileRecord.setValue(fieldName, databaseRecord.getValue(fieldName));
+                        }
+                     }
+                  }
+
+                  recordsToUpdate.add(fileRecord);
+                  continue fileRecordLoop;
+               }
+            }
+
+            ///////////////////////////////////////////////////////////////////////////////////////
+            // if we make it here, that means the record was not found, keep for logging warning //
+            ///////////////////////////////////////////////////////////////////////////////////////
+            nonMatchingRecords.add(fileRecord);
+         }
+      }
+
+      for(QRecord missingRecord : CollectionUtils.nonNullList(nonMatchingRecords))
+      {
+         String message = "Did not have a matching existing record.";
+         processSummaryWarningsAndErrorsRollup.addError(message, null);
+         addToErrorToExampleRowMap(message, missingRecord);
+      }
+
+      /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      // set up an insert-input, which will be used as input to the pre-customizer as well as for additional validations //
+      /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      UpdateInput updateInput = new UpdateInput(table.getName());
+      updateInput.setInputSource(QInputSource.USER);
+      updateInput.setRecords(recordsToUpdate);
+
+      //////////////////////////////////////////////////////////////////////
+      // load the pre-insert customizer and set it up, if there is one    //
+      // then we'll run it based on its WhenToRun value                   //
+      // we do this, in case it needs to, for example, adjust values that //
+      // are part of a unique key                                         //
+      //////////////////////////////////////////////////////////////////////
+      boolean                            didAlreadyRunCustomizer = false;
+      Optional<TableCustomizerInterface> preUpdateCustomizer     = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_UPDATE_RECORD.getRole());
+      if(preUpdateCustomizer.isPresent())
+      {
+         List<QRecord> recordsAfterCustomizer = preUpdateCustomizer.get().preUpdate(updateInput, records, true, Optional.of(oldRecords));
+         runBackendStepInput.setRecords(recordsAfterCustomizer);
+
+         /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         // so we used to have a comment here asking "do we care if the customizer runs both now, and in the validation below?" //
+         // when implementing Bulk Load V2, we were seeing that some customizers were adding errors to records, both now, and   //
+         // when they ran below.  so, at that time, we added this boolean, to track and avoid the double-run...                 //
+         // we could also imagine this being a setting on the pre-insert customizer, similar to its whenToRun attribute...      //
+         /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         didAlreadyRunCustomizer = true;
+      }
+
+      /////////////////////////////////////////////////////////////////////////////////
+      // run all validation from the insert action - in Preview mode (boolean param) //
+      /////////////////////////////////////////////////////////////////////////////////
+      updateInput.setRecords(recordsToUpdate);
+      UpdateAction updateAction = new UpdateAction();
+      updateAction.performValidations(updateInput, Optional.of(recordsToUpdate), didAlreadyRunCustomizer);
+      List<QRecord> validationResultRecords = updateInput.getRecords();
+
+      /////////////////////////////////////////////////////////////////
+      // look at validation results to build process summary results //
+      /////////////////////////////////////////////////////////////////
+      List<QRecord> outputRecords = new ArrayList<>();
+      for(QRecord record : validationResultRecords)
+      {
+         List<QErrorMessage> errorsFromAssociations = getErrorsFromAssociations(record);
+         if(CollectionUtils.nullSafeHasContents(errorsFromAssociations))
+         {
+            List<QErrorMessage> recordErrors = Objects.requireNonNullElseGet(record.getErrors(), () -> new ArrayList<>());
+            recordErrors.addAll(errorsFromAssociations);
+            record.setErrors(recordErrors);
+         }
+
+         if(CollectionUtils.nullSafeHasContents(record.getErrors()))
+         {
+            for(QErrorMessage error : record.getErrors())
+            {
+               if(error instanceof AbstractBulkLoadRollableValueError rollableValueError)
+               {
+                  processSummaryWarningsAndErrorsRollup.addError(rollableValueError.getMessageToUseAsProcessSummaryRollupKey(), null);
+                  addToErrorToExampleRowValueMap(rollableValueError, record);
+               }
+               else
+               {
+                  processSummaryWarningsAndErrorsRollup.addError(error.getMessage(), null);
+                  addToErrorToExampleRowMap(error.getMessage(), record);
+               }
+            }
+         }
+         else if(CollectionUtils.nullSafeHasContents(record.getWarnings()))
+         {
+            String message = record.getWarnings().get(0).getMessage();
+            processSummaryWarningsAndErrorsRollup.addWarning(message, null);
+            outputRecords.add(record);
+         }
+         else
+         {
+            okSummary.incrementCountAndAddPrimaryKey(null);
+            outputRecords.add(record);
+
+            for(Map.Entry<String, List<QRecord>> entry : CollectionUtils.nonNullMap(record.getAssociatedRecords()).entrySet())
+            {
+               String             associationName         = entry.getKey();
+               ProcessSummaryLine associationToInsertLine = associationsToInsertSummaries.computeIfAbsent(associationName, x -> new ProcessSummaryLine(Status.OK));
+               associationToInsertLine.incrementCount(CollectionUtils.nonNullList(entry.getValue()).size());
+            }
+         }
+      }
+
+      runBackendStepOutput.setRecords(outputRecords);
+      this.rowsProcessed += records.size();
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void handleBulkLoad(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput, List<QRecord> records, QTableMetaData table) throws QException
+   {
       /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
       // set up an insert-input, which will be used as input to the pre-customizer as well as for additional validations //
       /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@@ -209,7 +465,7 @@ public class BulkInsertTransformStep extends AbstractTransformStep
       Optional<TableCustomizerInterface> preInsertCustomizer     = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_INSERT_RECORD.getRole());
       if(preInsertCustomizer.isPresent())
       {
-         AbstractPreInsertCustomizer.WhenToRun whenToRun = preInsertCustomizer.get().whenToRunPreInsert(insertInput, true);
+         WhenToRun whenToRun = preInsertCustomizer.get().whenToRunPreInsert(insertInput, true);
          if(WhenToRun.BEFORE_ALL_VALIDATIONS.equals(whenToRun) || WhenToRun.BEFORE_UNIQUE_KEY_CHECKS.equals(whenToRun))
          {
             List<QRecord> recordsAfterCustomizer = preInsertCustomizer.get().preInsert(insertInput, records, true);
@@ -485,11 +741,13 @@ public class BulkInsertTransformStep extends AbstractTransformStep
       recordsProcessedLine.withPluralFutureMessage("records were");
       recordsProcessedLine.withPluralPastMessage("records were");
 
-      String noWarningsSuffix = processSummaryWarningsAndErrorsRollup.countWarnings() == 0 ? "" : " with no warnings";
-      okSummary.setSingularFutureMessage(tableLabel + " record will be inserted" + noWarningsSuffix + ".");
-      okSummary.setPluralFutureMessage(tableLabel + " records will be inserted" + noWarningsSuffix + ".");
-      okSummary.setSingularPastMessage(tableLabel + " record was inserted" + noWarningsSuffix + ".");
-      okSummary.setPluralPastMessage(tableLabel + " records were inserted" + noWarningsSuffix + ".");
+      boolean isBulkEdit       = BooleanUtils.isTrue(runBackendStepOutput.getValueBoolean("isBulkEdit"));
+      String  action           = isBulkEdit ? "updated" : "inserted";
+      String  noWarningsSuffix = processSummaryWarningsAndErrorsRollup.countWarnings() == 0 ? "" : " with no warnings";
+      okSummary.setSingularFutureMessage(tableLabel + " record will be " + action + noWarningsSuffix + ".");
+      okSummary.setPluralFutureMessage(tableLabel + " records will be " + action + noWarningsSuffix + ".");
+      okSummary.setSingularPastMessage(tableLabel + " record was " + action + noWarningsSuffix + ".");
+      okSummary.setPluralPastMessage(tableLabel + " records were " + action + noWarningsSuffix + ".");
       okSummary.pickMessage(isForResultScreen);
       okSummary.addSelfToListIfAnyCount(rs);
 
@@ -502,10 +760,10 @@ public class BulkInsertTransformStep extends AbstractTransformStep
             String         associationLabel = associationTable.getLabel();
 
             ProcessSummaryLine line = entry.getValue();
-            line.setSingularFutureMessage(associationLabel + " record will be inserted.");
-            line.setPluralFutureMessage(associationLabel + " records will be inserted.");
-            line.setSingularPastMessage(associationLabel + " record was inserted.");
-            line.setPluralPastMessage(associationLabel + " records were inserted.");
+            line.setSingularFutureMessage(associationLabel + " record will be " + action + ".");
+            line.setPluralFutureMessage(associationLabel + " records will be " + action + ".");
+            line.setSingularPastMessage(associationLabel + " record was " + action + ".");
+            line.setPluralPastMessage(associationLabel + " records were " + action + ".");
             line.pickMessage(isForResultScreen);
             line.addSelfToListIfAnyCount(rs);
          }
@@ -518,8 +776,8 @@ public class BulkInsertTransformStep extends AbstractTransformStep
 
          ukErrorSummary
             .withMessageSuffix(" inserted, because of duplicate values in a unique key on the fields (" + uniqueKey.getDescription(table) + "), with values"
-                               + (ukErrorSummary.areThereMoreSampleValues ? " such as: " : ": ")
-                               + StringUtils.joinWithCommasAndAnd(new ArrayList<>(ukErrorSummary.sampleValues)))
+               + (ukErrorSummary.areThereMoreSampleValues ? " such as: " : ": ")
+               + StringUtils.joinWithCommasAndAnd(new ArrayList<>(ukErrorSummary.sampleValues)))
 
             .withSingularFutureMessage(" record will not be")
             .withPluralFutureMessage(" records will not be")

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/mapping/BulkLoadMappingSuggester.java

@@ -54,7 +54,7 @@ public class BulkLoadMappingSuggester
    /***************************************************************************
     **
     ***************************************************************************/
-   public BulkLoadProfile suggestBulkLoadMappingProfile(BulkLoadTableStructure tableStructure, List<String> headerRow)
+   public BulkLoadProfile suggestBulkLoadMappingProfile(BulkLoadTableStructure tableStructure, List<String> headerRow, boolean isBulkEdit)
    {
       massagedHeadersWithoutNumbersToIndexMap = new LinkedHashMap<>();
       for(int i = 0; i < headerRow.size(); i++)
@@ -90,6 +90,7 @@ public class BulkLoadMappingSuggester
          .withVersion("v1")
          .withLayout(layout)
          .withHasHeaderRow(true)
+         .withIsBulkEdit(isBulkEdit)
          .withFieldList(fieldList);
 
       return (bulkLoadProfile);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/mapping/BulkLoadTableStructureBuilder.java

@@ -25,12 +25,19 @@ package com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.map
 import java.util.ArrayList;
 import java.util.Comparator;
 import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
+import com.kingsrook.qqq.backend.core.model.bulk.TableKeyFieldsPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Association;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.model.BulkLoadTableStructure;
@@ -44,12 +51,16 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  *******************************************************************************/
 public class BulkLoadTableStructureBuilder
 {
+   private static final QLogger LOG = QLogger.getLogger(BulkLoadTableStructureBuilder.class);
+
+
+
    /***************************************************************************
     **
     ***************************************************************************/
    public static BulkLoadTableStructure buildTableStructure(String tableName)
    {
-      return (buildTableStructure(tableName, null, null));
+      return (buildTableStructure(tableName, null, null, false));
    }
 
 
@@ -57,13 +68,24 @@ public class BulkLoadTableStructureBuilder
    /***************************************************************************
     **
     ***************************************************************************/
-   private static BulkLoadTableStructure buildTableStructure(String tableName, Association association, String parentAssociationPath)
+   public static BulkLoadTableStructure buildTableStructure(String tableName, Boolean isBulkEdit)
+   {
+      return (buildTableStructure(tableName, null, null, isBulkEdit));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static BulkLoadTableStructure buildTableStructure(String tableName, Association association, String parentAssociationPath, Boolean isBulkEdit)
    {
       QTableMetaData table = QContext.getQInstance().getTable(tableName);
 
       BulkLoadTableStructure tableStructure = new BulkLoadTableStructure();
       tableStructure.setTableName(tableName);
       tableStructure.setLabel(table.getLabel());
+      tableStructure.setIsBulkEdit(isBulkEdit);
 
       Set<String> associationJoinFieldNamesToExclude = new HashSet<>();
 
@@ -119,6 +141,30 @@ public class BulkLoadTableStructureBuilder
          }
       }
 
+      ////////////////////////////////////////////////////////
+      // for bulk edit, users can use the primary key field //
+      ////////////////////////////////////////////////////////
+      if(isBulkEdit)
+      {
+         fields.add(table.getField(table.getPrimaryKeyField()));
+
+         //////////////////////////////////////////////////////////////////////
+         // also make available what key fields are available for this table //
+         //////////////////////////////////////////////////////////////////////
+         try
+         {
+            SearchPossibleValueSourceInput input = new SearchPossibleValueSourceInput()
+               .withPossibleValueSourceName("tableKeyFields")
+               .withPathParamMap(Map.of("processName", tableName + ".bulkEditWithFile"));
+            List<QPossibleValue<String>> search = new TableKeyFieldsPossibleValueSource().search(input);
+            tableStructure.setPossibleKeyFields(new ArrayList<>(search.stream().map(QPossibleValue::getId).toList()));
+         }
+         catch(QException qe)
+         {
+            LOG.warn("Unable to retrieve possible key fields for table [" + tableName + "]", qe);
+         }
+      }
+
       fields.sort(Comparator.comparing(f -> ObjectUtils.requireNonNullElse(f.getLabel(), f.getName(), "")));
 
       for(Association childAssociation : CollectionUtils.nonNullList(table.getAssociations()))
@@ -131,8 +177,8 @@ public class BulkLoadTableStructureBuilder
          {
             String nextLevelPath =
                (StringUtils.hasContent(parentAssociationPath) ? parentAssociationPath + "." : "")
-               + (association != null ? association.getName() : "");
-            BulkLoadTableStructure associatedStructure = buildTableStructure(childAssociation.getAssociatedTableName(), childAssociation, nextLevelPath);
+                  + (association != null ? association.getName() : "");
+            BulkLoadTableStructure associatedStructure = buildTableStructure(childAssociation.getAssociatedTableName(), childAssociation, nextLevelPath, isBulkEdit);
             tableStructure.addAssociation(associatedStructure);
          }
       }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/model/BulkLoadProfile.java

@@ -37,6 +37,8 @@ public class BulkLoadProfile implements Serializable
    private Boolean hasHeaderRow;
    private String  layout;
    private String  version;
+   private Boolean isBulkEdit;
+   private String  keyFields;
 
 
 
@@ -132,6 +134,7 @@ public class BulkLoadProfile implements Serializable
    }
 
 
+
    /*******************************************************************************
     ** Getter for version
     *******************************************************************************/
@@ -162,4 +165,65 @@ public class BulkLoadProfile implements Serializable
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for isBulkEdit
+    *******************************************************************************/
+   public Boolean getIsBulkEdit()
+   {
+      return (this.isBulkEdit);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for isBulkEdit
+    *******************************************************************************/
+   public void setIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isBulkEdit
+    *******************************************************************************/
+   public BulkLoadProfile withIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for keyFields
+    *******************************************************************************/
+   public String getKeyFields()
+   {
+      return (this.keyFields);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for keyFields
+    *******************************************************************************/
+   public void setKeyFields(String keyFields)
+   {
+      this.keyFields = keyFields;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for keyFields
+    *******************************************************************************/
+   public BulkLoadProfile withKeyFields(String keyFields)
+   {
+      this.keyFields = keyFields;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/model/BulkLoadProfileField.java

@@ -35,6 +35,7 @@ public class BulkLoadProfileField
    private Integer                   columnIndex;
    private String                    headerName;
    private Serializable              defaultValue;
+   private Boolean                   clearIfEmpty;
    private Boolean                   doValueMapping;
    private Map<String, Serializable> valueMappings;
 
@@ -194,6 +195,7 @@ public class BulkLoadProfileField
    }
 
 
+
    /*******************************************************************************
     ** Getter for headerName
     *******************************************************************************/
@@ -224,4 +226,34 @@ public class BulkLoadProfileField
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for clearIfEmpty
+    *******************************************************************************/
+   public Boolean getClearIfEmpty()
+   {
+      return (this.clearIfEmpty);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for clearIfEmpty
+    *******************************************************************************/
+   public void setClearIfEmpty(Boolean clearIfEmpty)
+   {
+      this.clearIfEmpty = clearIfEmpty;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for clearIfEmpty
+    *******************************************************************************/
+   public BulkLoadProfileField withClearIfEmpty(Boolean clearIfEmpty)
+   {
+      this.clearIfEmpty = clearIfEmpty;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/bulk/insert/model/BulkLoadTableStructure.java

@@ -35,9 +35,12 @@ public class BulkLoadTableStructure implements Serializable
    private boolean isMain;
    private boolean isMany;
 
-   private String tableName;
-   private String label;
-   private String associationPath; // null/empty for main table, then associationName for a child, associationName.associationName for a grandchild
+   private String            tableName;
+   private String            label;
+   private String            associationPath; // null/empty for main table, then associationName for a child, associationName.associationName for a grandchild
+   private Boolean           isBulkEdit;
+   private String            keyFields;
+   private ArrayList<String> possibleKeyFields;
 
    private ArrayList<QFieldMetaData>         fields; // mmm, not marked as serializable (at this time) - is okay?
    private ArrayList<BulkLoadTableStructure> associations;
@@ -272,4 +275,98 @@ public class BulkLoadTableStructure implements Serializable
       }
       this.associations.add(association);
    }
+
+
+
+   /*******************************************************************************
+    ** Getter for isBulkEdit
+    *******************************************************************************/
+   public Boolean getIsBulkEdit()
+   {
+      return (this.isBulkEdit);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for isBulkEdit
+    *******************************************************************************/
+   public void setIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isBulkEdit
+    *******************************************************************************/
+   public BulkLoadTableStructure withIsBulkEdit(Boolean isBulkEdit)
+   {
+      this.isBulkEdit = isBulkEdit;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for keyFields
+    *******************************************************************************/
+   public String getKeyFields()
+   {
+      return (this.keyFields);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for keyFields
+    *******************************************************************************/
+   public void setKeyFields(String keyFields)
+   {
+      this.keyFields = keyFields;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for keyFields
+    *******************************************************************************/
+   public BulkLoadTableStructure withKeyFields(String keyFields)
+   {
+      this.keyFields = keyFields;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for possibleKeyFields
+    *******************************************************************************/
+   public ArrayList<String> getPossibleKeyFields()
+   {
+      return (this.possibleKeyFields);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for possibleKeyFields
+    *******************************************************************************/
+   public void setPossibleKeyFields(ArrayList<String> possibleKeyFields)
+   {
+      this.possibleKeyFields = possibleKeyFields;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for possibleKeyFields
+    *******************************************************************************/
+   public BulkLoadTableStructure withPossibleKeyFields(ArrayList<String> possibleKeyFields)
+   {
+      this.possibleKeyFields = possibleKeyFields;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/general/ProcessSummaryWarningsAndErrorsRollup.java

@@ -195,7 +195,7 @@ public class ProcessSummaryWarningsAndErrorsRollup
             {
                if(otherWarningsSummary == null)
                {
-                  otherWarningsSummary = new ProcessSummaryLine(Status.WARNING).withMessageSuffix("records had an other warning.");
+                  otherWarningsSummary = buildOtherWarningsSummary();
                }
                processSummaryLine = otherWarningsSummary;
             }
@@ -214,6 +214,27 @@ public class ProcessSummaryWarningsAndErrorsRollup
 
 
 
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private static ProcessSummaryLine buildOtherWarningsSummary()
+   {
+      return new ProcessSummaryLine(Status.WARNING).withMessageSuffix("records had an other warning.");
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public void resetWarnings()
+   {
+      warningSummaries.clear();
+      otherWarningsSummary = buildOtherWarningsSummary();
+   }
+
+
+
    /*******************************************************************************
     ** Wrapper around AlphaNumericComparator for ProcessSummaryLineInterface that
     ** extracts string messages out.

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedbulkloadprofiles/QuerySavedBulkLoadProfileProcess.java

@@ -45,6 +45,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.savedbulkloadprofiles.SavedBulkLoadProfile;
+import org.apache.commons.lang.BooleanUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -102,13 +103,29 @@ public class QuerySavedBulkLoadProfileProcess implements BackendStep
          }
          else
          {
-            String tableName = runBackendStepInput.getValueString("tableName");
+            String  tableName  = runBackendStepInput.getValueString("tableName");
+            boolean isBulkEdit = BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("isBulkEdit"));
 
             QueryInput input = new QueryInput();
             input.setTableName(SavedBulkLoadProfile.TABLE_NAME);
-            input.setFilter(new QQueryFilter()
+
+            QQueryFilter filter = new QQueryFilter()
                .withCriteria(new QFilterCriteria("tableName", QCriteriaOperator.EQUALS, tableName))
-               .withOrderBy(new QFilterOrderBy("label")));
+               .withOrderBy(new QFilterOrderBy("label"));
+
+            /////////////////////////////////////////////////////////////////////
+            // account for nulls here, so if is bulk edit, only look for true, //
+            // otherwise look for nulls or not equal to true                   //
+            /////////////////////////////////////////////////////////////////////
+            if(isBulkEdit)
+            {
+               filter.withCriteria(new QFilterCriteria("isBulkEdit", QCriteriaOperator.EQUALS, true));
+            }
+            else
+            {
+               filter.withCriteria(new QFilterCriteria("isBulkEdit", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true));
+            }
+            input.setFilter(filter);
 
             QueryOutput output = new QueryAction().execute(input);
             runBackendStepOutput.setRecords(output.getRecords());

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedbulkloadprofiles/StoreSavedBulkLoadProfileProcess.java

@@ -50,6 +50,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaD
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.savedbulkloadprofiles.SavedBulkLoadProfile;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import org.apache.commons.lang.BooleanUtils;
 
 
 /*******************************************************************************
@@ -87,9 +88,10 @@ public class StoreSavedBulkLoadProfileProcess implements BackendStep
 
       try
       {
-         String userId    = QContext.getQSession().getUser().getIdReference();
-         String tableName = runBackendStepInput.getValueString("tableName");
-         String label     = runBackendStepInput.getValueString("label");
+         String  userId     = QContext.getQSession().getUser().getIdReference();
+         String  tableName  = runBackendStepInput.getValueString("tableName");
+         String  label      = runBackendStepInput.getValueString("label");
+         Boolean isBulkEdit = BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("isBulkEdit"));
 
          String mappingJson = processMappingJson(runBackendStepInput.getValueString("mappingJson"));
 
@@ -98,6 +100,7 @@ public class StoreSavedBulkLoadProfileProcess implements BackendStep
             .withValue("mappingJson", mappingJson)
             .withValue("label", label)
             .withValue("tableName", tableName)
+            .withValue("isBulkEdit", isBulkEdit)
             .withValue("userId", userId);
 
          List<QRecord> savedBulkLoadProfileList;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/utils/RecordLookupHelper.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.processes.utils;
 
 
 import java.io.Serializable;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
@@ -51,13 +52,14 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  *******************************************************************************/
 public class RecordLookupHelper
 {
-   private Map<String, Map<Serializable, QRecord>> recordMaps = new HashMap<>();
+   private Map<String, Map<Serializable, QRecord>> recordMaps;
 
-   private Map<String, Map<Map<String, Serializable>, QRecord>> uniqueKeyMaps = new HashMap<>();
+   private Map<String, Map<Map<String, Serializable>, QRecord>> uniqueKeyMaps;
 
-   private Set<String> preloadedKeys = new HashSet<>();
+   private Set<String> preloadedKeys;
 
-   private Set<Pair<String, String>> disallowedOneOffLookups = new HashSet<>();
+   private Set<Pair<String, String>> disallowedOneOffLookups;
+   private boolean                   useSynchronizedCollections;
 
 
 
@@ -67,6 +69,33 @@ public class RecordLookupHelper
     *******************************************************************************/
    public RecordLookupHelper()
    {
+      this(false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public RecordLookupHelper(boolean useSynchronizedCollections)
+   {
+      this.useSynchronizedCollections = useSynchronizedCollections;
+
+      if(useSynchronizedCollections)
+      {
+         recordMaps = Collections.synchronizedMap(new HashMap<>());
+         uniqueKeyMaps = Collections.synchronizedMap(new HashMap<>());
+         preloadedKeys = Collections.synchronizedSet(new HashSet<>());
+         disallowedOneOffLookups = Collections.synchronizedSet(new HashSet<>());
+      }
+      else
+      {
+         recordMaps = new HashMap<>();
+         uniqueKeyMaps = new HashMap<>();
+         preloadedKeys = new HashSet<>();
+         disallowedOneOffLookups = new HashSet<>();
+      }
    }
 
 
@@ -77,7 +106,7 @@ public class RecordLookupHelper
    public QRecord getRecordByUniqueKey(String tableName, Map<String, Serializable> uniqueKey) throws QException
    {
       String                                  mapKey    = tableName + "." + uniqueKey.keySet().stream().sorted().collect(Collectors.joining(","));
-      Map<Map<String, Serializable>, QRecord> recordMap = uniqueKeyMaps.computeIfAbsent(mapKey, (k) -> new HashMap<>());
+      Map<Map<String, Serializable>, QRecord> recordMap = uniqueKeyMaps.computeIfAbsent(mapKey, (k) -> useSynchronizedCollections ? Collections.synchronizedMap(new HashMap<>()) : new HashMap<>());
 
       if(!recordMap.containsKey(uniqueKey))
       {
@@ -96,7 +125,7 @@ public class RecordLookupHelper
    public QRecord getRecordByKey(String tableName, String keyFieldName, Serializable key) throws QException
    {
       String                     mapKey    = tableName + "." + keyFieldName;
-      Map<Serializable, QRecord> recordMap = recordMaps.computeIfAbsent(mapKey, (k) -> new HashMap<>());
+      Map<Serializable, QRecord> recordMap = recordMaps.computeIfAbsent(mapKey, (k) -> useSynchronizedCollections ? Collections.synchronizedMap(new HashMap<>()) : new HashMap<>());
 
       ////////////////////////////////////////////////////////////
       // make sure we have they key object in the expected type //
@@ -150,7 +179,7 @@ public class RecordLookupHelper
    public void preloadRecords(String tableName, String keyFieldName, QQueryFilter filter) throws QException
    {
       String                     mapKey   = tableName + "." + keyFieldName;
-      Map<Serializable, QRecord> tableMap = recordMaps.computeIfAbsent(mapKey, s -> new HashMap<>());
+      Map<Serializable, QRecord> tableMap = recordMaps.computeIfAbsent(mapKey, s -> useSynchronizedCollections ? Collections.synchronizedMap(new HashMap<>()) : new HashMap<>());
       tableMap.putAll(GeneralProcessUtils.loadTableToMap(tableName, keyFieldName, filter));
    }
 
@@ -170,7 +199,7 @@ public class RecordLookupHelper
       }
 
       String                     mapKey   = tableName + "." + keyFieldName;
-      Map<Serializable, QRecord> tableMap = recordMaps.computeIfAbsent(mapKey, s -> new HashMap<>());
+      Map<Serializable, QRecord> tableMap = recordMaps.computeIfAbsent(mapKey, s -> useSynchronizedCollections ? Collections.synchronizedMap(new HashMap<>()) : new HashMap<>());
 
       QQueryFilter filter = new QQueryFilter(new QFilterCriteria(keyFieldName, QCriteriaOperator.IN, inList));
       tableMap.putAll(GeneralProcessUtils.loadTableToMap(tableName, keyFieldName, filter));

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/state/InMemoryStateProvider.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.state;
 import java.io.Serializable;
 import java.time.Instant;
 import java.time.temporal.ChronoUnit;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.Map;
 import java.util.Optional;
@@ -58,7 +59,7 @@ public class InMemoryStateProvider implements StateProviderInterface
     *******************************************************************************/
    private InMemoryStateProvider()
    {
-      this.map = new HashMap<>();
+      this.map = Collections.synchronizedMap(new HashMap<>());
 
       ///////////////////////////////////////////////////////////
       // Start a single thread executor to handle the cleaning //

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/utils/ClassPathUtils.java

@@ -61,6 +61,39 @@ public class ClassPathUtils
 
 
 
+   /*******************************************************************************
+    ** from https://stackoverflow.com/questions/520328/can-you-find-all-classes-in-a-package-using-reflection
+    **
+    *******************************************************************************/
+   public static List<Class<?>> getClassesContainingNameAndOfType(String nameContains, Class<?> type) throws IOException
+   {
+      List<Class<?>> classes = new ArrayList<>();
+      ClassLoader    loader  = Thread.currentThread().getContextClassLoader();
+
+      for(ClassPath.ClassInfo info : getTopLevelClasses(loader))
+      {
+         try
+         {
+            if(info.getName().contains(nameContains))
+            {
+               Class<?> testClass = info.load();
+               if(type.isAssignableFrom(testClass))
+               {
+                  classes.add(testClass);
+               }
+            }
+         }
+         catch(Throwable t)
+         {
+            // ignore - comes up for non-class entries, like module-info
+         }
+      }
+
+      return (classes);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/utils/ReflectiveBeanLikeClassUtils.java

@@ -0,0 +1,188 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.utils;
+
+
+import java.lang.reflect.AnnotatedParameterizedType;
+import java.lang.reflect.AnnotatedType;
+import java.lang.reflect.Method;
+import java.lang.reflect.Type;
+import java.math.BigDecimal;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.LocalTime;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.Locale;
+import java.util.Optional;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.data.QIgnore;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+
+
+/*******************************************************************************
+ ** Utilities for bean-like classes (e.g., QRecordEntity, QProcessPayload) that
+ ** use reflection to understand their bean-fields
+ *******************************************************************************/
+public class ReflectiveBeanLikeClassUtils
+{
+   private static final QLogger LOG = QLogger.getLogger(ReflectiveBeanLikeClassUtils.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static String getFieldNameFromGetter(Method getter)
+   {
+      String nameWithoutGet = getter.getName().replaceFirst("^get", "");
+      if(nameWithoutGet.length() == 1)
+      {
+         return (nameWithoutGet.toLowerCase(Locale.ROOT));
+      }
+      return (nameWithoutGet.substring(0, 1).toLowerCase(Locale.ROOT) + nameWithoutGet.substring(1));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static boolean isGetter(Method method, boolean allowAssociations)
+   {
+      return isGetter(method, allowAssociations, defaultAllowedTypes());
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static boolean isGetter(Method method, boolean allowAssociations, Collection<Class<?>> allowedTypes)
+   {
+      if(method.getParameterTypes().length == 0 && method.getName().matches("^get[A-Z].*"))
+      {
+         if(allowedTypes.contains(method.getReturnType()) || (allowAssociations && isSupportedAssociation(method.getReturnType(), method.getAnnotatedReturnType())))
+         {
+            return (true);
+         }
+         else
+         {
+            if(!method.getName().equals("getClass") && method.getAnnotation(QIgnore.class) == null)
+            {
+               LOG.debug("Method [" + method.getName() + "] in [" + method.getDeclaringClass().getSimpleName() + "] looks like a getter, but its return type, [" + method.getReturnType().getSimpleName() + "], isn't supported.");
+            }
+         }
+      }
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static Optional<Method> getSetterForGetter(Class<?> c, Method getter)
+   {
+      String setterName = getter.getName().replaceFirst("^get", "set");
+      for(Method method : c.getMethods())
+      {
+         if(method.getName().equals(setterName))
+         {
+            if(method.getParameterTypes().length == 1 && method.getParameterTypes()[0].equals(getter.getReturnType()))
+            {
+               return (Optional.of(method));
+            }
+            else
+            {
+               LOG.info("Method [" + method.getName() + "] looks like a setter for [" + getter.getName() + "], but its parameters, [" + Arrays.toString(method.getParameterTypes()) + "], don't match the getter's return type [" + getter.getReturnType() + "]");
+            }
+         }
+      }
+      return (Optional.empty());
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static Collection<Class<?>> defaultAllowedTypes()
+   {
+      /////////////////////////////////////////////
+      // note - this list has implications upon: //
+      // - QFieldType.fromClass                  //
+      // - QRecordEntityField.convertValueType   //
+      /////////////////////////////////////////////
+      return (Set.of(String.class,
+         Integer.class,
+         Long.class,
+         int.class,
+         Boolean.class,
+         boolean.class,
+         BigDecimal.class,
+         Instant.class,
+         LocalDate.class,
+         LocalTime.class,
+         byte[].class));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static boolean isSupportedAssociation(Class<?> returnType, AnnotatedType annotatedType)
+   {
+      Class<?> listTypeParam = getListTypeParam(returnType, annotatedType);
+      return (listTypeParam != null && QRecordEntity.class.isAssignableFrom(listTypeParam));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static Class<?> getListTypeParam(Class<?> listType, AnnotatedType annotatedType)
+   {
+      if(listType.equals(List.class))
+      {
+         if(annotatedType instanceof AnnotatedParameterizedType apt)
+         {
+            AnnotatedType[] annotatedActualTypeArguments = apt.getAnnotatedActualTypeArguments();
+            for(AnnotatedType annotatedActualTypeArgument : annotatedActualTypeArguments)
+            {
+               Type type = annotatedActualTypeArgument.getType();
+               if(type instanceof Class<?> c)
+               {
+                  return (c);
+               }
+            }
+         }
+      }
+
+      return (null);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/utils/StringUtils.java

@@ -23,9 +23,11 @@ package com.kingsrook.qqq.backend.core.utils;
 
 
 import java.util.Collection;
+import java.util.Collections;
 import java.util.List;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
+import com.ibm.icu.text.Transliterator;
 
 
 /*******************************************************************************
@@ -462,6 +464,17 @@ public class StringUtils
 
 
 
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static String replaceNonAsciiCharacters(String s)
+   {
+      Transliterator transliterator = Transliterator.getInstance("Any-Latin; Latin-ASCII");
+      return (transliterator.transliterate(s));
+   }
+
+
+
    /***************************************************************************
     **
     ***************************************************************************/
@@ -477,6 +490,24 @@ public class StringUtils
 
 
 
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static boolean safeEqualsIgnoreCase(String a, String b)
+   {
+      if(a == null && b == null)
+      {
+         return true;
+      }
+      if(a == null || b == null)
+      {
+         return false;
+      }
+      return (a.equalsIgnoreCase(b));
+   }
+
+
+
    /***************************************************************************
     **
     ***************************************************************************/
@@ -502,4 +533,59 @@ public class StringUtils
          return base + " (1)";
       }
    }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static String maskAndTruncate(String value)
+   {
+      return (maskAndTruncate(value, "** MASKED **", 6, 4));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static String maskAndTruncate(String value, String mask, int minLengthToMask, int charsToShowOnEnds)
+   {
+      if(!hasContent(value))
+      {
+         return ("");
+      }
+
+      if(value.length() < minLengthToMask || value.length() < 2 * charsToShowOnEnds)
+      {
+         return mask;
+      }
+
+      if(value.length() < charsToShowOnEnds * 3)
+      {
+         return (value.substring(0, charsToShowOnEnds) + mask);
+      }
+
+      return (value.substring(0, charsToShowOnEnds) + mask + value.substring(value.length() - charsToShowOnEnds));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static String nCopies(int n, String s)
+   {
+      return (nCopiesWithGlue(n, s, ""));
+   }
+
+
+
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   public static String nCopiesWithGlue(int n, String s, String glue)
+   {
+      return (StringUtils.join(glue, Collections.nCopies(n, s)));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/utils/Timer.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.utils;
 
 
+import java.time.Duration;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import org.apache.logging.log4j.Level;
 
@@ -79,9 +80,36 @@ public class Timer
     **
     *******************************************************************************/
    public void mark(String message)
+   {
+      mark(message, false);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void mark(String message, boolean prettyPrint)
    {
       long now = System.currentTimeMillis();
-      LOG.log(level, String.format("%s: Last [%5d] Total [%5d] %s", name, (now - last), (now - start), message));
+
+      if(!prettyPrint)
+      {
+         LOG.log(level, String.format("%s: Last [%5d] Total [%5d] %s", name, (now - last), (now - start), message));
+      }
+      else
+      {
+
+         Duration lastDuration  = Duration.ofMillis(now - last);
+         Duration totalDuration = Duration.ofMillis(now - start);
+
+         LOG.log(level, String.format(
+            "%s: Last [%d hours, %d minutes, %d seconds, %d milliseconds] Total [%d hours, %d minutes, %d seconds, %d milliseconds] %s",
+            name, lastDuration.toHours(), lastDuration.toMinutesPart(), lastDuration.toSecondsPart(), lastDuration.toMillisPart(),
+            totalDuration.toHours(), totalDuration.toMinutesPart(), totalDuration.toSecondsPart(), totalDuration.toMillisPart(),
+            message));
+      }
+
       last = now;
    }
 }