CE-938 move updatedFrontendStepList into state

CE-1240: added multi table widget type

docs/metaData/Joins.adoc

@@ -2,16 +2,57 @@
 == Joins
 include::../variables.adoc[]
 
-#TODO#
+A `QJoinMetaData` is a meta-data object that tells QQQ, essentially “it is possible for these 2 tables to join, here’s how to do it”.
+
+Joins can be used then, in an application, in a number of possible ways:
+
+* In a {link-table}, we can specify joins to be “exposed”, e.g., made available to users on a query screen
+* Also in a Table, as part of an “Association”, which sets up one table as a “parent” of another,
+such that you can store (and fetch) the child-records at the same time as the parent
+** A common use-case here may be an order & lineItem table -
+such that QQQ can generate an API uses to allow you to post an order and its lines in a single request, and they get stored all together
+* In defining the security field (record lock) on a table,
+sometimes, it isn’t a field directly on the table, but instead comes from a joined table (possibly even more than just 1 table away).
+** For example, maybe a lineItem table, doesn't have a clientId, but needs secured by that field
+- so its recordLock can specify a “joinNameChain” that describes how to get from lineItem to order.clientId
+* The `QueryAction` can take (through its QueryInput object) zero or more QueryJoin objects,
+which must make a reference (implicitly or explicitly) to a QJoinMetaData.
+See the section on <<QueryJoin,QueryJoins>> for more details.
 
 === QJoinMetaData
 Joins are defined in a QQQ Instance in a `*QJoinMetaData*` object.
 
-#TODO#
+In this object, we have the concept of a "leftTable" and a "rightTable".
+There isn't generally anything special about which table is on the "left" and which is on the "right".
+But the remaining pieces of the QJoinMetaData do all need to line-up with these sides.
+
+For example:
+
+* The Type (one-to-one, one-to-many, many-to-one) - where the leftTable comes first, and rightTable comes second
+(e.g., a one-to-many means 1-row in leftTable has many-rows in rightTable associated with it)
+* In a JoinOn object, the 1st field name given is from the leftTable;
+the second fieldName from the rightTable.
 
 *QJoinMetaData Properties:*
 
-* `name` - *String, Required* - Unique name for the join within the QQQ Instance. #todo infererences or conventions?#
-
-#TODO#
+* `name` - *String, Required* - Unique name for the join within the QQQ Instance.
+** One convention is to name joins based on (leftTable + "Join" + rightTable).
+** If you do not wish to define join names yourself, the method `withInferredName()`
+can be called (which defers to
+`public static String makeInferredJoinName(String leftTable, String rightTable)`),
+to create a name for the join following the (leftTable + "Join" + rightTable) convention.
+* `leftTable` - *String, Required* - Name of a {link-table} in the {link-instance}.
+* `rightTable` - *String, Required* - Name of a {link-table} in the {link-instance}.
+* `type` - *enum, Required* - cardinality between the two tables in the join.
+** e.g., `ONE_TO_ONE`, `ONE_TO_MANY` (indicating 1 record in the left table may join
+to many records in the right table), or `MANY_TO_ONE` (vice-versa).
+** Note that there is no MANY_TO_MANY option, as a many-to-many is built as multiple QJoinMetaData's
+going through the intermediary (intersection) table.
+* `joinOns` - *List<JoinOn>, Required* - fields used to join the tables.
+Note: In the 2-arg JoinOn constructor, the leftTable's field comes first.
+Alternatively, the no-arg constructor can be used along with `.withLeftField().withRightField()`
+* `orderBys` - *List<QFilterOrderBy>* - Optional list of order-by objects,
+used in some framework-generated queries using the join.
+The field names are assumed to come from the rightTable.
 
+#TODO# what else do we need here?

docs/metaData/Processes.adoc

@@ -16,6 +16,7 @@ Processes are defined in a QQQ Instance in a `*QProcessMetaData*` object.
 In addition to directly building a `QProcessMetaData` object setting its properties directly, there are a few common process patterns that provide *Builder* objects for ease-of-use.
 See StreamedETLWithFrontendProcess below for a common example
 
+[#_QProcessMetaData_Properties]
 *QProcessMetaData Properties:*
 
 * `name` - *String, Required* - Unique name for the process within the QQQ Instance.
@@ -30,12 +31,13 @@ See below for details.
 * `permissionRules` - *QPermissionRules object* - define the permission/access rules for the process.
 See {link-permissionRules} for details.
 * `steps` and `stepList` - *Map of String → <<QStepMetaData>>* and *List of QStepMetaData* - Defines the <<QFrontendStepMetaData,screens>> and <<QBackendStepMetaData,backend code>> that makes up the process.
-** `stepList` is the list of steps in the order that they will by default be executed.
-** `steps` is a map, including all steps from `stepList`, but which may also include steps which can used by the process if its backend steps make the decision to do so, at run-time.
+** `stepList` is the list of steps in the order that they will be executed
+(that is to say - this is the _default_ order of execution - but it can be customized - see <<_custom_process_flow>> for details).
+** `steps` is a map, including all steps from `stepList`, but which may also include steps which can used by the process if its backend steps make the decision to do so, at run-time (e.g., using <<_custom_process_flow>>).
 ** A process's steps are normally defined in one of two was:
 *** 1) by a single call to `.withStepList(List<QStepMetaData>)`, which internally adds each step into the `steps` map.
 *** 2) by multiple calls to `.addStep(QStepMetaData)`, which adds a step to both the `stepList` and `steps` map.
-** If a process also needs optional steps, they should be added by a call to `.addOptionalStep(QStepMetaData)`, which only places them in the `steps` map.
+** If a process also needs optional steps (for a <<_custom_process_flow>>), they should be added by a call to `.addOptionalStep(QStepMetaData)`, which only places them in the `steps` map.
 * `schedule` - *<<QScheduleMetaData>>* - set up the process to run automatically on the specified schedule.
 See below for details.
 * `minInputRecords` - *Integer* - #not used...#
@@ -214,3 +216,112 @@ But for some cases, doing page-level transactions can reduce long-transactions a
 * `withFields(List<QFieldMetaData> fieldList)` - Adds additional input fields to the preview step of the process.
 * `withBasepullConfiguration(BasepullConfiguration basepullConfiguration)` - Add a <<BasepullConfiguration>> to the process.
 * `withSchedule(QScheduleMetaData schedule)` - Add a <<QScheduleMetaData>> to the process.
+
+[#_custom_process_flow]
+==== Custom Process Flow
+As referenced in the definition of the <<_QProcessMetaData_Properties,QProcessMetaData Properties>>, by default, a process
+will execute each of its steps in-order, as defined in the `stepList` property.
+However, a Backend Step can customize this flow #todo - write more clearly here...
+
+There are generally 2 method to call (in a `BackendStep`) to do a dynamic flow:
+
+* `RunBackendStepOutput.setOverrideLastStepName(String stepName)`
+** QQQ's `RunProcessAction` keeps track of which step it "last" ran, e.g., to tell it which one to run next.
+However, if a step sets the `OverrideLastStepName` property in its output object,
+then the step named in that property becomes the effective "last" step,
+thus determining which step comes next.
+
+* `RunBackendStepOutput.updateStepList(List<String> stepNameList)`
+** Calling this method changes the process's runtime definition of steps to be executed.
+Thus allowing a completely custom flow.
+It should be noted, that the "last" step name (as tracked by QQQ within `RunProcessAction`)
+does need to be found in the new `stepNameList` - otherwise, the framework will not know where you were,
+for figuring out where to go next.
+
+[source,java]
+.Example of a defining process that can use a flexible flow:
+----
+// for a case like this, it would be recommended to define all step names in constants:
+public final static String STEP_START = "start";
+public final static String STEP_A     = "a";
+public final static String STEP_B     = "b";
+public final static String STEP_C     = "c";
+public final static String STEP_1     = "1";
+public final static String STEP_2     = "2";
+public final static String STEP_3     = "3";
+public final static String STEP_END   = "end";
+
+// also, to define the possible flows (lists of steps) in constants as well:
+public final static List<String> LETTERS_STEP_LIST = List.of(
+  STEP_START, STEP_A, STEP_B, STEP_C, STEP_END);
+
+public final static List<String> NUMBERS_STEP_LIST = List.of(
+  STEP_START, STEP_1, STEP_2, STEP_3, STEP_END);
+
+// when we define the process's meta-data, we only give a "skeleton" stepList -
+// we must at least have our starting step, and we may want at least one frontend step
+// for the UI to show some placeholder(s):
+QProcessMetaData process = new QProcessMetaData()
+   .withName(PROCESS_NAME)
+   .withStepList(List.of(
+      new QBackendStepMetaData().withName(STEP_START)
+         .withCode(new QCodeReference(/*...*/)),
+      new QFrontendStepMetaData()
+         .withName(STEP_END)
+   ));
+
+// the additional steps get added via `addOptionalStep`, which only puts them in
+// the process's stepMap, not its stepList!
+process.addOptionalStep(new QFrontendStepMetaData().withName(STEP_A));
+process.addOptionalStep(new QBackendStepMetaData().withName(STEP_B)
+   .withCode(new QCodeReference(/*...*/)));
+process.addOptionalStep(new QFrontendStepMetaData().withName(STEP_C));
+
+process.addOptionalStep(new QBackendStepMetaData().withName(STEP_1)
+   .withCode(new QCodeReference(/*...*/)));
+process.addOptionalStep(new QFrontendStepMetaData().withName(STEP_2));
+process.addOptionalStep(new QBackendStepMetaData().withName(STEP_3)
+   .withCode(new QCodeReference(/*...*/)));
+
+----
+
+[source,java]
+.Example of a process backend step adjusting the process's runtime flow:
+----
+/***************************************************************************
+** look at the value named "which". if it's "letters", then make the process
+** go through the stepList consisting of letters; else, update the step list
+** to be the "numbers" steps.
+**
+** Also - if the "skipSomeSteps" value is give as true, then set the
+** overrideLastStepName to skip again (in the letters case, skip past A, B
+** and C; in the numbers case, skip past 1 and 2).
+***************************************************************************/
+public static class StartStep implements BackendStep
+{
+  @Override
+  public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
+  {
+     Boolean skipSomeSteps = runBackendStepInput.getValueBoolean("skipSomeSteps");
+
+     if(runBackendStepInput.getValueString("which").equals("letters"))
+     {
+        runBackendStepOutput.updateStepList(LETTERS_STEP_LIST);
+        if(BooleanUtils.isTrue(skipSomeSteps))
+        {
+           runBackendStepOutput.setOverrideLastStepName(STEP_C);
+        }
+     }
+     else
+     {
+        runBackendStepOutput.updateStepList(NUMBERS_STEP_LIST);
+        if(BooleanUtils.isTrue(skipSomeSteps))
+        {
+           runBackendStepOutput.setOverrideLastStepName(STEP_2);
+        }
+     }
+  }
+}
+----
+
+

pom.xml

@@ -29,6 +29,7 @@
    <packaging>pom</packaging>
 
    <modules>
+      <module>qqq-bom</module>
       <module>qqq-backend-core</module>
       <module>qqq-backend-module-api</module>
       <module>qqq-backend-module-filesystem</module>
@@ -54,7 +55,7 @@
       <maven.compiler.showDeprecation>true</maven.compiler.showDeprecation>
       <maven.compiler.showWarnings>true</maven.compiler.showWarnings>
       <coverage.haltOnFailure>true</coverage.haltOnFailure>
-      <coverage.instructionCoveredRatioMinimum>0.80</coverage.instructionCoveredRatioMinimum>
+      <coverage.instructionCoveredRatioMinimum>0.75</coverage.instructionCoveredRatioMinimum>
       <coverage.classCoveredRatioMinimum>0.95</coverage.classCoveredRatioMinimum>
       <plugin.shade.phase>none</plugin.shade.phase>
    </properties>

qqq-backend-core/pom.xml

@@ -173,6 +173,19 @@
          <version>1.12.321</version>
       </dependency>
 
+      <dependency>
+         <groupId>com.amazonaws</groupId>
+         <artifactId>aws-java-sdk-ses</artifactId>
+         <version>1.12.705</version>
+      </dependency>
+
+      <dependency>
+         <groupId>cloud.localstack</groupId>
+         <artifactId>localstack-utils</artifactId>
+         <version>0.2.20</version>
+         <scope>test</scope>
+      </dependency>
+
       <dependency>
          <groupId>org.quartz-scheduler</groupId>
          <artifactId>quartz</artifactId>
@@ -186,6 +199,12 @@
          <version>2.23.0</version>
       </dependency>
 
+      <dependency>
+         <groupId>com.sun.mail</groupId>
+         <artifactId>jakarta.mail</artifactId>
+         <version>2.0.1</version>
+      </dependency>
+
       <!-- Common deps for all qqq modules -->
       <dependency>
          <groupId>org.apache.maven.plugins</groupId>

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/dashboard/widgets/ChildRecordListRenderer.java

@@ -36,6 +36,7 @@ import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.actions.values.QValueFormatter;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetOutput;
@@ -59,6 +60,7 @@ import com.kingsrook.qqq.backend.core.utils.JsonUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang.BooleanUtils;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -66,6 +68,9 @@ import org.apache.commons.lang.BooleanUtils;
  *******************************************************************************/
 public class ChildRecordListRenderer extends AbstractWidgetRenderer
 {
+   private static final QLogger LOG = QLogger.getLogger(ChildRecordListRenderer.class);
+
+
 
    /*******************************************************************************
     **
@@ -172,126 +177,134 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
    @Override
    public RenderWidgetOutput render(RenderWidgetInput input) throws QException
    {
-      String         widgetLabel = input.getQueryParams().get("widgetLabel");
-      String         joinName    = input.getQueryParams().get("joinName");
-      QJoinMetaData  join        = input.getInstance().getJoin(joinName);
-      String         id          = input.getQueryParams().get("id");
-      QTableMetaData leftTable   = input.getInstance().getTable(join.getLeftTable());
-      QTableMetaData rightTable  = input.getInstance().getTable(join.getRightTable());
-
-      Integer maxRows = null;
-      if(StringUtils.hasContent(input.getQueryParams().get("maxRows")))
+      try
       {
-         maxRows = ValueUtils.getValueAsInteger(input.getQueryParams().get("maxRows"));
-      }
-      else if(input.getWidgetMetaData().getDefaultValues().containsKey("maxRows"))
-      {
-         maxRows = ValueUtils.getValueAsInteger(input.getWidgetMetaData().getDefaultValues().containsKey("maxRows"));
-      }
+         String         widgetLabel = input.getQueryParams().get("widgetLabel");
+         String         joinName    = input.getQueryParams().get("joinName");
+         QJoinMetaData  join        = input.getInstance().getJoin(joinName);
+         String         id          = input.getQueryParams().get("id");
+         QTableMetaData leftTable   = input.getInstance().getTable(join.getLeftTable());
+         QTableMetaData rightTable  = input.getInstance().getTable(join.getRightTable());
 
-      //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-      // fetch the record that we're getting children for. e.g., the left-side of the join, with the input id                                     //
-      // but - only try this if we were given an id.  note, this widget could be called for on an INSERT screen, where we don't have a record yet //
-      // but we still want to be able to return all the other data in here that otherwise comes from the widget meta data, join, etc.             //
-      //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-      int          totalRows     = 0;
-      QRecord      primaryRecord = null;
-      QQueryFilter filter        = null;
-      QueryOutput  queryOutput   = new QueryOutput(new QueryInput());
-      if(StringUtils.hasContent(id))
-      {
-         GetInput getInput = new GetInput();
-         getInput.setTableName(join.getLeftTable());
-         getInput.setPrimaryKey(id);
-         GetOutput getOutput = new GetAction().execute(getInput);
-         primaryRecord = getOutput.getRecord();
-
-         if(primaryRecord == null)
+         Integer maxRows = null;
+         if(StringUtils.hasContent(input.getQueryParams().get("maxRows")))
          {
-            throw (new QNotFoundException("Could not find " + (leftTable == null ? "" : leftTable.getLabel()) + " with primary key " + id));
+            maxRows = ValueUtils.getValueAsInteger(input.getQueryParams().get("maxRows"));
+         }
+         else if(input.getWidgetMetaData().getDefaultValues().containsKey("maxRows"))
+         {
+            maxRows = ValueUtils.getValueAsInteger(input.getWidgetMetaData().getDefaultValues().containsKey("maxRows"));
          }
 
-         ////////////////////////////////////////////////////////////////////
-         // set up the query - for the table on the right side of the join //
-         ////////////////////////////////////////////////////////////////////
-         filter = new QQueryFilter();
-         for(JoinOn joinOn : join.getJoinOns())
+         //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         // fetch the record that we're getting children for. e.g., the left-side of the join, with the input id                                     //
+         // but - only try this if we were given an id.  note, this widget could be called for on an INSERT screen, where we don't have a record yet //
+         // but we still want to be able to return all the other data in here that otherwise comes from the widget meta data, join, etc.             //
+         //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         int          totalRows     = 0;
+         QRecord      primaryRecord = null;
+         QQueryFilter filter        = null;
+         QueryOutput  queryOutput   = new QueryOutput(new QueryInput());
+         if(StringUtils.hasContent(id))
          {
-            filter.addCriteria(new QFilterCriteria(joinOn.getRightField(), QCriteriaOperator.EQUALS, List.of(primaryRecord.getValue(joinOn.getLeftField()))));
-         }
-         filter.setOrderBys(join.getOrderBys());
-         filter.setLimit(maxRows);
+            GetInput getInput = new GetInput();
+            getInput.setTableName(join.getLeftTable());
+            getInput.setPrimaryKey(id);
+            GetOutput getOutput = new GetAction().execute(getInput);
+            primaryRecord = getOutput.getRecord();
 
-         QueryInput queryInput = new QueryInput();
-         queryInput.setTableName(join.getRightTable());
-         queryInput.setShouldTranslatePossibleValues(true);
-         queryInput.setShouldGenerateDisplayValues(true);
-         queryInput.setFilter(filter);
-         queryOutput = new QueryAction().execute(queryInput);
+            if(primaryRecord == null)
+            {
+               throw (new QNotFoundException("Could not find " + (leftTable == null ? "" : leftTable.getLabel()) + " with primary key " + id));
+            }
 
-         QValueFormatter.setBlobValuesToDownloadUrls(rightTable, queryOutput.getRecords());
-
-         totalRows = queryOutput.getRecords().size();
-         if(maxRows != null && (queryOutput.getRecords().size() == maxRows))
-         {
-            /////////////////////////////////////////////////////////////////////////////////////
-            // if the input said to only do some max, and the # of results we got is that max, //
-            // then do a count query, for displaying 1-n of <count>                            //
-            /////////////////////////////////////////////////////////////////////////////////////
-            CountInput countInput = new CountInput();
-            countInput.setTableName(join.getRightTable());
-            countInput.setFilter(filter);
-            totalRows = new CountAction().execute(countInput).getCount();
-         }
-      }
-
-      String tablePath   = input.getInstance().getTablePath(rightTable.getName());
-      String viewAllLink = tablePath == null ? null : (tablePath + "?filter=" + URLEncoder.encode(JsonUtils.toJson(filter), Charset.defaultCharset()));
-
-      ChildRecordListData widgetData = new ChildRecordListData(widgetLabel, queryOutput, rightTable, tablePath, viewAllLink, totalRows);
-
-      if(BooleanUtils.isTrue(ValueUtils.getValueAsBoolean(input.getQueryParams().get("canAddChildRecord"))))
-      {
-         widgetData.setCanAddChildRecord(true);
-
-         //////////////////////////////////////////////////////////
-         // new child records must have values from the join-ons //
-         //////////////////////////////////////////////////////////
-         Map<String, Serializable> defaultValuesForNewChildRecords = new HashMap<>();
-         if(primaryRecord != null)
-         {
+            ////////////////////////////////////////////////////////////////////
+            // set up the query - for the table on the right side of the join //
+            ////////////////////////////////////////////////////////////////////
+            filter = new QQueryFilter();
             for(JoinOn joinOn : join.getJoinOns())
             {
-               defaultValuesForNewChildRecords.put(joinOn.getRightField(), primaryRecord.getValue(joinOn.getLeftField()));
+               filter.addCriteria(new QFilterCriteria(joinOn.getRightField(), QCriteriaOperator.EQUALS, List.of(primaryRecord.getValue(joinOn.getLeftField()))));
+            }
+            filter.setOrderBys(join.getOrderBys());
+            filter.setLimit(maxRows);
+
+            QueryInput queryInput = new QueryInput();
+            queryInput.setTableName(join.getRightTable());
+            queryInput.setShouldTranslatePossibleValues(true);
+            queryInput.setShouldGenerateDisplayValues(true);
+            queryInput.setFilter(filter);
+            queryOutput = new QueryAction().execute(queryInput);
+
+            QValueFormatter.setBlobValuesToDownloadUrls(rightTable, queryOutput.getRecords());
+
+            totalRows = queryOutput.getRecords().size();
+            if(maxRows != null && (queryOutput.getRecords().size() == maxRows))
+            {
+               /////////////////////////////////////////////////////////////////////////////////////
+               // if the input said to only do some max, and the # of results we got is that max, //
+               // then do a count query, for displaying 1-n of <count>                            //
+               /////////////////////////////////////////////////////////////////////////////////////
+               CountInput countInput = new CountInput();
+               countInput.setTableName(join.getRightTable());
+               countInput.setFilter(filter);
+               totalRows = new CountAction().execute(countInput).getCount();
             }
          }
 
-         widgetData.setDefaultValuesForNewChildRecords(defaultValuesForNewChildRecords);
+         String tablePath   = input.getInstance().getTablePath(rightTable.getName());
+         String viewAllLink = tablePath == null ? null : (tablePath + "?filter=" + URLEncoder.encode(JsonUtils.toJson(filter), Charset.defaultCharset()));
 
-         Map<String, Serializable> widgetValues = input.getWidgetMetaData().getDefaultValues();
-         if(widgetValues.containsKey("disabledFieldsForNewChildRecords"))
+         ChildRecordListData widgetData = new ChildRecordListData(widgetLabel, queryOutput, rightTable, tablePath, viewAllLink, totalRows);
+
+         if(BooleanUtils.isTrue(ValueUtils.getValueAsBoolean(input.getQueryParams().get("canAddChildRecord"))))
          {
-            widgetData.setDisabledFieldsForNewChildRecords((Set<String>) widgetValues.get("disabledFieldsForNewChildRecords"));
-         }
-         else
-         {
-            /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-            // if there are no disabled fields specified - then normally any fields w/ a default value get implicitly disabled //
-            // but - if we didn't look-up the primary record, then we'll want to explicit disable fields from joins            //
-            /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-            if(primaryRecord == null)
+            widgetData.setCanAddChildRecord(true);
+
+            //////////////////////////////////////////////////////////
+            // new child records must have values from the join-ons //
+            //////////////////////////////////////////////////////////
+            Map<String, Serializable> defaultValuesForNewChildRecords = new HashMap<>();
+            if(primaryRecord != null)
             {
-               Set<String> implicitlyDisabledFields = new HashSet<>();
-               widgetData.setDisabledFieldsForNewChildRecords(implicitlyDisabledFields);
                for(JoinOn joinOn : join.getJoinOns())
                {
-                  implicitlyDisabledFields.add(joinOn.getRightField());
+                  defaultValuesForNewChildRecords.put(joinOn.getRightField(), primaryRecord.getValue(joinOn.getLeftField()));
+               }
+            }
+
+            widgetData.setDefaultValuesForNewChildRecords(defaultValuesForNewChildRecords);
+
+            Map<String, Serializable> widgetValues = input.getWidgetMetaData().getDefaultValues();
+            if(widgetValues.containsKey("disabledFieldsForNewChildRecords"))
+            {
+               widgetData.setDisabledFieldsForNewChildRecords((Set<String>) widgetValues.get("disabledFieldsForNewChildRecords"));
+            }
+            else
+            {
+               /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+               // if there are no disabled fields specified - then normally any fields w/ a default value get implicitly disabled //
+               // but - if we didn't look-up the primary record, then we'll want to explicit disable fields from joins            //
+               /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+               if(primaryRecord == null)
+               {
+                  Set<String> implicitlyDisabledFields = new HashSet<>();
+                  widgetData.setDisabledFieldsForNewChildRecords(implicitlyDisabledFields);
+                  for(JoinOn joinOn : join.getJoinOns())
+                  {
+                     implicitlyDisabledFields.add(joinOn.getRightField());
+                  }
                }
             }
          }
-      }
 
-      return (new RenderWidgetOutput(widgetData));
+         return (new RenderWidgetOutput(widgetData));
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error rendering child record list", e, logPair("widgetName", () -> input.getWidgetMetaData().getName()));
+         throw (e);
+      }
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/interfaces/QStorageInterface.java

@@ -30,7 +30,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.storage.StorageInput;
 
 /*******************************************************************************
  ** Interface for actions that a backend can perform, based on streaming data
- ** into the backend's storage. 
+ ** into the backend's storage.
  *******************************************************************************/
 public interface QStorageInterface
 {
@@ -46,4 +46,24 @@ public interface QStorageInterface
     *******************************************************************************/
    InputStream getInputStream(StorageInput storageInput) throws QException;
 
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   default void makePublic(StorageInput storageInput) throws QException
+   {
+      //////////
+      // noop //
+      //////////
+   }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   default String getDownloadURL(StorageInput storageInput) throws QException
+   {
+      return (null);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/messaging/SendMessageAction.java

@@ -0,0 +1,64 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.messaging;
+
+
+import com.kingsrook.qqq.backend.core.actions.AbstractQActionFunction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
+import com.kingsrook.qqq.backend.core.modules.messaging.MessagingProviderInterface;
+import com.kingsrook.qqq.backend.core.modules.messaging.QMessagingProviderDispatcher;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SendMessageAction extends AbstractQActionFunction<SendMessageInput, SendMessageOutput>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public SendMessageOutput execute(SendMessageInput input) throws QException
+   {
+      if(!StringUtils.hasContent(input.getMessagingProviderName()))
+      {
+         throw (new QException("Messaging provider name was not given in SendMessageInput."));
+      }
+
+      QMessagingProviderMetaData messagingProvider = QContext.getQInstance().getMessagingProvider(input.getMessagingProviderName());
+      if(messagingProvider == null)
+      {
+         throw (new QException("Messaging provider named [" + input.getMessagingProviderName() + "] was not found in this QInstance."));
+      }
+
+      MessagingProviderInterface messagingProviderInterface = new QMessagingProviderDispatcher().getMessagingProviderInterface(messagingProvider.getType());
+
+      return (messagingProviderInterface.sendMessage(input));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/metadata/JoinGraph.java

@@ -29,6 +29,7 @@ import java.util.List;
 import java.util.Objects;
 import java.util.Set;
 import java.util.TreeSet;
+import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
@@ -41,9 +42,19 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
  *******************************************************************************/
 public class JoinGraph
 {
-
    private Set<Edge> edges = new HashSet<>();
 
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   // as an instance grows, with the number of joins (say, more than 50?), especially as they may have a lot of connections, //
+   // it can become very very slow to process a full join graph (e.g., 10 seconds, maybe much worse, per Big-O...)           //
+   // also, it's not frequently useful to look at a join path that's more than a handful of tables long.                     //
+   // thus - this property exists - to limit the max length of a join path.  Keeping it small keeps instance enrichment      //
+   // and validation reasonably performant, at the possible cost of, some join-path that's longer than this limit may not    //
+   // be found - but - chances are, you don't want some 12-element join path to be used anyway, thus, this makes sense.      //
+   // but - it can be adjusted, per system property or ENV var.                                                              //
+   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   private int maxPathLength = new QMetaDataVariableInterpreter().getIntegerFromPropertyOrEnvironment("qqq.instance.joinGraph.maxPathLength", "QQQ_INSTANCE_JOIN_GRAPH_MAX_PATH_LENGTH", 3);
+
 
 
    /*******************************************************************************
@@ -303,6 +314,13 @@ public class JoinGraph
 
                if(otherTableName != null)
                {
+                  if(newPath.size() > maxPathLength)
+                  {
+                     ////////////////////////////////////////////////////////////////
+                     // performance hack.  see comment at maxPathLength definition //
+                     ////////////////////////////////////////////////////////////////
+                     continue;
+                  }
 
                   JoinConnectionList newConnectionList = connectionList.copy();
                   JoinConnection     joinConnection    = new JoinConnection(otherTableName, edge.joinName);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/processes/CancelProcessAction.java

@@ -0,0 +1,110 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2022.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.processes;
+
+
+import java.util.Optional;
+import java.util.UUID;
+import com.kingsrook.qqq.backend.core.actions.ActionHelper;
+import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessState;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.state.StateType;
+import com.kingsrook.qqq.backend.core.state.UUIDAndTypeStateKey;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Action handler for running the cancel step of a qqq process
+ *
+ *******************************************************************************/
+public class CancelProcessAction extends RunProcessAction
+{
+   private static final QLogger LOG = QLogger.getLogger(CancelProcessAction.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public RunProcessOutput execute(RunProcessInput runProcessInput) throws QException
+   {
+      ActionHelper.validateSession(runProcessInput);
+
+      QProcessMetaData process = runProcessInput.getInstance().getProcess(runProcessInput.getProcessName());
+      if(process == null)
+      {
+         throw new QBadRequestException("Process [" + runProcessInput.getProcessName() + "] is not defined in this instance.");
+      }
+
+      if(runProcessInput.getProcessUUID() == null)
+      {
+         throw (new QBadRequestException("Cannot cancel process - processUUID was not given."));
+      }
+
+      UUIDAndTypeStateKey    stateKey     = new UUIDAndTypeStateKey(UUID.fromString(runProcessInput.getProcessUUID()), StateType.PROCESS_STATUS);
+      Optional<ProcessState> processState = getState(runProcessInput.getProcessUUID());
+      if(processState.isEmpty())
+      {
+         throw (new QBadRequestException("Cannot cancel process - State for process UUID [" + runProcessInput.getProcessUUID() + "] was not found."));
+      }
+
+      RunProcessOutput runProcessOutput = new RunProcessOutput();
+      try
+      {
+         if(process.getCancelStep() != null)
+         {
+            LOG.info("Running cancel step for process", logPair("processName", process.getName()));
+            runBackendStep(runProcessInput, process, runProcessOutput, stateKey, process.getCancelStep(), process, processState.get());
+         }
+         else
+         {
+            LOG.debug("Process does not have a custom cancel step to run.", logPair("processName", process.getName()));
+         }
+      }
+      catch(QException qe)
+      {
+         ////////////////////////////////////////////////////////////
+         // upon exception (e.g., one thrown by a step), throw it. //
+         ////////////////////////////////////////////////////////////
+         throw (qe);
+      }
+      catch(Exception e)
+      {
+         throw (new QException("Error cancelling process", e));
+      }
+      finally
+      {
+         //////////////////////////////////////////////////////
+         // always put the final state in the process result //
+         //////////////////////////////////////////////////////
+         runProcessOutput.setProcessState(processState.get());
+      }
+
+      return (runProcessOutput);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/processes/QProcessCallbackFactory.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.actions.processes;
 
 
 import java.io.Serializable;
+import java.util.Collection;
 import java.util.Collections;
 import java.util.List;
 import java.util.Map;
@@ -114,4 +115,14 @@ public class QProcessCallbackFactory
       return (forFilter(new QQueryFilter().withCriteria(new QFilterCriteria(fieldName, QCriteriaOperator.EQUALS, value))));
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static QProcessCallback forPrimaryKeys(String fieldName, Collection<? extends Serializable> values)
+   {
+      return (forFilter(new QQueryFilter().withCriteria(new QFilterCriteria(fieldName, QCriteriaOperator.IN, values))));
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/processes/RunBackendStepAction.java

@@ -26,6 +26,7 @@ import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
+import java.util.Objects;
 import java.util.stream.Collectors;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
@@ -71,7 +72,17 @@ public class RunBackendStepAction
       QStepMetaData stepMetaData = process.getStep(runBackendStepInput.getStepName());
       if(stepMetaData == null)
       {
-         throw new QException("Step [" + runBackendStepInput.getStepName() + "] is not defined in the process [" + process.getName() + "]");
+         if(process.getCancelStep() != null && Objects.equals(process.getCancelStep().getName(), runBackendStepInput.getStepName()))
+         {
+            /////////////////////////////////////
+            // special case for cancel step... //
+            /////////////////////////////////////
+            stepMetaData = process.getCancelStep();
+         }
+         else
+         {
+            throw new QException("Step [" + runBackendStepInput.getStepName() + "] is not defined in the process [" + process.getName() + "]");
+         }
       }
 
       if(!(stepMetaData instanceof QBackendStepMetaData backendStepMetaData))

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/processes/RunProcessAction.java

@@ -82,7 +82,7 @@ public class RunProcessAction
    public static final String BASEPULL_THIS_RUNTIME_KEY = "basepullThisRuntimeKey";
    public static final String BASEPULL_LAST_RUNTIME_KEY = "basepullLastRuntimeKey";
    public static final String BASEPULL_TIMESTAMP_FIELD  = "basepullTimestampField";
-   public static final String BASEPULL_CONFIGURATION = "basepullConfiguration";
+   public static final String BASEPULL_CONFIGURATION    = "basepullConfiguration";
 
    ////////////////////////////////////////////////////////////////////////////////////////////////
    // indicator that the timestamp field should be updated - e.g., the execute step is finished. //
@@ -190,7 +190,25 @@ public class RunProcessAction
                // Run backend steps //
                ///////////////////////
                LOG.debug("Running backend step [" + step.getName() + "] in process [" + process.getName() + "]");
-               runBackendStep(runProcessInput, process, runProcessOutput, stateKey, backendStepMetaData, process, processState);
+               RunBackendStepOutput runBackendStepOutput = runBackendStep(runProcessInput, process, runProcessOutput, stateKey, backendStepMetaData, process, processState);
+
+               /////////////////////////////////////////////////////////////////////////////////////////
+               // if the step returned an override lastStepName, use that to determine how we proceed //
+               /////////////////////////////////////////////////////////////////////////////////////////
+               if(runBackendStepOutput.getOverrideLastStepName() != null)
+               {
+                  LOG.debug("Process step [" + lastStepName + "] returned an overrideLastStepName [" + runBackendStepOutput.getOverrideLastStepName() + "]!");
+                  lastStepName = runBackendStepOutput.getOverrideLastStepName();
+               }
+
+               /////////////////////////////////////////////////////////////////////////////////////////////
+               // similarly, if the step produced an updatedFrontendStepList, propagate that data outward //
+               /////////////////////////////////////////////////////////////////////////////////////////////
+               if(runBackendStepOutput.getUpdatedFrontendStepList() != null)
+               {
+                  LOG.debug("Process step [" + lastStepName + "] generated an updatedFrontendStepList [" + runBackendStepOutput.getUpdatedFrontendStepList().stream().map(s -> s.getName()).toList() + "]!");
+                  runProcessOutput.setUpdatedFrontendStepList(runBackendStepOutput.getUpdatedFrontendStepList());
+               }
             }
             else
             {
@@ -317,6 +335,13 @@ public class RunProcessAction
          ///////////////////////////////////////////////////
          runProcessInput.seedFromProcessState(optionalProcessState.get());
 
+         ///////////////////////////////////////////////////////////////////////////////////////////////////
+         // if we're restoring an old state, we can discard a previously stored updatedFrontendStepList - //
+         // it is only needed on the transitional edge from a backend-step to a frontend step, but not    //
+         // in the other directly                                                                         //
+         ///////////////////////////////////////////////////////////////////////////////////////////////////
+         optionalProcessState.get().setUpdatedFrontendStepList(null);
+
          ///////////////////////////////////////////////////////////////////////////
          // if there were values from the caller, put those (back) in the request //
          ///////////////////////////////////////////////////////////////////////////
@@ -339,7 +364,7 @@ public class RunProcessAction
    /*******************************************************************************
     ** Run a single backend step.
     *******************************************************************************/
-   private void runBackendStep(RunProcessInput runProcessInput, QProcessMetaData process, RunProcessOutput runProcessOutput, UUIDAndTypeStateKey stateKey, QBackendStepMetaData backendStep, QProcessMetaData qProcessMetaData, ProcessState processState) throws Exception
+   protected RunBackendStepOutput runBackendStep(RunProcessInput runProcessInput, QProcessMetaData process, RunProcessOutput runProcessOutput, UUIDAndTypeStateKey stateKey, QBackendStepMetaData backendStep, QProcessMetaData qProcessMetaData, ProcessState processState) throws Exception
    {
       RunBackendStepInput runBackendStepInput = new RunBackendStepInput(processState);
       runBackendStepInput.setProcessName(process.getName());
@@ -368,14 +393,16 @@ public class RunProcessAction
          runBackendStepInput.setBasepullLastRunTime((Instant) runProcessInput.getValues().get(BASEPULL_LAST_RUNTIME_KEY));
       }
 
-      RunBackendStepOutput lastFunctionResult = new RunBackendStepAction().execute(runBackendStepInput);
-      storeState(stateKey, lastFunctionResult.getProcessState());
+      RunBackendStepOutput runBackendStepOutput = new RunBackendStepAction().execute(runBackendStepInput);
+      storeState(stateKey, runBackendStepOutput.getProcessState());
 
-      if(lastFunctionResult.getException() != null)
+      if(runBackendStepOutput.getException() != null)
       {
-         runProcessOutput.setException(lastFunctionResult.getException());
-         throw (lastFunctionResult.getException());
+         runProcessOutput.setException(runBackendStepOutput.getException());
+         throw (runBackendStepOutput.getException());
       }
+
+      return (runBackendStepOutput);
    }
 
 
@@ -495,15 +522,15 @@ public class RunProcessAction
       String basepullKeyValue = (basepullConfiguration.getKeyValue() != null) ? basepullConfiguration.getKeyValue() : process.getName();
 
       //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-      // if backend specifies that it uses variants, look for that data in the session and append to our basepull key //
+      // if process specifies that it uses variants, look for that data in the session and append to our basepull key //
       //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-      if(process.getSchedule() != null && process.getVariantBackend() != null)
+      if(process.getVariantBackend() != null)
       {
          QSession         session         = QContext.getQSession();
          QBackendMetaData backendMetaData = QContext.getQInstance().getBackend(process.getVariantBackend());
          if(session.getBackendVariants() == null || !session.getBackendVariants().containsKey(backendMetaData.getVariantOptionsTableTypeValue()))
          {
-            LOG.info("Could not find Backend Variant information for Backend '" + backendMetaData.getName() + "'");
+            LOG.warn("Could not find Backend Variant information for Backend '" + backendMetaData.getName() + "'");
          }
          else
          {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/reporting/GenerateReportAction.java

@@ -557,7 +557,7 @@ public class GenerateReportAction extends AbstractQActionFunction<ReportInput, R
    /*******************************************************************************
     **
     *******************************************************************************/
-   private void setInputValuesInQueryFilter(ReportInput reportInput, QQueryFilter queryFilter)
+   private void setInputValuesInQueryFilter(ReportInput reportInput, QQueryFilter queryFilter) throws QException
    {
       if(queryFilter == null || queryFilter.getCriteria() == null)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/QueryAction.java

@@ -169,6 +169,7 @@ public class QueryAction
             nextLevelQueryInput.setTableName(association.getAssociatedTableName());
             nextLevelQueryInput.setIncludeAssociations(true);
             nextLevelQueryInput.setAssociationNamesToInclude(buildNextLevelAssociationNamesToInclude(association.getName(), queryInput.getAssociationNamesToInclude()));
+            nextLevelQueryInput.setTransaction(queryInput.getTransaction());
 
             QQueryFilter filter = new QQueryFilter();
             nextLevelQueryInput.setFilter(filter);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/ReplaceAction.java

@@ -157,6 +157,17 @@ public class ReplaceAction extends AbstractQActionFunction<ReplaceInput, Replace
             output.setDeleteOutput(deleteOutput);
          }
 
+         if(input.getSetPrimaryKeyInInsertedRecords())
+         {
+            for(int i = 0; i < insertList.size(); i++)
+            {
+               if(i < insertOutput.getRecords().size())
+               {
+                  insertList.get(i).setValue(primaryKeyField, insertOutput.getRecords().get(i).getValue(primaryKeyField));
+               }
+            }
+         }
+
          if(weOwnTheTransaction)
          {
             transaction.commit();

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/StorageAction.java

@@ -93,4 +93,28 @@ public class StorageAction
       QBackendModuleInterface  qModule                  = qBackendModuleDispatcher.getQBackendModule(backend);
       return (qModule);
    }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void makePublic(StorageInput storageInput) throws QException
+   {
+      QBackendModuleInterface qBackendModuleInterface = preAction(storageInput);
+      QStorageInterface       storageInterface        = qBackendModuleInterface.getStorageInterface();
+      storageInterface.makePublic(storageInput);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public String getDownloadURL(StorageInput storageInput) throws QException
+   {
+      QBackendModuleInterface qBackendModuleInterface = preAction(storageInput);
+      QStorageInterface       storageInterface        = qBackendModuleInterface.getStorageInterface();
+      return (storageInterface.getDownloadURL(storageInput));
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/helpers/QueryStatManager.java

@@ -214,71 +214,78 @@ public class QueryStatManager
     *******************************************************************************/
    public void add(QueryStat queryStat)
    {
-      if(queryStat == null)
+      try
       {
-         return;
-      }
-
-      if(active)
-      {
-         ////////////////////////////////////////////////////////////////////////////////////////
-         // set fields that we need to capture now (rather than when the thread to store runs) //
-         ////////////////////////////////////////////////////////////////////////////////////////
-         if(queryStat.getFirstResultTimestamp() == null)
+         if(queryStat == null)
          {
-            queryStat.setFirstResultTimestamp(Instant.now());
-         }
-
-         if(queryStat.getStartTimestamp() != null && queryStat.getFirstResultTimestamp() != null && queryStat.getFirstResultMillis() == null)
-         {
-            long millis = queryStat.getFirstResultTimestamp().toEpochMilli() - queryStat.getStartTimestamp().toEpochMilli();
-            queryStat.setFirstResultMillis((int) millis);
-         }
-
-         if(queryStat.getFirstResultMillis() != null && queryStat.getFirstResultMillis() < minMillisToStore)
-         {
-            //////////////////////////////////////////////////////////////
-            // discard this record if it's under the min millis setting //
-            //////////////////////////////////////////////////////////////
             return;
          }
 
-         if(queryStat.getSessionId() == null && QContext.getQSession() != null)
+         if(active)
          {
-            queryStat.setSessionId(QContext.getQSession().getUuid());
-         }
-
-         if(queryStat.getAction() == null)
-         {
-            if(!QContext.getActionStack().isEmpty())
+            ////////////////////////////////////////////////////////////////////////////////////////
+            // set fields that we need to capture now (rather than when the thread to store runs) //
+            ////////////////////////////////////////////////////////////////////////////////////////
+            if(queryStat.getFirstResultTimestamp() == null)
             {
-               queryStat.setAction(QContext.getActionStack().peek().getActionIdentity());
+               queryStat.setFirstResultTimestamp(Instant.now());
             }
-            else
+
+            if(queryStat.getStartTimestamp() != null && queryStat.getFirstResultTimestamp() != null && queryStat.getFirstResultMillis() == null)
             {
-               boolean   expected = false;
-               Exception e        = new Exception("Unexpected empty action stack");
-               for(StackTraceElement stackTraceElement : e.getStackTrace())
+               long millis = queryStat.getFirstResultTimestamp().toEpochMilli() - queryStat.getStartTimestamp().toEpochMilli();
+               queryStat.setFirstResultMillis((int) millis);
+            }
+
+            if(queryStat.getFirstResultMillis() != null && queryStat.getFirstResultMillis() < minMillisToStore)
+            {
+               //////////////////////////////////////////////////////////////
+               // discard this record if it's under the min millis setting //
+               //////////////////////////////////////////////////////////////
+               return;
+            }
+
+            if(queryStat.getSessionId() == null && QContext.getQSession() != null)
+            {
+               queryStat.setSessionId(QContext.getQSession().getUuid());
+            }
+
+            if(queryStat.getAction() == null)
+            {
+               if(QContext.getActionStack() != null && !QContext.getActionStack().isEmpty())
                {
-                  String className = stackTraceElement.getClassName();
-                  if(className.contains(QueryStatManagerInsertJob.class.getName()))
+                  queryStat.setAction(QContext.getActionStack().peek().getActionIdentity());
+               }
+               else
+               {
+                  boolean   expected = false;
+                  Exception e        = new Exception("Unexpected empty action stack");
+                  for(StackTraceElement stackTraceElement : e.getStackTrace())
                   {
-                     expected = true;
-                     break;
+                     String className = stackTraceElement.getClassName();
+                     if(className.contains(QueryStatManagerInsertJob.class.getName()))
+                     {
+                        expected = true;
+                        break;
+                     }
+                  }
+
+                  if(!expected)
+                  {
+                     LOG.debug(e);
                   }
                }
+            }
 
-               if(!expected)
-               {
-                  LOG.debug(e);
-               }
+            synchronized(this)
+            {
+               queryStats.add(queryStat);
             }
          }
-
-         synchronized(this)
-         {
-            queryStats.add(queryStat);
-         }
+      }
+      catch(Exception e)
+      {
+         LOG.debug("Error adding query stat", e);
       }
    }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceHelpContentManager.java

@@ -43,6 +43,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.help.HelpFormat;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpRole;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
@@ -111,6 +112,7 @@ public class QInstanceHelpContentManager
          String processName = nameValuePairs.get("process");
          String fieldName   = nameValuePairs.get("field");
          String sectionName = nameValuePairs.get("section");
+         String stepName = nameValuePairs.get("step");
          String widgetName  = nameValuePairs.get("widget");
          String slotName    = nameValuePairs.get("slot");
 
@@ -145,7 +147,7 @@ public class QInstanceHelpContentManager
          }
          else if(StringUtils.hasContent(processName))
          {
-            processHelpContentForProcess(key, processName, fieldName, roles, helpContent);
+            processHelpContentForProcess(key, processName, fieldName, stepName, roles, helpContent);
          }
          else if(StringUtils.hasContent(widgetName))
          {
@@ -208,6 +210,10 @@ public class QInstanceHelpContentManager
             optionalSection.get().removeHelpContent(roles);
          }
       }
+      else
+      {
+         LOG.info("Unrecognized key format for table help content", logPair("key", key));
+      }
    }
 
 
@@ -215,7 +221,7 @@ public class QInstanceHelpContentManager
    /*******************************************************************************
     **
     *******************************************************************************/
-   private static void processHelpContentForProcess(String key, String processName, String fieldName, Set<HelpRole> roles, QHelpContent helpContent)
+   private static void processHelpContentForProcess(String key, String processName, String fieldName, String stepName, Set<HelpRole> roles, QHelpContent helpContent)
    {
       QProcessMetaData process = QContext.getQInstance().getProcess(processName);
       if(process == null)
@@ -244,6 +250,30 @@ public class QInstanceHelpContentManager
             optionalField.get().removeHelpContent(roles);
          }
       }
+      else if(StringUtils.hasContent(stepName))
+      {
+         /////////////////////////////
+         // handle a process screen //
+         /////////////////////////////
+         QFrontendStepMetaData frontendStep = process.getFrontendStep(stepName);
+
+         if(frontendStep == null)
+         {
+            LOG.info("Unrecognized process step in help content", logPair("key", key));
+         }
+         else if(helpContent != null)
+         {
+            frontendStep.withHelpContent(helpContent);
+         }
+         else
+         {
+            frontendStep.removeHelpContent(roles);
+         }
+      }
+      else
+      {
+         LOG.info("Unrecognized key format for process help content", logPair("key", key));
+      }
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceValidator.java

@@ -109,6 +109,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import com.kingsrook.qqq.backend.core.utils.lambdas.UnsafeLambda;
 import org.quartz.CronExpression;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -151,6 +152,7 @@ public class QInstanceValidator
       // once, during the enrichment/validation work, so, capture it, and store it back in the instance. //
       /////////////////////////////////////////////////////////////////////////////////////////////////////
       JoinGraph joinGraph = null;
+      long      start     = System.currentTimeMillis();
       try
       {
          /////////////////////////////////////////////////////////////////////////////////////////////////
@@ -191,6 +193,9 @@ public class QInstanceValidator
          validateUniqueTopLevelNames(qInstance);
 
          runPlugins(QInstance.class, qInstance, qInstance);
+
+         long end = System.currentTimeMillis();
+         LOG.info("Validation (and enrichment) performance", logPair("millis", (end - start)));
       }
       catch(Exception e)
       {
@@ -209,6 +214,17 @@ public class QInstanceValidator
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void revalidate(QInstance qInstance) throws QInstanceValidationException
+   {
+      qInstance.setHasBeenValidated(null);
+      validate(qInstance);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -657,17 +673,20 @@ public class QInstanceValidator
             {
                if(assertCondition(CollectionUtils.nullSafeHasContents(exposedJoin.getJoinPath()), joinPrefix + "is missing a joinPath."))
                {
-                  joinConnectionsForTable = Objects.requireNonNullElseGet(joinConnectionsForTable, () -> joinGraph.getJoinConnections(table.getName()));
-
-                  boolean foundJoinConnection = false;
-                  for(JoinGraph.JoinConnectionList joinConnectionList : joinConnectionsForTable)
+                  if(joinGraph != null)
                   {
-                     if(joinConnectionList.matchesJoinPath(exposedJoin.getJoinPath()))
+                     joinConnectionsForTable = Objects.requireNonNullElseGet(joinConnectionsForTable, () -> joinGraph.getJoinConnections(table.getName()));
+
+                     boolean foundJoinConnection = false;
+                     for(JoinGraph.JoinConnectionList joinConnectionList : joinConnectionsForTable)
                      {
-                        foundJoinConnection = true;
+                        if(joinConnectionList.matchesJoinPath(exposedJoin.getJoinPath()))
+                        {
+                           foundJoinConnection = true;
+                        }
                      }
+                     assertCondition(foundJoinConnection, joinPrefix + "specified a joinPath [" + exposedJoin.getJoinPath() + "] which does not match a valid join connection in the instance.");
                   }
-                  assertCondition(foundJoinConnection, joinPrefix + "specified a joinPath [" + exposedJoin.getJoinPath() + "] which does not match a valid join connection in the instance.");
 
                   assertCondition(!usedJoinPaths.contains(exposedJoin.getJoinPath()), tablePrefix + "has more than one join with the joinPath: " + exposedJoin.getJoinPath());
                   usedJoinPaths.add(exposedJoin.getJoinPath());
@@ -1468,7 +1487,7 @@ public class QInstanceValidator
                                  warn("Error loading expectedType for field [" + fieldMetaData.getName() + "] in process [" + processName + "]: " + e.getMessage());
                               }
 
-                              validateSimpleCodeReference("Process " + processName + " code reference: ", codeReference, expectedClass);
+                              validateSimpleCodeReference("Process " + processName + " code reference:", codeReference, expectedClass);
                            }
                         }
                      }
@@ -1476,6 +1495,14 @@ public class QInstanceValidator
                }
             }
 
+            if(process.getCancelStep() != null)
+            {
+               if(assertCondition(process.getCancelStep().getCode() != null, "Cancel step is missing a code reference, in process " + processName))
+               {
+                  validateSimpleCodeReference("Process " + processName + " cancel step code reference: ", process.getCancelStep().getCode(), BackendStep.class);
+               }
+            }
+
             ///////////////////////////////////////////////////////////////////////////////
             // if the process has a schedule, make sure required schedule data populated //
             ///////////////////////////////////////////////////////////////////////////////
@@ -1487,7 +1514,11 @@ public class QInstanceValidator
 
             if(process.getVariantBackend() != null)
             {
-               assertCondition(qInstance.getBackend(process.getVariantBackend()) != null, "Process " + processName + ", a variant backend was not found named " + process.getVariantBackend());
+               if(qInstance.getBackends() != null)
+               {
+                  assertCondition(qInstance.getBackend(process.getVariantBackend()) != null, "Process " + processName + ", a variant backend was not found named " + process.getVariantBackend());
+               }
+
                assertCondition(process.getVariantRunStrategy() != null, "A variant run strategy was not set for process " + processName + " (which does specify a variant backend)");
             }
             else

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/logging/LogPair.java

@@ -79,7 +79,7 @@ public class LogPair
       }
       else if(value instanceof LogPair[] subLogPairs)
       {
-         String subLogPairsString = Arrays.stream(subLogPairs).map(LogPair::toString).collect(Collectors.joining(","));
+         String subLogPairsString = Arrays.stream(subLogPairs).filter(Objects::nonNull).map(LogPair::toString).collect(Collectors.joining(","));
          valueString = '{' + subLogPairsString + '}';
       }
       else if(value instanceof UnsafeSupplier<?, ?> us)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/logging/QLogger.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.logging;
 
 
+import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
@@ -147,6 +148,28 @@ public class QLogger
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public <T extends Throwable> T warnAndThrow(T t, LogPair... logPairs) throws T
+   {
+      warn(t.getMessage(), t, logPairs);
+      throw (t);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public <T extends Throwable> T errorAndThrow(T t, LogPair... logPairs) throws T
+   {
+      error(t.getMessage(), t, logPairs);
+      throw (t);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -595,7 +618,10 @@ public class QLogger
             {
                user = session.getUser().getIdReference();
             }
-            sessionLogPair = logPair("session", logPair("id", session.getUuid()), logPair("user", user));
+
+            LogPair variantsLogPair = getVariantsLogPair(session);
+
+            sessionLogPair = logPair("session", logPair("id", session.getUuid()), logPair("user", user), variantsLogPair);
          }
 
          try
@@ -615,6 +641,38 @@ public class QLogger
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static LogPair getVariantsLogPair(QSession session)
+   {
+      LogPair variantsLogPair = null;
+      try
+      {
+         if(session.getBackendVariants() != null)
+         {
+            LogPair[] variants = new LogPair[session.getBackendVariants().size()];
+
+            int i = 0;
+            for(Map.Entry<String, Serializable> entry : session.getBackendVariants().entrySet())
+            {
+               variants[i] = new LogPair(entry.getKey(), entry.getValue());
+            }
+
+            variantsLogPair = new LogPair("variants", variants);
+         }
+      }
+      catch(Exception e)
+      {
+         ////////////////
+         // leave null //
+         ////////////////
+      }
+      return variantsLogPair;
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/MetaDataProducerInterface.java

@@ -23,8 +23,8 @@ package com.kingsrook.qqq.backend.core.model;
 
 
 import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
 
 
 /*******************************************************************************
@@ -42,7 +42,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
  ** implement this interface.  or, same idea for a QRecordEntity that provides
  ** its own TableMetaData.
  *******************************************************************************/
-public interface MetaDataProducerInterface<T extends TopLevelMetaDataInterface>
+public interface MetaDataProducerInterface<T extends MetaDataProducerOutput>
 {
    int DEFAULT_SORT_ORDER = 500;
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/Attachment.java

@@ -0,0 +1,95 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class Attachment
+{
+   private byte[] contents;
+   private String name;
+
+
+
+   /*******************************************************************************
+    ** Getter for contents
+    *******************************************************************************/
+   public byte[] getContents()
+   {
+      return (this.contents);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for contents
+    *******************************************************************************/
+   public void setContents(byte[] contents)
+   {
+      this.contents = contents;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for contents
+    *******************************************************************************/
+   public Attachment withContents(byte[] contents)
+   {
+      this.contents = contents;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for name
+    *******************************************************************************/
+   public String getName()
+   {
+      return (this.name);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for name
+    *******************************************************************************/
+   public void setName(String name)
+   {
+      this.name = name;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for name
+    *******************************************************************************/
+   public Attachment withName(String name)
+   {
+      this.name = name;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/Content.java

@@ -0,0 +1,95 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class Content
+{
+   private String      body;
+   private ContentRole contentRole;
+
+
+
+   /*******************************************************************************
+    ** Getter for body
+    *******************************************************************************/
+   public String getBody()
+   {
+      return (this.body);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for body
+    *******************************************************************************/
+   public void setBody(String body)
+   {
+      this.body = body;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for body
+    *******************************************************************************/
+   public Content withBody(String body)
+   {
+      this.body = body;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for contentRole
+    *******************************************************************************/
+   public ContentRole getContentRole()
+   {
+      return (this.contentRole);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for contentRole
+    *******************************************************************************/
+   public void setContentRole(ContentRole contentRole)
+   {
+      this.contentRole = contentRole;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for contentRole
+    *******************************************************************************/
+   public Content withContentRole(ContentRole contentRole)
+   {
+      this.contentRole = contentRole;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/ContentRole.java

@@ -0,0 +1,39 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public interface ContentRole
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   enum Default implements ContentRole
+   {
+      DEFAULT
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/MultiParty.java

@@ -0,0 +1,92 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+import java.util.ArrayList;
+import java.util.List;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class MultiParty extends Party
+{
+   private List<Party> partyList;
+
+
+
+   /*******************************************************************************
+    ** Getter for partyList
+    *******************************************************************************/
+   public List<Party> getPartyList()
+   {
+      return (this.partyList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for partyList
+    *******************************************************************************/
+   public void setPartyList(List<Party> partyList)
+   {
+      this.partyList = partyList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for partyList
+    *******************************************************************************/
+   public MultiParty withPartyList(List<Party> partyList)
+   {
+      this.partyList = partyList;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MultiParty withParty(Party party)
+   {
+      addParty(party);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addParty(Party party)
+   {
+      if(this.partyList == null)
+      {
+         this.partyList = new ArrayList<>();
+      }
+      this.partyList.add(party);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/Party.java

@@ -0,0 +1,127 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class Party
+{
+   private String    label;
+   private String    address;
+   private PartyRole role;
+
+
+
+   /*******************************************************************************
+    ** Getter for label
+    *******************************************************************************/
+   public String getLabel()
+   {
+      return (this.label);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for label
+    *******************************************************************************/
+   public void setLabel(String label)
+   {
+      this.label = label;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for label
+    *******************************************************************************/
+   public Party withLabel(String label)
+   {
+      this.label = label;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for address
+    *******************************************************************************/
+   public String getAddress()
+   {
+      return (this.address);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for address
+    *******************************************************************************/
+   public void setAddress(String address)
+   {
+      this.address = address;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for address
+    *******************************************************************************/
+   public Party withAddress(String address)
+   {
+      this.address = address;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for role
+    *******************************************************************************/
+   public PartyRole getRole()
+   {
+      return (this.role);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for role
+    *******************************************************************************/
+   public void setRole(PartyRole role)
+   {
+      this.role = role;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for role
+    *******************************************************************************/
+   public Party withRole(PartyRole role)
+   {
+      this.role = role;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/PartyRole.java

@@ -0,0 +1,39 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public interface PartyRole
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   enum Default implements PartyRole
+   {
+      DEFAULT
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/SendMessageInput.java

@@ -0,0 +1,278 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SendMessageInput extends AbstractActionInput
+{
+   private String           messagingProviderName;
+   private Party            to;
+   private Party            from;
+   private String           subject;
+   private List<Content>    contentList;
+   private List<Attachment> attachmentList;
+
+
+
+   /*******************************************************************************
+    ** Getter for to
+    *******************************************************************************/
+   public Party getTo()
+   {
+      return (this.to);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for to
+    *******************************************************************************/
+   public void setTo(Party to)
+   {
+      this.to = to;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for to
+    *******************************************************************************/
+   public SendMessageInput withTo(Party to)
+   {
+      this.to = to;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for from
+    *******************************************************************************/
+   public Party getFrom()
+   {
+      return (this.from);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for from
+    *******************************************************************************/
+   public void setFrom(Party from)
+   {
+      this.from = from;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for from
+    *******************************************************************************/
+   public SendMessageInput withFrom(Party from)
+   {
+      this.from = from;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for subject
+    *******************************************************************************/
+   public String getSubject()
+   {
+      return (this.subject);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for subject
+    *******************************************************************************/
+   public void setSubject(String subject)
+   {
+      this.subject = subject;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for subject
+    *******************************************************************************/
+   public SendMessageInput withSubject(String subject)
+   {
+      this.subject = subject;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for contentList
+    *******************************************************************************/
+   public List<Content> getContentList()
+   {
+      return (this.contentList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for contentList
+    *******************************************************************************/
+   public void setContentList(List<Content> contentList)
+   {
+      this.contentList = contentList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for contentList
+    *******************************************************************************/
+   public SendMessageInput withContentList(List<Content> contentList)
+   {
+      this.contentList = contentList;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for attachmentList
+    *******************************************************************************/
+   public List<Attachment> getAttachmentList()
+   {
+      return (this.attachmentList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for attachmentList
+    *******************************************************************************/
+   public void setAttachmentList(List<Attachment> attachmentList)
+   {
+      this.attachmentList = attachmentList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for attachmentList
+    *******************************************************************************/
+   public SendMessageInput withAttachmentList(List<Attachment> attachmentList)
+   {
+      this.attachmentList = attachmentList;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public SendMessageInput withContent(Content content)
+   {
+      addContent(content);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addContent(Content content)
+   {
+      if(this.contentList == null)
+      {
+         this.contentList = new ArrayList<>();
+      }
+      this.contentList.add(content);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public SendMessageInput withAttachment(Attachment attachment)
+   {
+      addAttachment(attachment);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addAttachment(Attachment attachment)
+   {
+      if(this.attachmentList == null)
+      {
+         this.attachmentList = new ArrayList<>();
+      }
+      this.attachmentList.add(attachment);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for messagingProviderName
+    *******************************************************************************/
+   public String getMessagingProviderName()
+   {
+      return (this.messagingProviderName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for messagingProviderName
+    *******************************************************************************/
+   public void setMessagingProviderName(String messagingProviderName)
+   {
+      this.messagingProviderName = messagingProviderName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for messagingProviderName
+    *******************************************************************************/
+   public SendMessageInput withMessagingProviderName(String messagingProviderName)
+   {
+      this.messagingProviderName = messagingProviderName;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/SendMessageOutput.java

@@ -0,0 +1,33 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging;
+
+
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionOutput;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SendMessageOutput extends AbstractActionOutput
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/email/EmailContentRole.java

@@ -0,0 +1,35 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging.email;
+
+
+import com.kingsrook.qqq.backend.core.model.actions.messaging.ContentRole;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public enum EmailContentRole implements ContentRole
+{
+   TEXT,
+   HTML
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/messaging/email/EmailPartyRole.java

@@ -0,0 +1,38 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.messaging.email;
+
+
+import com.kingsrook.qqq.backend.core.model.actions.messaging.PartyRole;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public enum EmailPartyRole implements PartyRole
+{
+   TO,
+   CC,
+   BCC,
+   FROM,
+   REPLY_TO
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/ProcessState.java

@@ -29,6 +29,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Optional;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
 
 
 /*******************************************************************************
@@ -41,6 +42,11 @@ public class ProcessState implements Serializable
    private List<String>              stepList     = new ArrayList<>();
    private Optional<String>          nextStepName = Optional.empty();
 
+   /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   // maybe, remove this altogether - just let the frontend compute & send if needed... but how does it know last version...? //
+   /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+   private List<QFrontendStepMetaData> updatedFrontendStepList = null;
+
 
 
    /*******************************************************************************
@@ -139,4 +145,36 @@ public class ProcessState implements Serializable
    {
       this.stepList = stepList;
    }
+
+
+
+   /*******************************************************************************
+    ** Getter for updatedFrontendStepList
+    *******************************************************************************/
+   public List<QFrontendStepMetaData> getUpdatedFrontendStepList()
+   {
+      return (this.updatedFrontendStepList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for updatedFrontendStepList
+    *******************************************************************************/
+   public void setUpdatedFrontendStepList(List<QFrontendStepMetaData> updatedFrontendStepList)
+   {
+      this.updatedFrontendStepList = updatedFrontendStepList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for updatedFrontendStepList
+    *******************************************************************************/
+   public ProcessState withUpdatedFrontendStepList(List<QFrontendStepMetaData> updatedFrontendStepList)
+   {
+      this.updatedFrontendStepList = updatedFrontendStepList;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunBackendStepOutput.java

@@ -27,10 +27,14 @@ import java.math.BigDecimal;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QRuntimeException;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionOutput;
 import com.kingsrook.qqq.backend.core.model.actions.audits.AuditInput;
 import com.kingsrook.qqq.backend.core.model.actions.audits.AuditSingleInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
 
@@ -40,9 +44,13 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
  *******************************************************************************/
 public class RunBackendStepOutput extends AbstractActionOutput implements Serializable
 {
+   private String processName;
+
    private ProcessState processState;
    private Exception    exception; // todo - make optional
 
+   private String overrideLastStepName; // todo - does this need to go into state too??
+
    private List<AuditInput> auditInputList = new ArrayList<>();
 
 
@@ -78,6 +86,7 @@ public class RunBackendStepOutput extends AbstractActionOutput implements Serial
    public void seedFromRequest(RunBackendStepInput runBackendStepInput)
    {
       this.processState = runBackendStepInput.getProcessState();
+      this.processName = runBackendStepInput.getProcessName();
    }
 
 
@@ -312,4 +321,111 @@ public class RunBackendStepOutput extends AbstractActionOutput implements Serial
       auditInput.addAuditSingleInput(auditSingleInput);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for overrideLastStepName
+    *******************************************************************************/
+   public String getOverrideLastStepName()
+   {
+      return (this.overrideLastStepName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for overrideLastStepName
+    *******************************************************************************/
+   public void setOverrideLastStepName(String overrideLastStepName)
+   {
+      this.overrideLastStepName = overrideLastStepName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for overrideLastStepName
+    *******************************************************************************/
+   public RunBackendStepOutput withOverrideLastStepName(String overrideLastStepName)
+   {
+      this.overrideLastStepName = overrideLastStepName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void updateStepList(List<String> stepList)
+   {
+      getProcessState().setStepList(stepList);
+
+      if(processName == null)
+      {
+         throw (new QRuntimeException("ProcessName was not set in this object, therefore updateStepList cannot complete successfully.  Try to manually call setProcessName as a work around."));
+      }
+
+      QProcessMetaData processMetaData = QContext.getQInstance().getProcess(processName);
+
+      ArrayList<QFrontendStepMetaData> updatedFrontendStepList = new ArrayList<>(stepList.stream()
+         .map(name -> processMetaData.getStep(name))
+         .filter(step -> step instanceof QFrontendStepMetaData)
+         .map(step -> (QFrontendStepMetaData) step)
+         .toList());
+
+      setUpdatedFrontendStepList(updatedFrontendStepList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for processName
+    *******************************************************************************/
+   public String getProcessName()
+   {
+      return (this.processName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for processName
+    *******************************************************************************/
+   public void setProcessName(String processName)
+   {
+      this.processName = processName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for processName
+    *******************************************************************************/
+   public RunBackendStepOutput withProcessName(String processName)
+   {
+      this.processName = processName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for updatedFrontendStepList
+    *******************************************************************************/
+   public List<QFrontendStepMetaData> getUpdatedFrontendStepList()
+   {
+      return (this.processState.getUpdatedFrontendStepList());
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for updatedFrontendStepList
+    *******************************************************************************/
+   public void setUpdatedFrontendStepList(List<QFrontendStepMetaData> updatedFrontendStepList)
+   {
+      this.processState.setUpdatedFrontendStepList(updatedFrontendStepList);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/RunProcessOutput.java

@@ -32,6 +32,7 @@ import java.util.Map;
 import java.util.Optional;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
 
@@ -327,4 +328,25 @@ public class RunProcessOutput extends AbstractActionOutput implements Serializab
    {
       return exception;
    }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void setUpdatedFrontendStepList(List<QFrontendStepMetaData> updatedFrontendStepList)
+   {
+      this.processState.setUpdatedFrontendStepList(updatedFrontendStepList);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public List<QFrontendStepMetaData> getUpdatedFrontendStepList()
+   {
+      return this.processState.getUpdatedFrontendStepList();
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/insert/InsertInput.java

@@ -112,7 +112,7 @@ public class InsertInput extends AbstractTableActionInput
    /*******************************************************************************
     **
     *******************************************************************************/
-   public InsertInput withRecordEntities(List<QRecordEntity> recordEntityList)
+   public InsertInput withRecordEntities(List<? extends QRecordEntity> recordEntityList)
    {
       for(QRecordEntity recordEntity : CollectionUtils.nonNullList(recordEntityList))
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/JoinsContext.java

@@ -41,6 +41,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.security.MultiRecordSecurityLock;
+import com.kingsrook.qqq.backend.core.model.metadata.security.NullValueBehaviorUtil;
 import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLockFilters;
@@ -550,7 +551,7 @@ public class JoinsContext
             ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
             // handle user with no values -- they can only see null values, and only iff the lock's null-value behavior is ALLOW //
             ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-            if(RecordSecurityLock.NullValueBehavior.ALLOW.equals(recordSecurityLock.getNullValueBehavior()))
+            if(RecordSecurityLock.NullValueBehavior.ALLOW.equals(NullValueBehaviorUtil.getEffectiveNullValueBehavior(recordSecurityLock)))
             {
                lockCriteria.add(new QFilterCriteria(fieldName, QCriteriaOperator.IS_BLANK));
             }
@@ -569,7 +570,7 @@ public class JoinsContext
             // else, if user/session has some values, build an IN rule -                                                //
             // noting that if the lock's null-value behavior is ALLOW, then we actually want IS_NULL_OR_IN, not just IN //
             //////////////////////////////////////////////////////////////////////////////////////////////////////////////
-            if(RecordSecurityLock.NullValueBehavior.ALLOW.equals(recordSecurityLock.getNullValueBehavior()))
+            if(RecordSecurityLock.NullValueBehavior.ALLOW.equals(NullValueBehaviorUtil.getEffectiveNullValueBehavior(recordSecurityLock)))
             {
                lockCriteria.add(new QFilterCriteria(fieldName, QCriteriaOperator.IS_NULL_OR_IN, securityKeyValues));
             }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QQueryFilter.java

@@ -25,13 +25,18 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query;
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.Arrays;
+import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
 import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.AbstractFilterExpression;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.FilterVariableExpression;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
 
@@ -411,6 +416,88 @@ public class QQueryFilter implements Serializable, Cloneable
 
 
 
+   /*******************************************************************************
+    ** Replaces any FilterVariables' variableNames with one constructed from the field
+    ** name, criteria, and index, camel style
+    **
+    *******************************************************************************/
+   public void prepForBackend()
+   {
+      Map<String, Integer> fieldOperatorMap = new HashMap<>();
+      for(QFilterCriteria criterion : getCriteria())
+      {
+         if(criterion.getValues() != null)
+         {
+            int criteriaIndex = 1;
+            int valueIndex    = 0;
+            for(Serializable value : criterion.getValues())
+            {
+               ///////////////////////////////////////////////////////////////////////////////
+               // keep track of what the index is for this criterion, this way if there are //
+               // more than one with the same id/operator values, we can differentiate      //
+               ///////////////////////////////////////////////////////////////////////////////
+               String backendName = getBackendName(criterion, valueIndex);
+               if(!fieldOperatorMap.containsKey(backendName))
+               {
+                  fieldOperatorMap.put(backendName, criteriaIndex);
+               }
+               else
+               {
+                  criteriaIndex = fieldOperatorMap.get(backendName) + 1;
+                  fieldOperatorMap.put(backendName, criteriaIndex);
+               }
+
+               if(value instanceof FilterVariableExpression fve)
+               {
+                  if(criteriaIndex > 1)
+                  {
+                     backendName += criteriaIndex;
+                  }
+                  fve.setVariableName(backendName);
+               }
+
+               valueIndex++;
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** builds up a backend name for a field variable expression
+    **
+    *******************************************************************************/
+   private String getBackendName(QFilterCriteria criterion, int valueIndex)
+   {
+      StringBuilder backendName = new StringBuilder();
+      for(String fieldNameParts : criterion.getFieldName().split("\\."))
+      {
+         backendName.append(StringUtils.ucFirst(fieldNameParts));
+      }
+
+      for(String operatorParts : criterion.getOperator().name().split("_"))
+      {
+         backendName.append(StringUtils.ucFirst(operatorParts.toLowerCase()));
+      }
+
+      if(criterion.getOperator().equals(QCriteriaOperator.BETWEEN) || criterion.getOperator().equals(QCriteriaOperator.NOT_BETWEEN))
+      {
+         if(valueIndex == 0)
+         {
+            backendName.append("From");
+         }
+         else
+         {
+            backendName.append("To");
+         }
+      }
+
+      return (StringUtils.lcFirst(backendName.toString()));
+   }
+
+
+
    /*******************************************************************************
     ** Replace any criteria values that look like ${input.XXX} with the value of XXX
     ** from the supplied inputValues map.
@@ -419,8 +506,10 @@ public class QQueryFilter implements Serializable, Cloneable
     ** QQueryFilter - e.g., if it's one that defined in metaData, and that we don't
     ** want to be (permanently) changed!!
     *******************************************************************************/
-   public void interpretValues(Map<String, Serializable> inputValues)
+   public void interpretValues(Map<String, Serializable> inputValues) throws QException
    {
+      List<Exception> caughtExceptions = new ArrayList<>();
+
       QMetaDataVariableInterpreter variableInterpreter = new QMetaDataVariableInterpreter();
       variableInterpreter.addValueMap("input", inputValues);
       for(QFilterCriteria criterion : getCriteria())
@@ -431,24 +520,45 @@ public class QQueryFilter implements Serializable, Cloneable
 
             for(Serializable value : criterion.getValues())
             {
-               if(value instanceof AbstractFilterExpression<?>)
+               try
                {
-                  /////////////////////////////////////////////////////////////////////////
-                  // todo - do we want to try to interpret values within the expression? //
-                  // e.g., greater than now minus ${input.noOfDays}                      //
-                  /////////////////////////////////////////////////////////////////////////
-                  newValues.add(value);
+                  if(value instanceof AbstractFilterExpression<?>)
+                  {
+                     ///////////////////////////////////////////////////////////////////////
+                     // if a filter variable expression, evaluate the input values, which //
+                     // will replace the variables with the corresponding actual values   //
+                     ///////////////////////////////////////////////////////////////////////
+                     if(value instanceof FilterVariableExpression filterVariableExpression)
+                     {
+                        newValues.add(filterVariableExpression.evaluateInputValues(inputValues));
+                     }
+                     else
+                     {
+                        newValues.add(value);
+                     }
+                  }
+                  else
+                  {
+                     String       valueAsString    = ValueUtils.getValueAsString(value);
+                     Serializable interpretedValue = variableInterpreter.interpretForObject(valueAsString);
+                     newValues.add(interpretedValue);
+                  }
                }
-               else
+               catch(Exception e)
                {
-                  String       valueAsString    = ValueUtils.getValueAsString(value);
-                  Serializable interpretedValue = variableInterpreter.interpretForObject(valueAsString);
-                  newValues.add(interpretedValue);
+                  caughtExceptions.add(e);
                }
             }
             criterion.setValues(newValues);
          }
       }
+
+      if(!caughtExceptions.isEmpty())
+      {
+         String  message       = "Error interpreting filter values: " + StringUtils.joinWithCommasAndAnd(caughtExceptions.stream().map(e -> e.getMessage()).toList());
+         boolean allUserFacing = caughtExceptions.stream().allMatch(QUserFacingException.class::isInstance);
+         throw (allUserFacing ? new QUserFacingException(message) : new QException(message));
+      }
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QueryInput.java

@@ -25,6 +25,7 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.EnumSet;
+import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
@@ -37,7 +38,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterf
  ** Input data for the Query action
  **
  *******************************************************************************/
-public class QueryInput extends AbstractTableActionInput implements QueryOrGetInputInterface
+public class QueryInput extends AbstractTableActionInput implements QueryOrGetInputInterface, Cloneable
 {
    private QBackendTransaction transaction;
    private QQueryFilter        filter;
@@ -109,6 +110,40 @@ public class QueryInput extends AbstractTableActionInput implements QueryOrGetIn
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QueryInput clone() throws CloneNotSupportedException
+   {
+      QueryInput clone = (QueryInput) super.clone();
+
+      if(fieldsToTranslatePossibleValues != null)
+      {
+         clone.fieldsToTranslatePossibleValues = new HashSet<>(fieldsToTranslatePossibleValues);
+      }
+
+      if(queryJoins != null)
+      {
+         clone.queryJoins = new ArrayList<>(queryJoins);
+      }
+
+      if(clone.associationNamesToInclude != null)
+      {
+         clone.associationNamesToInclude = new HashSet<>(associationNamesToInclude);
+      }
+
+      if(queryHints != null)
+      {
+         clone.queryHints = EnumSet.noneOf(QueryHint.class);
+         clone.queryHints.addAll(queryHints);
+      }
+
+      return (clone);
+   }
+
+
+
    /*******************************************************************************
     ** Getter for filter
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/expressions/AbstractFilterExpression.java

@@ -23,6 +23,8 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions;
 
 
 import java.io.Serializable;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
 
 
 /*******************************************************************************
@@ -33,7 +35,17 @@ public abstract class AbstractFilterExpression<T extends Serializable> implement
    /*******************************************************************************
     **
     *******************************************************************************/
-   public abstract T evaluate();
+   public abstract T evaluate() throws QException;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public T evaluateInputValues(Map<String, Serializable> inputValues) throws QException
+   {
+      return (T) this;
+   }
 
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/expressions/FilterVariableExpression.java

@@ -0,0 +1,214 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2023.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions;
+
+
+import java.io.Serializable;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class FilterVariableExpression extends AbstractFilterExpression<Serializable>
+{
+   private String variableName;
+   private String fieldName;
+   private String operator;
+   private int    valueIndex = 0;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public Serializable evaluate() throws QException
+   {
+      throw (new QUserFacingException("Missing variable value."));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public Serializable evaluateInputValues(Map<String, Serializable> inputValues) throws QException
+   {
+      if(!inputValues.containsKey(variableName) || "".equals(ValueUtils.getValueAsString(inputValues.get(variableName))))
+      {
+         throw (new QUserFacingException("Missing value for variable: " + variableName));
+      }
+      return (inputValues.get(variableName));
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public FilterVariableExpression()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   private FilterVariableExpression(String fieldName, int valueIndex)
+   {
+      this.fieldName = fieldName;
+      this.valueIndex = valueIndex;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for valueIndex
+    *******************************************************************************/
+   public int getValueIndex()
+   {
+      return (this.valueIndex);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for valueIndex
+    *******************************************************************************/
+   public void setValueIndex(int valueIndex)
+   {
+      this.valueIndex = valueIndex;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for valueIndex
+    *******************************************************************************/
+   public FilterVariableExpression withValueIndex(int valueIndex)
+   {
+      this.valueIndex = valueIndex;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fieldName
+    *******************************************************************************/
+   public String getFieldName()
+   {
+      return (this.fieldName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for fieldName
+    *******************************************************************************/
+   public void setFieldName(String fieldName)
+   {
+      this.fieldName = fieldName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fieldName
+    *******************************************************************************/
+   public FilterVariableExpression withFieldName(String fieldName)
+   {
+      this.fieldName = fieldName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for variableName
+    *******************************************************************************/
+   public String getVariableName()
+   {
+      return (this.variableName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for variableName
+    *******************************************************************************/
+   public void setVariableName(String variableName)
+   {
+      this.variableName = variableName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for variableName
+    *******************************************************************************/
+   public FilterVariableExpression withVariableName(String variableName)
+   {
+      this.variableName = variableName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for operator
+    *******************************************************************************/
+   public String getOperator()
+   {
+      return (this.operator);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for operator
+    *******************************************************************************/
+   public void setOperator(String operator)
+   {
+      this.operator = operator;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for operator
+    *******************************************************************************/
+   public FilterVariableExpression withOperator(String operator)
+   {
+      this.operator = operator;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/expressions/Now.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions;
 
 
 import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
 
 
 /*******************************************************************************
@@ -35,7 +36,7 @@ public class Now extends AbstractFilterExpression<Instant>
     **
     *******************************************************************************/
    @Override
-   public Instant evaluate()
+   public Instant evaluate() throws QException
    {
       return (Instant.now());
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/expressions/NowWithOffset.java

@@ -28,6 +28,7 @@ import java.time.ZoneId;
 import java.time.ZoneOffset;
 import java.time.temporal.ChronoUnit;
 import java.util.concurrent.TimeUnit;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
 
 
 /*******************************************************************************
@@ -119,7 +120,7 @@ public class NowWithOffset extends AbstractFilterExpression<Instant>
     **
     *******************************************************************************/
    @Override
-   public Instant evaluate()
+   public Instant evaluate() throws QException
    {
       /////////////////////////////////////////////////////////////////////////////
       // Instant doesn't let us plus/minus WEEK, MONTH, or YEAR...               //

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/expressions/ThisOrLastPeriod.java

@@ -28,6 +28,7 @@ import java.time.LocalDateTime;
 import java.time.ZoneId;
 import java.time.temporal.ChronoField;
 import java.time.temporal.ChronoUnit;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.exceptions.QRuntimeException;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
@@ -95,7 +96,7 @@ public class ThisOrLastPeriod extends AbstractFilterExpression<Instant>
     **
     *******************************************************************************/
    @Override
-   public Instant evaluate()
+   public Instant evaluate() throws QException
    {
       ZoneId zoneId = ValueUtils.getSessionOrInstanceZoneId();
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/replace/ReplaceInput.java

@@ -39,8 +39,9 @@ public class ReplaceInput extends AbstractTableActionInput
    private UniqueKey           key;
    private List<QRecord>       records;
    private QQueryFilter        filter;
-   private boolean             performDeletes            = true;
-   private boolean             allowNullKeyValuesToEqual = false;
+   private boolean performDeletes                 = true;
+   private boolean allowNullKeyValuesToEqual      = false;
+   private boolean setPrimaryKeyInInsertedRecords = false;
 
    private boolean omitDmlAudit = false;
 
@@ -271,4 +272,35 @@ public class ReplaceInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for setPrimaryKeyInInsertedRecords
+    *******************************************************************************/
+   public boolean getSetPrimaryKeyInInsertedRecords()
+   {
+      return (this.setPrimaryKeyInInsertedRecords);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for setPrimaryKeyInInsertedRecords
+    *******************************************************************************/
+   public void setSetPrimaryKeyInInsertedRecords(boolean setPrimaryKeyInInsertedRecords)
+   {
+      this.setPrimaryKeyInInsertedRecords = setPrimaryKeyInInsertedRecords;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for setPrimaryKeyInInsertedRecords
+    *******************************************************************************/
+   public ReplaceInput withSetPrimaryKeyInInsertedRecords(boolean setPrimaryKeyInInsertedRecords)
+   {
+      this.setPrimaryKeyInInsertedRecords = setPrimaryKeyInInsertedRecords;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/update/UpdateInput.java

@@ -120,7 +120,7 @@ public class UpdateInput extends AbstractTableActionInput
    /*******************************************************************************
     **
     *******************************************************************************/
-   public UpdateInput withRecordEntities(List<QRecordEntity> recordEntityList)
+   public UpdateInput withRecordEntities(List<? extends QRecordEntity> recordEntityList)
    {
       for(QRecordEntity recordEntity : CollectionUtils.nonNullList(recordEntityList))
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/CompositeWidgetData.java

@@ -54,6 +54,7 @@ public class CompositeWidgetData extends AbstractBlockWidgetData<CompositeWidget
       /////////////////////////////////////////////////////////////
       // note, these are used in QQQ FMD CompositeWidgetData.tsx //
       /////////////////////////////////////////////////////////////
+      FLEX_COLUMN,
       FLEX_ROW_WRAPPED,
       FLEX_ROW_SPACE_BETWEEN,
       TABLE_SUB_ROW_DETAILS,

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/DynamicFormWidgetData.java

@@ -0,0 +1,188 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
+
+
+import java.util.List;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class DynamicFormWidgetData extends QWidgetData
+{
+   private List<QFieldMetaData> fieldList;
+
+   /////////////////////////////////////////////////////////////////////
+   // values for the fields -                                         //
+   // use a QRecord, so we can do "richer" things, like DisplayValues //
+   /////////////////////////////////////////////////////////////////////
+   private QRecord recordOfFieldValues;
+
+   /////////////////////////////////////////////////////
+   // if there are no fields, what message to display //
+   /////////////////////////////////////////////////////
+   private String noFieldsMessage;
+
+   ///////////////////////////////////////////////////////////////////////////////////
+   // what 1 field do we want to combine the dynamic fields into (as a JSON string) //
+   ///////////////////////////////////////////////////////////////////////////////////
+   private String mergedDynamicFormValuesIntoFieldName;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getType()
+   {
+      return WidgetType.DYNAMIC_FORM.getType();
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for fieldList
+    *******************************************************************************/
+   public List<QFieldMetaData> getFieldList()
+   {
+      return (this.fieldList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for fieldList
+    *******************************************************************************/
+   public void setFieldList(List<QFieldMetaData> fieldList)
+   {
+      this.fieldList = fieldList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for fieldList
+    *******************************************************************************/
+   public DynamicFormWidgetData withFieldList(List<QFieldMetaData> fieldList)
+   {
+      this.fieldList = fieldList;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for noFieldsMessage
+    *******************************************************************************/
+   public String getNoFieldsMessage()
+   {
+      return (this.noFieldsMessage);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for noFieldsMessage
+    *******************************************************************************/
+   public void setNoFieldsMessage(String noFieldsMessage)
+   {
+      this.noFieldsMessage = noFieldsMessage;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for noFieldsMessage
+    *******************************************************************************/
+   public DynamicFormWidgetData withNoFieldsMessage(String noFieldsMessage)
+   {
+      this.noFieldsMessage = noFieldsMessage;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for mergedDynamicFormValuesIntoFieldName
+    *******************************************************************************/
+   public String getMergedDynamicFormValuesIntoFieldName()
+   {
+      return (this.mergedDynamicFormValuesIntoFieldName);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for mergedDynamicFormValuesIntoFieldName
+    *******************************************************************************/
+   public void setMergedDynamicFormValuesIntoFieldName(String mergedDynamicFormValuesIntoFieldName)
+   {
+      this.mergedDynamicFormValuesIntoFieldName = mergedDynamicFormValuesIntoFieldName;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for mergedDynamicFormValuesIntoFieldName
+    *******************************************************************************/
+   public DynamicFormWidgetData withMergedDynamicFormValuesIntoFieldName(String mergedDynamicFormValuesIntoFieldName)
+   {
+      this.mergedDynamicFormValuesIntoFieldName = mergedDynamicFormValuesIntoFieldName;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for recordOfFieldValues
+    *******************************************************************************/
+   public QRecord getRecordOfFieldValues()
+   {
+      return (this.recordOfFieldValues);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for recordOfFieldValues
+    *******************************************************************************/
+   public void setRecordOfFieldValues(QRecord recordOfFieldValues)
+   {
+      this.recordOfFieldValues = recordOfFieldValues;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for recordOfFieldValues
+    *******************************************************************************/
+   public DynamicFormWidgetData withRecordOfFieldValues(QRecord recordOfFieldValues)
+   {
+      this.recordOfFieldValues = recordOfFieldValues;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/MultiTableData.java

@@ -0,0 +1,97 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2022.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
+
+
+import java.util.List;
+
+
+/*******************************************************************************
+ ** Model containing datastructure expected by frontend bar chart widget
+ **
+ *******************************************************************************/
+public class MultiTableData extends QWidgetData
+{
+   List<TableData> tableDataList;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MultiTableData()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MultiTableData(List<TableData> tableDataList)
+   {
+      setTableDataList(tableDataList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for type
+    **
+    *******************************************************************************/
+   public String getType()
+   {
+      return WidgetType.MULTI_TABLE.getType();
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tableDataList
+    *******************************************************************************/
+   public List<TableData> getTableDataList()
+   {
+      return (this.tableDataList);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tableDataList
+    *******************************************************************************/
+   public void setTableDataList(List<TableData> tableDataList)
+   {
+      this.tableDataList = tableDataList;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tableDataList
+    *******************************************************************************/
+   public MultiTableData withTableDataList(List<TableData> tableDataList)
+   {
+      this.tableDataList = tableDataList;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/WidgetType.java

@@ -27,10 +27,12 @@ package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
  *******************************************************************************/
 public enum WidgetType
 {
+   ///////////////////////////////////
+   // (generally) dashboard widgets //
+   ///////////////////////////////////
    ALERT("alert"),
    BAR_CHART("barChart"),
    CHART("chart"),
-   CHILD_RECORD_LIST("childRecordList"),
    DIVIDER("divider"),
    FIELD_VALUE_LIST("fieldValueList"),
    GENERIC("generic"),
@@ -40,20 +42,35 @@ public enum WidgetType
    SMALL_LINE_CHART("smallLineChart"),
    LOCATION("location"),
    MULTI_STATISTICS("multiStatistics"),
-   PARENT_WIDGET("parentWidget"),
+   MULTI_TABLE("multiTable"),
    PIE_CHART("pieChart"),
-   PROCESS("process"),
    QUICK_SIGHT_CHART("quickSightChart"),
    STATISTICS("statistics"),
    STACKED_BAR_CHART("stackedBarChart"),
    STEPPER("stepper"),
    TABLE("table"),
    USA_MAP("usaMap"),
+
+   ///////////////////////////////
+   // widget to house a process //
+   ///////////////////////////////
+   PROCESS("process"),
+
+   ///////////////////////
+   // container widgets //
+   ///////////////////////
+   PARENT_WIDGET("parentWidget"),
    COMPOSITE("composite"),
+
+   //////////////////////////////
+   // record view/edit widgets //
+   //////////////////////////////
+   CHILD_RECORD_LIST("childRecordList"),
+   DYNAMIC_FORM("dynamicForm"),
    DATA_BAG_VIEWER("dataBagViewer"),
-   SCRIPT_VIEWER("scriptViewer"),
-   REPORT_SETUP("reportSetup"),
-   PIVOT_TABLE_SETUP("pivotTableSetup");
+   PIVOT_TABLE_SETUP("pivotTableSetup"),
+   FILTER_AND_COLUMNS_SETUP("filterAndColumnsSetup"),
+   SCRIPT_VIEWER("scriptViewer");
 
 
    private final String type;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QField.java

@@ -49,6 +49,11 @@ public @interface QField
     *******************************************************************************/
    String backendName() default "";
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   boolean isPrimaryKey() default false;
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QIgnore.java

@@ -0,0 +1,39 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.data;
+
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/*******************************************************************************
+ ** Marker - that a piece of code should be ignored (e.g., a field not treated as
+ ** a @QField)
+ *******************************************************************************/
+@Target({ ElementType.FIELD, ElementType.METHOD })
+@Retention(RetentionPolicy.RUNTIME)
+public @interface QIgnore
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecord.java

@@ -35,12 +35,15 @@ import java.util.LinkedHashMap;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Map;
+import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.statusmessages.QErrorMessage;
 import com.kingsrook.qqq.backend.core.model.statusmessages.QWarningMessage;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang3.SerializationUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -462,6 +465,7 @@ public class QRecord implements Serializable
    }
 
 
+
    /*******************************************************************************
     ** Getter for a single field's value
     **
@@ -616,6 +620,22 @@ public class QRecord implements Serializable
 
 
 
+   /*******************************************************************************
+    ** Getter for errors
+    **
+    *******************************************************************************/
+   @JsonIgnore
+   public String getErrorsAsString()
+   {
+      if(CollectionUtils.nullSafeHasContents(errors))
+      {
+         return StringUtils.join("; ", errors.stream().map(e -> e.getMessage()).toList());
+      }
+      return ("");
+   }
+
+
+
    /*******************************************************************************
     ** Setter for errors
     **
@@ -732,6 +752,22 @@ public class QRecord implements Serializable
 
 
 
+   /*******************************************************************************
+    ** Getter for warnings
+    **
+    *******************************************************************************/
+   @JsonIgnore
+   public String getWarningsAsString()
+   {
+      if(CollectionUtils.nullSafeHasContents(warnings))
+      {
+         return StringUtils.join("; ", warnings.stream().map(e -> e.getMessage()).toList());
+      }
+      return ("");
+   }
+
+
+
    /*******************************************************************************
     ** Setter for warnings
     *******************************************************************************/
@@ -742,6 +778,18 @@ public class QRecord implements Serializable
 
 
 
+   /*******************************************************************************
+    ** Fluently Add one warning to this record
+    **
+    *******************************************************************************/
+   public QRecord withWarning(QWarningMessage warning)
+   {
+      addWarning(warning);
+      return (this);
+   }
+
+
+
    /*******************************************************************************
     ** Fluent setter for warnings
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEntity.java

@@ -218,6 +218,7 @@ public abstract class QRecordEntity
    }
 
 
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -296,7 +297,19 @@ public abstract class QRecordEntity
                   }
                   else
                   {
-                     LOG.debug("Skipping field without @QField annotation", logPair("class", c.getSimpleName()), logPair("fieldName", fieldName));
+                     Optional<QIgnore>      ignoreAnnotation      = getQIgnoreAnnotation(c, fieldName);
+                     Optional<QAssociation> associationAnnotation = getQAssociationAnnotation(c, fieldName);
+
+                     if(ignoreAnnotation.isPresent() || associationAnnotation.isPresent())
+                     {
+                        ////////////////////////////////////////////////////////////
+                        // silently skip if marked as an association or an ignore //
+                        ////////////////////////////////////////////////////////////
+                     }
+                     else
+                     {
+                        LOG.debug("Skipping field without @QField annotation", logPair("class", c.getSimpleName()), logPair("fieldName", fieldName));
+                     }
                   }
                }
                else
@@ -360,6 +373,16 @@ public abstract class QRecordEntity
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static Optional<QIgnore> getQIgnoreAnnotation(Class<? extends QRecordEntity> c, String ignoreName)
+   {
+      return (getAnnotationOnField(c, QIgnore.class, ignoreName));
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -419,9 +442,9 @@ public abstract class QRecordEntity
          }
          else
          {
-            if(!method.getName().equals("getClass"))
+            if(!method.getName().equals("getClass") && method.getAnnotation(QIgnore.class) == null)
             {
-               LOG.debug("Method [" + method.getName() + "] looks like a getter, but its return type, [" + method.getReturnType() + "], isn't supported.");
+               LOG.debug("Method [" + method.getName() + "] in [" + method.getDeclaringClass().getSimpleName() + "] looks like a getter, but its return type, [" + method.getReturnType().getSimpleName() + "], isn't supported.");
             }
          }
       }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/data/QRecordEnum.java

@@ -145,7 +145,7 @@ public interface QRecordEnum
          {
             if(!method.getName().equals("getClass") && !method.getName().equals("getDeclaringClass") && !method.getName().equals("getPossibleValueId"))
             {
-               LOG.debug("Method [" + method.getName() + "] looks like a getter, but its return type, [" + method.getReturnType() + "], isn't supported.");
+               LOG.debug("Method [" + method.getName() + "] in [" + method.getDeclaringClass().getSimpleName() + "] looks like a getter, but its return type, [" + method.getReturnType().getSimpleName() + "], isn't supported.");
             }
          }
       }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducer.java

@@ -30,7 +30,7 @@ import com.kingsrook.qqq.backend.core.model.MetaDataProducerInterface;
  ** MetaDataProducerHelper, to put point at a package full of these, and populate
  ** your whole QInstance.
  *******************************************************************************/
-public abstract class MetaDataProducer<T extends TopLevelMetaDataInterface> implements MetaDataProducerInterface<T>
+public abstract class MetaDataProducer<T extends MetaDataProducerOutput> implements MetaDataProducerInterface<T>
 {
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducerHelper.java

@@ -30,6 +30,7 @@ import java.util.Comparator;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import com.google.common.collect.ImmutableSet;
 import com.google.common.reflect.ClassPath;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -50,6 +51,8 @@ public class MetaDataProducerHelper
    private static Map<Class<?>, Integer> comparatorValuesByType = new HashMap<>();
    private static Integer                defaultComparatorValue;
 
+   private static ImmutableSet<ClassPath.ClassInfo> topLevelClasses;
+
    static
    {
       ////////////////////////////////////////////////////////////////////////////////////////
@@ -70,8 +73,6 @@ public class MetaDataProducerHelper
       comparatorValuesByType.put(QAppMetaData.class, 23);
    }
 
-
-
    /*******************************************************************************
     ** Recursively find all classes in the given package, that implement MetaDataProducerInterface
     ** run them, and add their output to the given qInstance.
@@ -156,7 +157,7 @@ public class MetaDataProducerHelper
          {
             try
             {
-               TopLevelMetaDataInterface metaData = producer.produce(instance);
+               MetaDataProducerOutput metaData = producer.produce(instance);
                if(metaData != null)
                {
                   metaData.addSelfToInstance(instance);
@@ -186,7 +187,7 @@ public class MetaDataProducerHelper
       List<Class<?>> classes = new ArrayList<>();
       ClassLoader    loader  = Thread.currentThread().getContextClassLoader();
 
-      for(ClassPath.ClassInfo info : ClassPath.from(loader).getTopLevelClasses())
+      for(ClassPath.ClassInfo info : getTopLevelClasses(loader))
       {
          if(info.getName().startsWith(packageName))
          {
@@ -197,4 +198,29 @@ public class MetaDataProducerHelper
       return (classes);
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static ImmutableSet<ClassPath.ClassInfo> getTopLevelClasses(ClassLoader loader) throws IOException
+   {
+      if(topLevelClasses == null)
+      {
+         topLevelClasses = ClassPath.from(loader).getTopLevelClasses();
+      }
+
+      return (topLevelClasses);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void clearTopLevelClassCache()
+   {
+      topLevelClasses = null;
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducerMultiOutput.java

@@ -0,0 +1,101 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+
+
+/*******************************************************************************
+ ** Output object for a MetaDataProducer, which contains multiple meta-data
+ ** objects.
+ *******************************************************************************/
+public class MetaDataProducerMultiOutput implements MetaDataProducerOutput
+{
+   private List<MetaDataProducerOutput> contents;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void addSelfToInstance(QInstance instance)
+   {
+      for(MetaDataProducerOutput metaDataProducerOutput : CollectionUtils.nonNullList(contents))
+      {
+         metaDataProducerOutput.addSelfToInstance(instance);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void add(MetaDataProducerOutput metaDataProducerOutput)
+   {
+      if(contents == null)
+      {
+         contents = new ArrayList<>();
+      }
+      contents.add(metaDataProducerOutput);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MetaDataProducerMultiOutput with(MetaDataProducerOutput metaDataProducerOutput)
+   {
+      add(metaDataProducerOutput);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public <T extends MetaDataProducerOutput> List<T> getEach(Class<T> c)
+   {
+      List<T> rs = new ArrayList<>();
+
+      for(MetaDataProducerOutput content : contents)
+      {
+         if(content instanceof MetaDataProducerMultiOutput multiOutput)
+         {
+            rs.addAll(multiOutput.getEach(c));
+         }
+         else if(c.isInstance(content))
+         {
+            rs.add(c.cast(content));
+         }
+      }
+
+      return (rs);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/MetaDataProducerOutput.java

@@ -0,0 +1,40 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata;
+
+
+/*******************************************************************************
+ ** Interface to mark objects that can be produced by a MetaDataProducer.
+ **
+ ** These would usually be TopLevelMetaData objects (a table, a process, etc)
+ ** but can also be a MetaDataProducerMultiOutput, to produce multiple objects
+ ** from one producer.
+ *******************************************************************************/
+public interface MetaDataProducerOutput
+{
+
+   /*******************************************************************************
+    ** call the appropriate methods on a QInstance to add ourselves to it.
+    *******************************************************************************/
+   void addSelfToInstance(QInstance instance);
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/QInstance.java

@@ -46,6 +46,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.frontend.AppTreeNode;
 import com.kingsrook.qqq.backend.core.model.metadata.frontend.AppTreeNodeType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
@@ -78,6 +79,7 @@ public class QInstance
    private QAuthenticationMetaData                  authentication      = null;
    private QBrandingMetaData                        branding            = null;
    private Map<String, QAutomationProviderMetaData> automationProviders = new HashMap<>();
+   private Map<String, QMessagingProviderMetaData> messagingProviders = new HashMap<>();
 
    ////////////////////////////////////////////////////////////////////////////////////////////
    // Important to use LinkedHashmap here, to preserve the order in which entries are added. //
@@ -739,6 +741,53 @@ public class QInstance
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addMessagingProvider(QMessagingProviderMetaData messagingProvider)
+   {
+      String name = messagingProvider.getName();
+      if(this.messagingProviders.containsKey(name))
+      {
+         throw (new IllegalArgumentException("Attempted to add a second messagingProvider with name: " + name));
+      }
+      this.messagingProviders.put(name, messagingProvider);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QMessagingProviderMetaData getMessagingProvider(String name)
+   {
+      return (this.messagingProviders.get(name));
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for messagingProviders
+    **
+    *******************************************************************************/
+   public Map<String, QMessagingProviderMetaData> getMessagingProviders()
+   {
+      return messagingProviders;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for messagingProviders
+    **
+    *******************************************************************************/
+   public void setMessagingProviders(Map<String, QMessagingProviderMetaData> messagingProviders)
+   {
+      this.messagingProviders = messagingProviders;
+   }
+
+
+
    /*******************************************************************************
     ** Getter for hasBeenValidated
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/TopLevelMetaDataInterface.java

@@ -26,7 +26,7 @@ package com.kingsrook.qqq.backend.core.model.metadata;
  ** Interface for meta-data classes that can be added directly (e.g, at the top
  ** level) to a QInstance (such as a QTableMetaData - not a QFieldMetaData).
  *******************************************************************************/
-public interface TopLevelMetaDataInterface
+public interface TopLevelMetaDataInterface extends MetaDataProducerOutput
 {
 
    /*******************************************************************************

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/QMessagingProviderMetaData.java

@@ -0,0 +1,110 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.TopLevelMetaDataInterface;
+
+
+/*******************************************************************************
+ ** Base class for qqq messaging-providers.  e.g., a connection to an outbound
+ ** email service, or, for example, Slack.
+ *******************************************************************************/
+public class QMessagingProviderMetaData implements TopLevelMetaDataInterface
+{
+   private String name;
+   private String type;
+
+
+
+   /*******************************************************************************
+    ** Getter for name
+    *******************************************************************************/
+   public String getName()
+   {
+      return (this.name);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for name
+    *******************************************************************************/
+   public void setName(String name)
+   {
+      this.name = name;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for name
+    *******************************************************************************/
+   public QMessagingProviderMetaData withName(String name)
+   {
+      this.name = name;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for type
+    *******************************************************************************/
+   public String getType()
+   {
+      return (this.type);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for type
+    *******************************************************************************/
+   public void setType(String type)
+   {
+      this.type = type;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for type
+    *******************************************************************************/
+   public QMessagingProviderMetaData withType(String type)
+   {
+      this.type = type;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void addSelfToInstance(QInstance qInstance)
+   {
+      qInstance.addMessagingProvider(this);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/email/EmailMessagingProvider.java

@@ -0,0 +1,56 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.email;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+import com.kingsrook.qqq.backend.core.modules.messaging.MessagingProviderInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class EmailMessagingProvider implements MessagingProviderInterface
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getType()
+   {
+      return (EmailMessagingProviderMetaData.TYPE);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public SendMessageOutput sendMessage(SendMessageInput sendMessageInput) throws QException
+   {
+      return new SendEmailAction().sendMessage(sendMessageInput);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/email/EmailMessagingProviderMetaData.java

@@ -0,0 +1,116 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.email;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
+import com.kingsrook.qqq.backend.core.modules.messaging.QMessagingProviderDispatcher;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class EmailMessagingProviderMetaData extends QMessagingProviderMetaData
+{
+   private String smtpServer;
+   private String smtpPort;
+
+   public static final String TYPE = "EMAIL";
+
+   static
+   {
+      QMessagingProviderDispatcher.registerMessagingProvider(new EmailMessagingProvider());
+   }
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public EmailMessagingProviderMetaData()
+   {
+      super();
+      setType(TYPE);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for smtpServer
+    *******************************************************************************/
+   public String getSmtpServer()
+   {
+      return (this.smtpServer);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for smtpServer
+    *******************************************************************************/
+   public void setSmtpServer(String smtpServer)
+   {
+      this.smtpServer = smtpServer;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for smtpServer
+    *******************************************************************************/
+   public EmailMessagingProviderMetaData withSmtpServer(String smtpServer)
+   {
+      this.smtpServer = smtpServer;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for smtpPort
+    *******************************************************************************/
+   public String getSmtpPort()
+   {
+      return (this.smtpPort);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for smtpPort
+    *******************************************************************************/
+   public void setSmtpPort(String smtpPort)
+   {
+      this.smtpPort = smtpPort;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for smtpPort
+    *******************************************************************************/
+   public EmailMessagingProviderMetaData withSmtpPort(String smtpPort)
+   {
+      this.smtpPort = smtpPort;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/email/SendEmailAction.java

@@ -0,0 +1,216 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.email;
+
+
+import java.io.UnsupportedEncodingException;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Properties;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Content;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.MultiParty;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Party;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.PartyRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailContentRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailPartyRole;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import jakarta.mail.Address;
+import jakarta.mail.Message;
+import jakarta.mail.Multipart;
+import jakarta.mail.Session;
+import jakarta.mail.Transport;
+import jakarta.mail.internet.AddressException;
+import jakarta.mail.internet.InternetAddress;
+import jakarta.mail.internet.MimeBodyPart;
+import jakarta.mail.internet.MimeMessage;
+import jakarta.mail.internet.MimeMultipart;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SendEmailAction
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public SendMessageOutput sendMessage(SendMessageInput sendMessageInput) throws QException
+   {
+      EmailMessagingProviderMetaData messagingProvider = (EmailMessagingProviderMetaData) QContext.getQInstance().getMessagingProvider(sendMessageInput.getMessagingProviderName());
+
+      /////////////////////////////////////////
+      // set up properties to make a session //
+      /////////////////////////////////////////
+      Properties properties = new Properties();
+      properties.setProperty("mail.smtp.host", messagingProvider.getSmtpServer());
+      properties.setProperty("mail.smtp.port", messagingProvider.getSmtpPort());
+      Session session = Session.getInstance(properties);
+
+      try
+      {
+         ////////////////////////////////////////////
+         // Construct a default MimeMessage object //
+         ////////////////////////////////////////////
+         MimeMessage emailMessage = new MimeMessage(session);
+         emailMessage.setSubject(sendMessageInput.getSubject());
+
+         Party to = sendMessageInput.getTo();
+         if(to instanceof MultiParty toMultiParty)
+         {
+            for(Party party : toMultiParty.getPartyList())
+            {
+               addRecipient(emailMessage, party);
+            }
+         }
+         else
+         {
+            addRecipient(emailMessage, to);
+         }
+
+         Party from = sendMessageInput.getFrom();
+         if(from instanceof MultiParty fromMultiParty)
+         {
+            for(Party party : fromMultiParty.getPartyList())
+            {
+               addSender(emailMessage, party);
+            }
+         }
+         else
+         {
+            addSender(emailMessage, from);
+         }
+
+         Multipart multipart = new MimeMultipart();
+         for(Content content : sendMessageInput.getContentList())
+         {
+            if(EmailContentRole.HTML.equals(content.getContentRole()))
+            {
+               MimeBodyPart mimeBodyPart = new MimeBodyPart();
+               mimeBodyPart.setContent(content.getBody(), "text/html; charset=utf-8");
+               multipart.addBodyPart(mimeBodyPart);
+            }
+            else if(EmailContentRole.TEXT.equals(content.getContentRole()))
+            {
+               MimeBodyPart mimeBodyPart = new MimeBodyPart();
+               mimeBodyPart.setContent(content.getBody(), "text/plain; charset=utf-8");
+               multipart.addBodyPart(mimeBodyPart);
+            }
+         }
+
+         emailMessage.setContent(multipart);
+
+         /////////////
+         // send it //
+         /////////////
+         Transport.send(emailMessage);
+         System.out.println("Message dispatched successfully...");
+      }
+      catch(Exception e)
+      {
+         throw (new QException("Error sending email", e));
+      }
+
+      return null;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void addSender(MimeMessage emailMessage, Party party) throws Exception
+   {
+      if(EmailPartyRole.REPLY_TO.equals(party.getRole()))
+      {
+         InternetAddress internetAddress = getInternetAddressFromParty(party);
+         Address[]       replyTo         = emailMessage.getReplyTo();
+         if(replyTo == null || replyTo.length == 0)
+         {
+            emailMessage.setReplyTo(new Address[] { internetAddress });
+         }
+         else
+         {
+            List<Address> replyToList = Arrays.asList(replyTo);
+            emailMessage.setReplyTo(replyToList.toArray(new Address[0]));
+         }
+      }
+      else if(party.getRole() == null || PartyRole.Default.DEFAULT.equals(party.getRole()) || EmailPartyRole.FROM.equals(party.getRole()))
+      {
+         emailMessage.setFrom(getInternetAddressFromParty(party));
+      }
+      else
+      {
+         throw (new QException("Unrecognized sender role [" + party.getRole() + "]"));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void addRecipient(MimeMessage emailMessage, Party party) throws Exception
+   {
+      Message.RecipientType recipientType;
+      if(EmailPartyRole.CC.equals(party.getRole()))
+      {
+         recipientType = Message.RecipientType.CC;
+      }
+      else if(EmailPartyRole.BCC.equals(party.getRole()))
+      {
+         recipientType = Message.RecipientType.BCC;
+      }
+      else if(party.getRole() == null || PartyRole.Default.DEFAULT.equals(party.getRole()) || EmailPartyRole.TO.equals(party.getRole()))
+      {
+         recipientType = Message.RecipientType.TO;
+      }
+      else
+      {
+         throw (new QException("Unrecognized recipient role [" + party.getRole() + "]"));
+      }
+
+      InternetAddress internetAddress = getInternetAddressFromParty(party);
+      emailMessage.addRecipient(recipientType, internetAddress);
+      System.out.println("add recipient: [" + recipientType + "] => [" + internetAddress + "]");
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static InternetAddress getInternetAddressFromParty(Party party) throws AddressException, UnsupportedEncodingException
+   {
+      InternetAddress internetAddress = new InternetAddress(party.getAddress());
+      if(StringUtils.hasContent(party.getLabel()))
+      {
+         internetAddress.setPersonal(party.getLabel());
+      }
+      return internetAddress;
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/ses/SESMessagingProvider.java

@@ -0,0 +1,56 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.ses;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+import com.kingsrook.qqq.backend.core.modules.messaging.MessagingProviderInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SESMessagingProvider implements MessagingProviderInterface
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getType()
+   {
+      return (SESMessagingProviderMetaData.TYPE);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public SendMessageOutput sendMessage(SendMessageInput sendMessageInput) throws QException
+   {
+      return new SendSESAction().sendMessage(sendMessageInput);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/ses/SESMessagingProviderMetaData.java

@@ -0,0 +1,148 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.ses;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
+import com.kingsrook.qqq.backend.core.modules.messaging.QMessagingProviderDispatcher;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SESMessagingProviderMetaData extends QMessagingProviderMetaData
+{
+   private String accessKey;
+   private String secretKey;
+   private String region;
+
+   public static final String TYPE = "SES";
+
+   static
+   {
+      QMessagingProviderDispatcher.registerMessagingProvider(new SESMessagingProvider());
+   }
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public SESMessagingProviderMetaData()
+   {
+      super();
+      setType(TYPE);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for accessKey
+    *******************************************************************************/
+   public String getAccessKey()
+   {
+      return (this.accessKey);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for accessKey
+    *******************************************************************************/
+   public void setAccessKey(String accessKey)
+   {
+      this.accessKey = accessKey;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for accessKey
+    *******************************************************************************/
+   public SESMessagingProviderMetaData withAccessKey(String accessKey)
+   {
+      this.accessKey = accessKey;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for secretKey
+    *******************************************************************************/
+   public String getSecretKey()
+   {
+      return (this.secretKey);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for secretKey
+    *******************************************************************************/
+   public void setSecretKey(String secretKey)
+   {
+      this.secretKey = secretKey;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for secretKey
+    *******************************************************************************/
+   public SESMessagingProviderMetaData withSecretKey(String secretKey)
+   {
+      this.secretKey = secretKey;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for region
+    *******************************************************************************/
+   public String getRegion()
+   {
+      return (this.region);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for region
+    *******************************************************************************/
+   public void setRegion(String region)
+   {
+      this.region = region;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for region
+    *******************************************************************************/
+   public SESMessagingProviderMetaData withRegion(String region)
+   {
+      this.region = region;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/messaging/ses/SendSESAction.java

@@ -0,0 +1,335 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.metadata.messaging.ses;
+
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import com.amazonaws.auth.AWSStaticCredentialsProvider;
+import com.amazonaws.auth.BasicAWSCredentials;
+import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
+import com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClientBuilder;
+import com.amazonaws.services.simpleemail.model.Body;
+import com.amazonaws.services.simpleemail.model.Content;
+import com.amazonaws.services.simpleemail.model.Destination;
+import com.amazonaws.services.simpleemail.model.Message;
+import com.amazonaws.services.simpleemail.model.SendEmailRequest;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.MultiParty;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Party;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.PartyRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailContentRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailPartyRole;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SendSESAction
+{
+   private static final QLogger LOG = QLogger.getLogger(SendSESAction.class);
+
+   private AmazonSimpleEmailService amazonSES;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public SendMessageOutput sendMessage(SendMessageInput sendMessageInput) throws QException
+   {
+      try
+      {
+         AmazonSimpleEmailService client = getAmazonSES(sendMessageInput);
+
+         ///////////////////////////////////
+         // build up a send email request //
+         ///////////////////////////////////
+         SendEmailRequest request = new SendEmailRequest()
+            .withSource(getSource(sendMessageInput))
+            .withReplyToAddresses(getReplyTos(sendMessageInput))
+            .withDestination(buildDestination(sendMessageInput))
+            .withMessage(buildMessage(sendMessageInput));
+
+         client.sendEmail(request);
+         LOG.info("SES Message [" + request.getMessage().getSubject().getData() + "] was sent to [" + request.getDestination().toString() + "].");
+      }
+      catch(Exception e)
+      {
+         String message = "An unexpected error occurred sending an SES message.";
+         throw (new QException(message, e));
+      }
+
+      return (null);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   Message buildMessage(SendMessageInput input) throws QException
+   {
+      ///////////////////////////////////////////////////////////////////////////////////////////////
+      // iterate over all contents of our input, looking for an HTML and Text version of the email //
+      ///////////////////////////////////////////////////////////////////////////////////////////////
+      Body body = new Body();
+      for(com.kingsrook.qqq.backend.core.model.actions.messaging.Content content : CollectionUtils.nonNullList(input.getContentList()))
+      {
+         if(EmailContentRole.TEXT.equals(content.getContentRole()))
+         {
+            body.setText(new Content().withCharset("UTF-8").withData(content.getBody()));
+         }
+         else if(EmailContentRole.HTML.equals(content.getContentRole()))
+         {
+            body.setHtml(new Content().withCharset("UTF-8").withData(content.getBody()));
+         }
+      }
+
+      ////////////////////////////////////////////////
+      // error if no text or html body was provided //
+      ////////////////////////////////////////////////
+      if(body.getText() == null && body.getHtml() == null)
+      {
+         throw (new QException("Cannot send SES message because neither a 'Text' nor an 'HTML' body was provided."));
+      }
+
+      ////////////////////////////////////////
+      // warning if no subject was provided //
+      ////////////////////////////////////////
+      Message message = new Message();
+      message.setBody(body);
+
+      /////////////////////////////////////
+      // warn if no subject was provided //
+      /////////////////////////////////////
+      if(input.getSubject() == null)
+      {
+         LOG.warn("Sending SES message with no subject.");
+      }
+      else
+      {
+         message.setSubject(new Content().withCharset("UTF-8").withData(input.getSubject()));
+      }
+
+      return (message);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   List<String> getReplyTos(SendMessageInput input) throws QException
+   {
+      ////////////////////////////
+      // no input, no reply tos //
+      ////////////////////////////
+      if(input == null)
+      {
+         return (Collections.emptyList());
+      }
+
+      ///////////////////////////////////////
+      // build up a list of froms if multi //
+      ///////////////////////////////////////
+      List<Party> partyList = getPartyListFromParty(input.getFrom());
+      if(partyList == null)
+      {
+         return (Collections.emptyList());
+      }
+
+      ///////////////////////////////
+      // only get reply to parties //
+      ///////////////////////////////
+      List<Party> replyToParties = partyList.stream().filter(p -> EmailPartyRole.REPLY_TO.equals(p.getRole())).toList();
+
+      //////////////////////////////////
+      // get addresses from reply tos //
+      //////////////////////////////////
+      List<String> replyTos = replyToParties.stream().map(Party::getAddress).toList();
+
+      /////////////////////////////
+      // return the from address //
+      /////////////////////////////
+      return (replyTos);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   String getSource(SendMessageInput input) throws QException
+   {
+      ///////////////////////////////
+      // error if no from provided //
+      ///////////////////////////////
+      if(input.getFrom() == null)
+      {
+         throw (new QException("Cannot send SES message because a FROM was not provided."));
+      }
+
+      ///////////////////////////////////////
+      // build up a list of froms if multi //
+      ///////////////////////////////////////
+      List<Party> partyList = getPartyListFromParty(input.getFrom());
+
+      ///////////////////////////////////////
+      // remove any roles that aren't FROM //
+      ///////////////////////////////////////
+      partyList.removeIf(p -> p.getRole() != null && !EmailPartyRole.FROM.equals(p.getRole()));
+
+      ///////////////////////////////////////////////////////////////////////////////////////////
+      // if no froms found, error, if more than one found, log a warning and use the first one //
+      ///////////////////////////////////////////////////////////////////////////////////////////
+      if(partyList.isEmpty())
+      {
+         throw (new QException("Cannot send SES message because a FROM was not provided."));
+      }
+      else if(partyList.size() > 1)
+      {
+         LOG.warn("More than one FROM value was found, will send using the first one found [" + partyList.get(0).getAddress() + "].");
+      }
+
+      /////////////////////////////
+      // return the from address //
+      /////////////////////////////
+      return (partyList.get(0).getAddress());
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   List<Party> getPartyListFromParty(Party party)
+   {
+      //////////////////////////////////////////////
+      // get all parties into one list of parties //
+      //////////////////////////////////////////////
+      List<Party> partyList = new ArrayList<>();
+      if(party != null)
+      {
+         if(party instanceof MultiParty toMultiParty)
+         {
+            partyList.addAll(toMultiParty.getPartyList());
+         }
+         else
+         {
+            partyList.add(party);
+         }
+      }
+      return (partyList);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   Destination buildDestination(SendMessageInput input) throws QException
+   {
+      ////////////////////////////////////////////////////////////////////
+      // iterate over the parties putting it the proper party type list //
+      ////////////////////////////////////////////////////////////////////
+      List<String> toList  = new ArrayList<>();
+      List<String> ccList  = new ArrayList<>();
+      List<String> bccList = new ArrayList<>();
+
+      List<Party> partyList = getPartyListFromParty(input.getTo());
+      for(Party party : partyList)
+      {
+         if(EmailPartyRole.CC.equals(party.getRole()))
+         {
+            ccList.add(party.getAddress());
+         }
+         else if(EmailPartyRole.BCC.equals(party.getRole()))
+         {
+            bccList.add(party.getAddress());
+         }
+         else if(party.getRole() == null || PartyRole.Default.DEFAULT.equals(party.getRole()) || EmailPartyRole.TO.equals(party.getRole()))
+         {
+            toList.add(party.getAddress());
+         }
+         else
+         {
+            throw (new QException("An unrecognized recipient role of [" + party.getRole() + "] was provided."));
+         }
+      }
+
+      //////////////////////////////////////////
+      // if no to addresses, this is an error //
+      //////////////////////////////////////////
+      if(toList.isEmpty())
+      {
+         throw (new QException("Cannot send SES message because no TO addresses were provided."));
+      }
+
+      /////////////////////////////////////////////
+      // build and return aws destination object //
+      /////////////////////////////////////////////
+      return (new Destination()
+         .withToAddresses(toList)
+         .withCcAddresses(ccList)
+         .withBccAddresses(bccList));
+   }
+
+
+
+   /*******************************************************************************
+    ** Set the amazonSES object.
+    *******************************************************************************/
+   public void setAmazonSES(AmazonSimpleEmailService amazonSES)
+   {
+      this.amazonSES = amazonSES;
+   }
+
+
+
+   /*******************************************************************************
+    ** Internal accessor for the amazonSES object - should always use this, not the field.
+    *******************************************************************************/
+   protected AmazonSimpleEmailService getAmazonSES(SendMessageInput sendMessageInput)
+   {
+      if(amazonSES == null)
+      {
+         SESMessagingProviderMetaData messagingProvider = (SESMessagingProviderMetaData) QContext.getQInstance().getMessagingProvider(sendMessageInput.getMessagingProviderName());
+
+         /////////////////////////////////////////////
+         // get credentials and build an SES client //
+         /////////////////////////////////////////////
+         BasicAWSCredentials credentials = new BasicAWSCredentials(messagingProvider.getAccessKey(), messagingProvider.getSecretKey());
+         amazonSES = AmazonSimpleEmailServiceClientBuilder.standard()
+            .withCredentials(new AWSStaticCredentialsProvider(credentials))
+            .withRegion(messagingProvider.getRegion()).build();
+      }
+
+      return amazonSES;
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QComponentType.java

@@ -36,6 +36,7 @@ public enum QComponentType
    RECORD_LIST,
    PROCESS_SUMMARY_RESULTS,
    GOOGLE_DRIVE_SELECT_FOLDER,
+   WIDGET,
    HTML;
    ///////////////////////////////////////////////////////////////////////////
    // keep these values in sync with QComponentType.ts in qqq-frontend-core //

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QFrontendStepMetaData.java

@@ -26,8 +26,12 @@ import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
 import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.kingsrook.qqq.backend.core.instances.QInstanceHelpContentManager;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
+import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
 
 
 /*******************************************************************************
@@ -43,6 +47,8 @@ public class QFrontendStepMetaData extends QStepMetaData
    private List<QFieldMetaData>             recordListFields;
    private Map<String, QFieldMetaData>      formFieldMap;
 
+   private List<QHelpContent> helpContents;
+
 
 
    /*******************************************************************************
@@ -340,4 +346,61 @@ public class QFrontendStepMetaData extends QStepMetaData
       return (rs);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for helpContents
+    *******************************************************************************/
+   public List<QHelpContent> getHelpContents()
+   {
+      return (this.helpContents);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for helpContents
+    *******************************************************************************/
+   public void setHelpContents(List<QHelpContent> helpContents)
+   {
+      this.helpContents = helpContents;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for helpContents
+    *******************************************************************************/
+   public QFrontendStepMetaData withHelpContents(List<QHelpContent> helpContents)
+   {
+      this.helpContents = helpContents;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for adding 1 helpContent
+    *******************************************************************************/
+   public QFrontendStepMetaData withHelpContent(QHelpContent helpContent)
+   {
+      if(this.helpContents == null)
+      {
+         this.helpContents = new ArrayList<>();
+      }
+
+      QInstanceHelpContentManager.putHelpContentInList(helpContent, this.helpContents);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** remove a single helpContent based on its set of roles
+    *******************************************************************************/
+   public void removeHelpContent(Set<HelpRole> roles)
+   {
+      QInstanceHelpContentManager.removeHelpContentByRoleSetFromList(roles, this.helpContents);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/processes/QProcessMetaData.java

@@ -60,6 +60,8 @@ public class QProcessMetaData implements QAppChildMetaData, MetaDataWithPermissi
    private List<QStepMetaData>        stepList; // these are the steps that are ran, by-default, in the order they are ran in
    private Map<String, QStepMetaData> steps; // this is the full map of possible steps
 
+   private QBackendStepMetaData cancelStep;
+
    private QIcon icon;
 
    private QScheduleMetaData schedule;
@@ -675,6 +677,7 @@ public class QProcessMetaData implements QAppChildMetaData, MetaDataWithPermissi
    }
 
 
+
    /*******************************************************************************
     ** Getter for variantRunStrategy
     *******************************************************************************/
@@ -736,4 +739,45 @@ public class QProcessMetaData implements QAppChildMetaData, MetaDataWithPermissi
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for the full map of all steps (not the step list!)
+    **
+    *******************************************************************************/
+   public Map<String, QStepMetaData> getAllSteps()
+   {
+      return steps;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for cancelStep
+    *******************************************************************************/
+   public QBackendStepMetaData getCancelStep()
+   {
+      return (this.cancelStep);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for cancelStep
+    *******************************************************************************/
+   public void setCancelStep(QBackendStepMetaData cancelStep)
+   {
+      this.cancelStep = cancelStep;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for cancelStep
+    *******************************************************************************/
+   public QProcessMetaData withCancelStep(QBackendStepMetaData cancelStep)
+   {
+      this.cancelStep = cancelStep;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/metadata/tables/QTableMetaData.java

@@ -111,6 +111,7 @@ public class QTableMetaData implements QAppChildMetaData, Serializable, MetaData
    private ShareableTableMetaData shareableTableMetaData;
 
 
+
    /*******************************************************************************
     ** Default constructor.
     *******************************************************************************/
@@ -158,11 +159,26 @@ public class QTableMetaData implements QAppChildMetaData, Serializable, MetaData
    public QTableMetaData withFieldsFromEntity(Class<? extends QRecordEntity> entityClass) throws QException
    {
       List<QRecordEntityField> recordEntityFieldList = QRecordEntity.getFieldList(entityClass);
+
+      boolean setPrimaryKey = false;
+
       for(QRecordEntityField recordEntityField : recordEntityFieldList)
       {
          QFieldMetaData field = new QFieldMetaData(recordEntityField.getGetter());
          addField(field);
+
+         if(recordEntityField.getFieldAnnotation().isPrimaryKey())
+         {
+            if(setPrimaryKey)
+            {
+               throw (new QException("Attempt to set more than one field as primary key (" + primaryKeyField + "," + field.getName() + ")."));
+            }
+
+            setPrimaryKeyField(field.getName());
+            setPrimaryKey = true;
+         }
       }
+
       return (this);
    }
 
@@ -624,6 +640,18 @@ public class QTableMetaData implements QAppChildMetaData, Serializable, MetaData
 
 
 
+   /*******************************************************************************
+    ** fluent setter for both recordLabelFormat and recordLabelFields
+    *******************************************************************************/
+   public QTableMetaData withRecordLabelFormatAndFields(String format, String... fields)
+   {
+      setRecordLabelFormat(format);
+      setRecordLabelFields(Arrays.asList(fields));
+      return (this);
+   }
+
+
+
    /*******************************************************************************
     ** Getter for recordLabelFields
     **
@@ -1388,6 +1416,7 @@ public class QTableMetaData implements QAppChildMetaData, Serializable, MetaData
    }
 
 
+
    /*******************************************************************************
     ** Getter for shareableTableMetaData
     *******************************************************************************/
@@ -1417,5 +1446,4 @@ public class QTableMetaData implements QAppChildMetaData, Serializable, MetaData
       return (this);
    }
 
-
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/querystats/QueryStat.java

@@ -28,6 +28,7 @@ import java.util.Set;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.data.QAssociation;
 import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QIgnore;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
@@ -77,7 +78,9 @@ public class QueryStat extends QRecordEntity
    ///////////////////////////////////////////////////////////
    // non-persistent fields - used to help build the record //
    ///////////////////////////////////////////////////////////
-   private String       tableName;
+   @QIgnore
+   private String tableName;
+
    private Set<String>  joinTableNames;
    private QQueryFilter queryFilter;
 
@@ -384,6 +387,7 @@ public class QueryStat extends QRecordEntity
    /*******************************************************************************
     ** Getter for queryFilter
     *******************************************************************************/
+   @QIgnore
    public QQueryFilter getQueryFilter()
    {
       return (this.queryFilter);
@@ -446,6 +450,7 @@ public class QueryStat extends QRecordEntity
    /*******************************************************************************
     ** Getter for joinTableNames
     *******************************************************************************/
+   @QIgnore
    public Set<String> getJoinTableNames()
    {
       return (this.joinTableNames);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/ReportValuesDynamicFormWidgetRenderer.java

@@ -0,0 +1,205 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.savedreports;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
+import com.kingsrook.qqq.backend.core.actions.reporting.GenerateReportAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.values.QPossibleValueTranslator;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.expressions.FilterVariableExpression;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.DynamicFormWidgetData;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.savedreports.SavedReportToReportMetaDataAdapter;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import org.json.JSONObject;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Note - exists under 2 names, for the RenderSavedReport process, and for the
+ ** ScheduledReport table
+ *******************************************************************************/
+public class ReportValuesDynamicFormWidgetRenderer extends AbstractWidgetRenderer
+{
+   private static final QLogger LOG = QLogger.getLogger(ReportValuesDynamicFormWidgetRenderer.class);
+
+   private QPossibleValueTranslator qPossibleValueTranslator;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+   {
+      try
+      {
+         List<QFieldMetaData> fieldList     = new ArrayList<>();
+         Map<String, String>  defaultValues = new HashMap<>();
+
+         //////////////////////////////////////////////////////////////////////////////
+         // read params to ultimately find the query filter that has variables in it //
+         //////////////////////////////////////////////////////////////////////////////
+         SavedReport savedReport = null;
+         if(input.getQueryParams().containsKey("savedReportId"))
+         {
+            QRecord record = new GetAction().executeForRecord(new GetInput(SavedReport.TABLE_NAME).withPrimaryKey(ValueUtils.getValueAsInteger(input.getQueryParams().get("savedReportId"))));
+            savedReport = new SavedReport(record);
+         }
+         else if(input.getQueryParams().containsKey("id"))
+         {
+            QRecord scheduledReportRecord = new GetAction().executeForRecord(new GetInput(ScheduledReport.TABLE_NAME).withPrimaryKey(ValueUtils.getValueAsInteger(input.getQueryParams().get("id"))));
+            QRecord record                = new GetAction().executeForRecord(new GetInput(SavedReport.TABLE_NAME).withPrimaryKey(ValueUtils.getValueAsInteger(scheduledReportRecord.getValueInteger("savedReportId"))));
+            savedReport = new SavedReport(record);
+
+            String inputValues = scheduledReportRecord.getValueString("inputValues");
+            if(StringUtils.hasContent(inputValues))
+            {
+               JSONObject jsonObject = JsonUtils.toJSONObject(inputValues);
+               for(String key : jsonObject.keySet())
+               {
+                  defaultValues.put(key, jsonObject.optString(key));
+               }
+            }
+         }
+         else
+         {
+            //////////////////////////////////
+            // return quietly w/ nothing... //
+            //////////////////////////////////
+            DynamicFormWidgetData widgetData = new DynamicFormWidgetData();
+            return new RenderWidgetOutput(widgetData);
+         }
+
+         QRecord recordOfFieldValues = new QRecord();
+
+         if(StringUtils.hasContent(savedReport.getQueryFilterJson()))
+         {
+            QQueryFilter   queryFilter = SavedReportToReportMetaDataAdapter.getQQueryFilter(savedReport.getQueryFilterJson());
+            QTableMetaData table       = QContext.getQInstance().getTable(savedReport.getTableName());
+
+            ///////////////////////////////////////////////////////////////////////////////////////////////
+            // find variables in the query filter; convert them to a list of fields for the dynamic form //
+            ///////////////////////////////////////////////////////////////////////////////////////////////
+            for(QFilterCriteria criteria : CollectionUtils.nonNullList(queryFilter.getCriteria()))
+            {
+               for(Serializable criteriaValue : CollectionUtils.nonNullList(criteria.getValues()))
+               {
+                  if(criteriaValue instanceof FilterVariableExpression filterVariableExpression)
+                  {
+                     GenerateReportAction.FieldAndJoinTable fieldAndJoinTable = GenerateReportAction.getFieldAndJoinTable(table, criteria.getFieldName());
+                     QFieldMetaData                         fieldMetaData     = fieldAndJoinTable.field().clone();
+
+                     /////////////////////////////////
+                     // make name & label for field //
+                     /////////////////////////////////
+                     String fieldName = filterVariableExpression.getVariableName();
+                     fieldMetaData.setName(fieldName);
+                     fieldMetaData.setLabel(QInstanceEnricher.nameToLabel(filterVariableExpression.getVariableName()));
+
+                     ////////////////////////////////////////////////////////////
+                     // in this use case, every field is required and editable //
+                     ////////////////////////////////////////////////////////////
+                     fieldMetaData.setIsRequired(true);
+                     fieldMetaData.setIsEditable(true);
+
+                     ///////////////////////////////////////////////////////////////////////
+                     // if we're in a context where there are values, then populate those //
+                     // e.g., a view screen instead of an edit screen, i think            //
+                     ///////////////////////////////////////////////////////////////////////
+                     if(defaultValues.containsKey(fieldName))
+                     {
+                        String value = defaultValues.get(fieldName);
+
+                        fieldMetaData.setDefaultValue(value);
+                        recordOfFieldValues.setValue(fieldName, value);
+
+                        //////////////////////////////////////////////////////
+                        // look up display values for possible value fields //
+                        //////////////////////////////////////////////////////
+                        if(StringUtils.hasContent(fieldMetaData.getPossibleValueSourceName()))
+                        {
+                           if(qPossibleValueTranslator == null)
+                           {
+                              qPossibleValueTranslator = new QPossibleValueTranslator();
+                           }
+                           String displayValue = qPossibleValueTranslator.translatePossibleValue(fieldMetaData, value);
+                           recordOfFieldValues.setDisplayValue(fieldName, displayValue);
+                        }
+                     }
+
+                     fieldList.add(fieldMetaData);
+                  }
+               }
+            }
+         }
+
+         ///////////////////////////////////
+         // make output object and return //
+         ///////////////////////////////////
+         DynamicFormWidgetData widgetData = new DynamicFormWidgetData();
+         widgetData.setFieldList(fieldList);
+         widgetData.setRecordOfFieldValues(recordOfFieldValues);
+         widgetData.setMergedDynamicFormValuesIntoFieldName("inputValues");
+
+         if(CollectionUtils.nullSafeIsEmpty(fieldList))
+         {
+            ///////////////////////////////////////////////
+            // actually don't show this for process mode //
+            ///////////////////////////////////////////////
+            if(!input.getQueryParams().containsKey("processName"))
+            {
+               widgetData.setNoFieldsMessage("This Report does not use any Variable Values");
+            }
+         }
+
+         return new RenderWidgetOutput(widgetData);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error rendering scheduled report values dynamic form widget", e, logPair("queryParams", String.valueOf(input.getQueryParams())));
+         throw (new QException("Error rendering scheduled report values dynamic form widget", e));
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/SavedReportTableCustomizer.java

@@ -34,13 +34,18 @@ import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.pivottable.PivotTableDefinition;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.pivottable.PivotTableGroupBy;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.pivottable.PivotTableValue;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+import com.kingsrook.qqq.backend.core.model.statusmessages.PermissionDeniedMessage;
 import com.kingsrook.qqq.backend.core.processes.implementations.savedreports.SavedReportToReportMetaDataAdapter;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.ObjectUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -67,11 +72,45 @@ public class SavedReportTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
    {
+      validateOwner(records, SavedReport.TABLE_NAME, "edit");
       return (preInsertOrUpdate(records));
    }
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      validateOwner(records, SavedReport.TABLE_NAME, "delete");
+      return (preInsertOrUpdate(records));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void validateOwner(List<QRecord> records, String tableName, String verb)
+   {
+      QTableMetaData tableMetaData = QContext.getQInstance().getTable(tableName);
+      String         currentUserId = ObjectUtils.tryElse(() -> QContext.getQSession().getUser().getIdReference(), null);
+      for(QRecord record : records)
+      {
+         if(record.getValue("userId") != null)
+         {
+            if(!record.getValue("userId").equals(currentUserId))
+            {
+               record.addError(new PermissionDeniedMessage("Only the owner of a " + tableMetaData.getLabel() + " may " + verb + " it."));
+            }
+         }
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -110,10 +149,12 @@ public class SavedReportTableCustomizer implements TableCustomizerInterface
          {
             try
             {
-               ////////////////////////////////////////////////////////////////
-               // nothing to validate on filter, other than, we can parse it //
-               ////////////////////////////////////////////////////////////////
-               SavedReportToReportMetaDataAdapter.getQQueryFilter(queryFilterJson);
+               /////////////////////////////////////////////////////////////////////////
+               // validate that we can parse the filter, then prep it for the backend //
+               /////////////////////////////////////////////////////////////////////////
+               QQueryFilter filter = SavedReportToReportMetaDataAdapter.getQQueryFilter(queryFilterJson);
+               filter.prepForBackend();
+               record.setValue("queryFilterJson", JsonUtils.toJson(filter));
             }
             catch(IOException e)
             {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/SavedReportsMetaDataProvider.java

@@ -25,9 +25,11 @@ package com.kingsrook.qqq.backend.core.model.savedreports;
 import java.util.List;
 import java.util.function.Consumer;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.DefaultWidgetRenderer;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormatPossibleValueEnum;
+import com.kingsrook.qqq.backend.core.model.common.TimeZonePossibleValueSourceMetaDataProvider;
 import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
@@ -54,6 +56,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
 import com.kingsrook.qqq.backend.core.processes.implementations.savedreports.RenderSavedReportMetaDataProducer;
+import com.kingsrook.qqq.backend.core.processes.implementations.savedreports.RunScheduledReportMetaDataProducer;
 
 
 /*******************************************************************************
@@ -63,8 +66,13 @@ public class SavedReportsMetaDataProvider
 {
    public static final String REPORT_STORAGE_TABLE_NAME = "reportStorage";
 
+   public static final String SAVED_REPORT_JOIN_SCHEDULED_REPORT    = "scheduledReportJoinSavedReport";
    public static final String SHARED_SAVED_REPORT_JOIN_SAVED_REPORT = "sharedSavedReportJoinSavedReport";
 
+   public static final String SCHEDULED_REPORT_VALUES_WIDGET      = "scheduledReportValuesWidget";
+   public static final String RENDER_REPORT_PROCESS_VALUES_WIDGET = "renderReportProcessValuesWidget";
+
+
 
    /*******************************************************************************
     **
@@ -73,6 +81,7 @@ public class SavedReportsMetaDataProvider
    {
       instance.addTable(defineSavedReportTable(recordTablesBackendName, backendDetailEnricher));
       instance.addTable(defineRenderedReportTable(recordTablesBackendName, backendDetailEnricher));
+
       instance.addPossibleValueSource(QPossibleValueSource.newForTable(SavedReport.TABLE_NAME));
       instance.addPossibleValueSource(QPossibleValueSource.newForEnum(ReportFormatPossibleValueEnum.NAME, ReportFormatPossibleValueEnum.values()));
       instance.addPossibleValueSource(QPossibleValueSource.newForEnum(RenderedReportStatus.NAME, RenderedReportStatus.values()));
@@ -85,10 +94,30 @@ public class SavedReportsMetaDataProvider
          .filter(f -> RenderSavedReportMetaDataProducer.FIELD_NAME_STORAGE_TABLE_NAME.equals(f.getName()))
          .findFirst()
          .ifPresent(f -> f.setDefaultValue(REPORT_STORAGE_TABLE_NAME));
+      instance.addWidget(defineRenderReportProcessValuesWidget());
 
       instance.addWidget(defineReportSetupWidget());
       instance.addWidget(definePivotTableSetupWidget());
 
+      ////////////////////////////////////////
+      // todo - param to enable scheduling? //
+      ////////////////////////////////////////
+      instance.addTable(defineScheduledReportTable(recordTablesBackendName, backendDetailEnricher));
+      QJoinMetaData join = defineSavedReportJoinScheduledReport();
+      instance.addJoin(join);
+      instance.addWidget(defineScheduledReportJoinSavedReportWidget(join));
+      QProcessMetaData scheduledReportSyncToScheduledJobProcess = new ScheduledReportSyncToScheduledJobProcess().produce(instance);
+      instance.addProcess(scheduledReportSyncToScheduledJobProcess);
+      instance.addWidget(defineScheduledReportValuesWidget());
+
+      QProcessMetaData runScheduledReportProcess = new RunScheduledReportMetaDataProducer().produce(instance);
+      instance.addProcess(runScheduledReportProcess);
+
+      if(instance.getPossibleValueSource(TimeZonePossibleValueSourceMetaDataProvider.NAME) == null)
+      {
+         instance.addPossibleValueSource(new TimeZonePossibleValueSourceMetaDataProvider().produce());
+      }
+
       /////////////////////////////////////
       // todo - param to enable sharing? //
       /////////////////////////////////////
@@ -102,6 +131,64 @@ public class SavedReportsMetaDataProvider
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QWidgetMetaDataInterface defineScheduledReportValuesWidget()
+   {
+      return new QWidgetMetaData()
+         .withName(SCHEDULED_REPORT_VALUES_WIDGET)
+         .withType(WidgetType.DYNAMIC_FORM.getType())
+         .withIsCard(true)
+         .withLabel("Variable Values")
+         .withCodeReference(new QCodeReference(ReportValuesDynamicFormWidgetRenderer.class));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QWidgetMetaDataInterface defineRenderReportProcessValuesWidget()
+   {
+      return new QWidgetMetaData()
+         .withName(RENDER_REPORT_PROCESS_VALUES_WIDGET)
+         .withType(WidgetType.DYNAMIC_FORM.getType())
+         .withIsCard(false)
+         .withDefaultValue("isEditable", true)
+         .withCodeReference(new QCodeReference(ReportValuesDynamicFormWidgetRenderer.class));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QJoinMetaData defineSavedReportJoinScheduledReport()
+   {
+      return (new QJoinMetaData()
+         .withName(SAVED_REPORT_JOIN_SCHEDULED_REPORT)
+         .withLeftTable(SavedReport.TABLE_NAME)
+         .withRightTable(ScheduledReport.TABLE_NAME)
+         .withType(JoinType.ONE_TO_MANY)
+         .withJoinOn(new JoinOn("id", "savedReportId")));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QWidgetMetaDataInterface defineScheduledReportJoinSavedReportWidget(QJoinMetaData join)
+   {
+      return ChildRecordListRenderer.widgetMetaDataBuilder(join)
+         .withLabel("Schedules")
+         .withCanAddChildRecord(true)
+         .getWidgetMetaData();
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -148,7 +235,7 @@ public class SavedReportsMetaDataProvider
          .withName("reportSetupWidget")
          .withLabel("Filters and Columns")
          .withIsCard(true)
-         .withType(WidgetType.REPORT_SETUP.getType())
+         .withType(WidgetType.FILTER_AND_COLUMNS_SETUP.getType())
          .withCodeReference(new QCodeReference(DefaultWidgetRenderer.class));
    }
 
@@ -187,6 +274,7 @@ public class SavedReportsMetaDataProvider
          .withSection(new QFieldSection("identity", new QIcon().withName("badge"), Tier.T1, List.of("id", "label", "tableName")))
          .withSection(new QFieldSection("filtersAndColumns", new QIcon().withName("table_chart"), Tier.T2).withLabel("Filters and Columns").withWidgetName("reportSetupWidget"))
          .withSection(new QFieldSection("pivotTable", new QIcon().withName("pivot_table_chart"), Tier.T2).withLabel("Pivot Table").withWidgetName("pivotTableSetupWidget"))
+         .withSection(new QFieldSection("schedule", new QIcon().withName("schedule"), Tier.T2).withWidgetName(SAVED_REPORT_JOIN_SCHEDULED_REPORT))
          .withSection(new QFieldSection("data", new QIcon().withName("text_snippet"), Tier.T2, List.of("queryFilterJson", "columnsJson", "pivotTableJson")).withIsHidden(true))
          .withSection(new QFieldSection("hidden", new QIcon().withName("text_snippet"), Tier.T2, List.of("inputFieldsJson", "userId")).withIsHidden(true))
          .withSection(new QFieldSection("dates", new QIcon().withName("calendar_month"), Tier.T3, List.of("createDate", "modifyDate")));
@@ -197,6 +285,7 @@ public class SavedReportsMetaDataProvider
 
       table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(SavedReportTableCustomizer.class));
       table.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(SavedReportTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.PRE_DELETE_RECORD, new QCodeReference(SavedReportTableCustomizer.class));
 
       table.withShareableTableMetaData(new ShareableTableMetaData()
          .withSharedRecordTableName(SharedSavedReport.TABLE_NAME)
@@ -278,4 +367,40 @@ public class SavedReportsMetaDataProvider
       return (table);
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private QTableMetaData defineScheduledReportTable(String backendName, Consumer<QTableMetaData> backendDetailEnricher) throws QException
+   {
+      QTableMetaData table = new QTableMetaData()
+         .withName(ScheduledReport.TABLE_NAME)
+         .withIcon(new QIcon().withName("schedule_send"))
+         .withRecordLabelFormat("%s (Schedule %s)")
+         .withRecordLabelFields("savedReportId", "id")
+         .withBackendName(backendName)
+         .withPrimaryKeyField("id")
+         .withFieldsFromEntity(ScheduledReport.class)
+         .withSection(new QFieldSection("identity", new QIcon().withName("badge"), Tier.T1, List.of("id", "savedReportId")))
+         .withSection(new QFieldSection("settings", new QIcon().withName("settings"), Tier.T2, List.of("cronExpression", "cronTimeZoneId", "isActive", "format")))
+         .withSection(new QFieldSection("email", new QIcon().withName("email"), Tier.T2, List.of("toAddresses", "subject")))
+         .withSection(new QFieldSection("variableValues", new QIcon().withName("data_object"), Tier.T2).withWidgetName(SCHEDULED_REPORT_VALUES_WIDGET))
+         .withSection(new QFieldSection("hidden", new QIcon().withName("visibility_off"), Tier.T2, List.of("inputValues", "userId")).withIsHidden(true))
+         .withSection(new QFieldSection("dates", new QIcon().withName("calendar_month"), Tier.T3, List.of("createDate", "modifyDate")));
+
+      if(backendDetailEnricher != null)
+      {
+         backendDetailEnricher.accept(table);
+      }
+
+      table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(ScheduledReportTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(ScheduledReportTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.POST_INSERT_RECORD, new QCodeReference(ScheduledReportTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.POST_UPDATE_RECORD, new QCodeReference(ScheduledReportTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.POST_DELETE_RECORD, new QCodeReference(ScheduledReportTableCustomizer.class));
+
+      return (table);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/ScheduledReport.java

@@ -0,0 +1,478 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.savedreports;
+
+
+import java.time.Instant;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormatPossibleValueEnum;
+import com.kingsrook.qqq.backend.core.model.common.TimeZonePossibleValueSourceMetaDataProvider;
+import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
+
+
+/*******************************************************************************
+ ** Entity bean for the scheduled report table
+ *******************************************************************************/
+public class ScheduledReport extends QRecordEntity
+{
+   public static final String TABLE_NAME = "scheduledReport";
+
+   @QField(isEditable = false)
+   private Integer id;
+
+   @QField(isEditable = false)
+   private Instant createDate;
+
+   @QField(isEditable = false)
+   private Instant modifyDate;
+
+   @QField(isRequired = true, possibleValueSourceName = SavedReport.TABLE_NAME)
+   private Integer savedReportId;
+
+   @QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR, isRequired = true)
+   private String cronExpression;
+
+   @QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR, possibleValueSourceName = TimeZonePossibleValueSourceMetaDataProvider.NAME, isRequired = true)
+   private String cronTimeZoneId;
+
+   @QField(isRequired = true, defaultValue = "true")
+   private Boolean isActive;
+
+   @QField(isRequired = true)
+   private String toAddresses;
+
+   @QField(isRequired = true, maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
+   private String subject;
+
+   @QField(isRequired = true, maxLength = 20, valueTooLongBehavior = ValueTooLongBehavior.ERROR, possibleValueSourceName = ReportFormatPossibleValueEnum.NAME)
+   private String format;
+
+   @QField()
+   private String inputValues;
+
+   @QField(maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR, dynamicDefaultValueBehavior = DynamicDefaultValueBehavior.USER_ID, label = "Owner")
+   private String userId;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public ScheduledReport()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public ScheduledReport(QRecord qRecord) throws QException
+   {
+      populateFromQRecord(qRecord);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for id
+    **
+    *******************************************************************************/
+   public Integer getId()
+   {
+      return id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for id
+    **
+    *******************************************************************************/
+   public void setId(Integer id)
+   {
+      this.id = id;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for createDate
+    **
+    *******************************************************************************/
+   public Instant getCreateDate()
+   {
+      return createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for createDate
+    **
+    *******************************************************************************/
+   public void setCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for modifyDate
+    **
+    *******************************************************************************/
+   public Instant getModifyDate()
+   {
+      return modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for modifyDate
+    **
+    *******************************************************************************/
+   public void setModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for id
+    *******************************************************************************/
+   public ScheduledReport withId(Integer id)
+   {
+      this.id = id;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for createDate
+    *******************************************************************************/
+   public ScheduledReport withCreateDate(Instant createDate)
+   {
+      this.createDate = createDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for modifyDate
+    *******************************************************************************/
+   public ScheduledReport withModifyDate(Instant modifyDate)
+   {
+      this.modifyDate = modifyDate;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for savedReportId
+    *******************************************************************************/
+   public Integer getSavedReportId()
+   {
+      return (this.savedReportId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for savedReportId
+    *******************************************************************************/
+   public void setSavedReportId(Integer savedReportId)
+   {
+      this.savedReportId = savedReportId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for savedReportId
+    *******************************************************************************/
+   public ScheduledReport withSavedReportId(Integer savedReportId)
+   {
+      this.savedReportId = savedReportId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for cronExpression
+    *******************************************************************************/
+   public String getCronExpression()
+   {
+      return (this.cronExpression);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for cronExpression
+    *******************************************************************************/
+   public void setCronExpression(String cronExpression)
+   {
+      this.cronExpression = cronExpression;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for cronExpression
+    *******************************************************************************/
+   public ScheduledReport withCronExpression(String cronExpression)
+   {
+      this.cronExpression = cronExpression;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for cronTimeZoneId
+    *******************************************************************************/
+   public String getCronTimeZoneId()
+   {
+      return (this.cronTimeZoneId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for cronTimeZoneId
+    *******************************************************************************/
+   public void setCronTimeZoneId(String cronTimeZoneId)
+   {
+      this.cronTimeZoneId = cronTimeZoneId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for cronTimeZoneId
+    *******************************************************************************/
+   public ScheduledReport withCronTimeZoneId(String cronTimeZoneId)
+   {
+      this.cronTimeZoneId = cronTimeZoneId;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for isActive
+    *******************************************************************************/
+   public Boolean getIsActive()
+   {
+      return (this.isActive);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for isActive
+    *******************************************************************************/
+   public void setIsActive(Boolean isActive)
+   {
+      this.isActive = isActive;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isActive
+    *******************************************************************************/
+   public ScheduledReport withIsActive(Boolean isActive)
+   {
+      this.isActive = isActive;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for toAddresses
+    *******************************************************************************/
+   public String getToAddresses()
+   {
+      return (this.toAddresses);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for toAddresses
+    *******************************************************************************/
+   public void setToAddresses(String toAddresses)
+   {
+      this.toAddresses = toAddresses;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for toAddresses
+    *******************************************************************************/
+   public ScheduledReport withToAddresses(String toAddresses)
+   {
+      this.toAddresses = toAddresses;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for subject
+    *******************************************************************************/
+   public String getSubject()
+   {
+      return (this.subject);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for subject
+    *******************************************************************************/
+   public void setSubject(String subject)
+   {
+      this.subject = subject;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for subject
+    *******************************************************************************/
+   public ScheduledReport withSubject(String subject)
+   {
+      this.subject = subject;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for format
+    *******************************************************************************/
+   public String getFormat()
+   {
+      return (this.format);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for format
+    *******************************************************************************/
+   public void setFormat(String format)
+   {
+      this.format = format;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for format
+    *******************************************************************************/
+   public ScheduledReport withFormat(String format)
+   {
+      this.format = format;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for inputValues
+    *******************************************************************************/
+   public String getInputValues()
+   {
+      return (this.inputValues);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for inputValues
+    *******************************************************************************/
+   public void setInputValues(String inputValues)
+   {
+      this.inputValues = inputValues;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for inputValues
+    *******************************************************************************/
+   public ScheduledReport withInputValues(String inputValues)
+   {
+      this.inputValues = inputValues;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for userId
+    *******************************************************************************/
+   public String getUserId()
+   {
+      return (this.userId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for userId
+    *******************************************************************************/
+   public void setUserId(String userId)
+   {
+      this.userId = userId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for userId
+    *******************************************************************************/
+   public ScheduledReport withUserId(String userId)
+   {
+      this.userId = userId;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/ScheduledReportSyncToScheduledJobProcess.java

@@ -0,0 +1,189 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.savedreports;
+
+
+import java.io.Serializable;
+import java.util.List;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormatPossibleValueEnum;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobParameter;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobType;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.processes.implementations.savedreports.RunScheduledReportMetaDataProducer;
+import com.kingsrook.qqq.backend.core.processes.implementations.tablesync.AbstractTableSyncTransformStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.tablesync.TableSyncProcess;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class ScheduledReportSyncToScheduledJobProcess extends AbstractTableSyncTransformStep implements MetaDataProducerInterface<QProcessMetaData>
+{
+   public static final String NAME = "scheduledReportSyncToScheduledJob";
+
+   public static final String SCHEDULER_NAME_FIELD_NAME = "schedulerName";
+
+   private static final QLogger LOG = QLogger.getLogger(ScheduledReportSyncToScheduledJobProcess.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QProcessMetaData produce(QInstance qInstance) throws QException
+   {
+      QProcessMetaData processMetaData = TableSyncProcess.processMetaDataBuilder(false)
+         .withName(NAME)
+         .withTableName(ScheduledReport.TABLE_NAME)
+
+         /////////////////////////////////////////
+         // todo - maybe - to keep 'em in sync? //
+         /////////////////////////////////////////
+         //.withBasepullConfiguration(CoreMetaDataProvider.getDefaultBasepullConfiguration("modifyDate", ONE_DAY_IN_HOURS)
+         //   .withSecondsToSubtractFromLastRunTimeForTimestampQuery(10 * 60))
+         // .withSchedule(new QScheduleMetaData()
+         //    .withRepeatSeconds(SYNC_BASEPULLS_INTERVAL_SECONDS))
+
+         .withSyncTransformStepClass(getClass())
+         .withReviewStepRecordFields(List.of(
+            new QFieldMetaData("savedReportId", QFieldType.INTEGER).withPossibleValueSourceName(SavedReport.TABLE_NAME),
+            new QFieldMetaData("cronExpression", QFieldType.STRING),
+            new QFieldMetaData("isActive", QFieldType.BOOLEAN),
+            new QFieldMetaData("toAddresses", QFieldType.STRING),
+            new QFieldMetaData("subject", QFieldType.STRING),
+            new QFieldMetaData("format", QFieldType.STRING).withPossibleValueSourceName(ReportFormatPossibleValueEnum.NAME)
+         ))
+         .getProcessMetaData();
+
+      processMetaData.getBackendStep(StreamedETLWithFrontendProcess.STEP_NAME_PREVIEW).getInputMetaData()
+         .withField(new QFieldMetaData(SCHEDULER_NAME_FIELD_NAME, QFieldType.STRING));
+
+      return (processMetaData);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QRecord populateRecordToStore(RunBackendStepInput runBackendStepInput, QRecord destinationRecord, QRecord sourceRecord) throws QException
+   {
+      ScheduledReport scheduledReport = new ScheduledReport(sourceRecord);
+      ScheduledJob    scheduledJob;
+
+      if(destinationRecord == null || destinationRecord.getValue("id") == null)
+      {
+         ////////////////////////////////////////////////////////////////////////
+         // need to do an insert - set lots of key values in the scheduled job //
+         ////////////////////////////////////////////////////////////////////////
+         scheduledJob = new ScheduledJob();
+         scheduledJob.setLabel("Scheduled Report " + scheduledReport.getId());
+         scheduledJob.setDescription("Job to run Scheduled Report Id " + scheduledReport.getId()
+            + " (which runs Report Id " + scheduledReport.getSavedReportId() + ")");
+         scheduledJob.setSchedulerName(runBackendStepInput.getValueString(SCHEDULER_NAME_FIELD_NAME));
+         scheduledJob.setType(ScheduledJobType.PROCESS.name());
+         scheduledJob.setForeignKeyType(getScheduledJobForeignKeyType());
+         scheduledJob.setForeignKeyValue(String.valueOf(scheduledReport.getId()));
+         scheduledJob.setJobParameters(List.of(
+            new ScheduledJobParameter().withKey("processName").withValue(getProcessNameScheduledJobParameter()),
+            new ScheduledJobParameter().withKey("recordId").withValue(ValueUtils.getValueAsString(scheduledReport.getId()))
+         ));
+      }
+      else
+      {
+         //////////////////////////////////////////////////////////////////////////////////
+         // else doing an update - populate scheduled job entity from destination record //
+         //////////////////////////////////////////////////////////////////////////////////
+         scheduledJob = new ScheduledJob(destinationRecord);
+      }
+
+      //////////////////////////////////////////////////////////////////////////////////
+      // these fields sync on insert and update                                       //
+      // todo - if no diffs, should we return null (to avoid changing quartz at all?) //
+      //////////////////////////////////////////////////////////////////////////////////
+      scheduledJob.setCronExpression(scheduledReport.getCronExpression());
+      scheduledJob.setCronTimeZoneId(scheduledReport.getCronTimeZoneId());
+      scheduledJob.setIsActive(scheduledReport.getIsActive());
+
+      return scheduledJob.toQRecord();
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   static String getScheduledJobForeignKeyType()
+   {
+      return "scheduledReport";
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static String getProcessNameScheduledJobParameter()
+   {
+      return RunScheduledReportMetaDataProducer.NAME;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   protected QQueryFilter getExistingRecordQueryFilter(RunBackendStepInput runBackendStepInput, List<Serializable> sourceKeyList)
+   {
+      return super.getExistingRecordQueryFilter(runBackendStepInput, sourceKeyList)
+         .withCriteria(new QFilterCriteria("foreignKeyType", QCriteriaOperator.EQUALS, getScheduledJobForeignKeyType()));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   protected SyncProcessConfig getSyncProcessConfig()
+   {
+      return new SyncProcessConfig(ScheduledReport.TABLE_NAME, "id", ScheduledJob.TABLE_NAME, "foreignKeyValue", true, true);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedreports/ScheduledReportTableCustomizer.java

@@ -0,0 +1,217 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.savedreports;
+
+
+import java.io.Serializable;
+import java.text.ParseException;
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
+import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValidationUtils;
+import org.quartz.CronScheduleBuilder;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class ScheduledReportTableCustomizer implements TableCustomizerInterface
+{
+   private static final QLogger LOG = QLogger.getLogger(ScheduledReportTableCustomizer.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      preInsertOrUpdate(records);
+      return (records);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      preInsertOrUpdate(records);
+      return (records);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void preInsertOrUpdate(List<QRecord> records)
+   {
+      for(QRecord record : records)
+      {
+         String cronExpression = record.getValueString("cronExpression");
+         try
+         {
+            CronScheduleBuilder.cronScheduleNonvalidatedExpression(cronExpression);
+         }
+         catch(ParseException e)
+         {
+            record.addError(new BadInputStatusMessage("Cron Expression [" + cronExpression + "] is not valid: " + e.getMessage()));
+         }
+
+         try
+         {
+            String toAddresses = record.getValueString("toAddresses");
+            if(StringUtils.hasContent(toAddresses))
+            {
+               ValidationUtils.parseAndValidateEmailAddresses(toAddresses);
+            }
+         }
+         catch(QUserFacingException ufe)
+         {
+            record.addError(new BadInputStatusMessage(ufe.getMessage()));
+         }
+         catch(Exception e)
+         {
+            record.addError(new BadInputStatusMessage("To Addresses is not valid: " + e.getMessage()));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
+   {
+      runSyncProcess(records);
+      return (records);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      runSyncProcess(records);
+      return (records);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void runSyncProcess(List<QRecord> records)
+   {
+      List<Serializable> scheduledReportIds = records.stream()
+         .filter(r -> CollectionUtils.nullSafeIsEmpty(r.getErrors()))
+         .map(r -> r.getValue("id")).toList();
+
+      if(CollectionUtils.nullSafeIsEmpty(scheduledReportIds))
+      {
+         return;
+      }
+
+      try
+      {
+         RunProcessInput runProcessInput = new RunProcessInput();
+         runProcessInput.setProcessName(ScheduledReportSyncToScheduledJobProcess.NAME);
+         runProcessInput.setCallback(QProcessCallbackFactory.forPrimaryKeys("id", scheduledReportIds));
+         runProcessInput.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
+         RunProcessOutput runProcessOutput = new RunProcessAction().execute(runProcessInput);
+
+         Serializable processSummary = runProcessOutput.getValue(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY);
+         System.out.println(processSummary);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error syncing scheduled reports to scheduled jobs", e, logPair("scheduledReportIds", scheduledReportIds));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
+   {
+      List<String> scheduledReportIds = records.stream()
+         .filter(r -> CollectionUtils.nullSafeIsEmpty(r.getErrors()))
+         .map(r -> r.getValueString("id")).toList();
+
+      if(scheduledReportIds.isEmpty())
+      {
+         return (records);
+      }
+
+      ///////////////////////////////////////////////////
+      // delete any corresponding scheduledJob records //
+      ///////////////////////////////////////////////////
+      try
+      {
+         DeleteOutput deleteOutput = new DeleteAction().execute(new DeleteInput(ScheduledJob.TABLE_NAME).withQueryFilter(new QQueryFilter()
+            .withCriteria(new QFilterCriteria("foreignKeyType", QCriteriaOperator.EQUALS, ScheduledReportSyncToScheduledJobProcess.getScheduledJobForeignKeyType()))
+            .withCriteria(new QFilterCriteria("foreignKeyValue", QCriteriaOperator.IN, scheduledReportIds))
+         ));
+
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error deleting scheduled jobs for scheduled reports", e);
+      }
+
+      return (records);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedviews/SavedViewTableCustomizer.java

@@ -0,0 +1,63 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.savedviews;
+
+
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.savedreports.SavedReportTableCustomizer;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class SavedViewTableCustomizer implements TableCustomizerInterface
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      SavedReportTableCustomizer.validateOwner(records, SavedView.TABLE_NAME, "edit");
+      return (records);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      SavedReportTableCustomizer.validateOwner(records, SavedView.TABLE_NAME, "delete");
+      return (records);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/savedviews/SavedViewsMetaDataProvider.java

@@ -24,10 +24,12 @@ package com.kingsrook.qqq.backend.core.model.savedviews;
 
 import java.util.List;
 import java.util.function.Consumer;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
@@ -99,6 +101,9 @@ public class SavedViewsMetaDataProvider
 
       table.getField("viewJson").withFieldAdornment(new FieldAdornment(AdornmentType.CODE_EDITOR).withValue(AdornmentType.CodeEditorValues.languageMode("json")));
 
+      table.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(SavedViewTableCustomizer.class));
+      table.withCustomizer(TableCustomizers.PRE_DELETE_RECORD, new QCodeReference(SavedViewTableCustomizer.class));
+
       if(backendDetailEnricher != null)
       {
          backendDetailEnricher.accept(table);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/scheduledjobs/ScheduledJob.java

@@ -31,6 +31,7 @@ import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.common.TimeZonePossibleValueSourceMetaDataProvider;
 import com.kingsrook.qqq.backend.core.model.data.QAssociation;
 import com.kingsrook.qqq.backend.core.model.data.QField;
+import com.kingsrook.qqq.backend.core.model.data.QIgnore;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat;
@@ -79,6 +80,12 @@ public class ScheduledJob extends QRecordEntity
    @QField(isRequired = true)
    private Boolean isActive;
 
+   @QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
+   private String foreignKeyType;
+
+   @QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
+   private String foreignKeyValue;
+
    @QAssociation(name = ScheduledJobParameter.TABLE_NAME)
    private List<ScheduledJobParameter> jobParameters;
 
@@ -428,6 +435,7 @@ public class ScheduledJob extends QRecordEntity
    /*******************************************************************************
     ** Getter for jobParameters - but a map of just the key=value pairs.
     *******************************************************************************/
+   @QIgnore
    public Map<String, String> getJobParametersMap()
    {
       if(CollectionUtils.nullSafeIsEmpty(this.jobParameters))
@@ -463,6 +471,7 @@ public class ScheduledJob extends QRecordEntity
    }
 
 
+
    /*******************************************************************************
     ** Getter for repeatSeconds
     *******************************************************************************/
@@ -493,4 +502,65 @@ public class ScheduledJob extends QRecordEntity
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for foreignKeyType
+    *******************************************************************************/
+   public String getForeignKeyType()
+   {
+      return (this.foreignKeyType);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for foreignKeyType
+    *******************************************************************************/
+   public void setForeignKeyType(String foreignKeyType)
+   {
+      this.foreignKeyType = foreignKeyType;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for foreignKeyType
+    *******************************************************************************/
+   public ScheduledJob withForeignKeyType(String foreignKeyType)
+   {
+      this.foreignKeyType = foreignKeyType;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for foreignKeyValue
+    *******************************************************************************/
+   public String getForeignKeyValue()
+   {
+      return (this.foreignKeyValue);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for foreignKeyValue
+    *******************************************************************************/
+   public void setForeignKeyValue(String foreignKeyValue)
+   {
+      this.foreignKeyValue = foreignKeyValue;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for foreignKeyValue
+    *******************************************************************************/
+   public ScheduledJob withForeignKeyValue(String foreignKeyValue)
+   {
+      this.foreignKeyValue = foreignKeyValue;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/scheduledjobs/ScheduledJobsMetaDataProvider.java

@@ -169,7 +169,7 @@ public class ScheduledJobsMetaDataProvider
          .withRecordLabelFields("label")
          .withSection(new QFieldSection("identity", new QIcon().withName("badge"), Tier.T1, List.of("id", "label", "description")))
          .withSection(new QFieldSection("schedule", new QIcon().withName("alarm"), Tier.T2, List.of("cronExpression", "cronTimeZoneId", "repeatSeconds")))
-         .withSection(new QFieldSection("settings", new QIcon().withName("tune"), Tier.T2, List.of("type", "isActive", "schedulerName")))
+         .withSection(new QFieldSection("settings", new QIcon().withName("tune"), Tier.T2, List.of("type", "isActive", "schedulerName", "foreignKeyType", "foreignKeyValue")))
          .withSection(new QFieldSection("parameters", new QIcon().withName("list"), Tier.T2).withWidgetName(JOB_PARAMETER_JOIN_NAME))
          .withSection(new QFieldSection("dates", new QIcon().withName("calendar_month"), Tier.T3, List.of("createDate", "modifyDate")));
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/scheduledjobs/customizers/ScheduledJobParameterTableCustomizer.java

@@ -33,6 +33,7 @@ import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
@@ -42,6 +43,7 @@ import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
 import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobParameter;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ListingHash;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
@@ -208,6 +210,17 @@ public class ScheduledJobParameterTableCustomizer implements TableCustomizerInte
                return (false);
             }
          }
+         else if(firstActionInStack.get() instanceof RunProcessInput runProcessInput)
+         {
+            String tableName = runProcessInput.getValueString("tableName");
+            if(StringUtils.hasContent(tableName))
+            {
+               if(!ScheduledJobParameter.TABLE_NAME.equals(tableName))
+               {
+                  return (false);
+               }
+            }
+         }
       }
       return (true);
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/scheduledjobs/customizers/ScheduledJobTableCustomizer.java

@@ -32,6 +32,7 @@ import java.util.Optional;
 import java.util.Set;
 import java.util.function.Predicate;
 import java.util.stream.Collectors;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -58,6 +59,7 @@ import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
  *******************************************************************************/
 public class ScheduledJobTableCustomizer implements TableCustomizerInterface
 {
+   private QBackendTransaction transaction = null;
 
    /*******************************************************************************
     **
@@ -65,6 +67,7 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
    {
+      transaction = insertInput.getTransaction();
       validateConditionalFields(records, Collections.emptyMap());
       return (records);
    }
@@ -77,6 +80,7 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
    {
+      transaction = insertInput.getTransaction();
       scheduleJobsForRecordList(records);
       return (records);
    }
@@ -89,6 +93,7 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
    {
+      transaction = updateInput.getTransaction();
       Map<Integer, QRecord> freshOldRecordsWithAssociationsMap = CollectionUtils.recordsToMap(freshlyQueryForRecordsWithAssociations(oldRecordList.get()), "id", Integer.class);
 
       validateConditionalFields(records, freshOldRecordsWithAssociationsMap);
@@ -169,6 +174,7 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
    {
+      transaction = updateInput.getTransaction();
       if(oldRecordList.isPresent())
       {
          Set<Integer> idsWithErrors = getRecordIdsWithErrors(records);
@@ -201,6 +207,7 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    @Override
    public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
    {
+      transaction = deleteInput.getTransaction();
       Set<Integer> idsWithErrors = getRecordIdsWithErrors(records);
       unscheduleJobsForRecordList(records, idsWithErrors);
       return (records);
@@ -262,12 +269,13 @@ public class ScheduledJobTableCustomizer implements TableCustomizerInterface
    /*******************************************************************************
     **
     *******************************************************************************/
-   private static List<QRecord> freshlyQueryForRecordsWithAssociations(List<QRecord> records) throws QException
+   private List<QRecord> freshlyQueryForRecordsWithAssociations(List<QRecord> records) throws QException
    {
       List<Integer> idList = records.stream().map(r -> r.getValueInteger("id")).toList();
 
       return new QueryAction().execute(new QueryInput(ScheduledJob.TABLE_NAME)
             .withIncludeAssociations(true)
+            .withTransaction(transaction)
             .withFilter(new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, idList))))
          .getRecords();
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/session/QSession.java

@@ -42,7 +42,7 @@ import com.kingsrook.qqq.backend.core.utils.collections.MutableMap;
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QSession implements Serializable
+public class QSession implements Serializable, Cloneable
 {
    private String idReference;
    private QUser  user;
@@ -68,6 +68,58 @@ public class QSession implements Serializable
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QSession clone() throws CloneNotSupportedException
+   {
+      QSession clone = (QSession) super.clone();
+
+      if(user != null)
+      {
+         clone.user = user.clone();
+      }
+
+      if(permissions != null)
+      {
+         clone.permissions = new HashSet<>();
+         clone.permissions.addAll(permissions);
+      }
+
+      if(securityKeyValues != null)
+      {
+         clone.securityKeyValues = new HashMap<>();
+         for(Map.Entry<String, List<Serializable>> entry : securityKeyValues.entrySet())
+         {
+            List<Serializable> cloneValues = entry.getValue() == null ? null : new ArrayList<>(entry.getValue());
+            clone.securityKeyValues.put(entry.getKey(), cloneValues);
+         }
+      }
+
+      if(backendVariants != null)
+      {
+         clone.backendVariants = new HashMap<>();
+         clone.backendVariants.putAll(backendVariants);
+      }
+
+      if(values != null)
+      {
+         clone.values = new HashMap<>();
+         clone.values.putAll(values);
+      }
+
+      if(valuesForFrontend != null)
+      {
+         clone.valuesForFrontend = new HashMap<>();
+         clone.valuesForFrontend.putAll(valuesForFrontend);
+      }
+
+      return (clone);
+   }
+
+
+
    /*******************************************************************************
     ** Default constructor, puts a uuid in the session
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/session/QUser.java

@@ -25,13 +25,24 @@ package com.kingsrook.qqq.backend.core.model.session;
 /*******************************************************************************
  **
  *******************************************************************************/
-public class QUser
+public class QUser implements Cloneable
 {
    private String idReference;
    private String fullName;
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QUser clone() throws CloneNotSupportedException
+   {
+      return (QUser) super.clone();
+   }
+
+
+
    /*******************************************************************************
     ** Getter for idReference
     **

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/QAuthenticationModuleCustomizerInterface.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.modules.authentication;
 
 import java.io.Serializable;
 import java.util.Map;
+import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 
@@ -62,4 +63,14 @@ public interface QAuthenticationModuleCustomizerInterface
       //////////
    }
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   default void customizeAutomatedSessionForUser(QInstance qInstance, QSession automatedSessionForUser, Serializable userId) throws QAuthenticationException
+   {
+      //////////
+      // noop //
+      //////////
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/QAuthenticationModuleInterface.java

@@ -22,12 +22,15 @@
 package com.kingsrook.qqq.backend.core.modules.authentication;
 
 
+import java.io.Serializable;
 import java.util.Map;
+import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.AccessTokenException;
 import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 import org.apache.commons.lang.NotImplementedException;
 
 
@@ -49,6 +52,27 @@ public interface QAuthenticationModuleInterface
    boolean isSessionValid(QInstance instance, QSession session);
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   default QSession createAutomatedSessionForUser(QInstance qInstance, Serializable userId) throws QAuthenticationException
+   {
+      try
+      {
+         QSession clone = QContext.getQSession().clone();
+         if(clone.getUser() != null)
+         {
+            clone.getUser().setIdReference(ValueUtils.getValueAsString(userId));
+         }
+         return clone;
+      }
+      catch(CloneNotSupportedException e)
+      {
+         throw (new QAuthenticationException("Cloning session failed", e));
+      }
+   }
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/authentication/implementations/Auth0AuthenticationModule.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.modules.authentication.implementations;
 
 
+import java.io.Serializable;
 import java.nio.charset.StandardCharsets;
 import java.security.interfaces.RSAPublicKey;
 import java.time.Duration;
@@ -622,7 +623,7 @@ public class Auth0AuthenticationModule implements QAuthenticationModuleInterface
       // set security keys in the session from the JWT //
       ///////////////////////////////////////////////////
       setSecurityKeysInSessionFromJwtPayload(qInstance, payload, qSession);
-      
+
       //////////////////////////////////////////////////////////////
       // allow customizer to do custom things here, if so desired //
       //////////////////////////////////////////////////////////////
@@ -1108,4 +1109,20 @@ public class Auth0AuthenticationModule implements QAuthenticationModuleInterface
          return (null);
       }
    }
+
+
+
+   /*******************************************************************************
+    ** e.g., if a scheduled job needs to run as a user (say, a report)...
+    *******************************************************************************/
+   @Override
+   public QSession createAutomatedSessionForUser(QInstance qInstance, Serializable userId) throws QAuthenticationException
+   {
+      QSession automatedSessionForUser = QAuthenticationModuleInterface.super.createAutomatedSessionForUser(qInstance, userId);
+      if(getCustomizer() != null)
+      {
+         getCustomizer().customizeAutomatedSessionForUser(qInstance, automatedSessionForUser, userId);
+      }
+      return (automatedSessionForUser);
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/backend/implementations/utils/BackendQueryFilterUtils.java

@@ -32,6 +32,8 @@ import java.util.ListIterator;
 import java.util.Map;
 import java.util.Objects;
 import java.util.concurrent.atomic.AtomicBoolean;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
@@ -50,6 +52,9 @@ import org.apache.commons.lang.NotImplementedException;
  *******************************************************************************/
 public class BackendQueryFilterUtils
 {
+   private static final QLogger LOG = QLogger.getLogger(BackendQueryFilterUtils.class);
+
+
 
    /*******************************************************************************
     ** Test if record matches filter.
@@ -138,7 +143,14 @@ public class BackendQueryFilterUtils
          Serializable criteriaValue = valueListIterator.next();
          if(criteriaValue instanceof AbstractFilterExpression<?> expression)
          {
-            valueListIterator.set(expression.evaluate());
+            try
+            {
+               valueListIterator.set(expression.evaluate());
+            }
+            catch(QException qe)
+            {
+               LOG.warn("Unexpected exception caught evaluating expression", qe);
+            }
          }
       }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/messaging/MessagingProviderInterface.java

@@ -0,0 +1,46 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.messaging;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public interface MessagingProviderInterface
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   String getType();
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   SendMessageOutput sendMessage(SendMessageInput sendMessageInput) throws QException;
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/modules/messaging/QMessagingProviderDispatcher.java

@@ -0,0 +1,123 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2022.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.modules.messaging;
+
+
+import java.util.HashMap;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.exceptions.QModuleDispatchException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
+
+
+/*******************************************************************************
+ ** This class is responsible for loading a messaging provider, by its name, and
+ ** returning an instance.
+ **
+ *******************************************************************************/
+public class QMessagingProviderDispatcher
+{
+   private static final QLogger LOG = QLogger.getLogger(QMessagingProviderDispatcher.class);
+
+   private static Map<String, String> typeToProviderClassNameMap;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QMessagingProviderDispatcher()
+   {
+      initBackendTypeToModuleClassNameMap();
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static void initBackendTypeToModuleClassNameMap()
+   {
+      if(typeToProviderClassNameMap != null)
+      {
+         return;
+      }
+
+      Map<String, String> newMap = new HashMap<>();
+
+      typeToProviderClassNameMap = newMap;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void registerMessagingProvider(MessagingProviderInterface messagingProviderInstance)
+   {
+      initBackendTypeToModuleClassNameMap();
+      String type = messagingProviderInstance.getType();
+      if(typeToProviderClassNameMap.containsKey(type))
+      {
+         LOG.info("Overwriting messagingProvider type [" + type + "] with [" + messagingProviderInstance.getClass() + "]");
+      }
+      typeToProviderClassNameMap.put(type, messagingProviderInstance.getClass().getName());
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MessagingProviderInterface getMessagingProviderInterface(QMessagingProviderMetaData messagingProviderMetaData) throws QModuleDispatchException
+   {
+      return (getMessagingProviderInterface(messagingProviderMetaData.getType()));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public MessagingProviderInterface getMessagingProviderInterface(String type) throws QModuleDispatchException
+   {
+      try
+      {
+         String className = typeToProviderClassNameMap.get(type);
+         if(className == null)
+         {
+            throw (new QModuleDispatchException("Unrecognized messaging provider type [" + type + "] in dispatcher."));
+         }
+
+         Class<?> moduleClass = Class.forName(className);
+         return (MessagingProviderInterface) moduleClass.getDeclaredConstructor().newInstance();
+      }
+      catch(QModuleDispatchException qmde)
+      {
+         throw (qmde);
+      }
+      catch(Exception e)
+      {
+         throw (new QModuleDispatchException("Error getting messaging provider of type: " + type, e));
+      }
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/etl/streamedwithfrontend/StreamedETLPreviewStep.java

@@ -144,6 +144,12 @@ public class StreamedETLPreviewStep extends BaseStreamedETLStep implements Backe
       BackendStepPostRunOutput postRunOutput = new BackendStepPostRunOutput(runBackendStepOutput);
       BackendStepPostRunInput  postRunInput  = new BackendStepPostRunInput(runBackendStepInput);
       transformStep.postRun(postRunInput, postRunOutput);
+
+      // todo figure out what kind of test we can get on this
+      if(postRunOutput.getUpdatedFrontendStepList() != null)
+      {
+         runBackendStepOutput.setUpdatedFrontendStepList(postRunOutput.getUpdatedFrontendStepList());
+      }
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/etl/streamedwithfrontend/StreamedETLValidateStep.java

@@ -141,6 +141,11 @@ public class StreamedETLValidateStep extends BaseStreamedETLStep implements Back
       BackendStepPostRunOutput postRunOutput = new BackendStepPostRunOutput(runBackendStepOutput);
       BackendStepPostRunInput  postRunInput  = new BackendStepPostRunInput(runBackendStepInput);
       transformStep.postRun(postRunInput, postRunOutput);
+
+      if(postRunOutput.getUpdatedFrontendStepList() != null)
+      {
+         runBackendStepOutput.setUpdatedFrontendStepList(postRunOutput.getUpdatedFrontendStepList());
+      }
    }
 
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedreports/RenderSavedReportExecuteStep.java

@@ -29,15 +29,25 @@ import java.time.LocalDate;
 import java.time.LocalTime;
 import java.time.ZoneId;
 import java.time.format.DateTimeFormatter;
+import java.util.ArrayList;
+import java.util.List;
 import java.util.Map;
 import java.util.UUID;
+import com.kingsrook.qqq.backend.core.actions.messaging.SendMessageAction;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.actions.reporting.GenerateReportAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.StorageAction;
 import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Content;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.MultiParty;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Party;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailContentRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailPartyRole;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.ReportDestination;
@@ -53,8 +63,10 @@ import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportMetaData;
 import com.kingsrook.qqq.backend.core.model.savedreports.RenderedReport;
 import com.kingsrook.qqq.backend.core.model.savedreports.RenderedReportStatus;
 import com.kingsrook.qqq.backend.core.model.savedreports.SavedReport;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ExceptionUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValidationUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -80,12 +92,29 @@ public class RenderSavedReportExecuteStep implements BackendStep
          ////////////////////////////////
          // read inputs, set up params //
          ////////////////////////////////
+         String       sesProviderName      = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.SES_PROVIDER_NAME);
+         String       fromEmailAddress     = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.FROM_EMAIL_ADDRESS);
+         String       replyToEmailAddress  = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.REPLY_TO_EMAIL_ADDRESS);
          String       storageTableName     = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.FIELD_NAME_STORAGE_TABLE_NAME);
          ReportFormat reportFormat         = ReportFormat.fromString(runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.FIELD_NAME_REPORT_FORMAT));
+         String       sendToEmailAddress   = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.FIELD_NAME_EMAIL_ADDRESS);
+         String       emailSubject         = runBackendStepInput.getValueString(RenderSavedReportMetaDataProducer.FIELD_NAME_EMAIL_SUBJECT);
          SavedReport  savedReport          = new SavedReport(runBackendStepInput.getRecords().get(0));
          String       downloadFileBaseName = getDownloadFileBaseName(runBackendStepInput, savedReport);
          String       storageReference     = LocalDate.now() + "/" + LocalTime.now().toString().replaceAll(":", "").replaceFirst("\\..*", "") + "/" + UUID.randomUUID() + "/" + downloadFileBaseName + "." + reportFormat.getExtension();
-         OutputStream outputStream         = new StorageAction().createOutputStream(new StorageInput(storageTableName).withReference(storageReference));
+
+         //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         // if sending an email (or emails), validate the addresses before doing anything so user gets error and can fix //
+         //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+         List<String> toEmailAddressList = new ArrayList<>();
+         if(StringUtils.hasContent(sendToEmailAddress))
+         {
+            toEmailAddressList = ValidationUtils.parseAndValidateEmailAddresses(sendToEmailAddress);
+         }
+
+         StorageAction storageAction = new StorageAction();
+         StorageInput  storageInput  = new StorageInput(storageTableName).withReference(storageReference);
+         OutputStream  outputStream  = storageAction.createOutputStream(storageInput);
 
          LOG.info("Starting to render a report", logPair("savedReportId", savedReport.getId()), logPair("tableName", savedReport.getTableName()), logPair("storageReference", storageReference));
          runBackendStepInput.getAsyncJobCallback().updateStatus("Generating Report");
@@ -116,6 +145,10 @@ public class RenderSavedReportExecuteStep implements BackendStep
             .withReportFormat(reportFormat)
             .withReportOutputStream(outputStream));
 
+         //////////////////////////////////////////////////////////
+         // todo variable-values                                 //
+         // actually, looks like they're coming in right here... //
+         //////////////////////////////////////////////////////////
          Map<String, Serializable> values = runBackendStepInput.getValues();
          reportInput.setInputValues(values);
 
@@ -132,10 +165,58 @@ public class RenderSavedReportExecuteStep implements BackendStep
             .withValue("rowCount", reportOutput.getTotalRecordCount())
          ));
 
-         runBackendStepOutput.addValue("downloadFileName", downloadFileBaseName + "." + reportFormat.getExtension());
+         String downloadFileName = downloadFileBaseName + "." + reportFormat.getExtension();
+         runBackendStepOutput.addValue("downloadFileName", downloadFileName);
          runBackendStepOutput.addValue("storageTableName", storageTableName);
          runBackendStepOutput.addValue("storageReference", storageReference);
          LOG.info("Completed rendering a report", logPair("savedReportId", savedReport.getId()), logPair("tableName", savedReport.getTableName()), logPair("storageReference", storageReference), logPair("rowCount", reportOutput.getTotalRecordCount()));
+
+         if(!toEmailAddressList.isEmpty() && CollectionUtils.nullSafeHasContents(QContext.getQInstance().getMessagingProviders()))
+         {
+            ///////////////////////////////////////////
+            // error if no from address was provided //
+            ///////////////////////////////////////////
+            if(!StringUtils.hasContent(fromEmailAddress))
+            {
+               String message = "Could not send an email because no from email address was provided.";
+               LOG.error(message);
+               throw (new QException(message));
+            }
+
+            ///////////////////////////////////////////////////////////
+            // since sending email, make s3 file publicly accessible //
+            ///////////////////////////////////////////////////////////
+            storageAction.makePublic(storageInput);
+
+            ////////////////////////////////////////////////
+            // add multiparty in case multiple recipients //
+            ////////////////////////////////////////////////
+            MultiParty recipients = new MultiParty();
+            for(String toAddress : toEmailAddressList)
+            {
+               recipients.addParty(new Party().withAddress(toAddress).withRole(EmailPartyRole.TO));
+            }
+
+            ///////////////
+            // add froms //
+            ///////////////
+            MultiParty froms = new MultiParty();
+            froms.addParty(new Party().withAddress(fromEmailAddress).withRole(EmailPartyRole.FROM));
+            if(StringUtils.hasContent(replyToEmailAddress))
+            {
+               froms.addParty(new Party().withAddress(replyToEmailAddress).withRole(EmailPartyRole.REPLY_TO));
+            }
+
+            String downloadURL = storageAction.getDownloadURL(storageInput);
+            new SendMessageAction().execute(new SendMessageInput()
+               .withMessagingProviderName(sesProviderName)
+               .withTo(recipients)
+               .withFrom(froms)
+               .withSubject(StringUtils.hasContent(emailSubject) ? emailSubject : downloadFileBaseName)
+               .withContent(new Content().withContentRole(EmailContentRole.TEXT).withBody("To download your report, open this URL in your browser: " + downloadURL))
+               .withContent(new Content().withContentRole(EmailContentRole.HTML).withBody("Link: <a target=\"_blank\" href=\"" + downloadURL + "\" download>" + downloadFileName + "</a>"))
+            );
+         }
       }
       catch(Exception e)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedreports/RenderSavedReportMetaDataProducer.java

@@ -25,11 +25,15 @@ package com.kingsrook.qqq.backend.core.processes.implementations.savedreports;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.MetaDataProducerInterface;
 import com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormatPossibleValueEnum;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.nocode.WidgetHtmlLine;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.NoCodeWidgetFrontendComponentMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
@@ -38,6 +42,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMet
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QRecordListMetaData;
 import com.kingsrook.qqq.backend.core.model.savedreports.SavedReport;
+import com.kingsrook.qqq.backend.core.model.savedreports.SavedReportsMetaDataProvider;
 
 
 /*******************************************************************************
@@ -47,8 +52,13 @@ public class RenderSavedReportMetaDataProducer implements MetaDataProducerInterf
 {
    public static final String NAME = "renderSavedReport";
 
+   public static final String SES_PROVIDER_NAME             = "sesProviderName";
+   public static final String FROM_EMAIL_ADDRESS            = "fromEmailAddress";
+   public static final String REPLY_TO_EMAIL_ADDRESS        = "replyToEmailAddress";
    public static final String FIELD_NAME_STORAGE_TABLE_NAME = "storageTableName";
    public static final String FIELD_NAME_REPORT_FORMAT      = "reportFormat";
+   public static final String FIELD_NAME_EMAIL_ADDRESS      = "reportDestinationEmailAddress";
+   public static final String FIELD_NAME_EMAIL_SUBJECT = "emailSubject";
 
 
 
@@ -63,26 +73,42 @@ public class RenderSavedReportMetaDataProducer implements MetaDataProducerInterf
          .withLabel("Render Report")
          .withTableName(SavedReport.TABLE_NAME)
          .withIcon(new QIcon().withName("print"))
+
          .addStep(new QBackendStepMetaData()
             .withName("pre")
             .withInputData(new QFunctionInputMetaData()
+               .withField(new QFieldMetaData(SES_PROVIDER_NAME, QFieldType.STRING))
+               .withField(new QFieldMetaData(FROM_EMAIL_ADDRESS, QFieldType.STRING))
+               .withField(new QFieldMetaData(REPLY_TO_EMAIL_ADDRESS, QFieldType.STRING))
                .withField(new QFieldMetaData(FIELD_NAME_STORAGE_TABLE_NAME, QFieldType.STRING))
                .withRecordListMetaData(new QRecordListMetaData().withTableName(SavedReport.TABLE_NAME)))
             .withCode(new QCodeReference(RenderSavedReportPreStep.class)))
+
          .addStep(new QFrontendStepMetaData()
             .withName("input")
             .withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM))
             .withFormField(new QFieldMetaData(FIELD_NAME_REPORT_FORMAT, QFieldType.STRING)
                .withPossibleValueSourceName(ReportFormatPossibleValueEnum.NAME)
-               .withIsRequired(true)))
+               .withIsRequired(true))
+            .withFormField(new QFieldMetaData(FIELD_NAME_EMAIL_ADDRESS, QFieldType.STRING).withLabel("Email To"))
+            .withFormField(new QFieldMetaData(FIELD_NAME_EMAIL_SUBJECT, QFieldType.STRING).withLabel("Email Subject"))
+            .withComponent(new QFrontendComponentMetaData().withType(QComponentType.WIDGET)
+               .withValue("widgetName", SavedReportsMetaDataProvider.RENDER_REPORT_PROCESS_VALUES_WIDGET)))
+
          .addStep(new QBackendStepMetaData()
             .withName("execute")
             .withInputData(new QFunctionInputMetaData().withRecordListMetaData(new QRecordListMetaData()
                .withTableName(SavedReport.TABLE_NAME)))
             .withCode(new QCodeReference(RenderSavedReportExecuteStep.class)))
+
          .addStep(new QFrontendStepMetaData()
             .withName("output")
-            .withComponent(new QFrontendComponentMetaData().withType(QComponentType.DOWNLOAD_FORM)));
+            .withComponent(new QFrontendComponentMetaData().withType(QComponentType.DOWNLOAD_FORM))
+            .withComponent(new NoCodeWidgetFrontendComponentMetaData()
+               .withOutput(new WidgetHtmlLine()
+                  .withCondition(new QFilterCriteria(FIELD_NAME_EMAIL_ADDRESS, QCriteriaOperator.IS_NOT_BLANK))
+                  .withVelocityTemplate(String.format("Report has been emailed to: ${%s}", FIELD_NAME_EMAIL_ADDRESS))))
+         );
 
       return (process);
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedreports/RenderSavedReportPreStep.java

@@ -63,17 +63,22 @@ public class RenderSavedReportPreStep implements BackendStep
       List<QRecord> records = runBackendStepInput.getRecords();
       if(!CollectionUtils.nullSafeHasContents(records))
       {
-         throw (new QUserFacingException("No report was selected or found to be rendered."));
+         throw (new QUserFacingException("No report was selected or found."));
       }
 
       if(records.size() > 1)
       {
-         throw (new QUserFacingException("You may only render 1 report at a time."));
+         throw (new QUserFacingException("You may only run 1 report at a time."));
       }
 
+      ///////////////////////////////////////////////////////////////////////////////////////
+      // put the savedReportId in values - this'll get passed into the widget, so it knows //
+      // what report we're working with, and thus what inputs to prompt for                //
+      // also put a value in just to help it know we're running the process                //
+      ///////////////////////////////////////////////////////////////////////////////////////
       SavedReport savedReport = new SavedReport(records.get(0));
-
-      // todo - check for inputs - set up the input screen...
+      runBackendStepOutput.addValue("savedReportId", savedReport.getId());
+      runBackendStepOutput.addValue("processName", runBackendStepInput.getProcessName());
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedreports/RunScheduledReportExecuteStep.java

@@ -0,0 +1,153 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.processes.implementations.savedreports;
+
+
+import java.util.List;
+import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
+import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.savedreports.ScheduledReport;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleInterface;
+import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import org.json.JSONObject;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class RunScheduledReportExecuteStep implements BackendStep
+{
+   private static final QLogger LOG = QLogger.getLogger(RunScheduledReportExecuteStep.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
+   {
+      Integer scheduledReportId = null;
+      try
+      {
+         List<QRecord> records = runBackendStepInput.getRecords();
+         if(!CollectionUtils.nullSafeHasContents(records))
+         {
+            throw (new QUserFacingException("No scheduled report was selected or found."));
+         }
+
+         ScheduledReport scheduledReport = new ScheduledReport(records.get(0));
+         scheduledReportId = scheduledReport.getId();
+
+         ////////////////////////////////////////////////////////////////////////////////////
+         // get the schedule's user - as that will drive the security key we need to apply //
+         ////////////////////////////////////////////////////////////////////////////////////
+         updateSessionForUser(scheduledReport.getUserId());
+
+         /////////////////////////////////////////////
+         // run the process that renders the report //
+         /////////////////////////////////////////////
+         RunProcessAction runProcessAction   = new RunProcessAction();
+         RunProcessInput  renderProcessInput = new RunProcessInput();
+         renderProcessInput.setProcessName(RenderSavedReportMetaDataProducer.NAME);
+         renderProcessInput.setCallback(QProcessCallbackFactory.forPrimaryKey("id", scheduledReport.getSavedReportId()));
+         renderProcessInput.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
+         renderProcessInput.setAsyncJobCallback(runBackendStepInput.getAsyncJobCallback());
+
+         renderProcessInput.addValue(RenderSavedReportMetaDataProducer.FIELD_NAME_REPORT_FORMAT, scheduledReport.getFormat());
+         renderProcessInput.addValue(RenderSavedReportMetaDataProducer.FIELD_NAME_EMAIL_ADDRESS, scheduledReport.getToAddresses());
+         renderProcessInput.addValue(RenderSavedReportMetaDataProducer.FIELD_NAME_EMAIL_SUBJECT, scheduledReport.getSubject());
+
+         if(StringUtils.hasContent(scheduledReport.getInputValues()))
+         {
+            /////////////////////////////////////////////////////////////////////////////////////
+            // if there are input values, pass them along on report input...                   //
+            // this could maybe be better (e.g., some object?), but, this is working initially //
+            /////////////////////////////////////////////////////////////////////////////////////
+            JSONObject jsonObject = JsonUtils.toJSONObject(scheduledReport.getInputValues());
+            for(String name : jsonObject.keySet())
+            {
+               renderProcessInput.addValue(name, jsonObject.optString(name));
+            }
+         }
+
+         RunProcessOutput renderProcessOutput = runProcessAction.execute(renderProcessInput);
+      }
+      catch(QUserFacingException ufe)
+      {
+         LOG.info("Error running scheduled report", ufe, logPair("id", scheduledReportId));
+         throw (ufe);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error running scheduled report", e, logPair("id", scheduledReportId));
+         throw (new QException("Error running scheduled report", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void updateSessionForUser(String userId) throws QException
+   {
+      try
+      {
+         QInstance                       qInstance                       = QContext.getQInstance();
+         QAuthenticationModuleDispatcher qAuthenticationModuleDispatcher = new QAuthenticationModuleDispatcher();
+         QAuthenticationModuleInterface  authenticationModule            = qAuthenticationModuleDispatcher.getQModule(qInstance.getAuthentication());
+
+         ///////////////////////////////////////
+         // create automated-session for user //
+         ///////////////////////////////////////
+         QSession session = authenticationModule.createAutomatedSessionForUser(qInstance, userId);
+
+         /////////////////////////////////////////////
+         // set that session in the current context //
+         /////////////////////////////////////////////
+         QContext.setQSession(session);
+      }
+      catch(Exception e)
+      {
+         LOG.warn("Error setting up user session for running scheduled report", e, logPair("userId", userId));
+         throw (new QException("Error setting up user session for running scheduled report", e));
+      }
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/processes/implementations/savedreports/RunScheduledReportMetaDataProducer.java

@@ -0,0 +1,76 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.processes.implementations.savedreports;
+
+
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.nocode.WidgetHtmlLine;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.NoCodeWidgetFrontendComponentMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QRecordListMetaData;
+import com.kingsrook.qqq.backend.core.model.savedreports.ScheduledReport;
+
+
+/*******************************************************************************
+ ** define process for rendering scheduled reports - that is - a thin layer on
+ ** top of rendering a saved report.
+ *******************************************************************************/
+public class RunScheduledReportMetaDataProducer implements MetaDataProducerInterface<QProcessMetaData>
+{
+   public static final String NAME = "runScheduledReport";
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public QProcessMetaData produce(QInstance qInstance) throws QException
+   {
+      QProcessMetaData process = new QProcessMetaData()
+         .withName(NAME)
+         .withLabel("Run Scheduled Report")
+         .withTableName(ScheduledReport.TABLE_NAME)
+         .withIcon(new QIcon().withName("print"))
+
+         .addStep(new QBackendStepMetaData()
+            .withName("execute")
+            .withInputData(new QFunctionInputMetaData().withRecordListMetaData(new QRecordListMetaData()
+               .withTableName(ScheduledReport.TABLE_NAME)))
+            .withCode(new QCodeReference(RunScheduledReportExecuteStep.class)))
+
+         .addStep(new QFrontendStepMetaData()
+            .withName("results")
+            .withComponent(new NoCodeWidgetFrontendComponentMetaData()
+               .withOutput(new WidgetHtmlLine().withVelocityTemplate("Success")))); // todo!!!
+
+      return (process);
+   }
+
+}