Fix iteration over form params (changed w/ addition of associations) - to handle empty values list

- Introduced SchedulableType, SchedulableRunner, SchedulableIdentity classes
- removed job-type specific methods from QScheduleManager and QSchedulerInterface
- Add scheduler-level management processes
- Change quartz to not change schedules during service startup
- re-added repeatSeconds to ScheduledJob

.circleci/config.yml

@@ -1,7 +1,7 @@
 version: 2.1
 
 orbs:
-  localstack: localstack/platform@1.0
+  localstack: localstack/platform@2.1
 
 commands:
   store_jacoco_site:

docs/.gitignore

@@ -0,0 +1 @@
+*.html

docs/actions/GetAction.adoc

@@ -0,0 +1,64 @@
+== GetAction
+include::../variables.adoc[]
+
+The `*GetAction*` is essentially a subset of the <<QueryAction>>, only specifically meant to get just a single record from a {link-table}.
+In SQL/RDBMS terms, it is analogous to a `SELECT` statement, where a single record may be found - or - it may not be found.
+
+For all tables, `GetAction` can do a lookup by primary key.
+In addition, for tables that define a `UniqueKey`, name/value pairs (in the form of a `Map`) can be used as input for `GetAction`.
+
+=== Examples
+[source,java]
+.Basic form - by primary key
+----
+GetInput input = new GetInput();
+input.setTableName("orders");
+input.setPrimaryKey(1);
+GetOutput output = new GetAction.execute(input);
+QRecord record = output.getRecord();
+----
+
+[source,java]
+.Secondary form - by Unique Key
+----
+GetInput input = new GetInput();
+input.setTableName("products");
+input.setUniqueKey(Map.of("storeId", 1701, "sku", "ABCD"));
+GetOutput output = new GetAction.execute(input);
+QRecord record = output.getRecord();
+----
+
+=== GetInput
+* `table` - *String, Required* - Name of the table being queried against.
+* `primaryKey` - *Serializable, Conditional* - Value for the primary key field of the table being queried.
+Type should match the table's primary key field's type.
+If a `primaryKey` is not given, then a `uniqueKey` must be given.
+* `uniqueKey` - *Map of String -> Serializable, Conditional* - Map of name-value pairs that define the record to be fetcheed.
+Keys in the map must be field names from the table being queried.
+Values in the map should should be of types that correspond to the fields.
+If a `primaryKey` is not given, then a `uniqueKey` must be given.
+If both `primaryKey` and `uniqueKey` are given, then `uniqueKey` is ignored.
+* `transaction` - *QBackendTransaction object* - Optional transaction object.
+** Behavior for this object is backend-dependant.
+In an RDBMS backend, this object is generally needed if you want your query to see data that may have been modified within the same transaction.
+
+* `shouldTranslatePossibleValues` - *boolean, default: false* - Controls whether any fields in the table with a *possibleValueSource* assigned to them should have those possible values looked up
+(e.g., to provide text translations in the generated records' `displayValues` map).
+** For example, if getting a record to present to a user, this would generally need to be *true*.
+But if getting a record as part of a process, then this can generally be left as *false*.
+* `shouldGenerateDisplayValues` - *boolean, default: false* - Controls whether field level *displayFormats* should be used to populate the generated records' `displayValues` map.
+** For example, if getting a record to present to a user, this would generally need to be *true*.
+But if getting a record as part of a process, then this can generally be left as *false*.
+* `shouldFetchHeavyFields` - *boolean, default: true* - Controls whether or not fields marked as `isHeavy` should be fetched & returned or not.
+* `shouldOmitHiddenFields` - *boolean, default: true* - Controls whether or not fields marked as `isHidden` should be included in the result or not.
+* `shouldMaskPassword` - *boolean, default: true* - Controls whether or not fields with `type` = `PASSWORD` should be masked, or if their actual values should be returned.
+* `queryJoins` - *List of <<QueryJoin>> objects* - Optional list of tables to be joined with the main table being queried.
+See QueryJoin under <<QueryAction>> for further details.
+* `includeAssociations` - *boolean, default: false* - Control whether or not records from tables defined as `associations` under the table being queried should be included in the result or not.
+* `associationNamesToInclude` - *Collection of String* - If `includeAssociations` is true, then this field can be used to limit which associated tables are included.
+If this field is null, then all associated tables are included.
+Otherwise, a table is only included if its name is in this collection.
+
+=== GetOutput
+* `record` - *QRecord* - The record that was specified by the input `primaryKey` or `uniqueKey`.
+Will be null if the record was not found.

docs/actions/InsertAction.adoc

@@ -0,0 +1,86 @@
+== InsertAction
+include::../variables.adoc[]
+
+To insert (add, create) new records into any {link-table}, the `*InsertAction*` is used.
+In SQL/RDBMS terms, it is analogous to a `INSERT` statement, where one or more records can be provided as input.
+
+=== Examples
+[source,java]
+.Canonical InsertAction invocation
+----
+InsertInput insertInput = new InsertInput();
+insertInput.setTableName("person");
+insertInput.setRecords(personRecordList);
+InsertOutput insertOutput = new InsertAction().execute(insertInput);
+List<QRecord> insertedPersonRecords = insertOutput.getRecords();
+----
+
+=== Details
+`InsertAction` does several things beyond just inserting records into the specified table.
+A high-level flow of its internal logic is:
+
+. For tables using an automation status field, set its value to `PENDING_INSERT_AUTOMATIONS` for all `records` that are going to be inserted.
+. Perform the following validations, which include running the table's `PRE_INSERT_CUSTOMIZER`, if one is defined, at the time that is specified by the customizer's `WhenToRun` property (default is `AFTER_ALL_VALIDATIONS`):
+.. Ensure that default values specified in the table's fields are present if needed.
+.. Apply per-field behaviors, as defined in {link-field} meta-data, such as truncating strings that are longer than their specified max.
+.. Check for unique key violations (done here instead of in the backend, to provide better error messaging, and to allow a subset of records to be stored while some fail).
+_We might want to make an input control in the future to specify that either the full input set should succeed or fail..._
+.. Validate that required fields (again, per {link-field} meta-data) are set, generating per-record errors if they are not.
+.. Validate any security fields in the records - e.g., ensure that the user has permission to insert records with the values they are attempting to insert.
+. Send the records to the table's backend module to actually insert them into the backend storage.
+. If the table has any associations defined, and if associated records are present, then recursively run `InsertAction` on the associated records.
+.. In particular, before these recursive `InsertAction` calls are made, values that were generated by the original insert may need to be propagated down into the associated records.
+*** For example, if inserting `order` and `lineItem` records, where a {link-join} exists between the two tables on `order.id` and `lineItem.orderId`, and `order.id` values were generated in the first `InsertAction`, then those values are propagated down into the associated `lineItem.orderId` fields.
+. If the {link-instance} has an `audit` table, then based on the {link-table}'s audit rules, audits about the inserted records are created.
+. If the table has a `POST_INSERT_CUSTOMIZER`, it is executed.
+
+=== Overloads
+
+`InsertAction` can be called in a few alternate forms, mostly just for convenience:
+
+[source,java]
+.If inserting a single record, get that record back instead of the InsertOutput:
+----
+InsertInput insertInput = new InsertInput();
+insertInput.setTableName("person");
+insertInput.setRecords(List.of(personRecord));
+QRecord insertedRecord = new InsertAction().executeForRecord(insertInput);
+
+// or more compactly, using InsertInput.withRecord (instead of withRecords)
+QRecord insertedRecord = new InsertAction()
+   .executeForRecord(new InsertInput("person").withRecord(personRecord));
+----
+
+[source,java]
+.Taking QRecordEntity objects as inputs instead of QRecords:
+----
+// insert a list of person entities:
+InsertInput insertInput = new InsertInput("person").withRecordEntities(personList);
+InsertOutput insertOutput = new InsertAction().execute(insertInput);
+
+// or for a single person entity (also mapping the output record back to an entity):
+Person insertedPerson = new Person(new InsertAction()
+   .executeForRecord(new InsertInput("person").withRecordEntity(person)));
+
+----
+
+=== InsertInput
+* `table` - *String, Required* - Name of the table that records are to be inserted into.
+* `records` - *List of QRecord, Required* - List of records to be inserted into the table.
+If the list is empty, the insert action does a `noop`.
+
+.Less common options
+* `inputSource` - *InputSource object, default: QInputSource.SYSTEM* - an indicator of what the source of the action is - generally, a `SYSTEM` triggered action, or a `USER` triggered action.
+** `InsertAction` will call the `shouldValidateRequiredFields()` method on this object to determine if it should validate that required fields on the records have values.
+Both `QInputSource.SYSTEM` and `QInputSource.USER` return `true` from this method, but application can define their own `InputSource` objects with different behavior.
+** In addition, this field can be used in pre- and post-insert customizers to drive further custom logic.
+* `skipUniqueKeyCheck` - *boolean, default: false* - control whether or not `InsertAction` should check for unique key violations before attempting to insert its records.
+** In a context where one has previously done this validation, or is okay letting the backend provide such checks, they may wish to avoid re-doing this work, and thus may set this property to `true`.
+* `omitDmlAudit` - *boolean, default: false* - control if the automatic DML audit that `InsertAction` generally performs should be omitted.
+* `auditContext` - *String* - optional message which can be included in DML audit messages, to give users more context about why the insert occurred.
+
+
+=== InsertOutput
+* `records` - *List of QRecord* - Copy of the input list of records, with details added based on the results of the input action.
+** If there were warnings or errors, the corresponding field (`warnings` or `errors`) will be set in the records.
+** If the insert action generated any values (such as a serial id or a default value), those values will be in the record's `fields` map.

docs/index.adoc

@@ -26,12 +26,12 @@ include::metaData/Reports.adoc[leveloffset=+1]
 include::metaData/Icons.adoc[leveloffset=+1]
 include::metaData/PermissionRules.adoc[leveloffset=+1]
 
-
 == Custom Application Code
 include::misc/QContext.adoc[leveloffset=+1]
 include::misc/QRecords.adoc[leveloffset=+1]
 include::misc/QRecordEntities.adoc[leveloffset=+1]
 include::misc/ProcessBackendSteps.adoc[leveloffset=+1]
+include::misc/RenderingWidgets.adoc[leveloffset=+1]
 
 === Table Customizers
 #todo#
@@ -61,3 +61,6 @@ include::actions/InsertAction.adoc[leveloffset=+1]
 == QQQ Default Implementations
 include::implementations/TableSync.adoc[leveloffset=+1]
 // later... include::actions/RenderTemplateAction.adoc[leveloffset=+1]
+
+== QQQ Utility Classes
+include::utilities/RecordLookupHelper.adoc[leveloffset=+1]

docs/justfile

@@ -0,0 +1,12 @@
+## https://github.com/casey/just
+
+default:
+   just --list
+
+build-index-html:
+   asciidoctor -a docinfo=shared index.adoc
+   line=$(grep 'Last updated' index.html) && sed -i "s/id=\"content\">/&$line/" index.html
+
+build-and-publish-index-html: build-index-html
+   scp index.html first-node:/mnt/first-volume/dkelkhoff/nginx/html/justinsgotskinnylegs.com/qqq-docs.html
+   @echo "Updated: https://justinsgotskinnylegs.com/qqq-docs.html"

docs/metaData/Icons.adoc

@@ -0,0 +1,20 @@
+[#Icons]
+== Icons
+include::../variables.adoc[]
+
+#TODO#
+
+=== QIcon
+Icons are defined in a QQQ Instance in a `*QIcon*` object.
+
+#TODO#
+
+*QIcon Properties:*
+
+* `name` - *String* - Name of an icon from the https://mui.com/material-ui/material-icons/[Material UI Icon set]
+** Note that icon names from the link above need to be converted from _CamelCase_ to _underscore_case_...
+* `path` - *String* - Path to a file served under the application's web root, to be used as the icon.
+
+_Either `name` or `path` must be specified. If both are given, then name is used._
+
+

docs/metaData/PermissionRules.adoc

@@ -0,0 +1,13 @@
+[#PermissionRules]
+== Permission Rules
+include::../variables.adoc[]
+
+#TODO#
+
+=== PermissionRule
+#TODO#
+
+*PermissionRule Properties:*
+
+#TODO#
+

docs/metaData/QInstance.adoc

@@ -0,0 +1,39 @@
+[#QInstance]
+== QInstance
+include::../variables.adoc[]
+
+An application in QQQ is defined as a set of Meta Data objects.  These objects are all stored together in an object called a `*QInstance*`.
+
+Currently, a `QInstance` must be programmatically constructed in java code - e.g., by constructing objects which get added to the QInstance, for example:
+
+[source,java]
+.Adding meta-data for two tables and one process to a QInstance
+----
+QInstance qInstance = new QInstance();
+qInstance.addTable(definePersonTable());
+qInstance.addTable(defineHomeTable());
+qInstance.addProcess(defineSendPersonHomeProcess());
+----
+
+It is on the QQQ roadmap to allow meta-data to be defined in a non-programmatic way, for example, in YAML or JSON files, or even from a dynamic data source (e.g. a document or relational database).
+
+The middleware and/or frontends being used in an application will drive how the `QInstance` is connected to the running server/process.
+For example, using the `qqq-middleware-javalin` module, a the `QJavalinImplementation` class () has a constructor which takes a `QInstance` as an argument:
+
+[source,java]
+.Starting a QQQ Javalin middleware server - passing a QInstance as a parameter to the new QJavalinImplementation
+----
+QJavalinImplementation qJavalinImplementation = new QJavalinImplementation(qInstance);
+Javalin service = Javalin.create();
+service.routes(qJavalinImplementation.getRoutes());
+service.start();
+----
+
+*QBackendMetaData Setup Methods:*
+
+These are the methods that one is most likely to use when setting up (defining) a `QInstance` object:
+
+* asdf
+
+*QBackendMetaData Usage Methods:*
+

docs/metaData/SecurtiyKeyTypes.adoc

@@ -2,16 +2,79 @@
 == Security Key Types
 include::../variables.adoc[]
 
-#TODO#
+In QQQ, record-level security is provided by using a lock & key metaphor.
+
+The use-case being handled here is:
+
+* A user has full permission on a table (query, insert, update, and delete).
+* However, they should only be allowed to read a sub-set of the rows in the table.
+** e.g., maybe it's a multi-tenant system, or the table has user-specific records.
+
+The lock & key metaphor is realized by the user being associated with one or more "Keys"
+(as values in their session), and records in tables being associated with one or more "Locks"
+(as values in fields).
+A user is only allowed to access records where the user's key(s) match the record's security lock(s).
+
+For a practical example, picture a multi-tenant Order Management System,where all orders are assigned to a "client".
+Users (customers) should only be able to see orders associated with the client which that user works for.
+
+In this scenario, the `order` table would have a "lock" on its `clientId` field.
+Customer-users would have a `clientId` key in their session.
+When the QQQ backend did a search for records (e.g., an SQL query) it would implicitly
+(without any code being written by the application developer) filter the table to only
+allow the user to see records with their `clientId`.
+
+To implement this scenario, the application would define the following pieces of meta-data:
+
+* At the QQQ-Instance level, a `SecurityKeyType`,
+to define a domain of possible locks & keys within an application.
+** An application can define multiple Security Key Types.
+For example, maybe `clientId` and `userId` as key types.
+* At the per-table level, a `RecordSecurityLock`,
+which references a security key type, and how that key type should be applied to the table.
+** For example, what field stores the `clientId` value in the `order` table.
+* Finally, when a user's session is constructed via a QQQ Authentication provider,
+security key values are set, based on data from the authentication backend.
+
+=== Additional Scenarios
+
+==== All Access Key
+A "super-user" may be allowed to access all records in a table regardless of their record locks,
+if the Security Key Type specifies an `allAccessKeyName`,
+and if the user has a key in their session with that key name, and a value of `true`.
+Going back to the lock & key metaphor, this can be thought of as a "skeleton key",
+that can unlock any record lock (of the security key's type).
+
+==== Null Value Behaviors
+In a record security lock, different behaviors can be defined for handling rows with a null key value.
+
+For example:
+
+* Sometimes orders may be loaded into the OMS system described above, where the application doesn't yet know what client the order belongs to.
+In this case, the application may need to ensure that such records, with a `null` value in `clientId` are hidden from customer-users,
+to avoid potentially leaking a different client's data.
+** This can be accomplished with a record security lock on the `order` table, with a `nullValueBehavior` of `DENY`.
+** Furthermore, if internal/admin users _should_ be given access to such records, then the security key type can be
+configured with a `nullValueBehaviorKeyName` (e.g., `"clientIdNullValueBehavior"`), which can be set per-user to allow
+access to records, overriding the table lock's specified `nullValueBehavior`.
+*** This could also be done by giving internal/admin users an `allAccessKey`, but sometimes that is not what is required.
+
+* Maybe a warehouse locations table is assigned a `clientId` once inventory for a client is placed in the location,
+at which point in time, only the client's users should be allowed to see the record.
+But, if no client has been assigned to the location, and `clientId` is `null`,
+then you may want to allow any user to see such records.
+** This can be accomplished with a record security lock on the Warehouse Locations table, with a `nullValueBehavior` of `ALLOW`.
 
 === QSecurityKeyType
 A Security Key Type is defined in a QQQ Instance in a `*QSecurityKeyType*` object.
 
-#TODO#
-
 *QSecurityKeyType Properties:*
 
-* `name` - *String, Required* - Unique name for the security key type within the QQQ Instance.
+* `name` - *String, Required* - Unique name for this security key type within the QQQ Instance.
+* `allAccessKeyName` - *String* - Optional name of the all-access security key associated with this key type.
+* `nullValueBehaviorKeyName` - *String* - Optional name of the null-value-behavior overriding security key associated with this key type.
+** Note, `name`, `allAccessKeyName`, and `nullValueBehaviorKeyName` are all checked against each other for uniqueness.
+A `QInstanceValidationException` will be thrown if any name collisions occur.
+* `possibleValueSourceName` - *String* - Optional reference to a possible value source from which value for the key can come.
 
-#TODO#
 

docs/metaData/Tables.adoc

@@ -244,3 +244,41 @@ QQQ provides the mechanism for UI's to present and manage such scripts (e.g., th
 * `scriptTypeId` - *Serializable (typically Integer), Required* - primary key value from the `"scriptType"` table in the instance, to designate the type of the Script.
 * `scriptTester` - *QCodeReference* - reference to a class which implements `TestScriptActionInterface`, that can be used by UI's for running an associated script to test it.
 
+
+
+=== Supplemental Meta Data
+==== QQQ Frontend Material Dashboard
+When running a QQQ application with the QQQ Frontend Material Dashboard module (QFMD),
+there are various pieces of supplemental meta-data which can be assigned to a Table,
+to modify some behaviors for the table in this UI.
+
+===== Default Quick Filter Field Names
+QFMD's table query has a "Basic" mode, which will always display a subset of the table's fields as quick-access filters.
+By default, the "Tier 1" fields on a table (e.g., fields in a Section that is marked as T1) will be used for this purpose.
+
+However, you can customize which fields are shown as the default quick-filter fields, by providing a list of field names in a
+`MaterialDashboardTableMetaData` object, placed in the table's `supplementalMetaData`.
+
+[source,java]
+----
+table.withSupplementalMetaData(new MaterialDashboardTableMetaData()
+    .withDefaultQuickFilterFieldNames(List.of("id", "warehouseId", "statusId", "orderDate")));
+----
+
+===== Go To Field Names
+QFMD has a feature where a table's query screen can include a "Go To" button,
+which a user can hit to open a modal popup, into which the user can enter a record's identifier,
+to be brought directly to the record matching that identifier.
+
+To use this feature, the table must have a List of `GotoFieldNames` set in its
+`MaterialDashboardTableMetaData` object in the table's `supplementalMetaData`.
+
+Each entry in this list is actually a list of fields, e.g., to account for a multi-value unique-key.
+
+[source,java]
+----
+table.withSupplementalMetaData(new MaterialDashboardTableMetaData()
+    .withGotoFieldNames(List.of(
+        List.of("id"),
+        List.of("partnerName", "partnerOrderId"))));
+----

docs/metaData/Tables.html

@@ -1,553 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-<head>
-<meta charset="UTF-8">
-<meta http-equiv="X-UA-Compatible" content="IE=edge">
-<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="generator" content="Asciidoctor 2.0.18">
-<title>QQQ Tables</title>
-<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
-<style>
-/*! Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */
-/* Uncomment the following line when using as a custom stylesheet */
-/* @import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"; */
-html{font-family:sans-serif;-webkit-text-size-adjust:100%}
-a{background:none}
-a:focus{outline:thin dotted}
-a:active,a:hover{outline:0}
-h1{font-size:2em;margin:.67em 0}
-b,strong{font-weight:bold}
-abbr{font-size:.9em}
-abbr[title]{cursor:help;border-bottom:1px dotted #dddddf;text-decoration:none}
-dfn{font-style:italic}
-hr{height:0}
-mark{background:#ff0;color:#000}
-code,kbd,pre,samp{font-family:monospace;font-size:1em}
-pre{white-space:pre-wrap}
-q{quotes:"\201C" "\201D" "\2018" "\2019"}
-small{font-size:80%}
-sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}
-sup{top:-.5em}
-sub{bottom:-.25em}
-img{border:0}
-svg:not(:root){overflow:hidden}
-figure{margin:0}
-audio,video{display:inline-block}
-audio:not([controls]){display:none;height:0}
-fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em}
-legend{border:0;padding:0}
-button,input,select,textarea{font-family:inherit;font-size:100%;margin:0}
-button,input{line-height:normal}
-button,select{text-transform:none}
-button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}
-button[disabled],html input[disabled]{cursor:default}
-input[type=checkbox],input[type=radio]{padding:0}
-button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}
-textarea{overflow:auto;vertical-align:top}
-table{border-collapse:collapse;border-spacing:0}
-*,::before,::after{box-sizing:border-box}
-html,body{font-size:100%}
-body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;line-height:1;position:relative;cursor:auto;-moz-tab-size:4;-o-tab-size:4;tab-size:4;word-wrap:anywhere;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased}
-a:hover{cursor:pointer}
-img,object,embed{max-width:100%;height:auto}
-object,embed{height:100%}
-img{-ms-interpolation-mode:bicubic}
-.left{float:left!important}
-.right{float:right!important}
-.text-left{text-align:left!important}
-.text-right{text-align:right!important}
-.text-center{text-align:center!important}
-.text-justify{text-align:justify!important}
-.hide{display:none}
-img,object,svg{display:inline-block;vertical-align:middle}
-textarea{height:auto;min-height:50px}
-select{width:100%}
-.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em}
-div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0}
-a{color:#2156a5;text-decoration:underline;line-height:inherit}
-a:hover,a:focus{color:#1d4b8f}
-a img{border:0}
-p{line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility}
-p aside{font-size:.875em;line-height:1.35;font-style:italic}
-h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em}
-h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0}
-h1{font-size:2.125em}
-h2{font-size:1.6875em}
-h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em}
-h4,h5{font-size:1.125em}
-h6{font-size:1em}
-hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em}
-em,i{font-style:italic;line-height:inherit}
-strong,b{font-weight:bold;line-height:inherit}
-small{font-size:60%;line-height:inherit}
-code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)}
-ul,ol,dl{line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit}
-ul,ol{margin-left:1.5em}
-ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0}
-ul.circle{list-style-type:circle}
-ul.disc{list-style-type:disc}
-ul.square{list-style-type:square}
-ul.circle ul:not([class]),ul.disc ul:not([class]),ul.square ul:not([class]){list-style:inherit}
-ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0}
-dl dt{margin-bottom:.3125em;font-weight:bold}
-dl dd{margin-bottom:1.25em}
-blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd}
-blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)}
-@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2}
-h1{font-size:2.75em}
-h2{font-size:2.3125em}
-h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em}
-h4{font-size:1.4375em}}
-table{background:#fff;margin-bottom:1.25em;border:1px solid #dedede;word-wrap:normal}
-table thead,table tfoot{background:#f7f8f7}
-table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left}
-table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)}
-table tr.even,table tr.alt{background:#f8f8f7}
-table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{line-height:1.6}
-h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em}
-h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400}
-.center{margin-left:auto;margin-right:auto}
-.stretch{width:100%}
-.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table}
-.clearfix::after,.float-group::after{clear:both}
-:not(pre).nobreak{word-wrap:normal}
-:not(pre).nowrap{white-space:nowrap}
-:not(pre).pre-wrap{white-space:pre-wrap}
-:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed}
-pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed}
-pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit}
-pre>code{display:block}
-pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal}
-em em{font-style:normal}
-strong strong{font-weight:400}
-.keyseq{color:rgba(51,51,51,.8)}
-kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;border-radius:3px;box-shadow:0 1px 0 rgba(0,0,0,.2),inset 0 0 0 .1em #fff;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap}
-.keyseq kbd:first-child{margin-left:0}
-.keyseq kbd:last-child{margin-right:0}
-.menuseq,.menuref{color:#000}
-.menuseq b:not(.caret),.menuref{font-weight:inherit}
-.menuseq{word-spacing:-.02em}
-.menuseq b.caret{font-size:1.25em;line-height:.8}
-.menuseq i.caret{font-weight:bold;text-align:center;width:.45em}
-b.button::before,b.button::after{position:relative;top:-1px;font-weight:400}
-b.button::before{content:"[";padding:0 3px 0 2px}
-b.button::after{content:"]";padding:0 2px 0 3px}
-p a>code:hover{color:rgba(0,0,0,.9)}
-#header,#content,#footnotes,#footer{width:100%;margin:0 auto;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em}
-#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table}
-#header::after,#content::after,#footnotes::after,#footer::after{clear:both}
-#content{margin-top:1.25em}
-#content::before{content:none}
-#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0}
-#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf}
-#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px}
-#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:flex;flex-flow:row wrap}
-#header .details span:first-child{margin-left:-.125em}
-#header .details span.email a{color:rgba(0,0,0,.85)}
-#header .details br{display:none}
-#header .details br+span::before{content:"\00a0\2013\00a0"}
-#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)}
-#header .details br+span#revremark::before{content:"\00a0|\00a0"}
-#header #revnumber{text-transform:capitalize}
-#header #revnumber::after{content:"\00a0"}
-#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem}
-#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em}
-#toc>ul{margin-left:.125em}
-#toc ul.sectlevel0>li>a{font-style:italic}
-#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0}
-#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none}
-#toc li{line-height:1.3334;margin-top:.3334em}
-#toc a{text-decoration:none}
-#toc a:active{text-decoration:underline}
-#toctitle{color:#7a2518;font-size:1.2em}
-@media screen and (min-width:768px){#toctitle{font-size:1.375em}
-body.toc2{padding-left:15em;padding-right:0}
-#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto}
-#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em}
-#toc.toc2>ul{font-size:.9em;margin-bottom:0}
-#toc.toc2 ul ul{margin-left:0;padding-left:1em}
-#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em}
-body.toc2.toc-right{padding-left:0;padding-right:15em}
-body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}}
-@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0}
-#toc.toc2{width:20em}
-#toc.toc2 #toctitle{font-size:1.375em}
-#toc.toc2>ul{font-size:.95em}
-#toc.toc2 ul ul{padding-left:1.25em}
-body.toc2.toc-right{padding-left:0;padding-right:20em}}
-#content #toc{border:1px solid #e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;border-radius:4px}
-#content #toc>:first-child{margin-top:0}
-#content #toc>:last-child{margin-bottom:0}
-#footer{max-width:none;background:rgba(0,0,0,.8);padding:1.25em}
-#footer-text{color:hsla(0,0%,100%,.8);line-height:1.44}
-#content{margin-bottom:.625em}
-.sect1{padding-bottom:.625em}
-@media screen and (min-width:768px){#content{margin-bottom:1.25em}
-.sect1{padding-bottom:1.25em}}
-.sect1:last-child{padding-bottom:0}
-.sect1+.sect1{border-top:1px solid #e7e7e9}
-#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400}
-#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em}
-#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible}
-#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none}
-#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221}
-details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em}
-details{margin-left:1.25rem}
-details>summary{cursor:pointer;display:block;position:relative;line-height:1.6;margin-bottom:.625rem;outline:none;-webkit-tap-highlight-color:transparent}
-details>summary::-webkit-details-marker{display:none}
-details>summary::before{content:"";border:solid transparent;border-left:solid;border-width:.3em 0 .3em .5em;position:absolute;top:.5em;left:-1.25rem;transform:translateX(15%)}
-details[open]>summary::before{border:solid transparent;border-top:solid;border-width:.5em .3em 0;transform:translateY(15%)}
-details>summary::after{content:"";width:1.25rem;height:1em;position:absolute;top:.3em;left:-1.25rem}
-.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic}
-table.tableblock.fit-content>caption.title{white-space:nowrap;width:0}
-.paragraph.lead>p,#preamble>.sectionbody>[class=paragraph]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)}
-.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%}
-.admonitionblock>table td.icon{text-align:center;width:80px}
-.admonitionblock>table td.icon img{max-width:none}
-.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase}
-.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6);word-wrap:anywhere}
-.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0}
-.exampleblock>.content{border:1px solid #e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;border-radius:4px}
-.exampleblock>.content>:first-child{margin-top:0}
-.exampleblock>.content>:last-child{margin-bottom:0}
-.sidebarblock{border:1px solid #dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;border-radius:4px}
-.sidebarblock>:first-child{margin-top:0}
-.sidebarblock>:last-child{margin-bottom:0}
-.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center}
-.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0}
-.literalblock pre,.listingblock>.content>pre{border-radius:4px;overflow-x:auto;padding:1em;font-size:.8125em}
-@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}}
-@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}}
-.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class=highlight],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8}
-.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)}
-.listingblock>.content{position:relative}
-.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5}
-.listingblock:hover code[data-lang]::before{display:block}
-.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5}
-.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"}
-.listingblock pre.highlightjs{padding:0}
-.listingblock pre.highlightjs>code{padding:1em;border-radius:4px}
-.listingblock pre.prettyprint{border-width:0}
-.prettyprint{background:#f7f7f8}
-pre.prettyprint .linenums{line-height:1.45;margin-left:2em}
-pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0}
-pre.prettyprint li code[data-lang]::before{opacity:1}
-pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none}
-table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none}
-table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal}
-table.linenotable td.code{padding-left:.75em}
-table.linenotable td.linenos,pre.pygments .linenos{border-right:1px solid;opacity:.35;padding-right:.5em;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}
-pre.pygments span.linenos{display:inline-block;margin-right:.75em}
-.quoteblock{margin:0 1em 1.25em 1.5em;display:table}
-.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em}
-.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify}
-.quoteblock blockquote{margin:0;padding:0;border:0}
-.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)}
-.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0}
-.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right}
-.verseblock{margin:0 1em 1.25em}
-.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans-serif;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility}
-.verseblock pre strong{font-weight:400}
-.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex}
-.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic}
-.quoteblock .attribution br,.verseblock .attribution br{display:none}
-.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)}
-.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none}
-.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0}
-.quoteblock.abstract{margin:0 1em 1.25em;display:block}
-.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center}
-.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf}
-.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0}
-.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem}
-.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;font-size:.85rem;text-align:left;margin-right:0}
-p.tableblock:last-child{margin-bottom:0}
-td.tableblock>.content{margin-bottom:1.25em;word-wrap:anywhere}
-td.tableblock>.content>:last-child{margin-bottom:-1.25em}
-table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede}
-table.grid-all>*>tr>*{border-width:1px}
-table.grid-cols>*>tr>*{border-width:0 1px}
-table.grid-rows>*>tr>*{border-width:1px 0}
-table.frame-all{border-width:1px}
-table.frame-ends{border-width:1px 0}
-table.frame-sides{border-width:0 1px}
-table.frame-none>colgroup+*>:first-child>*,table.frame-sides>colgroup+*>:first-child>*{border-top-width:0}
-table.frame-none>:last-child>:last-child>*,table.frame-sides>:last-child>:last-child>*{border-bottom-width:0}
-table.frame-none>*>tr>:first-child,table.frame-ends>*>tr>:first-child{border-left-width:0}
-table.frame-none>*>tr>:last-child,table.frame-ends>*>tr>:last-child{border-right-width:0}
-table.stripes-all>*>tr,table.stripes-odd>*>tr:nth-of-type(odd),table.stripes-even>*>tr:nth-of-type(even),table.stripes-hover>*>tr:hover{background:#f8f8f7}
-th.halign-left,td.halign-left{text-align:left}
-th.halign-right,td.halign-right{text-align:right}
-th.halign-center,td.halign-center{text-align:center}
-th.valign-top,td.valign-top{vertical-align:top}
-th.valign-bottom,td.valign-bottom{vertical-align:bottom}
-th.valign-middle,td.valign-middle{vertical-align:middle}
-table thead th,table tfoot th{font-weight:bold}
-tbody tr th{background:#f7f8f7}
-tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold}
-p.tableblock>code:only-child{background:none;padding:0}
-p.tableblock{font-size:1em}
-ol{margin-left:1.75em}
-ul li ol{margin-left:1.5em}
-dl dd{margin-left:1.125em}
-dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0}
-li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em}
-ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none}
-ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em}
-ul.unstyled,ol.unstyled{margin-left:0}
-li>p:empty:only-child::before{content:"";display:inline-block}
-ul.checklist>li>p:first-child{margin-left:-1em}
-ul.checklist>li>p:first-child>.fa-square-o:first-child,ul.checklist>li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em}
-ul.checklist>li>p:first-child>input[type=checkbox]:first-child{margin-right:.25em}
-ul.inline{display:flex;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em}
-ul.inline>li{margin-left:1.25em}
-.unstyled dl dt{font-weight:400;font-style:normal}
-ol.arabic{list-style-type:decimal}
-ol.decimal{list-style-type:decimal-leading-zero}
-ol.loweralpha{list-style-type:lower-alpha}
-ol.upperalpha{list-style-type:upper-alpha}
-ol.lowerroman{list-style-type:lower-roman}
-ol.upperroman{list-style-type:upper-roman}
-ol.lowergreek{list-style-type:lower-greek}
-.hdlist>table,.colist>table{border:0;background:none}
-.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none}
-td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em}
-td.hdlist1{font-weight:bold;padding-bottom:1.25em}
-td.hdlist2{word-wrap:anywhere}
-.literalblock+.colist,.listingblock+.colist{margin-top:-.5em}
-.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top}
-.colist td:not([class]):first-child img{max-width:none}
-.colist td:not([class]):last-child{padding:.25em 0}
-.thumb,.th{line-height:0;display:inline-block;border:4px solid #fff;box-shadow:0 0 0 1px #ddd}
-.imageblock.left{margin:.25em .625em 1.25em 0}
-.imageblock.right{margin:.25em 0 1.25em .625em}
-.imageblock>.title{margin-bottom:0}
-.imageblock.thumb,.imageblock.th{border-width:6px}
-.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em}
-.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0}
-.image.left{margin-right:.625em}
-.image.right{margin-left:.625em}
-a.image{text-decoration:none;display:inline-block}
-a.image object{pointer-events:none}
-sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super}
-sup.footnote a,sup.footnoteref a{text-decoration:none}
-sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline}
-#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em}
-#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0}
-#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em}
-#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em}
-#footnotes .footnote:last-of-type{margin-bottom:0}
-#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0}
-div.unbreakable{page-break-inside:avoid}
-.big{font-size:larger}
-.small{font-size:smaller}
-.underline{text-decoration:underline}
-.overline{text-decoration:overline}
-.line-through{text-decoration:line-through}
-.aqua{color:#00bfbf}
-.aqua-background{background:#00fafa}
-.black{color:#000}
-.black-background{background:#000}
-.blue{color:#0000bf}
-.blue-background{background:#0000fa}
-.fuchsia{color:#bf00bf}
-.fuchsia-background{background:#fa00fa}
-.gray{color:#606060}
-.gray-background{background:#7d7d7d}
-.green{color:#006000}
-.green-background{background:#007d00}
-.lime{color:#00bf00}
-.lime-background{background:#00fa00}
-.maroon{color:#600000}
-.maroon-background{background:#7d0000}
-.navy{color:#000060}
-.navy-background{background:#00007d}
-.olive{color:#606000}
-.olive-background{background:#7d7d00}
-.purple{color:#600060}
-.purple-background{background:#7d007d}
-.red{color:#bf0000}
-.red-background{background:#fa0000}
-.silver{color:#909090}
-.silver-background{background:#bcbcbc}
-.teal{color:#006060}
-.teal-background{background:#007d7d}
-.white{color:#bfbfbf}
-.white-background{background:#fafafa}
-.yellow{color:#bfbf00}
-.yellow-background{background:#fafa00}
-span.icon>.fa{cursor:default}
-a span.icon>.fa{cursor:inherit}
-.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default}
-.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c}
-.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111}
-.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900}
-.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400}
-.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000}
-.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);border-radius:50%;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold}
-.conum[data-value] *{color:#fff!important}
-.conum[data-value]+b{display:none}
-.conum[data-value]::after{content:attr(data-value)}
-pre .conum[data-value]{position:relative;top:-.125em}
-b.conum *{color:inherit!important}
-.conum:not([data-value]):empty{display:none}
-dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility}
-h1,h2,p,td.content,span.alt,summary{letter-spacing:-.01em}
-p strong,td.content strong,div.footnote strong{letter-spacing:-.005em}
-p,blockquote,dt,td.content,span.alt,summary{font-size:1.0625rem}
-p{margin-bottom:1.25rem}
-.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em}
-.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;box-shadow:0 1px 4px #e0e0dc}
-.print-only{display:none!important}
-@page{margin:1.25cm .75cm}
-@media print{*{box-shadow:none!important;text-shadow:none!important}
-html{font-size:80%}
-a{color:inherit!important;text-decoration:underline!important}
-a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important}
-a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em}
-abbr[title]{border-bottom:1px dotted}
-abbr[title]::after{content:" (" attr(title) ")"}
-pre,blockquote,tr,img,object,svg{page-break-inside:avoid}
-thead{display:table-header-group}
-svg{max-width:100%}
-p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3}
-h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid}
-#header,#content,#footnotes,#footer{max-width:none}
-#toc,.sidebarblock,.exampleblock>.content{background:none!important}
-#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important}
-body.book #header{text-align:center}
-body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em}
-body.book #header .details{border:0!important;display:block;padding:0!important}
-body.book #header .details span:first-child{margin-left:0!important}
-body.book #header .details br{display:block}
-body.book #header .details br+span::before{content:none!important}
-body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important}
-body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always}
-.listingblock code[data-lang]::before{display:block}
-#footer{padding:0 .9375em}
-.hide-on-print{display:none!important}
-.print-only{display:block!important}
-.hide-for-print{display:none!important}
-.show-for-print{display:inherit!important}}
-@media amzn-kf8,print{#header>h1:first-child{margin-top:1.25rem}
-.sect1{padding:0!important}
-.sect1+.sect1{border:0}
-#footer{background:none}
-#footer-text{color:rgba(0,0,0,.6);font-size:.9em}}
-@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}}
-</style>
-</head>
-<body class="article">
-<div id="header">
-</div>
-<div id="content">
-<div class="sect1">
-<h2 id="_qqq_tables">QQQ Tables</h2>
-<div class="sectionbody">
-<div class="paragraph">
-<p>The core type of object in a QQQ Instance is the Table.
-In the most common use-case, a QQQ Table may be the in-app representation of a Database table.
-That is, it is a collection of records (or rows) of data, each of which has a set of fields (or columns).</p>
-</div>
-<div class="paragraph">
-<p>QQQ also allows other types of data sources (<a href="Backends{relfilesuffix}">QQQ Backends</a>) to be used as tables, such as File systems, API&#8217;s, Java enums or objects, etc.
-All of these backend types present the same interfaces (both user-interfaces, and application programming interfaces), regardless of their backend type.</p>
-</div>
-<div class="sect2">
-<h3 id="_qtablemetadata">QTableMetaData</h3>
-<div class="paragraph">
-<p>Tables are defined in a QQQ Instance in a <code><strong>QTableMetaData</strong></code> object.
-All tables must reference a <a href="Backends{relfilesuffix}">QQQ Backend</a>, a list of fields that define the shape of records in the table, and additional data to describe how to work with the table within its backend.</p>
-</div>
-<div class="paragraph">
-<p><strong>QTableMetaData Properties:</strong></p>
-</div>
-<div class="ulist">
-<ul>
-<li>
-<p><code>name</code> - <strong>String, Required</strong> - Unique name for the table within the QQQ Instance.</p>
-</li>
-<li>
-<p><code>label</code> - <strong>String</strong> - User-facing label for the table, presented in User Interfaces.
-Inferred from <code>name</code> if not set.</p>
-</li>
-<li>
-<p><code>backendName</code> - <strong>String, Required</strong> - Name of a <a href="Backends{relfilesuffix}">QQQ Backend</a> in which this table&#8217;s data is managed.</p>
-</li>
-<li>
-<p><code>fields</code> - <strong>Map of String → <a href="Fields{relfilesuffix}">QQQ Field</a>, Required</strong> - The columns of data that make up all records in this table.</p>
-</li>
-<li>
-<p><code>primaryKeyField</code> - <strong>String, Conditional</strong> - Name of a <a href="Fields{relfilesuffix}">QQQ Field</a> that serves as the primary key (e.g., unique identifier) for records in this table.</p>
-</li>
-<li>
-<p><code>uniqueKeys</code> - <strong>List of UniqueKey</strong> - Definition of additional unique constraints (from an RDBMS point of view) from the table.
-e.g., sets of columns which must have unique values for each record in the table.</p>
-</li>
-<li>
-<p><code>backendDetails</code> - <strong>QTableBackendDetails or subclass</strong> - Additional data to configure the table within its <a href="Backends{relfilesuffix}">QQQ Backend</a>.</p>
-</li>
-<li>
-<p><code>automationDetails</code> - <strong>QTableAutomationDetails</strong> - Configuration of automated jobs that run against records in the table, e.g., upon insert or update.</p>
-</li>
-<li>
-<p><code>customizers</code> - <strong>Map of String → QCodeReference</strong> - References to custom code that are injected into standard table actions, that allow applications to customize certain parts of how the table works.</p>
-</li>
-<li>
-<p><code>parentAppName</code> - <strong>String</strong> - Name of a <a href="Apps{relfilesuffix}">QQQ App</a> that this table exists within.</p>
-</li>
-<li>
-<p><code>icon</code> - <strong>QIcon</strong> - Icon associated with this table in certain user interfaces.</p>
-</li>
-<li>
-<p><code>recordLabelFormat</code> - <strong>String</strong> - Java Format String, used with <code>recordLabelFields</code> to produce a label shown for records from the table.</p>
-</li>
-<li>
-<p><code>recordLabelFields</code> - <strong>List of String, Conditional</strong> - Used with <code>recordLabelFormat</code> to provide values for any format specifiers in the format string.
-These strings must be field names within the table.</p>
-<div class="ulist">
-<ul>
-<li>
-<p>Example of using <code>recordLabelFormat</code> and <code>recordLabelFields</code>:</p>
-</li>
-</ul>
-</div>
-</li>
-</ul>
-</div>
-<div class="listingblock">
-<div class="content">
-<pre class="CodeRay highlight"><code data-lang="java"><span style="color:#777">// given these fields in the table:</span>
-<span style="color:#080;font-weight:bold">new</span> QFieldMetaData(<span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">name</span><span style="color:#710">&quot;</span></span>, QFieldType.STRING)
-<span style="color:#080;font-weight:bold">new</span> QFieldMetaData(<span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">birthDate</span><span style="color:#710">&quot;</span></span>, QFieldType.DATE)
-
-<span style="color:#777">// We can produce a record label such as &quot;Darin Kelkhoff (1980-05-31)&quot; via:</span>
-.withRecordLabelFormat(<span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">%s (%s)</span><span style="color:#710">&quot;</span></span>)
-.withRecordLabelFields(<span style="color:#0a8;font-weight:bold">List</span>.of(<span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">name</span><span style="color:#710">&quot;</span></span>, <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">birthDate</span><span style="color:#710">&quot;</span></span>))</code></pre>
-</div>
-</div>
-<div class="ulist">
-<ul>
-<li>
-<p><code>sections</code> - <strong>List of QFieldSection</strong> - Mechanism to organize fields within user interfaces, into logical sections.
-If any sections are present in the table meta data, then all fields in the table must be listed in exactly 1 section.
-If no sections are defined, then instance enrichment will define default sections.</p>
-</li>
-<li>
-<p><code>associatedScripts</code> - <strong>List of AssociatedScript</strong> - Definition of user-defined scripts that can be associated with records within the table.</p>
-</li>
-<li>
-<p><code>enabledCapabilities</code> and <code>disabledCapabilities</code> - <strong>Set of Capability enum values</strong> - Overrides from the backend level, for capabilities that this table does or does not possess.</p>
-</li>
-</ul>
-</div>
-</div>
-</div>
-</div>
-</div>
-<div id="footer">
-<div id="footer-text">
-Last updated 2022-11-21 09:02:56 -0600
-</div>
-</div>
-</body>
-</html>

docs/metaData/Widgets.adoc

@@ -2,16 +2,50 @@
 == Widgets
 include::../variables.adoc[]
 
-#TODO#
+Widgets are the most customizable UI components in QQQ.
+They can be used either on App Home Screens (e.g., as Dashboard screens),
+or they can be included into Record View screens.
+
+QQQ defines several types of widgets, such as charts (pie, bar, line),
+numeric displays, application-populated tables, or even fully custom HTML.
 
 === QWidgetMetaData
 A Widget is defined in a QQQ Instance in a `*QWidgetMetaData*` object.
 
-#TODO#
-
 *QWidgetMetaData Properties:*
 
 * `name` - *String, Required* - Unique name for the widget within the QQQ Instance.
+* `type` - *String, Required* - Specifies the UI & data type for the widget.
+* `label` - *String* - User-facing header or title for a widget.
+* `tooltip` - *String* - Text contents to be placed in a tooltip associated with the widget's label in the UI.
+** Values should come from the `WidgetType` enum's `getType()` method (e.g., `WidgetType.BAR_CHART.getType()`)
+* `gridColumns` - *Integer* - for a desktop-sized screen, in a 12-based grid,
+how many columns the widget should consume.
+* `codeReference` - *QCodeReference, Required* - Reference to the custom code,
+a subclass of `AbstractWidgetRenderer`, which is responsible for loading data to render the widget.
+* `footerHTML` - *String* - HTML String, which, if present, will be displayed in the
+footer of the widget (not supported by all widget types).
+* `isCard` - *boolean, default false* #TODO#
+* `showReloadButton` - *boolean, default true* #TODO#
+* `showExportButton` - *boolean, default false* #TODO#
+* `dropdowns` - #TODO#
+* `storeDropdownSelections` - *boolean* #TODO#
+* `icons` - *Map<String, QIcon>* #TODO#
+* `defaultValues` - *Map<String, Serializable>* #TODO#
 
-#TODO#
+There are also some subclasses of `QWidgetMetaData`, for some specific widget types:
+
+*ParentWidgetMetaData Properties:*
+
+* `title` - *String* #TODO - how does this differ from label?#
+* `childWidgetNameList` - *List<String>, Required*
+* `childProcessNameList` - *List<String>* #TODO appears unused - check, and delete#
+* `laytoutType` - *enum of GRID or TABS, default GRID*
+
+*QNoCodeWidgetMetaData Properties:*
+
+* `values` - *List<AbstractWidgetValueSource>* #TODO#
+* `outputs` - *List<AbstractWidgetOutput>* #TODO#
+
+#TODO - Examples#
 

docs/misc/ProcessBackendSteps.adoc

@@ -0,0 +1,155 @@
+== Process Backend Steps
+include::../variables.adoc[]
+
+In many QQQ applications, much of the code that engineers write will take the form of Backend Steps for {link-processes}.
+Such code is defined in classes which implement the interface `BackendStep`.  
+This interface defines only a single method:
+[source,java]
+.BackendStep.java
+----
+public interface BackendStep
+{
+   /*******************************************************************************
+    ** Execute the backend step - using the request as input, and the result as output.
+    **
+    *******************************************************************************/
+   void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException;
+}
+
+----
+
+Process backend steps have access to state information - specifically, a list of records, and a map of name=value pairs - in the input & output objects.
+This state data is persisted by QQQ between steps (e.g., if a frontend step is presented to a user between backend steps).
+
+
+=== RunBackendStepInput
+All input data to the step is available in the `RunBackendStepInput` object.
+Key methods in this class are:
+
+* `getRecords()` - Returns the List of <<QRecords>> that are currently being acted on in the process.
+* `getValues()` - Returns a Map of String -> Serializable; name=value pairs that are the state-data of the process.
+** Values can be added to this state from a process's meta-data, from a screen, or from another backend step.
+* `getValue(String fieldName)` - Returns a specific value, by name, from the process's state.
+** This method has several variations that return the value as a specific type, such as `getValueString`, `getValueInteger`, `getValueBoolean`...
+* `getAsyncJobCallback()` - Accessor for an `AsyncJobCallback` object, which provides a way for the process backend step to communicate about its status or progress with a user running the process in a frontend.
+Provides methods:
+** `updateStatus(String message)` - Where general status messages can be given.
+For example, `"Loading census data"`
+** `updateStatus(int current, int total)` - For updating a progress meter.
+e.g., "47 of 1701" would be display by calling `.updateStatus(47, 1701)`
+* `getFrontendStepBehavior()` - Enum, indicating what should happen when a frontend step is encountered as the process's next step to run.
+Possible values are:
+** `BREAK` - Indicates that the process's execution should be suspended, so that the screen represented by the frontend step can be presented to a user.
+This would be the expected behavior if a process is being run by a user from a UI.
+** `SKIP` - Indicates that frontend steps should be skipped.
+This would be the expected behavior if a process is running from a scheduled job (without a user present to drive it), for example.
+** `FAIL` - Indicates that the process should end with an exception if a frontend step is encountered.
+** A backend step may want to act differently based on its frontendStepBehavior.
+For example, additional data may be looked up for displaying to a user if the behavior is `BREAK`.
+* `getBasepullLastRunTime()` - For <<BasepullConfiguration,Basepull>> processes, this is the `Instant` stored in the basepull table as the process's last run time.
+
+=== RunBackendStepOutput
+All output from a process step should be placed in its `RunBackendStepOutput` object (and/or stored to a backend, as appropriate).
+Key methods in this class are:
+
+* `addValue(String fieldName, Serializable value)` - Adds a single named value to the process's state, overwriting it the value if it already exists.
+* `addRecord(QRecord record)` - Add a `<<QRecord>>` to the process's output.
+* `addAuditSingleInput(AuditSingleInput auditSingleInput)` - Add a new entry to the process's list of audit inputs, to be stored at the completion of the process.
+** An `AuditSingleInput` object can most easily be built with the constructor: `AuditSingleInput(String tableName, QRecord record, String auditMessage)`.
+** Additional audit details messages (sub-bullets that accompany the primary `auditMessage`) can be added to an `AuditSingleInput` via the `addDetail(String message)` method.
+** _Note that at this time, the automatic storing of these audits is only provided by the execute step of a StreamedETLWithFrontendProcesses._
+
+=== Example
+
+[source,java]
+.Example of a BackendStep
+----
+/*******************************************************************************
+ ** For the "person" table's "Add Age" process -
+ ** For each input person record, add the specified yearsToAdd to their age.
+ *******************************************************************************/
+public class AddAge implements BackendStep
+{
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput)
+   {
+      /////////////////////////////////////////////////////////////////
+      // get the yearsToAdd input field value from the process input //
+      /////////////////////////////////////////////////////////////////
+      Integer yearsToAdd = runBackendStepInput.getValueInteger("yearsToAdd");
+      int totalYearsAdded = 0;
+
+      ///////////////////////////////////////////////////
+      // loop over the records passed into the process //
+      ///////////////////////////////////////////////////
+      for(QRecord record : runBackendStepInput.getRecords())
+      {
+         Integer age = record.getValueInteger("age");
+         age += yearsToAdd;
+         totalYearsAdded += yearsToAdd;
+
+         ////////////////////////////////////////////////////////////////////////////////////////////
+         // update the record with the new "age" value.                                            //
+         // note that this update record object will implicitly be available to the process's next //
+         // backend step, via the sharing of the processState object.                              //
+         ////////////////////////////////////////////////////////////////////////////////////////////
+         record.setValue("age", age);
+      }
+
+      /////////////////////////////////////////
+      // set an output value for the process //
+      /////////////////////////////////////////
+      runBackendStepOutput.addValue("totalYearsAdded", totalYearsAdded);
+   }
+}
+
+----
+
+
+=== Backend Steps for StreamedETLWithFrontendProcesses
+
+For <<StreamedETLWithFrontendProcess>> type processes, backend steps are defined a little bit differently than they are for other process types.
+In this type of process, the process meta-data defines 3 backend steps which are built-in to QQQ, and which do not have any custom application logic.
+These steps are:
+
+* `StreamedETLPreviewStep`
+* `StreamedETLValidateStep`
+* `StreamedETLExecuteStep`
+
+For custom application logic to be implemented in a StreamedETLWithFrontendProcesses, an application engineer must define (up to) 3 backend step classes which are loaded by the steps listed above.
+These application-defined steps must extend specific classes (which themselves implement the `BackendStep` interface), to provide the needed logic of this style of process.
+These steps are:
+
+* *Extract* - a subclass of `AbstractExtractStep` - is responsible for Extracting records from the source table.
+** For this step, we can often use the QQQ-provided class `ExtractViaQueryStep`, or sometimes a subclass of it.
+** The Extract step is called before the Preview, Validate, and Result screens, though for the Preview screen, it is set to only extract a small number of records (10).
+* *Transform* - a subclass of `AbstractTransformStep` - is responsible for applying the majority of the business logic of the process.
+In ETL terminology, this is the "Transform" action - which means applying some type of logical transformation an input record (found by the Extract step) to generate an output record (stored by the Load step).
+** A Transform step's `run` method will be called, potentially, multiple times, each time with a page of records in the `runBackendStepInput` parameter.
+** This method is responsible for adding records to the `runBackendStepOutput`, which will then be passed to the *Load* step.
+** This class is also responsible for implementing the method `getProcessSummary`, which provides the data to the *Validate* screen.
+** The run method will generally update ProcessSummaryLine objects to facilitate this functionality.
+** The Transform step is called before the Preview, Validate, and Result screens, consuming all records selected by the Extract step.
+* *Load* - a subclass of `AbstractLoadStep` - is responsible for the Load function of the ETL job.
+_A quick word on terminology - this step is actually doing what we are more likely to think of as storing data - which feels like the opposite of “loading” - but we use the name Load to keep in line with the ETL naming convention…_
+** The Load step is ONLY called before the Result screen is presented (possibly after Preview, if the user chose to skip validation, otherwise, after validation).
+** Similar to the Transform step, the Load step's `run` method will be called potentially multiple times, with pages of records in its input.
+** As such, the Load step is generally the only step where data writes should occur.
+*** e.g., a Transform step should not do any writes, as it will be called when the user is going to the Preview & Validate screens - e.g., before the user confirmed that they want to execute the action!
+** A common pattern is that the Load step just needs to insert or update the list of records output by the Transform step, in which case the QQQ-provided `LoadViaInsertStep` or `LoadViaUpdateStep` can be used, but custom use-cases can be built as well.
+
+Another distinction between StreamedELTWithFrontendProcess steps and general QQQ process backend steps, is that the list of records in the input & output objects is NOT shared for StreamedELTWithFrontendProcess steps.
+The direct implication of this is, that a Transform step MUST explicitly call `output.addRecord()` for any records that it wants to pass along to the Load step.
+
+==== Example
+
+[source,java]
+.Examples of a Transform and Load step for a StreamedELTWithFrontendProcess
+----
+   // todo!
+----
+
+#todo: more details on these 3 specialized types of process steps (e.g., method to overload, when stuff like pre-action is called; how summaries work).#

docs/misc/QContext.adoc

@@ -0,0 +1,30 @@
+== QContext
+include::../variables.adoc[]
+
+The class `QContext` contains a collection of thread-local variables, to define the current context of the QQQ code that is currently running.
+For example, what `QInstance` (meta-data container) is being used, what `QSession` (user attributes) is active, etc.
+
+Most of the time, main-line application code does not need to worry about setting up the `QContext` - although unit-test code can is a common exception to that rule.
+This is because all of QQQ's entry-points into execution (e.g., web context handlers, CLI startup methods, schedulers, and multi-threaded executors) take care of initializing the context.
+
+It is more common though, for application code to need to get data from the context, such as the current session or any piece of meta-data from the QInstance.
+The methods to access data from the `QContext` are fairly straightforward:
+
+=== Examples
+==== Get a QTableMetaData from the active QInstance
+[source,java]
+----
+QTableMeataData myTable = QContext.getQInstance().getTable("myTable");
+for(QFieldMeataData field : myTable.getFields().values())
+{
+   // ...
+}
+----
+
+==== Get a security key value for the current user session
+[source,java]
+----
+QSession           session   = QContext.getQSession();
+List<Serializable> clientIds = session.getSecurityKeyValues("clientId");
+----
+

docs/misc/QRecordEntities.adoc

@@ -0,0 +1,116 @@
+== QRecordEntities
+include::../variables.adoc[]
+
+While `<<QRecords>>` provide a flexible mechanism for passing around record data in a QQQ Application, they have one big disadvantage from the point-of-view of a Java Application:
+They do not provide a mechanism to ensure compile-time checks of field names or field types.
+
+As such, an alternative mechanism exists, which allows records in a QQQ application to be worked with following a more familiar Java Bean (Entity Bean) like pattern.
+This mechanism is known as a `QRecordEntity`.
+
+Specifically speaking, `QRecordEntity` is an abstract base class, whose purpose is to be the base class for entity-bean classes.
+Using reflection, `QRecordEntity` is able to convert objects back and forth from `QRecord` to specific entity-bean subtypes.
+
+For example, the method `QRecordEntity::toQRecord()` converts a record entity object into a `QRecord`.
+
+Inversely, `QRecordEntity::populateFromQRecord(QRecord record)` sets fields in a record entity object, based on the values in the supplied `QRecord`.
+It is conventional for a subclass of `QRecordEntity` to have both a no-arg (default) constructor, and a constructor that takes a `QRecord` as a parameter, and calls `populateFromQRecord`.
+
+In addition to these constructors, a `QRecordEntity` subclass will generally contain:
+
+* A `public static final String TABLE_NAME`, used throughout the application as a constant reference to the name for the {link-table}.
+* A series of `private` fields, corresponding to the fields in the table that the entity represents.
+** If these fields are annotated as `@QField()`, then the {link-table} meta-data for the table that the entity represents can in part be inferred by QQQ, by using the method `QTableMetaData::withFieldsFromEntity`.
+* `getX()`, `setX()`, and `withX()` methods for all of the entity's fields.
+
+=== Examples
+[source,java]
+.Example Definition of a QRecordEntity subclass:  Person.java
+----
+/*******************************************************************************
+ ** QRecordEntity for the person table.
+ *******************************************************************************/
+public class Person extends QRecordEntity
+{
+   public static final String TABLE_NAME = "person";
+
+   @QField(isEditable = false)
+   private Integer id;
+
+   @QField()
+   private String firstName;
+
+   @QField()
+   private String lastName;
+
+   @QField()
+   private Integer age;
+
+   // all other fields
+
+   /*******************************************************************************
+    ** Default constructor
+    *******************************************************************************/
+   public Person()
+   {
+   }
+
+
+   /*******************************************************************************
+    ** Constructor that takes a QRecord
+    *******************************************************************************/
+   public Person(QRecord record)
+   {
+      populateFromQRecord(record);
+   }
+
+
+   /*******************************************************************************
+    ** Custom method to concatenate firstName and lastName
+    *******************************************************************************/
+   public String getFullName()
+   {
+      //////////////////////////////////////////////////////////////////////
+      // if there were more than an example, we could do some null-checks //
+      // here to avoid silly output like "Bobby null" :)                  //
+      //////////////////////////////////////////////////////////////////////
+      return (firstName + " " + lastName);
+   }
+
+   // all getter, setter, and fluent setter (wither) methods
+}
+----
+
+The core ORM actions and process-execution actions of QQQ work with `QRecords`.
+However, application engineers may want to apply patterns like the following example, to gain the compile-time safety of `QRecordEntities`:
+
+[source,java]
+.Example Usage of a QRecordEntity
+----
+//////////////////////////////////////////////////////////////////////
+// assume we're working with the "person" table & entity from above //
+//////////////////////////////////////////////////////////////////////
+List<QRecord> recordsToUpdate = new ArrayList<>();
+for(QRecord record : inputRecordList)
+{
+   /////////////////////////////////////////////////////////////////////////////
+   // call the constructor that copies values from the record into the entity //
+   /////////////////////////////////////////////////////////////////////////////
+   Person person = new Person(record);
+
+   ////////////////////////////////////////////////
+   // call a custom method defined in the entity //
+   ////////////////////////////////////////////////
+   LOG.info("Processing: " + person.getFullName());
+
+   /////////////////////////////////////////////////////////////
+   // age is an Integer, so no type-conversion is needed here //
+   /////////////////////////////////////////////////////////////
+   person.setAge(person.getAge() + 1);
+
+   ///////////////////////////////////////////////////////////////////////////
+   // to pass the updated records to UpdateAction, convert them to QRecords //
+   ///////////////////////////////////////////////////////////////////////////
+   recordsToUpdate.add(person.toQRecord());
+}
+----
+

docs/misc/QRecords.adoc

@@ -0,0 +1,68 @@
+== QRecords
+include::../variables.adoc[]
+
+Almost all code inside a QQQ application will be dealing with *Records* (aka Tuples or Rows).
+That is: a collection of named values, representing a single Entity, Fact, or, Row from a {link-table}.
+
+The class that QQQ uses to work with records is called:  `QRecord`.
+
+=== Values
+At its core, a `QRecord` is a wrapper around a `Map<String, Serializable> values`.
+These are the *actual* values for the fields in the table for the record.
+That is, direct representations of the values as they are stored in the {link-backend}.
+
+The keys in the `values` map are names from the {link-fields} in the {link-table}.
+
+The values in `values` map are declared as `Serializable` (to help ensure the serializability of the `QRecord` as a whole).
+In practice, their types will be based on the `QFieldType` of the {link-field} that they correspond to.
+That will typically be one of: `String`, `Integer`, `Boolean`, `BigDecimal`, `Instant`, `LocalDate`, `LocalTime`, or `byte[]`.
+Be aware that `null` values may be in the `values` map, especially/per if the backend/table support `null`.
+
+To work with the `values` map, the following methods are provided:
+
+* `setValue(String fieldName, Serializable value)` - Sets a value for the specified field in the record.
+** Overloaded as `setValue(String fieldName, Object value)` - For some cases where the value may not be known to be `Serializable`.
+In this overload, if the value is `null` or `Serializable`, the primary version of `setValue` is called.
+Otherwise, the `value` is passed through `String::valueOf`, and the result is stored.
+** Overloaded as `setValue(QFieldMetaData field, Serializable value)` - which simply defers to the primary version of `setValue`, passing `field.getName()` as the first parameter.
+* `removeValue(String fieldName)` - Remove the given field from the `values` map.
+** Note that in some situations this is an important distinction from having a `null` value in the map (See <<UpdateAction)>>).
+* `setValues(Map<String, Serializable> values)` - Sets the full map of `values`.
+* `getValues()` - Returns the full map of `values`.
+* `getValue(String fieldName)` - Returns the value for the named field - possibly `null` - as a `Serializable`.
+* Several type-specific variations of `getValueX(String fieldName)`, where internally, values will be not exactly type-cast, but effectively converted (if possible) to the requested type.
+These conversions are done using the `ValueUtils.getValueAsX(Object)` methods.
+These methods are generally the preferred/cleanest way to get record values in application code, when it is needed in a type-specific way .
+** `getValueString(String fieldName)`
+** `getValueInteger(String fieldName)`
+** `getValueBoolean(String fieldName)`
+** `getValueBigDecimal(String fieldName)`
+** `getValueInstant(String fieldName)`
+** `getValueLocalDate(String fieldName)`
+** `getValueLocalTime(String fieldName)`
+** `getValueByteArray(String fieldName)`
+
+=== Display Values
+In addition to the `values` map, a `QRecord` contains another map called `displayValues`, which only stores `String` values.
+That is to say, values for other types are stringified, based on their {link-field}'s type and `displayFormat` property.
+In addition, fields which have a `possibleValueSource` property will have their translated values set in the `displayValues` map.
+
+By default, a `QRecord` will not have its `displayValues` populated.
+To populate `displayValues`, the <<QueryAction>> and <<GetAction>> classes take a property in their inputs called `shouldGenerateDisplayValues`, which must be set to `true` to generate `displayValues`.
+In addition, these two actions also have a property `shouldTranslatePossibleValues` in their inputs, which needs to be set to `true` if possible-value lookups are to be performed.
+
+As an alternative to the `shouldGenerateDisplayValues` and `shouldTranslatePossibleValues` inputs to <<QueryAction>> and <<GetAction>>, one can directly call the `QValueFormatter.setDisplayValuesInRecords` and/or `qPossibleValueTranslator.translatePossibleValuesInRecords` methods.
+Or, for special cases, `setDisplayValue(String fieldName, String displayValue)` or `setDisplayValues(Map<String, String> displayValues)` can be called directly.
+
+=== Backend Details
+Sometimes a backend may want to place additional data in a `QRecord` that doesn't exactly correspond to a field.
+To do this, the `Map<String, Serializable> backendDetails` member is used.
+
+For example, an API backend may store the full JSON `String` that came from the API as a backend detail in a `QRecord`.
+Or fields that are marked as `isHeavy`, if the full (heavy) value of the field hasn't been fetched, then the lengths of any such heavy fields may be stored in `backendDetails`.
+
+=== Errors and Warnings
+#todo#
+
+=== Associated Records
+#todo#

docs/misc/RenderingWidgets.adoc

@@ -0,0 +1,224 @@
+== Rendering Widgets
+include::../variables.adoc[]
+
+=== WidgetRenderer classes
+In general, to fully implement a Widget, you must define its `QWidgetMetaData`,
+and supply a subclass of `AbstractWidgetRenderer`, to provide the data to the widget.
+(Note the "No Code" category of widgets, which are an exception to this generalization).
+
+The only method required in a subclass of `AbstractWidgetRenderer` is:
+
+   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+
+The fields available in `RenderWidgetInput` are:
+
+- `Map<String, String> queryParams` - These are parameters supplied by the frontend, for example,
+if a user selected values from dropdowns to control a dimension of your widget, those name/value
+pairs would be in this map.  Similarly, if your widget is being included on a record view screen, then
+the record's primary key will be in this map.
+- `QWidgetMetaDataInterface widgetMetaData` - This is the meta-data for the widget being
+rendered.  This can be useful in case you are using the same renderer class for multiple widgets.
+
+The only field in `RenderWidgetOutput` is:
+
+- `QWidgetData widgetData` - This is a base class, with several attributes, and more importantly,
+several subclasses, specific to the type of widget that is being rendered.
+
+==== Widget-Type Specific Rendering Details
+Different widget types expect & require different types of values to be set in the `RenderWidgetOutput` by their renderers.
+
+===== Pie Chart
+The `WidgetType.PIE_CHART` requires an object of type `ChartData`.
+The fields on this type are:
+
+* `chartData` an instance of `ChartData.Data`, which has the following fields:
+** `labels` - *List<String>, required* - the labels for the slices of the pie.
+** `datasets` - *List<Dataset> required* - the data for each slice of the pie.
+For a Pie chart, only a single entry in this list is used
+(other chart types using `ChartData` may support more than 1 entry in this list).
+Fields in this object are:
+*** `label` - *String, required* - a label to describe the dataset as a whole.
+e.g., "Orders" for a pie showing orders of different statuses.
+*** `data` - *List<Number>, required* - the data points for each slice of the pie.
+*** `color` - *String* - HTML color for the slice
+*** `urls` - *List<String>* - Optional URLs for slices of the pie to link to when clicked.
+*** `backgroundColors` - *List<String>* - Optional HTML color codes for each slice of the pie.
+
+[source,java]
+.Pie chart widget example
+----
+// meta data
+new QWidgetMetaData()
+    .withName("pieChartExample")
+    .withType(WidgetType.PIE_CHART.getType())
+    .withGridColumns(4)
+    .withIsCard(true)
+    .withLabel("Pie Chart Example")
+    .withCodeReference(new QCodeReference(PieChartExampleRenderer.class));
+
+// renderer
+private List<String> labels = new ArrayList<>();
+private List<String> colors = new ArrayList<>();
+private List<Number> data   = new ArrayList<>();
+
+/*******************************************************************************
+** helper method - to add values for a slice to the lists
+*******************************************************************************/
+private void addSlice(String label, String color, Number datum)
+{
+    labels.add(label);
+    colors.add(color);
+    data.add(datum);
+}
+
+/*******************************************************************************
+** main method of the widget renderer
+*******************************************************************************/
+@Override
+public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+{
+   addSlice("Apple", "#FF0000", 100);
+   addSlice("Orange", "#FF8000", 150);
+   addSlice("Banana", "#FFFF00", 75);
+   addSlice("Lime", "#00FF00", 100);
+   addSlice("Blueberry", "#0000FF", 200);
+
+   ChartData chartData = new ChartData()
+      .withChartData(new ChartData.Data()
+         .withLabels(labels)
+         .withDatasets(List.of(
+            new ChartData.Data.Dataset()
+               .withLabel("Flavor")
+               .withData(data)
+               .withBackgroundColors(colors)
+               .withUrls(urls))));
+
+   return (new RenderWidgetOutput(chartData));
+}
+
+----
+
+===== Bar Chart
+#todo#
+
+===== Stacked Bar Chart
+#todo#
+
+===== Horizontal Bar Chart
+#todo#
+
+===== Child Record List
+#todo#
+
+===== Line Chart
+#todo#
+
+===== Small Line Chart
+#todo#
+
+===== Statistics
+#todo#
+
+===== Parent Widget
+#todo#
+
+===== Composite
+A `WidgetType.COMPOSITE` is built by using one or more smaller elements, known as `Blocks`.
+Note that `Blocks` can also be used in the data of some other widget types
+(specifically, within a cell of a Table-type widget, or (in the future?) as a header above a pie or bar chart).
+
+A composite widget renderer must return data of type `CompositeWidgetData`,
+which has the following fields:
+
+* `blocks` - *List<AbstractBlockWidgetData>, required* - The blocks (1 or more) being composited together to make the widget.
+See below for details on the specific Block types.
+* `styleOverrides` - *Map<String, Serializable>* - Optional map of CSS attributes
+(named following javascriptStyleCamelCase) to apply to the `<div>` element that wraps the rendered blocks.
+* `layout` - *Layout enum* - Optional specifier for how the blocks should be laid out.
+e.g., predefined sets of CSS attributes to achieve specific layouts.
+** Note that some blocks are designed to work within composites with specific layouts.
+Look for matching names, such as `Layout.BADGES_WRAPPER` to go with `NumberIconBadgeBlock`.
+
+[source,java]
+.Composite widget example - consisting of 3 Progress Bar Blocks, and one Divider Block
+----
+// meta data
+new QWidgetMetaData()
+    .withName("compositeExample")
+    .withType(WidgetType.COMPOSITE.getType())
+    .withGridColumns(4)
+    .withIsCard(true)
+    .withLabel("Composite Example")
+    .withCodeReference(new QCodeReference(CompositeExampleRenderer.class));
+
+// renderer
+public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+{
+   CompositeWidgetData data = new CompositeWidgetData();
+
+   data.addBlock(new ProgressBarBlockData()
+      .withValues(new ProgressBarValues()
+         .withHeading("Blocks")
+         .withPercent(new BigDecimal("78.5"))));
+
+   data.addBlock(new ProgressBarBlockData()
+      .withValues(new ProgressBarValues()
+         .withHeading("Progress")
+         .withPercent(new BigDecimal(0))));
+
+   data.addBlock(new DividerBlockData());
+
+   data.addBlock(new ProgressBarBlockData()
+      .withStyles(new ProgressBarStyles().withBarColor("#C0C000"))
+      .withValues(new ProgressBarValues()
+         .withHeading("Custom Color")
+         .withPercent(new BigDecimal("75.3"))));
+
+   return (new RenderWidgetOutput(data));
+}
+----
+
+
+===== Table
+#todo#
+
+===== HTML
+#todo#
+
+===== Divider
+#todo#
+
+===== Process
+#todo#
+
+===== Stepper
+#todo#
+
+===== Data Bag Viewer
+#todo#
+
+===== Script Viewer
+#todo#
+
+=== Block-type Specific Rendering Details
+For Composite-type widgets (or other widgets which can include blocks),
+there are specific data classes required to be returned by the widget renderer.
+
+Each block type defines a subclass of `AbstractBlockWidgetData`,
+which is a generic class with 3 type parameters:
+
+* `V` - an implementation of `BlockValuesInterface` - to define the type of values that the block uses.
+* `S` - an implementation of `BlockSlotsInterface` (expected to be an `enum`) - to define the "slots" in the block,
+that can have Tooltips and/or Links applied to them.
+* `SX` - an implementation of `BlockStylesInterface` - to define the types of style customizations that the block supports.
+
+These type parameters are designed to ensure type-safety for the application developer,
+to ensure that only
+
+=== Additional Tips
+
+* To make a Dashboard page (e.g., an App consisting of Widgets) with a parent widget use the parent widget's label as the page's label:
+** On the `QAppMetaData` that contains the Parent widget, call
+`.withSupplementalMetaData(new MaterialDashboardAppMetaData().withShowAppLabelOnHomeScreen(false))`.
+** In the Parent widget's renderer, on the `ParentWidgetData`, call `setLabel("My Label")` and
+`setIsLabelPageTitle(true)`.

docs/utilities/RecordLookupHelper.adoc

@@ -0,0 +1,272 @@
+== RecordLookupHelper
+include::../variables.adoc[]
+
+`RecordLookupHelper` is a utility class that exists to help you... lookup records :)
+
+OK, I'll try to give a little more context:
+
+=== Motivation 1:  Performance
+One of the most significant performance optimizations that the team behind QQQ has found time and time again,
+is to minimize the number of times you have to perform an I/O operation.
+To just say it more plainly:
+Make fewer calls to your database (or other backend).
+
+This is part of why the DML actions in QQQ (InsertAction, UpdateAction, DeleteAction) are all written to work on multiple records:
+If you've got to insert 1,000 records, the performance difference between doing that as 1,000 SQL INSERT statements vs. just 1 statement cannot be overstated.
+
+Similarly then, for looking up records:
+If we can do 1 round-trip to the database backend - that is - 1 query to fetch _n_ records,
+then in almost all cases it will be significantly faster than doing _n_ queries, one-by-one, for those _n_ records.
+
+The primary reason why `RecordLookupHelper` exists is to help you cut down on the number of times you have to make a round-trip to a backend data store to fetch records within a process.
+
+[sidebar]
+This basically is version of caching, isn't it?
+Take a set of data from "far away" (e.g., database), and bring it "closer" (local or instance variables), for faster access.
+So we may describe this as a "cache" through the rest of this document.
+
+=== Motivation 2:  Convenience
+
+So, given that one wants to try to minimize the number of queries being executed to look up data in a QQQ processes,
+one can certainly do this "by-hand" in each process that they write.
+
+Doing this kind of record caching in a QQQ Process `BackendStep` may be done as:
+
+* Adding a `Map<Integer, QRecord>` as a field in your class.
+* Setting up and running a `QueryAction`, with a filter based on the collection of the keys you need to look up, then iterating over (or streaming) the results into the map field.
+* Getting values out of the map when you need to use them (dealing with missing values as needed).
+
+That's not so bad, but, it does get a little verbose, especially if you're going to have several such caches in your class.
+
+As such, the second reason that `RecordLookupHelper` exists, is to be a reusable and convenient way to do this kind of optimization,
+by providing methods to perform the bulk query & map building operation described above,
+while also providing some convenient methods for accessing such data after it's been fetched.
+In addition, a single instance of `RecordLookupHelper` can provide this service for multiple tables at once
+(e.g., so you don't need to add a field to your class for each type of data that you're trying to cache).
+
+=== Use Cases
+==== Preload records, then access them
+Scenario:
+
+* We're writing a process `BackendStep` that uses `shipment` records as input.
+* We need to know the `order` record associated with each `shipment` (via an `orderId` foreign key), for some business logic that isn't germaine to the explanation of `RecordLookupHelper`.
+* We also to access some field on the `shippingPartner` record assigned to each `shipment`.
+** Note that here, the `shipment` table has a `partnerCode` field, which relates to the `code` unique-key in the `shippingPartner` table.
+** It's also worth mentioning, we only have a handful of `shippingPartner` records in our database, and we never expect to have very many more than that.
+
+[source,java]
+.Example of a process step using a RecordLookupHelper to preload records
+----
+public class MyShipmentProcessStep implements BackendStep
+{
+   // Add a new RecordLookupHelper field, which will "cache" both orders and shippingPartners
+   private RecordLookupHelper recordLookupHelper = new RecordLookupHelper();
+
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException;
+   {
+      // lookup the full shippingPartner table (because it's cheap & easy to do so)
+      // use the partner's "code" as the key field (e.g,. they key in the helper's internal map).
+      recordLookupHelper.preloadRecords("shippingPartner", "code");
+
+      // get all of the orderIds from the input shipments
+      List<Serializable> orderIds = input.getRecords().stream()
+         .map(r -> r.getValue("id")).toList();
+
+      // fetch the orders related to by these shipments
+      recordLookupHelper.preloadRecords("order", "id", orderIds);
+
+      for(QRecord shipment : input.getRecords())
+      {
+         // get someConfigField from the shippingPartner assigned to the shipment
+         Boolean someConfig = recordLookupHelper.getRecordValue("shippingPartner", "someConfigField", "code", shipment.getValue("partnerCode"));
+
+         // get the order record assigned to the shipment
+         QRecord order = recordLookupHelper.getRecordByKey("order", "id", shipment.getValue("orderId"));
+      }
+   }
+}
+----
+
+==== Lazy fetching records
+Scenario:
+
+* We have a `BackendStep` that is taking in `purchaseOrderHeader` records, from an API partner.
+* For each record, we need to make an API call to the partner to fetch the `purchaseOrderLine` records under that header.
+** In this contrived example, the partner's API forces us to do these lookups order-by-order...
+* Each `purchaseOrderLine` that we fetch will have a `sku` on it - a reference to our `item` table.
+** We need to look up each `item` to apply some business logic.
+** We assume there are very many item records in the backend database, so we don't want to pre-load the full table.
+Also, we don't know what `sku` values we will need until we fetch the `purchaseOrderLine`.
+
+This is a situation where we can use `RecordLookupHelper` to lazily fetch the `item` records as we discover them,
+and it will take care of not re-fetching ones that it has already loaded.
+
+[source,java]
+.Example of a process step using a RecordLookupHelper to lazy fetch records
+----
+public class MyPurchaseOrderProcessStep implements BackendStep
+{
+   // Add a new RecordLookupHelper field, which will "cache" lazy-loaded item records
+   private RecordLookupHelper recordLookupHelper = new RecordLookupHelper();
+
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException;
+   {
+      for(QRecord poHeader : input.getRecords())
+      {
+         // fetch the lines under the header
+         Serializable poNo = poHeader.getValue("poNo");
+         List<QRecord> poLines = new QueryAction().execute(new QueryInput("purchaseOrderLine")
+            .withFilter(new QQueryFilter(new QFilterCriteria("poNo", EQUALS, poNo))));
+
+         for(QRecord poLine : poLines)
+         {
+            // use recordLookupHelper to lazy-load item records by SKU.
+            QRecord item = recordLookupHelper.getRecordByKey("item", "sku", poLine.getValue("sku"));
+
+            // business logic related to item performed here.
+         }
+      }
+   }
+}
+----
+
+In this example, we will be doing exactly 1 query on the `item` table for each unique `sku` that is found across all of the `poLine` records we process.
+That is to say, if the same `sku` appears on only 1 `poLine`, or if it appears on 100 `poLines`, we will still only query once for that `sku`.
+
+A slight tweak could be made to the example above, to make 1 `item` table lookup for each `poHeader` record:
+
+[source,java]
+.Tweaked example doing 1 item lookup per poLine
+----
+// continuing from above, after the List<QRecord> poLines has been built
+
+// get all of the skus from the lines
+List<Serializable> skus = poLines.stream().map(r -> r.getValue("sku")).toList();
+
+// preload the items for the skus
+recordLookupHelper.preloadRecords("item", "sku", new QQueryFilter(new QFilterCriteria("sku", IN, skus)));
+
+for(QRecord poLine : poLines)
+{
+   // get the items from the helper
+   QRecord item = recordLookupHelper.getRecordByKey("item", "sku", poLine.getValue("sku"));
+
+----
+
+In this example, we've made a trade-off:  We will query the `item` table exactly 1 time for each `poHeader` that we process.
+However, if the same `sku` is on every PO that we process, we will end up fetching it multiple times.
+
+This could end up being better or worse than the previous example, depending on the distribution of the data we are dealing with.
+
+A further tweak, a hybrid approach, could potentially reap the benefits of both of these examples (at the tradeoff of, more code, more complexity):
+
+[source,java]
+.Tweaked example doing 1 item lookup per poLine, but only for not-previously-encountered skus
+----
+// Critically - we must tell our recordLookupHelper to NOT do any one-off lookups in this table
+recordLookupHelper.setMayNotDoOneOffLookups("item", "sku");
+
+// continuing from above, after the List<QRecord> poLines has been built
+
+// get all of the skus from the lines
+List<Serializable> skus = poLines.stream().map(r -> r.getValue("sku")).toList();
+
+// determine which skus have not yet been loaded - e.g., they are not in the recordLookupHelper.
+// this is why we needed to tell it above not to do one-off lookups; else it would lazy-load each sku here.
+List<Serializable> skusToLoad = new ArrayList<>();
+for(Serializable sku : skus)
+{
+   if(recordLookupHelper.getRecordByKey("item", "sku", sku) == null)
+   {
+      skusToLoad.add(sku);
+   }
+}
+
+// preload the item records for any skus that are still needed
+if(!skusToLoad.isEmpty())
+{
+   recordLookupHelper.preloadRecords("item", "sku",
+      new QQueryFilter(new QFilterCriteria("sku", IN, skusToLoad)));
+}
+
+// continue as above
+
+----
+
+In this example, we will start by querying the `item` table once for each `poHeader`, but,
+if we eventually encounter a PO where all of its `skus` have already been loaded, then we may be able to avoid any `item` queries for such a PO.
+
+=== Implementation Details
+
+* Internally, an instance of `RecordLookupHelper` maintains a number of `Maps`,
+with QQQ table names and field names as keys.
+* The accessing/lazy-fetching methods (e.g., any method whose name starts with `getRecord`)
+all begin by looking in these internal maps for the `tableName` and `keyFieldName` that they take as parameters.
+** If they find an entry in the maps, then it is used for producing a return value.
+** If they do not find an entry, then they will perform the a `QueryAction`,
+to try to fetch the requested record from the table's backend.
+*** Unless the `setMayNotDoOneOffLookups` method has been called for the `(tableName, keyFieldName)` pair.
+
+=== Full API
+
+==== Methods for accessing and lazy-fetching
+* `getRecordByKey(String tableName, String keyFieldName, Serializable key)`
+
+Get a `QRecord` from `tableName`, where `keyFieldName` = `key`.
+
+* `getRecordValue(String tableName, String requestedField, String keyFieldName, Serializable key)`
+
+Get the field `requestedField` from the record in `tableName`, where `keyFieldName` = `key`, as a `Serializable`.
+If the record is not found, `null` is returned.
+
+* `getRecordValue(String tableName, String requestedField, String keyFieldName, Serializable key, Class<T> type)`
+
+Get the field `requestedField` from the record in `tableName`, where `keyFieldName` = `key`, as an instance of `type`.
+If the record is not found, `null` is returned.
+
+* `getRecordId(String tableName, String keyFieldName, Serializable key)`
+
+Get the primary key of the record in `tableName`, where `keyFieldName` = `key`, as a `Serializable`.
+If the record is not found, `null` is returned.
+
+* `getRecordId(String tableName, String keyFieldName, Serializable key, Class<T> type)`
+
+Get the primary key of the record in `tableName`, where `keyFieldName` = `key`, as an instance of `type`.
+If the record is not found, `null` is returned.
+
+* `getRecordByUniqueKey(String tableName, Map<String, Serializable> uniqueKey)`
+
+Get a `QRecord` from `tableName`, where the record matches the field/value pairs in `uniqueKey`.
+
+_Note: this method does not use the same internal map as the rest of the class.
+As such, it does not take advantage of any data fetched via the preload methods.
+It is only used for caching lazy-fetches._
+
+==== Methods for preloading
+* `preloadRecords(String tableName, String keyFieldName)`
+
+Query for all records from `tableName`, storing them in an internal map keyed by the field `keyFieldName`.
+
+* `preloadRecords(String tableName, String keyFieldName, QQueryFilter filter)`
+
+Query for records matching `filter` from `tableName`,
+storing them in an internal map keyed by the field `keyFieldName`.
+
+* `preloadRecords(String tableName, String keyFieldName, List<Serializable> inList)`
+
+Query for records with the field `keyFieldName` having a value in `inList` from `tableName`,
+storing them in an internal map keyed by the field `keyFieldName`.
+
+==== Config Methods
+* `setMayNotDoOneOffLookups(String tableName, String fieldName)`
+
+For cases where you know that you have preloaded records for `tableName`, keyed by `fieldName`,
+and you know that some of the keys may not have been found,
+so you want to avoid doing a query when a missed key is found in one of the `getRecord...` methods,
+then if you call this method, an internal flag is set, which will prevent any such one-off lookups.
+
+In other words, if this method has been called for a `(tableName, fieldName)` pair,
+then the `getRecord...` methods will only look in the internal map for records,
+and no queries will be performed to look for records.

pom.xml

@@ -80,12 +80,12 @@
          <dependency>
             <groupId>org.apache.logging.log4j</groupId>
             <artifactId>log4j-api</artifactId>
-            <version>2.17.1</version>
+            <version>2.23.0</version>
          </dependency>
          <dependency>
             <groupId>org.apache.logging.log4j</groupId>
             <artifactId>log4j-core</artifactId>
-            <version>2.17.1</version>
+            <version>2.23.0</version>
          </dependency>
          <dependency>
             <groupId>org.junit.jupiter</groupId>
@@ -330,28 +330,28 @@ fi
 
    <!-- mvn javadoc:aggregate -->
    <reporting>
-    <plugins>
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-javadoc-plugin</artifactId>
-        <version>3.6.2</version>
-        <reportSets>
-          <reportSet>
-            <id>aggregate</id>
-            <inherited>false</inherited>
-            <reports>
-              <report>aggregate</report>
-            </reports>
-          </reportSet>
-          <reportSet>
-            <id>default</id>
-            <reports>
-              <report>javadoc</report>
-            </reports>
-          </reportSet>
-        </reportSets>
-      </plugin>
-    </plugins>
+      <plugins>
+         <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-javadoc-plugin</artifactId>
+            <version>3.6.2</version>
+            <reportSets>
+               <reportSet>
+                  <id>aggregate</id>
+                  <inherited>false</inherited>
+                  <reports>
+                     <report>aggregate</report>
+                  </reports>
+               </reportSet>
+               <reportSet>
+                  <id>default</id>
+                  <reports>
+                     <report>javadoc</report>
+                  </reports>
+               </reportSet>
+            </reportSets>
+         </plugin>
+      </plugins>
    </reporting>
 
    <repositories>

qqq-backend-core/pom.xml

@@ -163,6 +163,19 @@
          <version>1.12.321</version>
       </dependency>
 
+      <dependency>
+         <groupId>org.quartz-scheduler</groupId>
+         <artifactId>quartz</artifactId>
+         <version>2.3.2</version>
+      </dependency>
+
+      <!-- Many of the deps we bring in use slf4j.  This dep maps slf4j to our logger, log4j -->
+      <dependency>
+         <groupId>org.apache.logging.log4j</groupId>
+         <artifactId>log4j-slf4j-impl</artifactId>
+         <version>2.23.0</version>
+      </dependency>
+
       <!-- Common deps for all qqq modules -->
       <dependency>
          <groupId>org.apache.maven.plugins</groupId>

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/async/AsyncJobManager.java

@@ -159,7 +159,8 @@ public class AsyncJobManager
    private <T extends Serializable> T runAsyncJob(String jobName, AsyncJob<T> asyncJob, UUIDAndTypeStateKey uuidAndTypeStateKey, AsyncJobStatus asyncJobStatus)
    {
       String originalThreadName = Thread.currentThread().getName();
-      Thread.currentThread().setName("Job:" + jobName + ":" + uuidAndTypeStateKey.getUuid().toString().substring(0, 8));
+      // Thread.currentThread().setName("Job:" + jobName + ":" + uuidAndTypeStateKey.getUuid().toString().substring(0, 8));
+      Thread.currentThread().setName("Job:" + jobName);
       try
       {
          LOG.debug("Starting job " + uuidAndTypeStateKey.getUuid());

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/async/AsyncRecordPipeLoop.java

@@ -83,6 +83,15 @@ public class AsyncRecordPipeLoop
       long    jobStartTime          = System.currentTimeMillis();
       boolean everCalledConsumer    = false;
 
+      ////////////////////////////////////////////////////////////////////////////
+      // in case the pipe capacity has been made very small (useful in tests!), //
+      // then make the minRecordsToConsume match it.                            //
+      ////////////////////////////////////////////////////////////////////////////
+      if(recordPipe.getCapacity() < minRecordsToConsume)
+      {
+         minRecordsToConsume = recordPipe.getCapacity();
+      }
+
       while(jobState.equals(AsyncJobState.RUNNING))
       {
          if(recordPipe.countAvailableRecords() < minRecordsToConsume)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/audits/AuditAction.java

@@ -295,7 +295,7 @@ public class AuditAction extends AbstractQActionFunction<AuditInput, AuditOutput
          }
          catch(Exception e)
          {
-            LOG.error("Error performing an audit", e);
+            LOG.warn("Error performing an audit", e);
          }
       }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/audits/DMLAuditAction.java

@@ -78,6 +78,7 @@ public class DMLAuditAction extends AbstractQActionFunction<DMLAuditInput, DMLAu
    private static Set<String> loggedUnauditableTableNames = new HashSet<>();
 
 
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -186,7 +187,7 @@ public class DMLAuditAction extends AbstractQActionFunction<DMLAuditInput, DMLAu
       }
       catch(Exception e)
       {
-         LOG.error("Error performing DML audit", e, logPair("type", String.valueOf(dmlType)), logPair("table", table.getName()));
+         LOG.warn("Error performing DML audit", e, logPair("type", String.valueOf(dmlType)), logPair("table", table.getName()));
       }
 
       return (output);
@@ -210,6 +211,19 @@ public class DMLAuditAction extends AbstractQActionFunction<DMLAuditInput, DMLAu
          contextSuffix.append(" ").append(input.getAuditContext());
       }
 
+      //////////////////////////////////////////////////////////////
+      // look for a context value place directly into the session //
+      //////////////////////////////////////////////////////////////
+      QSession qSession = QContext.getQSession();
+      if(qSession != null)
+      {
+         String sessionContext = qSession.getValue(AUDIT_CONTEXT_FIELD_NAME);
+         if(StringUtils.hasContent(sessionContext))
+         {
+            contextSuffix.append(" ").append(sessionContext);
+         }
+      }
+
       /////////////////////////////////////////////////////////////////////////////////////
       // note process label (and a possible context from the process's state) if present //
       /////////////////////////////////////////////////////////////////////////////////////
@@ -233,17 +247,20 @@ public class DMLAuditAction extends AbstractQActionFunction<DMLAuditInput, DMLAu
       ///////////////////////////////////////////////////
       // use api label & version if present in session //
       ///////////////////////////////////////////////////
-      QSession qSession   = QContext.getQSession();
-      String   apiVersion = qSession.getValue("apiVersion");
-      if(apiVersion != null)
+      if(qSession != null)
       {
-         String apiLabel = qSession.getValue("apiLabel");
-         if(!StringUtils.hasContent(apiLabel))
+         String apiVersion = qSession.getValue("apiVersion");
+         if(apiVersion != null)
          {
-            apiLabel = "API";
+            String apiLabel = qSession.getValue("apiLabel");
+            if(!StringUtils.hasContent(apiLabel))
+            {
+               apiLabel = "API";
+            }
+            contextSuffix.append(" via ").append(apiLabel).append(" Version: ").append(apiVersion);
          }
-         contextSuffix.append(" via ").append(apiLabel).append(" Version: ").append(apiVersion);
       }
+
       return (contextSuffix.toString());
    }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/automation/AutomationStatus.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.actions.automation;
 
 
+import java.util.Objects;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
 
 
@@ -55,6 +56,30 @@ public enum AutomationStatus implements PossibleValueEnum<Integer>
 
 
 
+   /*******************************************************************************
+    ** Get instance by id
+    **
+    *******************************************************************************/
+   public static AutomationStatus getById(Integer id)
+   {
+      if(id == null)
+      {
+         return (null);
+      }
+
+      for(AutomationStatus value : AutomationStatus.values())
+      {
+         if(Objects.equals(value.id, id))
+         {
+            return (value);
+         }
+      }
+
+      return (null);
+   }
+
+
+
    /*******************************************************************************
     ** Getter for id
     **
@@ -106,10 +131,10 @@ public enum AutomationStatus implements PossibleValueEnum<Integer>
    public String getInsertOrUpdate()
    {
       return switch(this)
-         {
-            case PENDING_INSERT_AUTOMATIONS, RUNNING_INSERT_AUTOMATIONS, FAILED_INSERT_AUTOMATIONS -> "Insert";
-            case PENDING_UPDATE_AUTOMATIONS, RUNNING_UPDATE_AUTOMATIONS, FAILED_UPDATE_AUTOMATIONS -> "Update";
-            case OK -> "";
-         };
+      {
+         case PENDING_INSERT_AUTOMATIONS, RUNNING_INSERT_AUTOMATIONS, FAILED_INSERT_AUTOMATIONS -> "Insert";
+         case PENDING_UPDATE_AUTOMATIONS, RUNNING_UPDATE_AUTOMATIONS, FAILED_UPDATE_AUTOMATIONS -> "Update";
+         case OK -> "";
+      };
    }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/automation/RecordAutomationStatusUpdater.java

@@ -22,30 +22,40 @@
 package com.kingsrook.qqq.backend.core.actions.automation;
 
 
+import java.io.Serializable;
+import java.time.Duration;
+import java.time.temporal.ChronoUnit;
 import java.util.Collections;
+import java.util.HashSet;
 import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.automation.TableTrigger;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.AutomationStatusTrackingType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.QTableAutomationDetails;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.TableAutomationAction;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.TriggerEvent;
-import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
 import org.apache.commons.lang.NotImplementedException;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -55,19 +65,37 @@ public class RecordAutomationStatusUpdater
 {
    private static final QLogger LOG = QLogger.getLogger(RecordAutomationStatusUpdater.class);
 
+   ///////////////////////////////////////////////////////////////////////////////////////////////////////
+   // feature flag - by default, will be true - before setting records to PENDING_UPDATE_AUTOMATIONS,   //
+   // we will fetch them (if we didn't take them in from the caller, which, UpdateAction does if its    //
+   // backend supports it), to check their current automationStatus - and if they are currently PENDING //
+   // or RUNNING inserts or updates, we won't update them.                                              //
+   ///////////////////////////////////////////////////////////////////////////////////////////////////////
+   private static boolean allowPreUpdateFetch = new QMetaDataVariableInterpreter().getBooleanFromPropertyOrEnvironment("qqq.recordAutomationStatusUpdater.allowPreUpdateFetch", "QQQ_RECORD_AUTOMATION_STATUS_UPDATER_ALLOW_PRE_UPDATE_FETCH", true);
+
+   ///////////////////////////////////////////////////////////////////////////////////////////////
+   // feature flag - by default, we'll memoize the check for triggers - but we can turn it off. //
+   ///////////////////////////////////////////////////////////////////////////////////////////////
+   private static boolean memoizeCheckForTriggers = new QMetaDataVariableInterpreter().getBooleanFromPropertyOrEnvironment("qqq.recordAutomationStatusUpdater.memoizeCheckForTriggers", "QQQ_RECORD_AUTOMATION_STATUS_UPDATER_MEMOIZE_CHECK_FOR_TRIGGERS", true);
+
+   private static Memoization<Key, Boolean> areThereTableTriggersForTableMemoization = new Memoization<Key, Boolean>().withTimeout(Duration.of(60, ChronoUnit.SECONDS));
+
 
 
    /*******************************************************************************
     ** for a list of records from a table, set their automation status - based on
     ** how the table is configured.
     *******************************************************************************/
-   public static boolean setAutomationStatusInRecords(QSession session, QTableMetaData table, List<QRecord> records, AutomationStatus automationStatus)
+   public static boolean setAutomationStatusInRecords(QTableMetaData table, List<QRecord> records, AutomationStatus automationStatus, QBackendTransaction transaction, List<QRecord> oldRecordList)
    {
       if(table == null || table.getAutomationDetails() == null || CollectionUtils.nullSafeIsEmpty(records))
       {
          return (false);
       }
 
+      QTableAutomationDetails automationDetails   = table.getAutomationDetails();
+      Set<Serializable>       pkeysWeMayNotUpdate = new HashSet<>();
+
       ///////////////////////////////////////////////////////////////////////////////////////////////////
       // In case an automation is running, and it updates records - don't let those records be marked  //
       // as PENDING_UPDATE_AUTOMATIONS... this is meant to avoid having a record's automation update   //
@@ -81,12 +109,60 @@ public class RecordAutomationStatusUpdater
          for(StackTraceElement stackTraceElement : e.getStackTrace())
          {
             String className = stackTraceElement.getClassName();
-            if(className.contains("com.kingsrook.qqq.backend.core.actions.automation") && !className.equals(RecordAutomationStatusUpdater.class.getName()) && !className.endsWith("Test"))
+            if(className.contains(RecordAutomationStatusUpdater.class.getPackageName()) && !className.equals(RecordAutomationStatusUpdater.class.getName()) && !className.endsWith("Test") && !className.contains("Test$"))
             {
                LOG.debug("Avoiding re-setting automation status to PENDING_UPDATE while running an automation");
                return (false);
             }
          }
+
+         ///////////////////////////////////////////////////////////////////////////////
+         // if table uses field-in-table status tracking, then check the old records, //
+         // before we set them to pending-updates, to avoid losing other pending or   //
+         // running status information.  We will allow moving  from OK or the 2       //
+         // failed statuses into pending-updates - which seems right.                 //
+         // This is added to fix cases where an update that comes in before insert    //
+         // -automations have run, will cause the pending-insert status to be missed. //
+         ///////////////////////////////////////////////////////////////////////////////
+         if(automationDetails.getStatusTracking() != null && AutomationStatusTrackingType.FIELD_IN_TABLE.equals(automationDetails.getStatusTracking().getType()))
+         {
+            try
+            {
+               if(CollectionUtils.nullSafeIsEmpty(oldRecordList))
+               {
+                  ///////////////////////////////////////////////////////////////////////////////////////////////
+                  // if we didn't get the oldRecordList as input (though UpdateAction should usually pass it?) //
+                  // then check feature-flag if we're allowed to do a lookup here & now.  If so, then do.      //
+                  ///////////////////////////////////////////////////////////////////////////////////////////////
+                  if(allowPreUpdateFetch)
+                  {
+                     List<Serializable> pkeysToLookup = records.stream().map(r -> r.getValue(table.getPrimaryKeyField())).toList();
+                     oldRecordList = new QueryAction().execute(new QueryInput(table.getName())
+                        .withFilter(new QQueryFilter(new QFilterCriteria(table.getPrimaryKeyField(), QCriteriaOperator.IN, pkeysToLookup)))
+                        .withTransaction(transaction)
+                     ).getRecords();
+                  }
+               }
+
+               for(QRecord freshRecord : CollectionUtils.nonNullList(oldRecordList))
+               {
+                  Serializable recordStatus = freshRecord.getValue(automationDetails.getStatusTracking().getFieldName());
+                  if(AutomationStatus.PENDING_INSERT_AUTOMATIONS.getId().equals(recordStatus)
+                     || AutomationStatus.PENDING_UPDATE_AUTOMATIONS.getId().equals(recordStatus)
+                     || AutomationStatus.RUNNING_INSERT_AUTOMATIONS.getId().equals(recordStatus)
+                     || AutomationStatus.RUNNING_UPDATE_AUTOMATIONS.getId().equals(recordStatus))
+                  {
+                     Serializable primaryKey = freshRecord.getValue(table.getPrimaryKeyField());
+                     LOG.debug("May not update automation status", logPair("table", table.getName()), logPair("id", primaryKey), logPair("currentStatus", recordStatus), logPair("requestedStatus", automationStatus.getId()));
+                     pkeysWeMayNotUpdate.add(primaryKey);
+                  }
+               }
+            }
+            catch(QException qe)
+            {
+               LOG.error("Error checking existing automation status before setting new automation status - more records will be updated than maybe should be...", qe);
+            }
+         }
       }
 
       ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@@ -98,19 +174,15 @@ public class RecordAutomationStatusUpdater
          automationStatus = AutomationStatus.OK;
       }
 
-      QTableAutomationDetails automationDetails = table.getAutomationDetails();
       if(automationDetails.getStatusTracking() != null && AutomationStatusTrackingType.FIELD_IN_TABLE.equals(automationDetails.getStatusTracking().getType()))
       {
          for(QRecord record : records)
          {
-            ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-            // todo - seems like there's some case here, where if an order was in PENDING_INSERT, but then some other job updated the record, that we'd  //
-            //  lose that pending status, which would be a Bad Thing™...                                                                                 //
-            //  problem is - we may not have the full record in here, so we can't necessarily check the record to see what status it's currently in...   //
-            ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-
-            record.setValue(automationDetails.getStatusTracking().getFieldName(), automationStatus.getId());
-            // todo - another field - for the automation timestamp??
+            if(!pkeysWeMayNotUpdate.contains(record.getValue(table.getPrimaryKeyField())))
+            {
+               record.setValue(automationDetails.getStatusTracking().getFieldName(), automationStatus.getId());
+               // todo - another field - for the automation timestamp??
+            }
          }
       }
 
@@ -188,11 +260,29 @@ public class RecordAutomationStatusUpdater
          return (false);
       }
 
+      if(memoizeCheckForTriggers)
+      {
+         ///////////////////////////////////////////////////////////////////////////////////////
+         // as within the lookup method, error on the side of "yes, maybe there are triggers" //
+         ///////////////////////////////////////////////////////////////////////////////////////
+         Optional<Boolean> result = areThereTableTriggersForTableMemoization.getResult(new Key(table, triggerEvent), key -> lookupIfThereAreTriggersForTable(table, triggerEvent));
+         return result.orElse(true);
+      }
+      else
+      {
+         return lookupIfThereAreTriggersForTable(table, triggerEvent);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static Boolean lookupIfThereAreTriggersForTable(QTableMetaData table, TriggerEvent triggerEvent)
+   {
       try
       {
-         ///////////////////
-         // todo - cache? //
-         ///////////////////
          CountInput countInput = new CountInput();
          countInput.setTableName(TableTrigger.TABLE_NAME);
          countInput.setFilter(new QQueryFilter(
@@ -207,6 +297,7 @@ public class RecordAutomationStatusUpdater
          ///////////////////////////////////////////////////////////////////////////////////////////////////////////
          // if the count query failed, we're a bit safer to err on the side of "yeah, there might be automations" //
          ///////////////////////////////////////////////////////////////////////////////////////////////////////////
+         LOG.warn("Error looking if there are triggers for table", e, logPair("tableName", table.getName()));
          return (true);
       }
    }
@@ -217,12 +308,12 @@ public class RecordAutomationStatusUpdater
     ** for a list of records, update their automation status and actually Update the
     ** backend as well.
     *******************************************************************************/
-   public static void setAutomationStatusInRecordsAndUpdate(QInstance instance, QSession session, QTableMetaData table, List<QRecord> records, AutomationStatus automationStatus) throws QException
+   public static void setAutomationStatusInRecordsAndUpdate(QTableMetaData table, List<QRecord> records, AutomationStatus automationStatus, QBackendTransaction transaction) throws QException
    {
       QTableAutomationDetails automationDetails = table.getAutomationDetails();
       if(automationDetails != null && AutomationStatusTrackingType.FIELD_IN_TABLE.equals(automationDetails.getStatusTracking().getType()))
       {
-         boolean didSetStatusField = setAutomationStatusInRecords(session, table, records, automationStatus);
+         boolean didSetStatusField = setAutomationStatusInRecords(table, records, automationStatus, transaction, null);
          if(didSetStatusField)
          {
             UpdateInput updateInput = new UpdateInput();
@@ -237,6 +328,7 @@ public class RecordAutomationStatusUpdater
                .withValue(table.getPrimaryKeyField(), r.getValue(table.getPrimaryKeyField()))
                .withValue(automationDetails.getStatusTracking().getFieldName(), r.getValue(automationDetails.getStatusTracking().getFieldName()))).toList());
             updateInput.setAreAllValuesBeingUpdatedTheSame(true);
+            updateInput.setTransaction(transaction);
             updateInput.setOmitDmlAudit(true);
 
             new UpdateAction().execute(updateInput);
@@ -250,4 +342,8 @@ public class RecordAutomationStatusUpdater
       }
    }
 
+
+
+   private record Key(QTableMetaData table, TriggerEvent triggerEvent) {}
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/automation/polling/PollingAutomationPerTableRunner.java

@@ -28,6 +28,7 @@ import java.util.Comparator;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import java.util.Optional;
 import java.util.function.Supplier;
 import java.util.stream.Collectors;
 import com.kingsrook.qqq.backend.core.actions.async.AsyncRecordPipeLoop;
@@ -60,18 +61,21 @@ import com.kingsrook.qqq.backend.core.model.automation.TableTrigger;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.AutomationStatusTrackingType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.QTableAutomationDetails;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.TableAutomationAction;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.TriggerEvent;
-import com.kingsrook.qqq.backend.core.model.savedfilters.SavedFilter;
+import com.kingsrook.qqq.backend.core.model.savedviews.SavedView;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.JsonUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
 import org.apache.commons.lang.NotImplementedException;
+import org.json.JSONObject;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -128,6 +132,11 @@ public class PollingAutomationPerTableRunner implements Runnable
        *******************************************************************************/
       String tableName();
 
+      /*******************************************************************************
+       **
+       *******************************************************************************/
+      QTableAutomationDetails tableAutomationDetails();
+
       /*******************************************************************************
        **
        *******************************************************************************/
@@ -139,7 +148,7 @@ public class PollingAutomationPerTableRunner implements Runnable
    /*******************************************************************************
     ** Wrapper for a pair of (tableName, automationStatus)
     *******************************************************************************/
-   public record TableActions(String tableName, AutomationStatus status) implements TableActionsInterface
+   public record TableActions(String tableName, QTableAutomationDetails tableAutomationDetails, AutomationStatus status) implements TableActionsInterface
    {
       /*******************************************************************************
        **
@@ -155,7 +164,7 @@ public class PollingAutomationPerTableRunner implements Runnable
     ** extended version of TableAction, for sharding use-case - adds the shard
     ** details.
     *******************************************************************************/
-   public record ShardedTableActions(String tableName, AutomationStatus status, String shardByFieldName, Serializable shardValue, String shardLabel) implements TableActionsInterface
+   public record ShardedTableActions(String tableName, QTableAutomationDetails tableAutomationDetails, AutomationStatus status, String shardByFieldName, Serializable shardValue, String shardLabel) implements TableActionsInterface
    {
       /*******************************************************************************
        **
@@ -194,8 +203,8 @@ public class PollingAutomationPerTableRunner implements Runnable
                   {
                      Serializable shardId = record.getValue(automationDetails.getShardIdFieldName());
                      String       label   = record.getValueString(automationDetails.getShardLabelFieldName());
-                     tableActionList.add(new ShardedTableActions(table.getName(), AutomationStatus.PENDING_INSERT_AUTOMATIONS, automationDetails.getShardByFieldName(), shardId, label));
-                     tableActionList.add(new ShardedTableActions(table.getName(), AutomationStatus.PENDING_UPDATE_AUTOMATIONS, automationDetails.getShardByFieldName(), shardId, label));
+                     tableActionList.add(new ShardedTableActions(table.getName(), automationDetails, AutomationStatus.PENDING_INSERT_AUTOMATIONS, automationDetails.getShardByFieldName(), shardId, label));
+                     tableActionList.add(new ShardedTableActions(table.getName(), automationDetails, AutomationStatus.PENDING_UPDATE_AUTOMATIONS, automationDetails.getShardByFieldName(), shardId, label));
                   }
                }
                catch(Exception e)
@@ -205,11 +214,11 @@ public class PollingAutomationPerTableRunner implements Runnable
             }
             else
             {
-               ///////////////////////////////////////////////////////////////////
-               // for non-sharded, we just need tabler name & automation status //
-               ///////////////////////////////////////////////////////////////////
-               tableActionList.add(new TableActions(table.getName(), AutomationStatus.PENDING_INSERT_AUTOMATIONS));
-               tableActionList.add(new TableActions(table.getName(), AutomationStatus.PENDING_UPDATE_AUTOMATIONS));
+               //////////////////////////////////////////////////////////////////
+               // for non-sharded, we just need table name & automation status //
+               //////////////////////////////////////////////////////////////////
+               tableActionList.add(new TableActions(table.getName(), automationDetails, AutomationStatus.PENDING_INSERT_AUTOMATIONS));
+               tableActionList.add(new TableActions(table.getName(), automationDetails, AutomationStatus.PENDING_UPDATE_AUTOMATIONS));
             }
          }
       }
@@ -251,12 +260,11 @@ public class PollingAutomationPerTableRunner implements Runnable
 
       try
       {
-         QSession session = sessionSupplier != null ? sessionSupplier.get() : new QSession();
-         processTableInsertOrUpdate(instance.getTable(tableActions.tableName()), session, tableActions.status());
+         processTableInsertOrUpdate(instance.getTable(tableActions.tableName()), tableActions.status());
       }
       catch(Exception e)
       {
-         LOG.warn("Error running automations", e);
+         LOG.warn("Error running automations", e, logPair("tableName", tableActions.tableName()), logPair("status", tableActions.status()));
       }
       finally
       {
@@ -270,7 +278,7 @@ public class PollingAutomationPerTableRunner implements Runnable
    /*******************************************************************************
     ** Query for and process records that have a PENDING_INSERT or PENDING_UPDATE status on a given table.
     *******************************************************************************/
-   public void processTableInsertOrUpdate(QTableMetaData table, QSession session, AutomationStatus automationStatus) throws QException
+   public void processTableInsertOrUpdate(QTableMetaData table, AutomationStatus automationStatus) throws QException
    {
       /////////////////////////////////////////////////////////////////////////
       // get the actions to run against this table in this automation status //
@@ -300,7 +308,9 @@ public class PollingAutomationPerTableRunner implements Runnable
             AutomationStatusTrackingType statusTrackingType = automationDetails.getStatusTracking().getType();
             if(AutomationStatusTrackingType.FIELD_IN_TABLE.equals(statusTrackingType))
             {
-               queryInput.setFilter(new QQueryFilter().withCriteria(new QFilterCriteria(automationDetails.getStatusTracking().getFieldName(), QCriteriaOperator.EQUALS, List.of(automationStatus.getId()))));
+               QQueryFilter filter = new QQueryFilter().withCriteria(new QFilterCriteria(automationDetails.getStatusTracking().getFieldName(), QCriteriaOperator.EQUALS, List.of(automationStatus.getId())));
+               addOrderByToQueryFilter(table, automationStatus, filter);
+               queryInput.setFilter(filter);
             }
             else
             {
@@ -321,7 +331,7 @@ public class PollingAutomationPerTableRunner implements Runnable
          }, () ->
          {
             List<QRecord> records = recordPipe.consumeAvailableRecords();
-            applyActionsToRecords(session, table, records, actions, automationStatus);
+            applyActionsToRecords(table, records, actions, automationStatus);
             return (records.size());
          }
       );
@@ -329,6 +339,38 @@ public class PollingAutomationPerTableRunner implements Runnable
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   static void addOrderByToQueryFilter(QTableMetaData table, AutomationStatus automationStatus, QQueryFilter filter)
+   {
+      ////////////////////////////////////////////////////////////////////////////////////
+      // look for a field in the table with either create-date or modify-date behavior, //
+      // based on if doing insert or update automations                                 //
+      ////////////////////////////////////////////////////////////////////////////////////
+      DynamicDefaultValueBehavior dynamicDefaultValueBehavior = automationStatus.equals(AutomationStatus.PENDING_INSERT_AUTOMATIONS) ? DynamicDefaultValueBehavior.CREATE_DATE : DynamicDefaultValueBehavior.MODIFY_DATE;
+      Optional<QFieldMetaData> field = table.getFields().values().stream()
+         .filter(f -> dynamicDefaultValueBehavior.equals(f.getBehaviorOrDefault(QContext.getQInstance(), DynamicDefaultValueBehavior.class)))
+         .findFirst();
+
+      if(field.isPresent())
+      {
+         //////////////////////////////////////////////////////////////////////
+         // if a create/modify date field was found, order by it (ascending) //
+         //////////////////////////////////////////////////////////////////////
+         filter.addOrderBy(new QFilterOrderBy(field.get().getName()));
+      }
+      else
+      {
+         ////////////////////////////////////
+         // else, order by the primary key //
+         ////////////////////////////////////
+         filter.addOrderBy(new QFilterOrderBy(table.getPrimaryKeyField()));
+      }
+   }
+
+
+
    /*******************************************************************************
     ** get the actions to run against a table in an automation status.  both from
     ** metaData and tableTriggers/data.
@@ -388,13 +430,15 @@ public class PollingAutomationPerTableRunner implements Runnable
                if(filterId != null)
                {
                   GetInput getInput = new GetInput();
-                  getInput.setTableName(SavedFilter.TABLE_NAME);
+                  getInput.setTableName(SavedView.TABLE_NAME);
                   getInput.setPrimaryKey(filterId);
                   GetOutput getOutput = new GetAction().execute(getInput);
                   if(getOutput.getRecord() != null)
                   {
-                     SavedFilter savedFilter = new SavedFilter(getOutput.getRecord());
-                     filter = JsonUtils.toObject(savedFilter.getFilterJson(), QQueryFilter.class);
+                     SavedView  savedView   = new SavedView(getOutput.getRecord());
+                     JSONObject viewJson    = new JSONObject(savedView.getViewJson());
+                     JSONObject queryFilter = viewJson.getJSONObject("queryFilter");
+                     filter = JsonUtils.toObject(queryFilter.toString(), QQueryFilter.class);
                   }
                }
 
@@ -427,7 +471,7 @@ public class PollingAutomationPerTableRunner implements Runnable
     ** table's actions against them - IF they are found to match the action's filter
     ** (assuming it has one - if it doesn't, then all records match).
     *******************************************************************************/
-   private void applyActionsToRecords(QSession session, QTableMetaData table, List<QRecord> records, List<TableAutomationAction> actions, AutomationStatus automationStatus) throws QException
+   private void applyActionsToRecords(QTableMetaData table, List<QRecord> records, List<TableAutomationAction> actions, AutomationStatus automationStatus) throws QException
    {
       if(CollectionUtils.nullSafeIsEmpty(records))
       {
@@ -437,7 +481,7 @@ public class PollingAutomationPerTableRunner implements Runnable
       ///////////////////////////////////////////////////
       // mark the records as RUNNING their automations //
       ///////////////////////////////////////////////////
-      RecordAutomationStatusUpdater.setAutomationStatusInRecordsAndUpdate(instance, session, table, records, pendingToRunningStatusMap.get(automationStatus));
+      RecordAutomationStatusUpdater.setAutomationStatusInRecordsAndUpdate(table, records, pendingToRunningStatusMap.get(automationStatus), null);
 
       ///////////////////////////////////////////////////////////////////////////////////////////////////////////////
       // foreach action - run it against the records (but only if they match the action's filter, if there is one) //
@@ -455,13 +499,15 @@ public class PollingAutomationPerTableRunner implements Runnable
       ////////////////////////////////////////
       // update status on all these records //
       ////////////////////////////////////////
-      if(anyActionsFailed)
+      AutomationStatus statusToUpdateTo = anyActionsFailed ? pendingToFailedStatusMap.get(automationStatus) : AutomationStatus.OK;
+      try
       {
-         RecordAutomationStatusUpdater.setAutomationStatusInRecordsAndUpdate(instance, session, table, records, pendingToFailedStatusMap.get(automationStatus));
+         RecordAutomationStatusUpdater.setAutomationStatusInRecordsAndUpdate(table, records, statusToUpdateTo, null);
       }
-      else
+      catch(Exception e)
       {
-         RecordAutomationStatusUpdater.setAutomationStatusInRecordsAndUpdate(instance, session, table, records, AutomationStatus.OK);
+         LOG.warn("Error updating automationStatus after running automations", logPair("tableName", table), logPair("count", records.size()), logPair("status", statusToUpdateTo));
+         throw (e);
       }
    }
 
@@ -491,7 +537,7 @@ public class PollingAutomationPerTableRunner implements Runnable
       }
       catch(Exception e)
       {
-         LOG.warn("Caught exception processing records on " + table + " for action " + action, e);
+         LOG.warn("Caught exception processing automations", e, logPair("tableName", table), logPair("action", action.getName()));
          return (true);
       }
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPostDeleteCustomizer.java

@@ -47,12 +47,24 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  ** records that the delete action marked in error - the user might want to do
  ** something special with them (idk, try some other way to delete them?)
  *******************************************************************************/
-public abstract class AbstractPostDeleteCustomizer
+public abstract class AbstractPostDeleteCustomizer implements TableCustomizerInterface
 {
    protected DeleteInput deleteInput;
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
+   {
+      this.deleteInput = deleteInput;
+      return apply(records);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPostInsertCustomizer.java

@@ -42,12 +42,24 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  **
  ** Note that the full insertInput is available as a field in this class.
  *******************************************************************************/
-public abstract class AbstractPostInsertCustomizer
+public abstract class AbstractPostInsertCustomizer implements TableCustomizerInterface
 {
    protected InsertInput insertInput;
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
+   {
+      this.insertInput = insertInput;
+      return (apply(records));
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPostQueryCustomizer.java

@@ -23,16 +23,29 @@ package com.kingsrook.qqq.backend.core.actions.customizers;
 
 
 import java.util.List;
-import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 
 
 /*******************************************************************************
  **
  *******************************************************************************/
-public abstract class AbstractPostQueryCustomizer
+public abstract class AbstractPostQueryCustomizer implements TableCustomizerInterface
 {
-   protected AbstractTableActionInput input;
+   protected QueryOrGetInputInterface input;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records) throws QException
+   {
+      input = queryInput;
+      return apply(records);
+   }
 
 
 
@@ -47,7 +60,7 @@ public abstract class AbstractPostQueryCustomizer
     ** Getter for input
     **
     *******************************************************************************/
-   public AbstractTableActionInput getInput()
+   public QueryOrGetInputInterface getInput()
    {
       return (input);
    }
@@ -58,7 +71,7 @@ public abstract class AbstractPostQueryCustomizer
     ** Setter for input
     **
     *******************************************************************************/
-   public void setInput(AbstractTableActionInput input)
+   public void setInput(QueryOrGetInputInterface input)
    {
       this.input = input;
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPostUpdateCustomizer.java

@@ -26,6 +26,7 @@ import java.io.Serializable;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Optional;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -48,7 +49,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  ** available (if the backend supports it) - both as a list (`getOldRecordList`)
  ** and as a memoized (by this class) map of primaryKey to record (`getOldRecordMap`).
  *******************************************************************************/
-public abstract class AbstractPostUpdateCustomizer
+public abstract class AbstractPostUpdateCustomizer implements TableCustomizerInterface
 {
    protected UpdateInput   updateInput;
    protected List<QRecord> oldRecordList;
@@ -57,6 +58,19 @@ public abstract class AbstractPostUpdateCustomizer
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      this.updateInput = updateInput;
+      this.oldRecordList = oldRecordList.orElse(null);
+      return apply(records);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPreDeleteCustomizer.java

@@ -50,7 +50,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  ** Note that the full deleteInput is available as a field in this class.
  **
  *******************************************************************************/
-public abstract class AbstractPreDeleteCustomizer
+public abstract class AbstractPreDeleteCustomizer implements TableCustomizerInterface
 {
    protected DeleteInput deleteInput;
 
@@ -58,6 +58,19 @@ public abstract class AbstractPreDeleteCustomizer
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      this.deleteInput = deleteInput;
+      this.isPreview = isPreview;
+      return apply(records);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPreInsertCustomizer.java

@@ -47,7 +47,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  **
  ** Note that the full insertInput is available as a field in this class.
  *******************************************************************************/
-public abstract class AbstractPreInsertCustomizer
+public abstract class AbstractPreInsertCustomizer implements TableCustomizerInterface
 {
    protected InsertInput insertInput;
 
@@ -70,6 +70,30 @@ public abstract class AbstractPreInsertCustomizer
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      this.insertInput = insertInput;
+      this.isPreview = isPreview;
+      return (apply(records));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public WhenToRun whenToRunPreInsert(InsertInput insertInput, boolean isPreview)
+   {
+      return getWhenToRun();
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/AbstractPreUpdateCustomizer.java

@@ -26,6 +26,7 @@ import java.io.Serializable;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Optional;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -53,7 +54,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
  ** available (if the backend supports it) - both as a list (`getOldRecordList`)
  ** and as a memoized (by this class) map of primaryKey to record (`getOldRecordMap`).
  *******************************************************************************/
-public abstract class AbstractPreUpdateCustomizer
+public abstract class AbstractPreUpdateCustomizer implements TableCustomizerInterface
 {
    protected UpdateInput   updateInput;
    protected List<QRecord> oldRecordList;
@@ -63,6 +64,20 @@ public abstract class AbstractPreUpdateCustomizer
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      this.updateInput = updateInput;
+      this.isPreview = isPreview;
+      this.oldRecordList = oldRecordList.orElse(null);
+      return apply(records);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/QCodeLoader.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.actions.customizers;
 
 
+import java.lang.reflect.Constructor;
 import java.util.Optional;
 import java.util.function.Function;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationHandler;
@@ -34,42 +35,35 @@ import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeType;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.TableAutomationAction;
+import com.kingsrook.qqq.backend.core.utils.lambdas.UnsafeFunction;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
  ** Utility to load code for running QQQ customizers.
+ **
+ ** TODO - redo all to go through method that memoizes class & constructor
+ ** lookup.  That memoziation causes 1,000,000 such calls to go from ~500ms
+ ** to ~100ms.
  *******************************************************************************/
 public class QCodeLoader
 {
    private static final QLogger LOG = QLogger.getLogger(QCodeLoader.class);
 
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   public static <T, R> Optional<Function<T, R>> getTableCustomizerFunction(QTableMetaData table, String customizerName)
-   {
-      Optional<QCodeReference> codeReference = table.getCustomizer(customizerName);
-      if(codeReference.isPresent())
-      {
-         return (Optional.ofNullable(QCodeLoader.getFunction(codeReference.get())));
-      }
-      return (Optional.empty());
-   }
+   private static Memoization<String, Constructor<?>> constructorMemoization = new Memoization<>();
 
 
 
    /*******************************************************************************
     **
     *******************************************************************************/
-   public static <T> Optional<T> getTableCustomizer(Class<T> expectedClass, QTableMetaData table, String customizerName)
+   public static Optional<TableCustomizerInterface> getTableCustomizer(QTableMetaData table, String customizerName)
    {
       Optional<QCodeReference> codeReference = table.getCustomizer(customizerName);
       if(codeReference.isPresent())
       {
-         return (Optional.ofNullable(QCodeLoader.getAdHoc(expectedClass, codeReference.get())));
+         return (Optional.ofNullable(QCodeLoader.getAdHoc(TableCustomizerInterface.class, codeReference.get())));
       }
       return (Optional.empty());
    }
@@ -175,8 +169,21 @@ public class QCodeLoader
 
       try
       {
-         Class<?> customizerClass = Class.forName(codeReference.getName());
-         return ((T) customizerClass.getConstructor().newInstance());
+         Optional<Constructor<?>> constructor = constructorMemoization.getResultThrowing(codeReference.getName(), (UnsafeFunction<String, Constructor<?>, Exception>) s ->
+         {
+            Class<?> customizerClass = Class.forName(codeReference.getName());
+            return customizerClass.getConstructor();
+         });
+
+         if(constructor.isPresent())
+         {
+            return ((T) constructor.get().newInstance());
+         }
+         else
+         {
+            LOG.error("Could not get constructor for code reference", logPair("codeReference", codeReference));
+            return (null);
+         }
       }
       catch(Exception e)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/TableCustomizerInterface.java

@@ -0,0 +1,202 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.customizers;
+
+
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Common interface used by all (core) TableCustomizer types (e.g., post-query,
+ ** and {pre,post}-{insert,update,delete}.
+ **
+ ** Note that the abstract-base classes for each action still exist, though have
+ ** been back-ported to be implementors of this interface.  The action classes
+ ** will now expect this type, and call this type's methods.
+ **
+ *******************************************************************************/
+public interface TableCustomizerInterface
+{
+   QLogger LOG = QLogger.getLogger(TableCustomizerInterface.class);
+
+
+   /*******************************************************************************
+    ** custom actions to run after a query (or get!) takes place.
+    **
+    *******************************************************************************/
+   default List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records) throws QException
+   {
+      LOG.info("A default implementation of postQuery is running...  Probably not expected!", logPair("tableName", queryInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    ** custom actions before an insert takes place.
+    **
+    ** It's important for implementations to be aware of the isPreview field, which
+    ** is set to true when the code is running to give users advice, e.g., on a review
+    ** screen - vs. being false when the action is ACTUALLY happening.  So, if you're doing
+    ** things like storing data, you don't want to do that if isPreview is true!!
+    **
+    ** General implementation would be, to iterate over the records (the inputs to
+    ** the insert action), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records
+    ** - possibly manipulating values (`setValue`)
+    ** - possibly throwing an exception - if you really don't want the insert operation to continue.
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) that you want to go on to the backend implementation class.
+    *******************************************************************************/
+   default List<QRecord> preInsert(InsertInput insertInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      LOG.info("A default implementation of preInsert is running...  Probably not expected!", logPair("tableName", insertInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   default AbstractPreInsertCustomizer.WhenToRun whenToRunPreInsert(InsertInput insertInput, boolean isPreview)
+   {
+      return (AbstractPreInsertCustomizer.WhenToRun.AFTER_ALL_VALIDATIONS);
+   }
+
+
+   /*******************************************************************************
+    ** custom actions after an insert takes place.
+    **
+    ** General implementation would be, to iterate over the records (the outputs of
+    ** the insert action), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records
+    ** - possibly throwing an exception - though doing so won't stop the update, and instead
+    **   will just set a warning on all of the updated records...
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) that you want to go back to the caller.
+    *******************************************************************************/
+   default List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
+   {
+      LOG.info("A default implementation of postInsert is running...  Probably not expected!", logPair("tableName", insertInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    ** custom actions before an update takes place.
+    **
+    ** It's important for implementations to be aware of the isPreview field, which
+    ** is set to true when the code is running to give users advice, e.g., on a review
+    ** screen - vs. being false when the action is ACTUALLY happening.  So, if you're doing
+    ** things like storing data, you don't want to do that if isPreview is true!!
+    **
+    ** General implementation would be, to iterate over the records (the inputs to
+    ** the update action), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records
+    ** - possibly manipulating values (`setValue`)
+    ** - possibly throwing an exception - if you really don't want the update operation to continue.
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) that you want to go on to the backend implementation class.
+    **
+    ** Note, "old records" (e.g., with values freshly fetched from the backend) will be
+    ** available (if the backend supports it)
+    *******************************************************************************/
+   default List<QRecord> preUpdate(UpdateInput updateInput, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      LOG.info("A default implementation of preUpdate is running...  Probably not expected!", logPair("tableName", updateInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    ** custom actions after an update takes place.
+    **
+    ** General implementation would be, to iterate over the records (the outputs of
+    ** the update action), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records?
+    ** - possibly throwing an exception - though doing so won't stop the update, and instead
+    **   will just set a warning on all of the updated records...
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) that you want to go back to the caller.
+    **
+    ** Note, "old records" (e.g., with values freshly fetched from the backend) will be
+    ** available (if the backend supports it).
+    *******************************************************************************/
+   default List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
+   {
+      LOG.info("A default implementation of postUpdate is running...  Probably not expected!", logPair("tableName", updateInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    ** Custom actions before a delete takes place.
+    **
+    ** It's important for implementations to be aware of the isPreview param, which
+    ** is set to true when the code is running to give users advice, e.g., on a review
+    ** screen - vs. being false when the action is ACTUALLY happening.  So, if you're doing
+    ** things like storing data, you don't want to do that if isPreview is true!!
+    **
+    ** General implementation would be, to iterate over the records (which the DeleteAction
+    ** would look up based on the inputs to the delete action), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records
+    ** - possibly throwing an exception - if you really don't want the delete operation to continue.
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) - this is how errors
+    **   and warnings are propagated to the DeleteAction.  Note that any records with
+    **   an error will NOT proceed to the backend's delete interface - but those with
+    **   warnings will.
+    *******************************************************************************/
+   default List<QRecord> preDelete(DeleteInput deleteInput, List<QRecord> records, boolean isPreview) throws QException
+   {
+      LOG.info("A default implementation of preDelete is running...  Probably not expected!", logPair("tableName", deleteInput.getTableName()));
+      return (records);
+   }
+
+
+   /*******************************************************************************
+    ** Custom actions after a delete takes place.
+    **
+    ** General implementation would be, to iterate over the records (ones which didn't
+    ** have a delete error), and look at their values:
+    ** - possibly adding Errors (`addError`) or Warnings (`addWarning`) to the records?
+    ** - possibly throwing an exception - though doing so won't stop the delete, and instead
+    **   will just set a warning on all of the deleted records...
+    ** - doing "whatever else" you may want to do.
+    ** - returning the list of records (can be the input list) that you want to go back
+    **   to the caller - this is how errors and warnings are propagated .
+    *******************************************************************************/
+   default List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
+   {
+      LOG.info("A default implementation of postDelete is running...  Probably not expected!", logPair("tableName", deleteInput.getTableName()));
+      return (records);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/customizers/TableCustomizers.java

@@ -29,13 +29,13 @@ package com.kingsrook.qqq.backend.core.actions.customizers;
  *******************************************************************************/
 public enum TableCustomizers
 {
-   POST_QUERY_RECORD("postQueryRecord", AbstractPostQueryCustomizer.class),
-   PRE_INSERT_RECORD("preInsertRecord", AbstractPreInsertCustomizer.class),
-   POST_INSERT_RECORD("postInsertRecord", AbstractPostInsertCustomizer.class),
-   PRE_UPDATE_RECORD("preUpdateRecord", AbstractPreUpdateCustomizer.class),
-   POST_UPDATE_RECORD("postUpdateRecord", AbstractPostUpdateCustomizer.class),
-   PRE_DELETE_RECORD("preDeleteRecord", AbstractPreDeleteCustomizer.class),
-   POST_DELETE_RECORD("postDeleteRecord", AbstractPostDeleteCustomizer.class);
+   POST_QUERY_RECORD("postQueryRecord", TableCustomizerInterface.class),
+   PRE_INSERT_RECORD("preInsertRecord", TableCustomizerInterface.class),
+   POST_INSERT_RECORD("postInsertRecord", TableCustomizerInterface.class),
+   PRE_UPDATE_RECORD("preUpdateRecord", TableCustomizerInterface.class),
+   POST_UPDATE_RECORD("postUpdateRecord", TableCustomizerInterface.class),
+   PRE_DELETE_RECORD("preDeleteRecord", TableCustomizerInterface.class),
+   POST_DELETE_RECORD("postDeleteRecord", TableCustomizerInterface.class);
 
 
    private final String   role;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/dashboard/AbstractHTMLWidgetRenderer.java

@@ -41,6 +41,7 @@ import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.QQueryFilterDeduper;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
@@ -176,11 +177,13 @@ public abstract class AbstractHTMLWidgetRenderer extends AbstractWidgetRenderer
       {
          return (totalString);
       }
+      filter = QQueryFilterDeduper.dedupeFilter(filter);
       return ("<a href='" + tablePath + "?filter=" + JsonUtils.toJson(filter) + "'>" + totalString + "</a>");
    }
 
 
 
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -192,6 +195,7 @@ public abstract class AbstractHTMLWidgetRenderer extends AbstractWidgetRenderer
          return;
       }
 
+      filter = QQueryFilterDeduper.dedupeFilter(filter);
       urls.add(tablePath + "?filter=" + JsonUtils.toJson(filter));
    }
 
@@ -208,6 +212,7 @@ public abstract class AbstractHTMLWidgetRenderer extends AbstractWidgetRenderer
          return (null);
       }
 
+      filter = QQueryFilterDeduper.dedupeFilter(filter);
       return (tablePath + "?filter=" + JsonUtils.toJson(filter));
    }
 
@@ -224,6 +229,7 @@ public abstract class AbstractHTMLWidgetRenderer extends AbstractWidgetRenderer
          return (null);
       }
 
+      filter = QQueryFilterDeduper.dedupeFilter(filter);
       return (tablePath + "?filter=" + URLEncoder.encode(JsonUtils.toJson(filter), Charset.defaultCharset()));
    }
 
@@ -326,6 +332,7 @@ public abstract class AbstractHTMLWidgetRenderer extends AbstractWidgetRenderer
       }
 
       String tablePath = QContext.getQInstance().getTablePath(tableName);
+      filter = QQueryFilterDeduper.dedupeFilter(filter);
       return (tablePath + "/" + processName + "?recordsParam=filterJSON&filterJSON=" + URLEncoder.encode(JsonUtils.toJson(filter), StandardCharsets.UTF_8));
    }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/dashboard/widgets/Aggregate2DTableWidgetRenderer.java

@@ -0,0 +1,202 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.actions.dashboard.widgets;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.QFilterOrderByAggregate;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.QFilterOrderByGroupBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+
+/*******************************************************************************
+ ** Generic widget that does an aggregate query, and presents its results
+ ** as a table, using group-by values as both row & column labels.
+ *******************************************************************************/
+public class Aggregate2DTableWidgetRenderer extends AbstractWidgetRenderer
+{
+   private static final QLogger LOG = QLogger.getLogger(Aggregate2DTableWidgetRenderer.class);
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+   {
+      Map<String, Serializable> values = input.getWidgetMetaData().getDefaultValues();
+
+      String         tableName   = ValueUtils.getValueAsString(values.get("tableName"));
+      String         valueField  = ValueUtils.getValueAsString(values.get("valueField"));
+      String         rowField    = ValueUtils.getValueAsString(values.get("rowField"));
+      String         columnField = ValueUtils.getValueAsString(values.get("columnField"));
+      QTableMetaData table       = QContext.getQInstance().getTable(tableName);
+
+      AggregateInput aggregateInput = new AggregateInput();
+      aggregateInput.setTableName(tableName);
+
+      // todo - allow input of "list of columns" (e.g., in case some miss sometimes, or as a version of filter)
+      // todo - max rows, max cols?
+
+      // todo - from input map
+      QQueryFilter filter = new QQueryFilter();
+      aggregateInput.setFilter(filter);
+
+      Aggregate aggregate = new Aggregate(valueField, AggregateOperator.COUNT);
+      aggregateInput.withAggregate(aggregate);
+
+      GroupBy rowGroupBy    = new GroupBy(table.getField(rowField));
+      GroupBy columnGroupBy = new GroupBy(table.getField(columnField));
+      aggregateInput.withGroupBy(rowGroupBy);
+      aggregateInput.withGroupBy(columnGroupBy);
+
+      String orderBys = ValueUtils.getValueAsString(values.get("orderBys"));
+      if(StringUtils.hasContent(orderBys))
+      {
+         for(String orderBy : orderBys.split(","))
+         {
+            switch(orderBy)
+            {
+               case "row" -> filter.addOrderBy(new QFilterOrderByGroupBy(rowGroupBy));
+               case "column" -> filter.addOrderBy(new QFilterOrderByGroupBy(columnGroupBy));
+               case "value" -> filter.addOrderBy(new QFilterOrderByAggregate(aggregate));
+               default -> LOG.warn("Unrecognized orderBy: " + orderBy);
+            }
+         }
+      }
+
+      AggregateOutput aggregateOutput = new AggregateAction().execute(aggregateInput);
+
+      Map<Serializable, Map<Serializable, Serializable>> data       = new LinkedHashMap<>();
+      Set<Serializable>                                  columnsSet = new LinkedHashSet<>();
+
+      for(AggregateResult result : aggregateOutput.getResults())
+      {
+         Serializable column = result.getGroupByValue(columnGroupBy);
+         Serializable row    = result.getGroupByValue(rowGroupBy);
+         Serializable value  = result.getAggregateValue(aggregate);
+
+         Map<Serializable, Serializable> rowMap = data.computeIfAbsent(row, (k) -> new LinkedHashMap<>());
+         rowMap.put(column, value);
+         columnsSet.add(column);
+      }
+
+      // todo - possible values from rows, cols
+
+      ////////////////////////////////////
+      // setup datastructures for table //
+      ////////////////////////////////////
+      List<Map<String, Object>> tableRows    = new ArrayList<>();
+      List<TableData.Column>    tableColumns = new ArrayList<>();
+      tableColumns.add(new TableData.Column("default", table.getField(rowField).getLabel(), "_row", "2fr", "left"));
+
+      for(Serializable column : columnsSet)
+      {
+         tableColumns.add(new TableData.Column("default", String.valueOf(column) /* todo display value */, String.valueOf(column), "1fr", "right"));
+      }
+
+      tableColumns.add(new TableData.Column("default", "Total", "_total", "1fr", "right"));
+
+      TableData tableData = new TableData(null, tableColumns, tableRows)
+         .withRowsPerPage(100)
+         .withFixedStickyLastRow(false)
+         .withHidePaginationDropdown(true);
+
+      Map<Serializable, Integer> columnSums = new HashMap<>();
+      int                        grandTotal = 0;
+      for(Map.Entry<Serializable, Map<Serializable, Serializable>> rowEntry : data.entrySet())
+      {
+         Map<String, Object> rowMap = new HashMap<>();
+         tableRows.add(rowMap);
+
+         rowMap.put("_row", rowEntry.getKey() /* todo display value */);
+         int rowTotal = 0;
+         for(Serializable column : columnsSet)
+         {
+            Serializable value = rowEntry.getValue().get(column);
+            if(value == null)
+            {
+               value = 0; // todo?
+            }
+
+            Integer valueAsInteger = Objects.requireNonNullElse(ValueUtils.getValueAsInteger(value), 0);
+            rowTotal += valueAsInteger;
+            columnSums.putIfAbsent(column, 0);
+            columnSums.put(column, columnSums.get(column) + valueAsInteger);
+
+            rowMap.put(String.valueOf(column), value); // todo format commas?
+         }
+
+         rowMap.put("_total", rowTotal);
+         grandTotal += rowTotal;
+      }
+
+      ///////////////
+      // total row //
+      ///////////////
+      Map<String, Object> totalRowMap = new HashMap<>();
+      tableRows.add(totalRowMap);
+
+      totalRowMap.put("_row", "Total");
+      int rowTotal = 0;
+      for(Serializable column : columnsSet)
+      {
+         Serializable value = columnSums.get(column);
+         if(value == null)
+         {
+            value = 0; // todo?
+         }
+
+         totalRowMap.put(String.valueOf(column), value); // todo format commas?
+      }
+
+      totalRowMap.put("_total", grandTotal);
+
+      return (new RenderWidgetOutput(tableData));
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/dashboard/widgets/ChildRecordListRenderer.java

@@ -137,7 +137,7 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
        *******************************************************************************/
       public Builder withCanAddChildRecord(boolean b)
       {
-         widgetMetaData.withDefaultValue("canAddChildRecord", true);
+         widgetMetaData.withDefaultValue("canAddChildRecord", b);
          return (this);
       }
 
@@ -151,6 +151,17 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
          widgetMetaData.withDefaultValue("disabledFieldsForNewChildRecords", new HashSet<>(disabledFieldsForNewChildRecords));
          return (this);
       }
+
+
+
+      /*******************************************************************************
+       **
+       *******************************************************************************/
+      public Builder withManageAssociationName(String manageAssociationName)
+      {
+         widgetMetaData.withDefaultValue("manageAssociationName", manageAssociationName);
+         return (this);
+      }
    }
 
 
@@ -178,52 +189,60 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
          maxRows = ValueUtils.getValueAsInteger(input.getWidgetMetaData().getDefaultValues().containsKey("maxRows"));
       }
 
-      ////////////////////////////////////////////////////////
-      // fetch the record that we're getting children for.  //
-      // e.g., the left-side of the join, with the input id //
-      ////////////////////////////////////////////////////////
-      GetInput getInput = new GetInput();
-      getInput.setTableName(join.getLeftTable());
-      getInput.setPrimaryKey(id);
-      GetOutput getOutput = new GetAction().execute(getInput);
-      QRecord   record    = getOutput.getRecord();
-
-      if(record == null)
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      // fetch the record that we're getting children for. e.g., the left-side of the join, with the input id                                     //
+      // but - only try this if we were given an id.  note, this widget could be called for on an INSERT screen, where we don't have a record yet //
+      // but we still want to be able to return all the other data in here that otherwise comes from the widget meta data, join, etc.             //
+      //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      int          totalRows     = 0;
+      QRecord      primaryRecord = null;
+      QQueryFilter filter        = null;
+      QueryOutput  queryOutput   = new QueryOutput(new QueryInput());
+      if(StringUtils.hasContent(id))
       {
-         throw (new QNotFoundException("Could not find " + (leftTable == null ? "" : leftTable.getLabel()) + " with primary key " + id));
-      }
+         GetInput getInput = new GetInput();
+         getInput.setTableName(join.getLeftTable());
+         getInput.setPrimaryKey(id);
+         GetOutput getOutput = new GetAction().execute(getInput);
+         primaryRecord = getOutput.getRecord();
 
-      ////////////////////////////////////////////////////////////////////
-      // set up the query - for the table on the right side of the join //
-      ////////////////////////////////////////////////////////////////////
-      QQueryFilter filter = new QQueryFilter();
-      for(JoinOn joinOn : join.getJoinOns())
-      {
-         filter.addCriteria(new QFilterCriteria(joinOn.getRightField(), QCriteriaOperator.EQUALS, List.of(record.getValue(joinOn.getLeftField()))));
-      }
-      filter.setOrderBys(join.getOrderBys());
-      filter.setLimit(maxRows);
+         if(primaryRecord == null)
+         {
+            throw (new QNotFoundException("Could not find " + (leftTable == null ? "" : leftTable.getLabel()) + " with primary key " + id));
+         }
 
-      QueryInput queryInput = new QueryInput();
-      queryInput.setTableName(join.getRightTable());
-      queryInput.setShouldTranslatePossibleValues(true);
-      queryInput.setShouldGenerateDisplayValues(true);
-      queryInput.setFilter(filter);
-      QueryOutput queryOutput = new QueryAction().execute(queryInput);
+         ////////////////////////////////////////////////////////////////////
+         // set up the query - for the table on the right side of the join //
+         ////////////////////////////////////////////////////////////////////
+         filter = new QQueryFilter();
+         for(JoinOn joinOn : join.getJoinOns())
+         {
+            filter.addCriteria(new QFilterCriteria(joinOn.getRightField(), QCriteriaOperator.EQUALS, List.of(primaryRecord.getValue(joinOn.getLeftField()))));
+         }
+         filter.setOrderBys(join.getOrderBys());
+         filter.setLimit(maxRows);
 
-      QValueFormatter.setBlobValuesToDownloadUrls(rightTable, queryOutput.getRecords());
+         QueryInput queryInput = new QueryInput();
+         queryInput.setTableName(join.getRightTable());
+         queryInput.setShouldTranslatePossibleValues(true);
+         queryInput.setShouldGenerateDisplayValues(true);
+         queryInput.setFilter(filter);
+         queryOutput = new QueryAction().execute(queryInput);
 
-      int totalRows = queryOutput.getRecords().size();
-      if(maxRows != null && (queryOutput.getRecords().size() == maxRows))
-      {
-         /////////////////////////////////////////////////////////////////////////////////////
-         // if the input said to only do some max, and the # of results we got is that max, //
-         // then do a count query, for displaying 1-n of <count>                            //
-         /////////////////////////////////////////////////////////////////////////////////////
-         CountInput countInput = new CountInput();
-         countInput.setTableName(join.getRightTable());
-         countInput.setFilter(filter);
-         totalRows = new CountAction().execute(countInput).getCount();
+         QValueFormatter.setBlobValuesToDownloadUrls(rightTable, queryOutput.getRecords());
+
+         totalRows = queryOutput.getRecords().size();
+         if(maxRows != null && (queryOutput.getRecords().size() == maxRows))
+         {
+            /////////////////////////////////////////////////////////////////////////////////////
+            // if the input said to only do some max, and the # of results we got is that max, //
+            // then do a count query, for displaying 1-n of <count>                            //
+            /////////////////////////////////////////////////////////////////////////////////////
+            CountInput countInput = new CountInput();
+            countInput.setTableName(join.getRightTable());
+            countInput.setFilter(filter);
+            totalRows = new CountAction().execute(countInput).getCount();
+         }
       }
 
       String tablePath   = input.getInstance().getTablePath(rightTable.getName());
@@ -239,10 +258,14 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
          // new child records must have values from the join-ons //
          //////////////////////////////////////////////////////////
          Map<String, Serializable> defaultValuesForNewChildRecords = new HashMap<>();
-         for(JoinOn joinOn : join.getJoinOns())
+         if(primaryRecord != null)
          {
-            defaultValuesForNewChildRecords.put(joinOn.getRightField(), record.getValue(joinOn.getLeftField()));
+            for(JoinOn joinOn : join.getJoinOns())
+            {
+               defaultValuesForNewChildRecords.put(joinOn.getRightField(), primaryRecord.getValue(joinOn.getLeftField()));
+            }
          }
+
          widgetData.setDefaultValuesForNewChildRecords(defaultValuesForNewChildRecords);
 
          Map<String, Serializable> widgetValues = input.getWidgetMetaData().getDefaultValues();
@@ -250,6 +273,22 @@ public class ChildRecordListRenderer extends AbstractWidgetRenderer
          {
             widgetData.setDisabledFieldsForNewChildRecords((Set<String>) widgetValues.get("disabledFieldsForNewChildRecords"));
          }
+         else
+         {
+            /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+            // if there are no disabled fields specified - then normally any fields w/ a default value get implicitly disabled //
+            // but - if we didn't look-up the primary record, then we'll want to explicit disable fields from joins            //
+            /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+            if(primaryRecord == null)
+            {
+               Set<String> implicitlyDisabledFields = new HashSet<>();
+               widgetData.setDisabledFieldsForNewChildRecords(implicitlyDisabledFields);
+               for(JoinOn joinOn : join.getJoinOns())
+               {
+                  implicitlyDisabledFields.add(joinOn.getRightField());
+               }
+            }
+         }
       }
 
       return (new RenderWidgetOutput(widgetData));

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/dashboard/widgets/DateTimeGroupBy.java

@@ -297,4 +297,21 @@ public enum DateTimeGroupBy
       ZonedDateTime zoned = instant.atZone(zoneId);
       return (zoned.plus(noOfChronoUnitsToAdd, chronoUnitToAdd).toInstant());
    }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static DateTimeFormatter sqlDateFormatToSelectedDateTimeFormatter(String sqlDateFormat)
+   {
+      for(DateTimeGroupBy value : values())
+      {
+         if(value.sqlDateFormat.equals(sqlDateFormat))
+         {
+            return (value.selectedStringFormatter);
+         }
+      }
+      return null;
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/permissions/PermissionsHelper.java

@@ -34,15 +34,18 @@ import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput;
+import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.DenyBehavior;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithName;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
@@ -333,9 +336,25 @@ public class PermissionsHelper
          QPermissionRules rules    = getEffectivePermissionRules(tableMetaData, instance);
          String           baseName = getEffectivePermissionBaseName(rules, tableMetaData.getName());
 
-         for(TablePermissionSubType permissionSubType : TablePermissionSubType.values())
+         QBackendMetaData backend = instance.getBackend(tableMetaData.getBackendName());
+         if(tableMetaData.isCapabilityEnabled(backend, Capability.TABLE_INSERT))
          {
-            addEffectiveAvailablePermission(rules, permissionSubType, rs, baseName, tableMetaData, "Table");
+            addEffectiveAvailablePermission(rules, TablePermissionSubType.INSERT, rs, baseName, tableMetaData, "Table");
+         }
+
+         if(tableMetaData.isCapabilityEnabled(backend, Capability.TABLE_UPDATE))
+         {
+            addEffectiveAvailablePermission(rules, TablePermissionSubType.EDIT, rs, baseName, tableMetaData, "Table");
+         }
+
+         if(tableMetaData.isCapabilityEnabled(backend, Capability.TABLE_DELETE))
+         {
+            addEffectiveAvailablePermission(rules, TablePermissionSubType.DELETE, rs, baseName, tableMetaData, "Table");
+         }
+
+         if(tableMetaData.isCapabilityEnabled(backend, Capability.TABLE_QUERY) || tableMetaData.isCapabilityEnabled(backend, Capability.TABLE_GET))
+         {
+            addEffectiveAvailablePermission(rules, TablePermissionSubType.READ, rs, baseName, tableMetaData, "Table");
          }
       }
 
@@ -369,7 +388,10 @@ public class PermissionsHelper
       {
          QPermissionRules rules    = getEffectivePermissionRules(widgetMetaData, instance);
          String           baseName = getEffectivePermissionBaseName(rules, widgetMetaData.getName());
-         addEffectiveAvailablePermission(rules, PrivatePermissionSubType.HAS_ACCESS, rs, baseName, widgetMetaData, "Widget");
+         if(!rules.getLevel().equals(PermissionLevel.NOT_PROTECTED))
+         {
+            addEffectiveAvailablePermission(rules, PrivatePermissionSubType.HAS_ACCESS, rs, baseName, widgetMetaData, "Widget");
+         }
       }
 
       return (rs);

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/processes/RunProcessAction.java

@@ -497,10 +497,10 @@ public class RunProcessAction
       //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
       // if backend specifies that it uses variants, look for that data in the session and append to our basepull key //
       //////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-      if(process.getSchedule() != null && process.getSchedule().getVariantBackend() != null)
+      if(process.getSchedule() != null && process.getVariantBackend() != null)
       {
          QSession         session         = QContext.getQSession();
-         QBackendMetaData backendMetaData = QContext.getQInstance().getBackend(process.getSchedule().getVariantBackend());
+         QBackendMetaData backendMetaData = QContext.getQInstance().getBackend(process.getVariantBackend());
          if(session.getBackendVariants() == null || !session.getBackendVariants().containsKey(backendMetaData.getVariantOptionsTableTypeValue()))
          {
             LOG.info("Could not find Backend Variant information for Backend '" + backendMetaData.getName() + "'");

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/reporting/GenerateReportAction.java

@@ -73,6 +73,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.aggregates.AggregatesInterface;
 import com.kingsrook.qqq.backend.core.utils.aggregates.BigDecimalAggregates;
 import com.kingsrook.qqq.backend.core.utils.aggregates.IntegerAggregates;
+import com.kingsrook.qqq.backend.core.utils.aggregates.LongAggregates;
 
 
 /*******************************************************************************
@@ -273,7 +274,7 @@ public class GenerateReportAction
       RunBackendStepOutput  transformStepOutput = null;
       if(tableView != null && tableView.getRecordTransformStep() != null)
       {
-         transformStep = QCodeLoader.getBackendStep(AbstractTransformStep.class, tableView.getRecordTransformStep());
+         transformStep = QCodeLoader.getAdHoc(AbstractTransformStep.class, tableView.getRecordTransformStep());
 
          transformStepInput = new RunBackendStepInput();
          transformStepInput.setValues(reportInput.getInputValues());
@@ -553,6 +554,12 @@ public class GenerateReportAction
             AggregatesInterface<Integer> fieldAggregates = (AggregatesInterface<Integer>) aggregatesMap.computeIfAbsent(field.getName(), (name) -> new IntegerAggregates());
             fieldAggregates.add(record.getValueInteger(field.getName()));
          }
+         else if(field.getType().equals(QFieldType.LONG))
+         {
+            @SuppressWarnings("unchecked")
+            AggregatesInterface<Long> fieldAggregates = (AggregatesInterface<Long>) aggregatesMap.computeIfAbsent(field.getName(), (name) -> new LongAggregates());
+            fieldAggregates.add(record.getValueLong(field.getName()));
+         }
          else if(field.getType().equals(QFieldType.DECIMAL))
          {
             @SuppressWarnings("unchecked")

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/reporting/RecordPipe.java

@@ -44,7 +44,8 @@ public class RecordPipe
    private static final long BLOCKING_SLEEP_MILLIS = 100;
    private static final long MAX_SLEEP_LOOP_MILLIS = 300_000; // 5 minutes
 
-   private ArrayBlockingQueue<QRecord> queue = new ArrayBlockingQueue<>(1_000);
+   private int capacity = 1_000;
+   private ArrayBlockingQueue<QRecord> queue = new ArrayBlockingQueue<>(capacity);
 
    private boolean isTerminated = false;
 
@@ -72,6 +73,7 @@ public class RecordPipe
     *******************************************************************************/
    public RecordPipe(Integer overrideCapacity)
    {
+      this.capacity = overrideCapacity;
       queue = new ArrayBlockingQueue<>(overrideCapacity);
    }
 
@@ -213,4 +215,14 @@ public class RecordPipe
       this.postRecordActions = postRecordActions;
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for capacity
+    **
+    *******************************************************************************/
+   public int getCapacity()
+   {
+      return capacity;
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/scripts/RunAdHocRecordScriptAction.java

@@ -30,6 +30,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Optional;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
+import com.kingsrook.qqq.backend.core.actions.audits.DMLAuditAction;
 import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
@@ -47,12 +48,14 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryJoin;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.code.AdHocScriptCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.scripts.Script;
 import com.kingsrook.qqq.backend.core.model.scripts.ScriptRevision;
 import com.kingsrook.qqq.backend.core.model.scripts.ScriptsMetaDataProvider;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
 
 
 /*******************************************************************************
@@ -65,6 +68,8 @@ public class RunAdHocRecordScriptAction
    private Map<Integer, ScriptRevision> scriptRevisionCacheByScriptRevisionId = new HashMap<>();
    private Map<Integer, ScriptRevision> scriptRevisionCacheByScriptId         = new HashMap<>();
 
+   private static Memoization<Integer, Script> scriptMemoizationById = new Memoization<>();
+
 
 
    /*******************************************************************************
@@ -85,6 +90,12 @@ public class RunAdHocRecordScriptAction
             throw (new QException("Script revision was not found."));
          }
 
+         Optional<Script> script = getScript(scriptRevision);
+
+         QContext.getQSession().setValue(DMLAuditAction.AUDIT_CONTEXT_FIELD_NAME, script.isPresent()
+            ? "via Script \"%s\"".formatted(script.get().getName())
+            : "via Script id " + scriptRevision.getScriptId());
+
          ////////////////////////////
          // figure out the records //
          ////////////////////////////
@@ -124,6 +135,10 @@ public class RunAdHocRecordScriptAction
       {
          output.setException(Optional.of(e));
       }
+      finally
+      {
+         QContext.getQSession().removeValue(DMLAuditAction.AUDIT_CONTEXT_FIELD_NAME);
+      }
    }
 
 
@@ -217,4 +232,44 @@ public class RunAdHocRecordScriptAction
       throw (new QException("Code reference did not contain a scriptRevision, scriptRevisionId, or scriptId"));
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private Optional<Script> getScript(ScriptRevision scriptRevision)
+   {
+      if(scriptRevision == null || scriptRevision.getScriptId() == null)
+      {
+         return (Optional.empty());
+      }
+
+      try
+      {
+         return scriptMemoizationById.getResult(scriptRevision.getScriptId(), scriptId ->
+         {
+            try
+            {
+               QRecord scriptRecord = new GetAction().executeForRecord(new GetInput(Script.TABLE_NAME).withPrimaryKey(scriptRevision.getScriptId()));
+               if(scriptRecord != null)
+               {
+                  Script script = new Script(scriptRecord);
+                  scriptMemoizationById.storeResult(scriptRevision.getScriptId(), script);
+                  return (script);
+               }
+            }
+            catch(Exception e)
+            {
+               LOG.info("");
+            }
+
+            return (null);
+         });
+      }
+      catch(Exception e)
+      {
+         return (Optional.empty());
+      }
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/DeleteAction.java

@@ -35,9 +35,8 @@ import java.util.Set;
 import java.util.stream.Collectors;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
 import com.kingsrook.qqq.backend.core.actions.audits.DMLAuditAction;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostDeleteCustomizer;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreDeleteCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.interfaces.DeleteInterface;
 import com.kingsrook.qqq.backend.core.actions.tables.helpers.ValidateRecordSecurityLockHelper;
@@ -250,7 +249,7 @@ public class DeleteAction
       //////////////////////////////////////////////////////////////
       // finally, run the post-delete customizer, if there is one //
       //////////////////////////////////////////////////////////////
-      Optional<AbstractPostDeleteCustomizer> postDeleteCustomizer = QCodeLoader.getTableCustomizer(AbstractPostDeleteCustomizer.class, table, TableCustomizers.POST_DELETE_RECORD.getRole());
+      Optional<TableCustomizerInterface> postDeleteCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_DELETE_RECORD.getRole());
       if(postDeleteCustomizer.isPresent() && oldRecordList.isPresent())
       {
          ////////////////////////////////////////////////////////////////////////////
@@ -260,8 +259,7 @@ public class DeleteAction
 
          try
          {
-            postDeleteCustomizer.get().setDeleteInput(deleteInput);
-            List<QRecord> postCustomizerResult = postDeleteCustomizer.get().apply(recordsForCustomizer);
+            List<QRecord> postCustomizerResult = postDeleteCustomizer.get().postDelete(deleteInput, recordsForCustomizer);
 
             ///////////////////////////////////////////////////////
             // check if any records got errors in the customizer //
@@ -327,13 +325,11 @@ public class DeleteAction
       ///////////////////////////////////////////////////////////////////////////
       // after all validations, run the pre-delete customizer, if there is one //
       ///////////////////////////////////////////////////////////////////////////
-      Optional<AbstractPreDeleteCustomizer> preDeleteCustomizer = QCodeLoader.getTableCustomizer(AbstractPreDeleteCustomizer.class, table, TableCustomizers.PRE_DELETE_RECORD.getRole());
+      Optional<TableCustomizerInterface> preDeleteCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_DELETE_RECORD.getRole());
       List<QRecord>                         customizerResult    = oldRecordList.get();
       if(preDeleteCustomizer.isPresent())
       {
-         preDeleteCustomizer.get().setDeleteInput(deleteInput);
-         preDeleteCustomizer.get().setIsPreview(isPreview);
-         customizerResult = preDeleteCustomizer.get().apply(oldRecordList.get());
+         customizerResult = preDeleteCustomizer.get().preDelete(deleteInput, oldRecordList.get(), isPreview);
       }
 
       /////////////////////////////////////////////////////////////////////////

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/GetAction.java

@@ -27,8 +27,8 @@ import java.util.List;
 import java.util.Map;
 import java.util.Optional;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostQueryCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.interfaces.GetInterface;
 import com.kingsrook.qqq.backend.core.actions.tables.helpers.GetActionCacheHelper;
@@ -49,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
 import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleInterface;
+import com.kingsrook.qqq.backend.core.utils.ObjectUtils;
 
 
 /*******************************************************************************
@@ -57,7 +58,7 @@ import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleInterface;
  *******************************************************************************/
 public class GetAction
 {
-   private Optional<AbstractPostQueryCustomizer> postGetRecordCustomizer;
+   private Optional<TableCustomizerInterface> postGetRecordCustomizer;
 
    private GetInput                 getInput;
    private QPossibleValueTranslator qPossibleValueTranslator;
@@ -87,7 +88,7 @@ public class GetAction
          throw (new QException("Requested to Get a record from an unrecognized table: " + getInput.getTableName()));
       }
 
-      postGetRecordCustomizer = QCodeLoader.getTableCustomizer(AbstractPostQueryCustomizer.class, table, TableCustomizers.POST_QUERY_RECORD.getRole());
+      postGetRecordCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_QUERY_RECORD.getRole());
       this.getInput = getInput;
 
       QBackendModuleDispatcher qBackendModuleDispatcher = new QBackendModuleDispatcher();
@@ -107,9 +108,11 @@ public class GetAction
       }
 
       GetOutput getOutput;
+      boolean usingDefaultGetInterface = false;
       if(getInterface == null)
       {
          getInterface = new DefaultGetInterface();
+         usingDefaultGetInterface = true;
       }
 
       getInterface.validateInput(getInput);
@@ -123,10 +126,11 @@ public class GetAction
          new GetActionCacheHelper().handleCaching(getInput, getOutput);
       }
 
-      ////////////////////////////////////////////////////////
-      // if the record is found, perform post-actions on it //
-      ////////////////////////////////////////////////////////
-      if(getOutput.getRecord() != null)
+      ////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      // if the record is found, perform post-actions on it                                                         //
+      // unless the defaultGetInterface was used - as it just does a query, and the query will do the post-actions. //
+      ////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+      if(getOutput.getRecord() != null && !usingDefaultGetInterface)
       {
          getOutput.setRecord(postRecordActions(getOutput.getRecord()));
       }
@@ -202,7 +206,7 @@ public class GetAction
       }
       else
       {
-         throw (new QException("No primaryKey or uniqueKey was passed to Get"));
+         throw (new QException("Unable to get " + ObjectUtils.tryElse(() -> queryInput.getTable().getLabel(), queryInput.getTableName()) + ".  Missing required input."));
       }
 
       queryInput.setFilter(filter);
@@ -216,12 +220,12 @@ public class GetAction
     ** Run the necessary actions on a record.  This may include setting display values,
     ** translating possible values, and running post-record customizations.
     *******************************************************************************/
-   public QRecord postRecordActions(QRecord record)
+   public QRecord postRecordActions(QRecord record) throws QException
    {
       QRecord returnRecord = record;
       if(this.postGetRecordCustomizer.isPresent())
       {
-         returnRecord = postGetRecordCustomizer.get().apply(List.of(record)).get(0);
+         returnRecord = postGetRecordCustomizer.get().postQuery(getInput, List.of(record)).get(0);
       }
 
       if(getInput.getShouldTranslatePossibleValues())

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/InsertAction.java

@@ -28,6 +28,7 @@ import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
+import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
 import java.util.stream.Collectors;
@@ -37,9 +38,9 @@ import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.actions.audits.DMLAuditAction;
 import com.kingsrook.qqq.backend.core.actions.automation.AutomationStatus;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationStatusUpdater;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostInsertCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreInsertCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.interfaces.InsertInterface;
 import com.kingsrook.qqq.backend.core.actions.tables.helpers.UniqueKeyHelper;
@@ -168,13 +169,12 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
       //////////////////////////////////////////////////////////////
       // finally, run the post-insert customizer, if there is one //
       //////////////////////////////////////////////////////////////
-      Optional<AbstractPostInsertCustomizer> postInsertCustomizer = QCodeLoader.getTableCustomizer(AbstractPostInsertCustomizer.class, table, TableCustomizers.POST_INSERT_RECORD.getRole());
+      Optional<TableCustomizerInterface> postInsertCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_INSERT_RECORD.getRole());
       if(postInsertCustomizer.isPresent())
       {
          try
          {
-            postInsertCustomizer.get().setInsertInput(insertInput);
-            insertOutput.setRecords(postInsertCustomizer.get().apply(insertOutput.getRecords()));
+            insertOutput.setRecords(postInsertCustomizer.get().postInsert(insertInput, insertOutput.getRecords()));
          }
          catch(Exception e)
          {
@@ -232,31 +232,29 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
       // load the pre-insert customizer and set it up, if there is one //
       // then we'll run it based on its WhenToRun value                //
       ///////////////////////////////////////////////////////////////////
-      Optional<AbstractPreInsertCustomizer> preInsertCustomizer = QCodeLoader.getTableCustomizer(AbstractPreInsertCustomizer.class, table, TableCustomizers.PRE_INSERT_RECORD.getRole());
+      Optional<TableCustomizerInterface> preInsertCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_INSERT_RECORD.getRole());
       if(preInsertCustomizer.isPresent())
       {
-         preInsertCustomizer.get().setInsertInput(insertInput);
-         preInsertCustomizer.get().setIsPreview(isPreview);
-         runPreInsertCustomizerIfItIsTime(insertInput, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_ALL_VALIDATIONS);
+         runPreInsertCustomizerIfItIsTime(insertInput, isPreview, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_ALL_VALIDATIONS);
       }
 
       setDefaultValuesInRecords(table, insertInput.getRecords());
 
-      ValueBehaviorApplier.applyFieldBehaviors(ValueBehaviorApplier.Action.INSERT, insertInput.getInstance(), table, insertInput.getRecords());
+      ValueBehaviorApplier.applyFieldBehaviors(ValueBehaviorApplier.Action.INSERT, insertInput.getInstance(), table, insertInput.getRecords(), null);
 
-      runPreInsertCustomizerIfItIsTime(insertInput, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_UNIQUE_KEY_CHECKS);
+      runPreInsertCustomizerIfItIsTime(insertInput, isPreview, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_UNIQUE_KEY_CHECKS);
       setErrorsIfUniqueKeyErrors(insertInput, table);
 
-      runPreInsertCustomizerIfItIsTime(insertInput, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_REQUIRED_FIELD_CHECKS);
+      runPreInsertCustomizerIfItIsTime(insertInput, isPreview, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_REQUIRED_FIELD_CHECKS);
       if(insertInput.getInputSource().shouldValidateRequiredFields())
       {
          validateRequiredFields(insertInput);
       }
 
-      runPreInsertCustomizerIfItIsTime(insertInput, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_SECURITY_CHECKS);
+      runPreInsertCustomizerIfItIsTime(insertInput, isPreview, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.BEFORE_SECURITY_CHECKS);
       ValidateRecordSecurityLockHelper.validateSecurityFields(insertInput.getTable(), insertInput.getRecords(), ValidateRecordSecurityLockHelper.Action.INSERT);
 
-      runPreInsertCustomizerIfItIsTime(insertInput, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.AFTER_ALL_VALIDATIONS);
+      runPreInsertCustomizerIfItIsTime(insertInput, isPreview, preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun.AFTER_ALL_VALIDATIONS);
    }
 
 
@@ -290,13 +288,13 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
    /*******************************************************************************
     **
     *******************************************************************************/
-   private void runPreInsertCustomizerIfItIsTime(InsertInput insertInput, Optional<AbstractPreInsertCustomizer> preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun whenToRun) throws QException
+   private void runPreInsertCustomizerIfItIsTime(InsertInput insertInput, boolean isPreview, Optional<TableCustomizerInterface> preInsertCustomizer, AbstractPreInsertCustomizer.WhenToRun whenToRun) throws QException
    {
       if(preInsertCustomizer.isPresent())
       {
-         if(whenToRun.equals(preInsertCustomizer.get().getWhenToRun()))
+         if(whenToRun.equals(preInsertCustomizer.get().whenToRunPreInsert(insertInput, isPreview)))
          {
-            insertInput.setRecords(preInsertCustomizer.get().apply(insertInput.getRecords()));
+            insertInput.setRecords(preInsertCustomizer.get().preInsert(insertInput, insertInput.getRecords(), isPreview));
          }
       }
    }
@@ -321,7 +319,7 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
             {
                if(record.getValue(requiredField.getName()) == null || (requiredField.getType().isStringLike() && record.getValueString(requiredField.getName()).trim().equals("")))
                {
-                  record.addError(new BadInputStatusMessage("Missing value in required field: " + requiredField.getLabel()));
+                  record.addError(new BadInputStatusMessage("Missing value in required field: " + Objects.requireNonNullElse(requiredField.getLabel(), requiredField.getName())));
                }
             }
          }
@@ -444,7 +442,7 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
     *******************************************************************************/
    private void setAutomationStatusField(InsertInput insertInput)
    {
-      RecordAutomationStatusUpdater.setAutomationStatusInRecords(insertInput.getSession(), insertInput.getTable(), insertInput.getRecords(), AutomationStatus.PENDING_INSERT_AUTOMATIONS);
+      RecordAutomationStatusUpdater.setAutomationStatusInRecords(insertInput.getTable(), insertInput.getRecords(), AutomationStatus.PENDING_INSERT_AUTOMATIONS, insertInput.getTransaction(), null);
    }
 
 
@@ -455,7 +453,7 @@ public class InsertAction extends AbstractQActionFunction<InsertInput, InsertOut
    private QBackendModuleInterface getBackendModuleInterface(QBackendMetaData backend) throws QException
    {
       QBackendModuleDispatcher qBackendModuleDispatcher = new QBackendModuleDispatcher();
-      QBackendModuleInterface qModule = qBackendModuleDispatcher.getQBackendModule(backend);
+      QBackendModuleInterface  qModule                  = qBackendModuleDispatcher.getQBackendModule(backend);
       return (qModule);
    }
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/QueryAction.java

@@ -31,8 +31,8 @@ import java.util.Map;
 import java.util.Optional;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.ActionHelper;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostQueryCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.interfaces.QueryInterface;
 import com.kingsrook.qqq.backend.core.actions.reporting.BufferedRecordPipe;
@@ -73,7 +73,7 @@ public class QueryAction
 {
    private static final QLogger LOG = QLogger.getLogger(QueryAction.class);
 
-   private Optional<AbstractPostQueryCustomizer> postQueryRecordCustomizer;
+   private Optional<TableCustomizerInterface> postQueryRecordCustomizer;
 
    private QueryInput               queryInput;
    private QueryInterface           queryInterface;
@@ -100,7 +100,7 @@ public class QueryAction
       }
 
       QBackendMetaData backend = queryInput.getBackend();
-      postQueryRecordCustomizer = QCodeLoader.getTableCustomizer(AbstractPostQueryCustomizer.class, table, TableCustomizers.POST_QUERY_RECORD.getRole());
+      postQueryRecordCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_QUERY_RECORD.getRole());
       this.queryInput = queryInput;
 
       if(queryInput.getRecordPipe() != null)
@@ -264,7 +264,7 @@ public class QueryAction
    {
       if(this.postQueryRecordCustomizer.isPresent())
       {
-         records = postQueryRecordCustomizer.get().apply(records);
+         records = postQueryRecordCustomizer.get().postQuery(queryInput, records);
       }
 
       if(queryInput.getShouldTranslatePossibleValues())

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/ReplaceAction.java

@@ -136,19 +136,22 @@ public class ReplaceAction extends AbstractQActionFunction<ReplaceInput, Replace
          UpdateOutput updateOutput = new UpdateAction().execute(updateInput);
          output.setUpdateOutput(updateOutput);
 
-         QQueryFilter deleteFilter = new QQueryFilter(new QFilterCriteria(primaryKeyField, QCriteriaOperator.NOT_IN, primaryKeysToKeep));
-         if(input.getFilter() != null)
+         if(input.getPerformDeletes())
          {
-            deleteFilter.addSubFilter(input.getFilter());
-         }
+            QQueryFilter deleteFilter = new QQueryFilter(new QFilterCriteria(primaryKeyField, QCriteriaOperator.NOT_IN, primaryKeysToKeep));
+            if(input.getFilter() != null)
+            {
+               deleteFilter.addSubFilter(input.getFilter());
+            }
 
-         DeleteInput deleteInput = new DeleteInput();
-         deleteInput.setTableName(table.getName());
-         deleteInput.setQueryFilter(deleteFilter);
-         deleteInput.setTransaction(transaction);
-         deleteInput.setOmitDmlAudit(input.getOmitDmlAudit());
-         DeleteOutput deleteOutput = new DeleteAction().execute(deleteInput);
-         output.setDeleteOutput(deleteOutput);
+            DeleteInput deleteInput = new DeleteInput();
+            deleteInput.setTableName(table.getName());
+            deleteInput.setQueryFilter(deleteFilter);
+            deleteInput.setTransaction(transaction);
+            deleteInput.setOmitDmlAudit(input.getOmitDmlAudit());
+            DeleteOutput deleteOutput = new DeleteAction().execute(deleteInput);
+            output.setDeleteOutput(deleteOutput);
+         }
 
          if(weOwnTheTransaction)
          {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/UpdateAction.java

@@ -34,9 +34,8 @@ import com.kingsrook.qqq.backend.core.actions.ActionHelper;
 import com.kingsrook.qqq.backend.core.actions.audits.DMLAuditAction;
 import com.kingsrook.qqq.backend.core.actions.automation.AutomationStatus;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationStatusUpdater;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostUpdateCustomizer;
-import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreUpdateCustomizer;
 import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.interfaces.UpdateInterface;
 import com.kingsrook.qqq.backend.core.actions.tables.helpers.ValidateRecordSecurityLockHelper;
@@ -57,6 +56,8 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldBehavior;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
@@ -72,6 +73,7 @@ import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
 import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleInterface;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import org.apache.commons.lang.BooleanUtils;
 import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
@@ -113,7 +115,6 @@ public class UpdateAction
    public UpdateOutput execute(UpdateInput updateInput) throws QException
    {
       ActionHelper.validateSession(updateInput);
-      setAutomationStatusField(updateInput);
 
       QTableMetaData table = updateInput.getTable();
 
@@ -130,6 +131,16 @@ public class UpdateAction
       ////////////////////////////////////////////////////////////////////////////////
       Optional<List<QRecord>> oldRecordList = fetchOldRecords(updateInput, updateInterface);
 
+      ///////////////////////////////////////////////////////////////////////////////////////
+      // allow caller to specify that we don't want to trigger automations. this isn't     //
+      // isn't expected to be used much - by design, only for the process that is meant to //
+      // heal automation status, so that it can force us into status=Pending-inserts       //
+      ///////////////////////////////////////////////////////////////////////////////////////
+      if(!updateInput.getOmitTriggeringAutomations())
+      {
+         setAutomationStatusField(updateInput, oldRecordList);
+      }
+
       performValidations(updateInput, oldRecordList, false);
 
       ////////////////////////////////////
@@ -180,14 +191,12 @@ public class UpdateAction
       //////////////////////////////////////////////////////////////
       // finally, run the post-update customizer, if there is one //
       //////////////////////////////////////////////////////////////
-      Optional<AbstractPostUpdateCustomizer> postUpdateCustomizer = QCodeLoader.getTableCustomizer(AbstractPostUpdateCustomizer.class, table, TableCustomizers.POST_UPDATE_RECORD.getRole());
+      Optional<TableCustomizerInterface> postUpdateCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.POST_UPDATE_RECORD.getRole());
       if(postUpdateCustomizer.isPresent())
       {
          try
          {
-            postUpdateCustomizer.get().setUpdateInput(updateInput);
-            oldRecordList.ifPresent(l -> postUpdateCustomizer.get().setOldRecordList(l));
-            updateOutput.setRecords(postUpdateCustomizer.get().apply(updateOutput.getRecords()));
+            updateOutput.setRecords(postUpdateCustomizer.get().postUpdate(updateInput, updateOutput.getRecords(), oldRecordList));
          }
          catch(Exception e)
          {
@@ -235,7 +244,13 @@ public class UpdateAction
       /////////////////////////////
       // run standard validators //
       /////////////////////////////
-      ValueBehaviorApplier.applyFieldBehaviors(ValueBehaviorApplier.Action.UPDATE, updateInput.getInstance(), table, updateInput.getRecords());
+      Set<FieldBehavior<?>> behaviorsToOmit = null;
+      if(BooleanUtils.isTrue(updateInput.getOmitModifyDateUpdate()))
+      {
+         behaviorsToOmit = Set.of(DynamicDefaultValueBehavior.MODIFY_DATE);
+      }
+
+      ValueBehaviorApplier.applyFieldBehaviors(ValueBehaviorApplier.Action.UPDATE, updateInput.getInstance(), table, updateInput.getRecords(), behaviorsToOmit);
       validatePrimaryKeysAreGiven(updateInput);
 
       if(oldRecordList.isPresent())
@@ -255,13 +270,10 @@ public class UpdateAction
       ///////////////////////////////////////////////////////////////////////////
       // after all validations, run the pre-update customizer, if there is one //
       ///////////////////////////////////////////////////////////////////////////
-      Optional<AbstractPreUpdateCustomizer> preUpdateCustomizer = QCodeLoader.getTableCustomizer(AbstractPreUpdateCustomizer.class, table, TableCustomizers.PRE_UPDATE_RECORD.getRole());
+      Optional<TableCustomizerInterface> preUpdateCustomizer = QCodeLoader.getTableCustomizer(table, TableCustomizers.PRE_UPDATE_RECORD.getRole());
       if(preUpdateCustomizer.isPresent())
       {
-         preUpdateCustomizer.get().setUpdateInput(updateInput);
-         preUpdateCustomizer.get().setIsPreview(isPreview);
-         oldRecordList.ifPresent(l -> preUpdateCustomizer.get().setOldRecordList(l));
-         updateInput.setRecords(preUpdateCustomizer.get().apply(updateInput.getRecords()));
+         updateInput.setRecords(preUpdateCustomizer.get().preUpdate(updateInput, updateInput.getRecords(), isPreview, oldRecordList));
       }
    }
 
@@ -561,9 +573,9 @@ public class UpdateAction
    /*******************************************************************************
     ** If the table being updated uses an automation-status field, populate it now.
     *******************************************************************************/
-   private void setAutomationStatusField(UpdateInput updateInput)
+   private void setAutomationStatusField(UpdateInput updateInput, Optional<List<QRecord>> oldRecordList)
    {
-      RecordAutomationStatusUpdater.setAutomationStatusInRecords(updateInput.getSession(), updateInput.getTable(), updateInput.getRecords(), AutomationStatus.PENDING_UPDATE_AUTOMATIONS);
+      RecordAutomationStatusUpdater.setAutomationStatusInRecords(updateInput.getTable(), updateInput.getRecords(), AutomationStatus.PENDING_UPDATE_AUTOMATIONS, updateInput.getTransaction(), oldRecordList.orElse(null));
    }
 
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/tables/helpers/ValidateRecordSecurityLockHelper.java

@@ -42,6 +42,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.security.NullValueBehaviorUtil;
 import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLockFilters;
@@ -230,7 +231,7 @@ public class ValidateRecordSecurityLockHelper
                   {
                      for(QRecord inputRecord : inputRecords)
                      {
-                        if(RecordSecurityLock.NullValueBehavior.DENY.equals(recordSecurityLock.getNullValueBehavior()))
+                        if(RecordSecurityLock.NullValueBehavior.DENY.equals(NullValueBehaviorUtil.getEffectiveNullValueBehavior(recordSecurityLock)))
                         {
                            inputRecord.addError(new PermissionDeniedMessage("You do not have permission to " + action.name().toLowerCase() + " this record - the referenced " + leftMostJoinTable.getLabel() + " was not found."));
                         }
@@ -298,7 +299,7 @@ public class ValidateRecordSecurityLockHelper
          /////////////////////////////////////////////////////////////////
          // handle null values - error if the NullValueBehavior is DENY //
          /////////////////////////////////////////////////////////////////
-         if(RecordSecurityLock.NullValueBehavior.DENY.equals(recordSecurityLock.getNullValueBehavior()))
+         if(RecordSecurityLock.NullValueBehavior.DENY.equals(NullValueBehaviorUtil.getEffectiveNullValueBehavior(recordSecurityLock)))
          {
             String lockLabel = CollectionUtils.nullSafeHasContents(recordSecurityLock.getJoinNameChain()) ? recordSecurityLock.getSecurityKeyType() : table.getField(recordSecurityLock.getFieldName()).getLabel();
             record.addError(new PermissionDeniedMessage("You do not have permission to " + action.name().toLowerCase() + " a record without a value in the field: " + lockLabel));

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/QPossibleValueTranslator.java

@@ -269,6 +269,10 @@ public class QPossibleValueTranslator
          {
             value = ValueUtils.getValueAsInteger(value);
          }
+         if(field.getType().equals(QFieldType.LONG) && !(value instanceof Long))
+         {
+            value = ValueUtils.getValueAsLong(value);
+         }
       }
       catch(QValueException e)
       {

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/QValueFormatter.java

@@ -490,6 +490,13 @@ public class QValueFormatter
             {
                adornmentValues = fileDownloadAdornment.get().getValues();
             }
+            else
+            {
+               ///////////////////////////////////////////////////////
+               // don't change blobs unless they are file-downloads //
+               ///////////////////////////////////////////////////////
+               continue;
+            }
 
             String fileNameField    = ValueUtils.getValueAsString(adornmentValues.get(AdornmentType.FileDownloadValues.FILE_NAME_FIELD));
             String fileNameFormat   = ValueUtils.getValueAsString(adornmentValues.get(AdornmentType.FileDownloadValues.FILE_NAME_FORMAT));

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/actions/values/ValueBehaviorApplier.java

@@ -23,6 +23,7 @@ package com.kingsrook.qqq.backend.core.actions.values;
 
 
 import java.util.List;
+import java.util.Set;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldBehavior;
@@ -32,7 +33,7 @@ import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 
 
 /*******************************************************************************
- ** Utility class to apply value behaviors to records.  
+ ** Utility class to apply value behaviors to records.
  *******************************************************************************/
 public class ValueBehaviorApplier
 {
@@ -51,7 +52,7 @@ public class ValueBehaviorApplier
    /*******************************************************************************
     **
     *******************************************************************************/
-   public static void applyFieldBehaviors(Action action, QInstance instance, QTableMetaData table, List<QRecord> recordList)
+   public static void applyFieldBehaviors(Action action, QInstance instance, QTableMetaData table, List<QRecord> recordList, Set<FieldBehavior<?>> behaviorsToOmit)
    {
       if(CollectionUtils.nullSafeIsEmpty(recordList))
       {
@@ -62,7 +63,7 @@ public class ValueBehaviorApplier
       {
          for(FieldBehavior<?> fieldBehavior : CollectionUtils.nonNullCollection(field.getBehaviors()))
          {
-            fieldBehavior.apply(action, recordList, instance, table, field);
+            fieldBehavior.apply(action, recordList, instance, table, field, behaviorsToOmit);
          }
       }
    }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/context/CapturedContext.java

@@ -34,5 +34,11 @@ import com.kingsrook.qqq.backend.core.model.session.QSession;
  *******************************************************************************/
 public record CapturedContext(QInstance qInstance, QSession qSession, QBackendTransaction qBackendTransaction, Stack<AbstractActionInput> actionStack)
 {
-
+   /*******************************************************************************
+    ** Simpler constructor
+    *******************************************************************************/
+   public CapturedContext(QInstance qInstance, QSession qSession)
+   {
+      this(qInstance, qSession, null, null);
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/context/QContext.java

@@ -22,6 +22,9 @@
 package com.kingsrook.qqq.backend.core.context;
 
 
+import java.io.Serializable;
+import java.util.HashMap;
+import java.util.Map;
 import java.util.Optional;
 import java.util.Stack;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
@@ -31,6 +34,8 @@ import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.utils.lambdas.UnsafeVoidVoidMethod;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
 
 
 /*******************************************************************************
@@ -47,6 +52,8 @@ public class QContext
    private static ThreadLocal<QBackendTransaction>        qBackendTransactionThreadLocal = new ThreadLocal<>();
    private static ThreadLocal<Stack<AbstractActionInput>> actionStackThreadLocal         = new ThreadLocal<>();
 
+   private static ThreadLocal<Map<String, Serializable>> objectsThreadLocal = new ThreadLocal<>();
+
 
 
    /*******************************************************************************
@@ -100,6 +107,25 @@ public class QContext
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static <T extends Throwable> void withTemporaryContext(CapturedContext context, UnsafeVoidVoidMethod<T> method) throws T
+   {
+      CapturedContext originalContext = QContext.capture();
+      try
+      {
+         QContext.init(context);
+         method.run();
+      }
+      finally
+      {
+         QContext.init(originalContext);
+      }
+   }
+
+
+
    /*******************************************************************************
     ** Init a new thread with the context captured from a different thread.  e.g.,
     ** when starting some async task.
@@ -132,6 +158,7 @@ public class QContext
       qSessionThreadLocal.remove();
       qBackendTransactionThreadLocal.remove();
       actionStackThreadLocal.remove();
+      objectsThreadLocal.remove();
    }
 
 
@@ -259,4 +286,94 @@ public class QContext
 
       return (Optional.of(actionStackThreadLocal.get().get(0)));
    }
+
+
+
+   /*******************************************************************************
+    ** get one named object from the Context for the current thread.  may return null.
+    *******************************************************************************/
+   public static Serializable getObject(String key)
+   {
+      if(objectsThreadLocal.get() == null)
+      {
+         return null;
+      }
+      return objectsThreadLocal.get().get(key);
+   }
+
+
+
+   /*******************************************************************************
+    ** get one named object from the Context for the current thread, cast to the
+    ** specified type if possible.  if not found, or wrong type, empty is returned.
+    *******************************************************************************/
+   public static <T extends Serializable> Optional<T> getObject(String key, Class<T> type)
+   {
+      Serializable object = getObject(key);
+
+      if(type.isInstance(object))
+      {
+         return Optional.of(type.cast(object));
+      }
+      else if(object == null)
+      {
+         return Optional.empty();
+      }
+      else
+      {
+         LOG.warn("Unexpected type of object found in session under key [" + key + "]",
+            logPair("expectedType", type.getName()),
+            logPair("actualType", object.getClass().getName())
+         );
+         return Optional.empty();
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** put a named object into the Context for the current thread.
+    *******************************************************************************/
+   public static void setObject(String key, Serializable object)
+   {
+      if(objectsThreadLocal.get() == null)
+      {
+         objectsThreadLocal.set(new HashMap<>());
+      }
+      objectsThreadLocal.get().put(key, object);
+   }
+
+
+
+   /*******************************************************************************
+    ** remove a named object from the Context of the current thread.
+    *******************************************************************************/
+   public static void removeObject(String key)
+   {
+      if(objectsThreadLocal.get() != null)
+      {
+         objectsThreadLocal.get().remove(key);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** get the full map of named objects for the current thread (possibly null).
+    *******************************************************************************/
+   public static Map<String, Serializable> getObjects()
+   {
+      return objectsThreadLocal.get();
+   }
+
+
+
+   /*******************************************************************************
+    ** fully replace the map of named objets for the current thread.
+    *******************************************************************************/
+   public static void setObjects(Map<String, Serializable> objects)
+   {
+      objectsThreadLocal.set(objects);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceEnricher.java

@@ -1030,6 +1030,50 @@ public class QInstanceEnricher
 
 
 
+   /*******************************************************************************
+    ** Do a default mapping from an underscore_style field name to a camelCase name.
+    **
+    ** Examples:
+    ** <ul>
+    **   <li>word_another_word_more_words -> wordAnotherWordMoreWords</li>
+    **   <li>l_ul_ul_ul -> lUlUlUl</li>
+    **   <li>tla_first -> tlaFirst</li>
+    **   <li>word_then_tla_in_middle -> wordThenTlaInMiddle</li>
+    **   <li>end_with_tla -> endWithTla</li>
+    **   <li>tla_and_another_tla -> tlaAndAnotherTla</li>
+    **   <li>ALL_CAPS -> allCaps</li>
+    ** </ul>
+    *******************************************************************************/
+   public static String inferNameFromBackendName(String backendName)
+   {
+      StringBuilder rs = new StringBuilder();
+
+      ////////////////////////////////////////////////////////////////////////////////////////
+      // build a list of words in the name, then join them with _ and lower-case the result //
+      ////////////////////////////////////////////////////////////////////////////////////////
+      String[] words = backendName.toLowerCase(Locale.ROOT).split("_");
+      for(int i = 0; i < words.length; i++)
+      {
+         String word = words[i];
+         if(i == 0)
+         {
+            rs.append(word);
+         }
+         else
+         {
+            rs.append(word.substring(0, 1).toUpperCase());
+            if(word.length() > 1)
+            {
+               rs.append(word.substring(1));
+            }
+         }
+      }
+
+      return (rs.toString());
+   }
+
+
+
    /*******************************************************************************
     ** If a app didn't have any sections, generate "sensible defaults"
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceHelpContentManager.java

@@ -30,12 +30,14 @@ import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.helpcontent.HelpContent;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpFormat;
 import com.kingsrook.qqq.backend.core.model.metadata.help.HelpRole;
@@ -109,6 +111,8 @@ public class QInstanceHelpContentManager
          String processName = nameValuePairs.get("process");
          String fieldName   = nameValuePairs.get("field");
          String sectionName = nameValuePairs.get("section");
+         String widgetName  = nameValuePairs.get("widget");
+         String slotName    = nameValuePairs.get("slot");
 
          ///////////////////////////////////////////////////////////
          // build a help content meta-data object from the record //
@@ -137,89 +141,16 @@ public class QInstanceHelpContentManager
          ///////////////////////////////////////////////////////////////////////////////////
          if(StringUtils.hasContent(tableName))
          {
-            QTableMetaData table = qInstance.getTable(tableName);
-            if(table == null)
-            {
-               LOG.info("Unrecognized table in help content", logPair("key", key));
-               return;
-            }
-
-            if(StringUtils.hasContent(fieldName))
-            {
-               //////////////////////////
-               // handle a table field //
-               //////////////////////////
-               QFieldMetaData field = table.getFields().get(fieldName);
-               if(field == null)
-               {
-                  LOG.info("Unrecognized table field in help content", logPair("key", key));
-                  return;
-               }
-
-               if(helpContent != null)
-               {
-                  field.withHelpContent(helpContent);
-               }
-               else
-               {
-                  field.removeHelpContent(roles);
-               }
-            }
-            else if(StringUtils.hasContent(sectionName))
-            {
-               ////////////////////////////
-               // handle a table section //
-               ////////////////////////////
-               Optional<QFieldSection> optionalSection = table.getSections().stream().filter(s -> sectionName.equals(s.getName())).findFirst();
-               if(optionalSection.isEmpty())
-               {
-                  LOG.info("Unrecognized table section in help content", logPair("key", key));
-                  return;
-               }
-
-               if(helpContent != null)
-               {
-                  optionalSection.get().withHelpContent(helpContent);
-               }
-               else
-               {
-                  optionalSection.get().removeHelpContent(roles);
-               }
-            }
+            processHelpContentForTable(key, tableName, sectionName, fieldName, roles, helpContent);
          }
          else if(StringUtils.hasContent(processName))
          {
-            QProcessMetaData process = qInstance.getProcess(processName);
-            if(process == null)
-            {
-               LOG.info("Unrecognized process in help content", logPair("key", key));
-               return;
-            }
+            processHelpContentForProcess(key, processName, fieldName, roles, helpContent);
+         }
+         else if(StringUtils.hasContent(widgetName))
+         {
+            processHelpContentForWidget(key, widgetName, slotName, helpContent);
 
-            if(StringUtils.hasContent(fieldName))
-            {
-               ////////////////////////////
-               // handle a process field //
-               ////////////////////////////
-               Optional<QFieldMetaData> optionalField = CollectionUtils.mergeLists(process.getInputFields(), process.getOutputFields())
-                  .stream().filter(f -> fieldName.equals(f.getName()))
-                  .findFirst();
-
-               if(optionalField.isEmpty())
-               {
-                  LOG.info("Unrecognized process field in help content", logPair("key", key));
-                  return;
-               }
-
-               if(helpContent != null)
-               {
-                  optionalField.get().withHelpContent(helpContent);
-               }
-               else
-               {
-                  optionalField.get().removeHelpContent(roles);
-               }
-            }
          }
       }
       catch(Exception e)
@@ -230,6 +161,131 @@ public class QInstanceHelpContentManager
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static void processHelpContentForTable(String key, String tableName, String sectionName, String fieldName, Set<HelpRole> roles, QHelpContent helpContent)
+   {
+      QTableMetaData table = QContext.getQInstance().getTable(tableName);
+      if(table == null)
+      {
+         LOG.info("Unrecognized table in help content", logPair("key", key));
+      }
+      else if(StringUtils.hasContent(fieldName))
+      {
+         //////////////////////////
+         // handle a table field //
+         //////////////////////////
+         QFieldMetaData field = table.getFields().get(fieldName);
+         if(field == null)
+         {
+            LOG.info("Unrecognized table field in help content", logPair("key", key));
+         }
+         else if(helpContent != null)
+         {
+            field.withHelpContent(helpContent);
+         }
+         else
+         {
+            field.removeHelpContent(roles);
+         }
+      }
+      else if(StringUtils.hasContent(sectionName))
+      {
+         ////////////////////////////
+         // handle a table section //
+         ////////////////////////////
+         Optional<QFieldSection> optionalSection = table.getSections().stream().filter(s -> sectionName.equals(s.getName())).findFirst();
+         if(optionalSection.isEmpty())
+         {
+            LOG.info("Unrecognized table section in help content", logPair("key", key));
+         }
+         else if(helpContent != null)
+         {
+            optionalSection.get().withHelpContent(helpContent);
+         }
+         else
+         {
+            optionalSection.get().removeHelpContent(roles);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static void processHelpContentForProcess(String key, String processName, String fieldName, Set<HelpRole> roles, QHelpContent helpContent)
+   {
+      QProcessMetaData process = QContext.getQInstance().getProcess(processName);
+      if(process == null)
+      {
+         LOG.info("Unrecognized process in help content", logPair("key", key));
+      }
+      else if(StringUtils.hasContent(fieldName))
+      {
+         ////////////////////////////
+         // handle a process field //
+         ////////////////////////////
+         Optional<QFieldMetaData> optionalField = CollectionUtils.mergeLists(process.getInputFields(), process.getOutputFields())
+            .stream().filter(f -> fieldName.equals(f.getName()))
+            .findFirst();
+
+         if(optionalField.isEmpty())
+         {
+            LOG.info("Unrecognized process field in help content", logPair("key", key));
+         }
+         else if(helpContent != null)
+         {
+            optionalField.get().withHelpContent(helpContent);
+         }
+         else
+         {
+            optionalField.get().removeHelpContent(roles);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static void processHelpContentForWidget(String key, String widgetName, String slotName, QHelpContent helpContent)
+   {
+      QWidgetMetaDataInterface widget = QContext.getQInstance().getWidget(widgetName);
+      if(!StringUtils.hasContent(slotName))
+      {
+         LOG.info("Missing slot name in help content", logPair("key", key));
+      }
+      else if(widget == null)
+      {
+         LOG.info("Unrecognized widget in help content", logPair("key", key));
+      }
+      else
+      {
+         Map<String, QHelpContent> widgetHelpContent = widget.getHelpContent();
+         if(widgetHelpContent == null)
+         {
+            widgetHelpContent = new HashMap<>();
+         }
+
+         if(helpContent != null)
+         {
+            widgetHelpContent.put(slotName, helpContent);
+         }
+         else
+         {
+            widgetHelpContent.remove(slotName);
+         }
+
+         widget.setHelpContent(widgetHelpContent);
+      }
+   }
+
+
+
    /*******************************************************************************
     ** add a help content object to a list - replacing an entry in the list with the
     ** same roles if one is found.

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/QInstanceValidator.java

@@ -24,8 +24,11 @@ package com.kingsrook.qqq.backend.core.instances;
 
 import java.io.Serializable;
 import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
 import java.lang.reflect.Modifier;
+import java.text.ParseException;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.Collections;
 import java.util.HashSet;
 import java.util.List;
@@ -33,15 +36,18 @@ import java.util.Map;
 import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
+import java.util.TimeZone;
 import java.util.function.Supplier;
 import java.util.stream.Stream;
 import com.kingsrook.qqq.backend.core.actions.automation.RecordAutomationHandler;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
 import com.kingsrook.qqq.backend.core.actions.metadata.JoinGraph;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.actions.scripts.TestScriptActionInterface;
 import com.kingsrook.qqq.backend.core.actions.values.QCustomPossibleValueProvider;
 import com.kingsrook.qqq.backend.core.exceptions.QInstanceValidationException;
+import com.kingsrook.qqq.backend.core.instances.validation.plugins.QInstanceValidatorPluginInterface;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
@@ -50,8 +56,12 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryJoin;
 import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.automation.QAutomationProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeType;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.ParentWidgetMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
@@ -61,21 +71,27 @@ import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppChildMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QSupplementalProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.queues.QQueueMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.queues.QQueueProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.queues.SQSQueueProviderMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportDataSource;
 import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportField;
+import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.reporting.QReportView;
 import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock;
+import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
 import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.AssociatedScript;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Association;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QSupplementalTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
@@ -84,9 +100,13 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.Automatio
 import com.kingsrook.qqq.backend.core.model.metadata.tables.automation.QTableAutomationDetails;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.cache.CacheOf;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.cache.CacheUseCase;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
+import com.kingsrook.qqq.backend.core.utils.ListingHash;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+import com.kingsrook.qqq.backend.core.utils.lambdas.UnsafeLambda;
+import org.quartz.CronExpression;
 
 
 /*******************************************************************************
@@ -105,6 +125,8 @@ public class QInstanceValidator
 
    private boolean printWarnings = false;
 
+   private static ListingHash<Class<?>, QInstanceValidatorPluginInterface<?>> validatorPlugins = new ListingHash<>();
+
    private List<String> errors = new ArrayList<>();
 
 
@@ -151,11 +173,13 @@ public class QInstanceValidator
       try
       {
          validateBackends(qInstance);
+         validateAuthentication(qInstance);
          validateAutomationProviders(qInstance);
          validateTables(qInstance, joinGraph);
          validateProcesses(qInstance);
          validateReports(qInstance);
          validateApps(qInstance);
+         validateWidgets(qInstance);
          validatePossibleValueSources(qInstance);
          validateQueuesAndProviders(qInstance);
          validateJoins(qInstance);
@@ -163,6 +187,8 @@ public class QInstanceValidator
          validateSupplementalMetaData(qInstance);
 
          validateUniqueTopLevelNames(qInstance);
+
+         runPlugins(QInstance.class, qInstance, qInstance);
       }
       catch(Exception e)
       {
@@ -181,6 +207,57 @@ public class QInstanceValidator
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void addValidatorPlugin(QInstanceValidatorPluginInterface<?> plugin)
+   {
+      Optional<Method> validateMethod = Arrays.stream(plugin.getClass().getDeclaredMethods())
+         .filter(m -> m.getName().equals("validate")
+            && m.getParameterCount() == 3
+            && !m.getParameterTypes()[0].equals(Object.class)
+            && m.getParameterTypes()[1].equals(QInstance.class)
+            && m.getParameterTypes()[2].equals(QInstanceValidator.class)
+         ).findFirst();
+
+      if(validateMethod.isPresent())
+      {
+         Class<?> parameterType = validateMethod.get().getParameterTypes()[0];
+         validatorPlugins.add(parameterType, plugin);
+      }
+      else
+      {
+         LOG.warn("Could not find validate method on validator plugin [" + plugin.getClass().getName() + "] (to infer type being validated) - this plugin will not be used.");
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void removeAllValidatorPlugins()
+   {
+      validatorPlugins.clear();
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private <T> void runPlugins(Class<T> c, T t, QInstance qInstance)
+   {
+      for(QInstanceValidatorPluginInterface<?> plugin : CollectionUtils.nonNullList(validatorPlugins.get(c)))
+      {
+         @SuppressWarnings("unchecked")
+         QInstanceValidatorPluginInterface<T> processPlugin = (QInstanceValidatorPluginInterface<T>) plugin;
+         processPlugin.validate(t, qInstance, this);
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -189,6 +266,8 @@ public class QInstanceValidator
       for(QSupplementalInstanceMetaData supplementalInstanceMetaData : CollectionUtils.nonNullMap(qInstance.getSupplementalMetaData()).values())
       {
          supplementalInstanceMetaData.validate(qInstance, this);
+
+         runPlugins(QSupplementalInstanceMetaData.class, supplementalInstanceMetaData, qInstance);
       }
    }
 
@@ -205,18 +284,29 @@ public class QInstanceValidator
          if(assertCondition(StringUtils.hasContent(securityKeyType.getName()), "Missing name for a securityKeyType"))
          {
             assertCondition(Objects.equals(name, securityKeyType.getName()), "Inconsistent naming for securityKeyType: " + name + "/" + securityKeyType.getName() + ".");
-            assertCondition(!usedNames.contains(name), "More than one SecurityKeyType with name (or allAccessKeyName) of: " + name);
+
+            String duplicateNameMessagePrefix = "More than one SecurityKeyType with name (or allAccessKeyName or nullValueBehaviorKeyName) of: ";
+            assertCondition(!usedNames.contains(name), duplicateNameMessagePrefix + name);
             usedNames.add(name);
+
             if(StringUtils.hasContent(securityKeyType.getAllAccessKeyName()))
             {
-               assertCondition(!usedNames.contains(securityKeyType.getAllAccessKeyName()), "More than one SecurityKeyType with name (or allAccessKeyName) of: " + securityKeyType.getAllAccessKeyName());
+               assertCondition(!usedNames.contains(securityKeyType.getAllAccessKeyName()), duplicateNameMessagePrefix + securityKeyType.getAllAccessKeyName());
                usedNames.add(securityKeyType.getAllAccessKeyName());
             }
 
+            if(StringUtils.hasContent(securityKeyType.getNullValueBehaviorKeyName()))
+            {
+               assertCondition(!usedNames.contains(securityKeyType.getNullValueBehaviorKeyName()), duplicateNameMessagePrefix + securityKeyType.getNullValueBehaviorKeyName());
+               usedNames.add(securityKeyType.getNullValueBehaviorKeyName());
+            }
+
             if(StringUtils.hasContent(securityKeyType.getPossibleValueSourceName()))
             {
                assertCondition(qInstance.getPossibleValueSource(securityKeyType.getPossibleValueSourceName()) != null, "Unrecognized possibleValueSourceName in securityKeyType: " + name);
             }
+
+            runPlugins(QSecurityKeyType.class, securityKeyType, qInstance);
          }
       });
    }
@@ -263,6 +353,8 @@ public class QInstanceValidator
                assertNoException(() -> qInstance.getTable(join.getRightTable()).getField(orderBy.getFieldName()), "Field name " + orderBy.getFieldName() + " in orderBy for join " + joinName + " is not a defined field in the right-table " + join.getRightTable());
             }
          }
+
+         runPlugins(QJoinMetaData.class, join, qInstance);
       });
    }
 
@@ -326,6 +418,8 @@ public class QInstanceValidator
                assertCondition(StringUtils.hasContent(sqsQueueProvider.getBaseURL()), "Missing baseURL for SQSQueueProvider: " + name);
                assertCondition(StringUtils.hasContent(sqsQueueProvider.getRegion()), "Missing region for SQSQueueProvider: " + name);
             }
+
+            runPlugins(QQueueProviderMetaData.class, queueProvider, qInstance);
          });
       }
 
@@ -340,6 +434,13 @@ public class QInstanceValidator
             {
                assertCondition(qInstance.getProcesses() != null && qInstance.getProcess(queue.getProcessName()) != null, "Unrecognized processName for queue: " + name);
             }
+
+            if(queue.getSchedule() != null)
+            {
+               validateScheduleMetaData(queue.getSchedule(), qInstance, "SQSQueueProvider " + name + ", schedule: ");
+            }
+
+            runPlugins(QQueueMetaData.class, queue, qInstance);
          });
       }
    }
@@ -358,6 +459,8 @@ public class QInstanceValidator
             assertCondition(Objects.equals(backendName, backend.getName()), "Inconsistent naming for backend: " + backendName + "/" + backend.getName() + ".");
 
             backend.performValidation(this);
+
+            runPlugins(QBackendMetaData.class, backend, qInstance);
          });
       }
    }
@@ -375,12 +478,33 @@ public class QInstanceValidator
          {
             assertCondition(Objects.equals(name, automationProvider.getName()), "Inconsistent naming for automationProvider: " + name + "/" + automationProvider.getName() + ".");
             assertCondition(automationProvider.getType() != null, "Missing type for automationProvider: " + name);
+
+            runPlugins(QAutomationProviderMetaData.class, automationProvider, qInstance);
          });
       }
    }
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void validateAuthentication(QInstance qInstance)
+   {
+      QAuthenticationMetaData authentication = qInstance.getAuthentication();
+      if(authentication != null)
+      {
+         if(authentication.getCustomizer() != null)
+         {
+            validateSimpleCodeReference("Instance Authentication meta data customizer ", authentication.getCustomizer(), QAuthenticationModuleCustomizerInterface.class);
+         }
+
+         runPlugins(QAuthenticationMetaData.class, authentication, qInstance);
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -493,6 +617,13 @@ public class QInstanceValidator
             validateTableRecordSecurityLocks(qInstance, table);
             validateTableAssociations(qInstance, table);
             validateExposedJoins(qInstance, joinGraph, table);
+
+            for(QSupplementalTableMetaData supplementalTableMetaData : CollectionUtils.nonNullMap(table.getSupplementalMetaData()).values())
+            {
+               supplementalTableMetaData.validate(qInstance, table, this);
+            }
+
+            runPlugins(QTableMetaData.class, table, qInstance);
          });
       }
    }
@@ -890,6 +1021,11 @@ public class QInstanceValidator
          assertCondition(qInstance.getAutomationProvider(providerName) != null, " has an unrecognized providerName: " + providerName);
       }
 
+      if(automationDetails.getSchedule() != null)
+      {
+         validateScheduleMetaData(automationDetails.getSchedule(), qInstance, prefix + " automationDetails, schedule: ");
+      }
+
       //////////////////////////////////
       // validate the status tracking //
       //////////////////////////////////
@@ -1277,13 +1413,17 @@ public class QInstanceValidator
             if(process.getSchedule() != null)
             {
                QScheduleMetaData schedule = process.getSchedule();
-               assertCondition(schedule.getRepeatMillis() != null || schedule.getRepeatSeconds() != null, "Either repeat millis or repeat seconds must be set on schedule in process " + processName);
+               validateScheduleMetaData(schedule, qInstance, "Process " + processName + ", schedule: ");
+            }
 
-               if(schedule.getVariantBackend() != null)
-               {
-                  assertCondition(qInstance.getBackend(schedule.getVariantBackend()) != null, "A variant backend was not found for " + schedule.getVariantBackend());
-                  assertCondition(schedule.getVariantRunStrategy() != null, "A variant run strategy was not set for " + schedule.getVariantBackend() + " on schedule in process " + processName);
-               }
+            if(process.getVariantBackend() != null)
+            {
+               assertCondition(qInstance.getBackend(process.getVariantBackend()) != null, "Process " + processName + ", a variant backend was not found named " + process.getVariantBackend());
+               assertCondition(process.getVariantRunStrategy() != null, "A variant run strategy was not set for process " + processName + " (which does specify a variant backend)");
+            }
+            else
+            {
+               assertCondition(process.getVariantRunStrategy() == null, "A variant run strategy was set for process " + processName + " (which isn't allowed, since it does not specify a variant backend)");
             }
 
             for(QSupplementalProcessMetaData supplementalProcessMetaData : CollectionUtils.nonNullMap(process.getSupplementalMetaData()).values())
@@ -1291,12 +1431,57 @@ public class QInstanceValidator
                supplementalProcessMetaData.validate(qInstance, process, this);
             }
 
+            runPlugins(QProcessMetaData.class, process, qInstance);
          });
       }
    }
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void validateScheduleMetaData(QScheduleMetaData schedule, QInstance qInstance, String prefix)
+   {
+      boolean isRepeat = schedule.getRepeatMillis() != null || schedule.getRepeatSeconds() != null;
+      boolean isCron = StringUtils.hasContent(schedule.getCronExpression());
+      assertCondition(isRepeat || isCron, prefix + " either repeatMillis or repeatSeconds or cronExpression must be set");
+      assertCondition(!(isRepeat && isCron), prefix + " both a repeat time and cronExpression may not be set");
+
+      if(isCron)
+      {
+         boolean hasDelay = schedule.getInitialDelayMillis() != null || schedule.getInitialDelaySeconds() != null;
+         assertCondition(!hasDelay, prefix + " a cron schedule may not have an initial delay");
+
+         try
+         {
+            CronExpression.validateExpression(schedule.getCronExpression());
+         }
+         catch(ParseException pe)
+         {
+            errors.add(prefix + " invalid cron expression: " + pe.getMessage());
+         }
+
+         if(assertCondition(StringUtils.hasContent(schedule.getCronTimeZoneId()), prefix + " a cron schedule must specify a cronTimeZoneId"))
+         {
+            String[] availableIDs = TimeZone.getAvailableIDs();
+            Optional<String> first = Arrays.stream(availableIDs).filter(id -> id.equals(schedule.getCronTimeZoneId())).findFirst();
+            assertCondition(first.isPresent(), prefix + " unrecognized cronTimeZoneId: " + schedule.getCronTimeZoneId());
+         }
+      }
+      else
+      {
+         assertCondition(!StringUtils.hasContent(schedule.getCronTimeZoneId()), prefix + " a non-cron schedule must not specify a cronTimeZoneId");
+      }
+
+      if(assertCondition(StringUtils.hasContent(schedule.getSchedulerName()), prefix + " is missing a scheduler name"))
+      {
+         assertCondition(qInstance.getScheduler(schedule.getSchedulerName()) != null, prefix + " is referencing an unknown scheduler name: " + schedule.getSchedulerName());
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -1397,6 +1582,8 @@ public class QInstanceValidator
                   // view.getTitleFormat(); view.getTitleFields(); // validate these match?
                }
             }
+
+            runPlugins(QReportMetaData.class, report, qInstance);
          });
       }
    }
@@ -1517,9 +1704,9 @@ public class QInstanceValidator
                }
             }
 
-            //////////////////////////////////////////
-            // validate field sections in the table //
-            //////////////////////////////////////////
+            ////////////////////////////////////////
+            // validate field sections in the app //
+            ////////////////////////////////////////
             Set<String> childNamesInSections = new HashSet<>();
             if(app.getSections() != null)
             {
@@ -1539,12 +1726,57 @@ public class QInstanceValidator
                   }
                }
             }
+
+            //////////////////////
+            // validate widgets //
+            //////////////////////
+            for(String widgetName : CollectionUtils.nonNullList(app.getWidgets()))
+            {
+               assertCondition(qInstance.getWidget(widgetName) != null, "App " + appName + " widget " + widgetName + " is not a recognized widget.");
+            }
+
+            runPlugins(QAppMetaData.class, app, qInstance);
          });
       }
    }
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private void validateWidgets(QInstance qInstance)
+   {
+      if(CollectionUtils.nullSafeHasContents(qInstance.getWidgets()))
+      {
+         qInstance.getWidgets().forEach((widgetName, widget) ->
+            {
+               assertCondition(Objects.equals(widgetName, widget.getName()), "Inconsistent naming for widget: " + widgetName + "/" + widget.getName() + ".");
+
+               if(assertCondition(widget.getCodeReference() != null, "Missing codeReference for widget: " + widgetName))
+               {
+                  validateSimpleCodeReference("Widget " + widgetName + " code reference: ", widget.getCodeReference(), AbstractWidgetRenderer.class);
+               }
+
+               if(widget instanceof ParentWidgetMetaData parentWidgetMetaData)
+               {
+                  if(assertCondition(CollectionUtils.nullSafeHasContents(parentWidgetMetaData.getChildWidgetNameList()), "Missing child widgets for parent widget: " + widget.getName()))
+                  {
+                     for(String childWidgetName : parentWidgetMetaData.getChildWidgetNameList())
+                     {
+                        assertCondition(qInstance.getWidget(childWidgetName) != null, "Unrecognized child widget name [" + childWidgetName + "] in parent widget: " + widget.getName());
+                     }
+                  }
+               }
+
+               runPlugins(QWidgetMetaDataInterface.class, widget, qInstance);
+            }
+         );
+      }
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -1623,6 +1855,8 @@ public class QInstanceValidator
                   }
                   default -> errors.add("Unexpected possibleValueSource type: " + possibleValueSource.getType());
                }
+
+               runPlugins(QPossibleValueSource.class, possibleValueSource, qInstance);
             }
          });
       }
@@ -1784,20 +2018,6 @@ public class QInstanceValidator
 
 
 
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @FunctionalInterface
-   interface UnsafeLambda
-   {
-      /*******************************************************************************
-       **
-       *******************************************************************************/
-      void run() throws Exception;
-   }
-
-
-
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/validation/plugins/BasepullExtractStepValidator.java

@@ -0,0 +1,87 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.validation.plugins;
+
+
+import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.basepull.BasepullExtractStepInterface;
+
+
+/*******************************************************************************
+ ** instance validator plugin, to ensure that a process which is a basepull uses
+ ** an extract step marked for basepulls.
+ *******************************************************************************/
+public class BasepullExtractStepValidator implements QInstanceValidatorPluginInterface<QProcessMetaData>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void validate(QProcessMetaData process, QInstance qInstance, QInstanceValidator qInstanceValidator)
+   {
+      ///////////////////////////////////////////////////////////////////////////
+      // if there's no basepull config on the process, don't do any validation //
+      ///////////////////////////////////////////////////////////////////////////
+      if(process.getBasepullConfiguration() == null)
+      {
+         return;
+      }
+
+      //////////////////////////////////////////////////////////////////////////////////////////
+      // try to find an input field in the process, w/ a defaultValue that's a QCodeReference //
+      // and is an instance of BasepullExtractStepInterface                                   //
+      //////////////////////////////////////////////////////////////////////////////////////////
+      boolean foundBasepullExtractStep = false;
+      for(QFieldMetaData field : process.getInputFields())
+      {
+         if(field.getDefaultValue() != null && field.getDefaultValue() instanceof QCodeReference codeReference)
+         {
+            try
+            {
+               BasepullExtractStepInterface extractStep = QCodeLoader.getAdHoc(BasepullExtractStepInterface.class, codeReference);
+               if(extractStep != null)
+               {
+                  foundBasepullExtractStep = true;
+               }
+            }
+            catch(Exception e)
+            {
+               //////////////////////////////////////////////////////
+               // ok, just means we haven't found our extract step //
+               //////////////////////////////////////////////////////
+            }
+         }
+      }
+
+      ///////////////////////////////////////////////////////////
+      // validate we could find a BasepullExtractStepInterface //
+      ///////////////////////////////////////////////////////////
+      qInstanceValidator.assertCondition(foundBasepullExtractStep, "Process [" + process.getName() + "] has a basepullConfiguration, but does not have a field with a default value that is a BasepullExtractStepInterface CodeReference");
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/instances/validation/plugins/QInstanceValidatorPluginInterface.java

@@ -0,0 +1,41 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.instances.validation.plugins;
+
+
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+
+
+/*******************************************************************************
+ ** Interface for additional / optional q instance validators.  Some will be
+ ** provided by QQQ - others can be defined by applications.
+ *******************************************************************************/
+public interface QInstanceValidatorPluginInterface<T>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   void validate(T object, QInstance qInstance, QInstanceValidator qInstanceValidator);
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/logging/CollectedLogMessage.java

@@ -0,0 +1,164 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.logging;
+
+
+import org.apache.logging.log4j.Level;
+import org.json.JSONException;
+import org.json.JSONObject;
+
+
+/*******************************************************************************
+ ** A log message, which can be "collected" by the QCollectingLogger.
+ *******************************************************************************/
+public class CollectedLogMessage
+{
+   private Level     level;
+   private String    message;
+   private Throwable exception;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public CollectedLogMessage()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String toString()
+   {
+      return "CollectedLogMessage{level=" + level + ", message='" + message + '\'' + ", exception=" + exception + '}';
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for message
+    *******************************************************************************/
+   public String getMessage()
+   {
+      return (this.message);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for message
+    *******************************************************************************/
+   public void setMessage(String message)
+   {
+      this.message = message;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for message
+    *******************************************************************************/
+   public CollectedLogMessage withMessage(String message)
+   {
+      this.message = message;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for exception
+    *******************************************************************************/
+   public Throwable getException()
+   {
+      return (this.exception);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for exception
+    *******************************************************************************/
+   public void setException(Throwable exception)
+   {
+      this.exception = exception;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for exception
+    *******************************************************************************/
+   public CollectedLogMessage withException(Throwable exception)
+   {
+      this.exception = exception;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for level
+    **
+    *******************************************************************************/
+   public Level getLevel()
+   {
+      return level;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for level
+    **
+    *******************************************************************************/
+   public void setLevel(Level level)
+   {
+      this.level = level;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for level
+    **
+    *******************************************************************************/
+   public CollectedLogMessage withLevel(Level level)
+   {
+      this.level = level;
+      return (this);
+   }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public JSONObject getMessageAsJSONObject() throws JSONException
+   {
+      return (new JSONObject(getMessage()));
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/logging/QCollectingLogger.java

@@ -0,0 +1,155 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.logging;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Properties;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.Marker;
+import org.apache.logging.log4j.message.Message;
+import org.apache.logging.log4j.message.ObjectMessage;
+import org.apache.logging.log4j.message.SimpleMessageFactory;
+import org.apache.logging.log4j.simple.SimpleLogger;
+import org.apache.logging.log4j.util.PropertiesUtil;
+
+
+/*******************************************************************************
+ ** QQQ log4j implementation, used within a QLogger, to "collect" log messages
+ ** in an internal list - the idea being - for tests, to assert that logs happened.
+ *******************************************************************************/
+public class QCollectingLogger extends SimpleLogger
+{
+   private List<CollectedLogMessage> collectedMessages = new ArrayList<>();
+
+   ///////////////////////////////////////////////////////////////////////////////////////////////////////////
+   // just in case one of these gets activated, and left on, put a limit on how many messages we'll collect //
+   ///////////////////////////////////////////////////////////////////////////////////////////////////////////
+   private int capacity = 100;
+
+   private Logger logger;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public QCollectingLogger(Logger logger)
+   {
+      super(logger.getName(), logger.getLevel(), false, false, true, false, "", new SimpleMessageFactory(), new PropertiesUtil(new Properties()), System.out);
+      this.logger = logger;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public void logMessage(String fqcn, Level level, Marker marker, Message message, Throwable throwable)
+   {
+      ////////////////////////////////////////////
+      // add this log message to our collection //
+      ////////////////////////////////////////////
+      collectedMessages.add(new CollectedLogMessage()
+         .withLevel(level)
+         .withMessage(message.getFormattedMessage())
+         .withException(throwable));
+
+      ////////////////////////////////////////////////////////////////////////////////////////
+      // if we've gone over our capacity, remove the 1st entry until we're back at capacity //
+      ////////////////////////////////////////////////////////////////////////////////////////
+      while(collectedMessages.size() > capacity)
+      {
+         collectedMessages.remove(0);
+      }
+
+      //////////////////////////////////////////////////////////////////////
+      // update the message that we log to indicate that we collected it. //
+      // if it looks like JSON, insert as a name:value pair; else text.   //
+      //////////////////////////////////////////////////////////////////////
+      String formattedMessage = message.getFormattedMessage();
+      String updatedMessage;
+      if(formattedMessage.startsWith("{"))
+      {
+         updatedMessage = """
+            {"collected":true,""" + formattedMessage.substring(1);
+      }
+      else
+      {
+         updatedMessage = "[Collected] " + formattedMessage;
+      }
+      ObjectMessage myMessage = new ObjectMessage(updatedMessage);
+
+      ///////////////////////////////////////////////////////////////////////////////////////
+      // log the message with the original log4j logger, with our slightly updated message //
+      ///////////////////////////////////////////////////////////////////////////////////////
+      logger.logMessage(level, marker, fqcn, null, myMessage, throwable);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for logger
+    **
+    *******************************************************************************/
+   public void setLogger(Logger logger)
+   {
+      this.logger = logger;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for collectedMessages
+    **
+    *******************************************************************************/
+   public List<CollectedLogMessage> getCollectedMessages()
+   {
+      return collectedMessages;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void clear()
+   {
+      this.collectedMessages.clear();
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for capacity
+    **
+    *******************************************************************************/
+   public void setCapacity(int capacity)
+   {
+      this.capacity = capacity;
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/logging/QLogger.java

@@ -119,6 +119,34 @@ public class QLogger
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static QCollectingLogger activateCollectingLoggerForClass(Class<?> c)
+   {
+      Logger            loggerFromLogManager = LogManager.getLogger(c);
+      QCollectingLogger collectingLogger     = new QCollectingLogger(loggerFromLogManager);
+
+      QLogger qLogger = getLogger(c);
+      qLogger.setLogger(collectingLogger);
+
+      return collectingLogger;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static void deactivateCollectingLoggerForClass(Class<?> c)
+   {
+      Logger  loggerFromLogManager = LogManager.getLogger(c);
+      QLogger qLogger              = getLogger(c);
+      qLogger.setLogger(loggerFromLogManager);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -518,7 +546,7 @@ public class QLogger
    /*******************************************************************************
     **
     *******************************************************************************/
-   private String makeJsonString(String message, Throwable t, List<LogPair> logPairList)
+   protected String makeJsonString(String message, Throwable t, List<LogPair> logPairList)
    {
       if(logPairList == null)
       {
@@ -620,4 +648,15 @@ public class QLogger
       exceptionList.get(0).setHasLoggedLevel(level);
       return (level);
    }
+
+
+
+   /*******************************************************************************
+    ** Setter for logger
+    **
+    *******************************************************************************/
+   private void setLogger(Logger logger)
+   {
+      this.logger = logger;
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/audits/AuditDetailAccumulator.java

@@ -0,0 +1,156 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.actions.audits;
+
+
+import java.io.Serializable;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
+
+
+/*******************************************************************************
+ ** Object to accumulate multiple audit-details to be recorded under a single
+ ** audit per-record, within a process step.  Especially useful if/when the
+ ** process step spreads its work out through multiple classes.
+ **
+ ** Pattern of usage looks like:
+ **
+ ** <pre>
+ ** // declare as a field (or local) w/ message for the audit headers
+ ** private AuditDetailAccumulator auditDetailAccumulator = new AuditDetailAccumulator("Audit header message");
+ **
+ **  // put into thread context
+ **  AuditDetailAccumulator.setInContext(auditDetailAccumulator);
+ **
+ **  // add a detail message for a record
+ **  auditDetailAccumulator.addAuditDetail(tableName, record, "Detail message");
+ **
+ **  // in another class, get the accumulator from context and safely add a detail message
+ **  AuditDetailAccumulator.getFromContext().ifPresent(ada -> ada.addAuditDetail(tableName, record, "More Details"));
+ **
+ **  // at the end of a step run/runOnePage method, add the accumulated audit details to step output
+ **  auditDetailAccumulator.getAccumulatedAuditSingleInputs().forEach(runBackendStepOutput::addAuditSingleInput);
+ **  auditDetailAccumulator.clear();
+ ** </pre>
+ *******************************************************************************/
+public class AuditDetailAccumulator implements Serializable
+{
+   private static final QLogger LOG = QLogger.getLogger(AuditDetailAccumulator.class);
+
+   private static final String objectKey = AuditDetailAccumulator.class.getSimpleName();
+
+   private String header;
+
+   private Map<TableNameAndPrimaryKey, AuditSingleInput> recordAuditInputMap = new HashMap<>();
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public AuditDetailAccumulator(String header)
+   {
+      this.header = header;
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void setInContext()
+   {
+      QContext.setObject(objectKey, this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public static Optional<AuditDetailAccumulator> getFromContext()
+   {
+      return QContext.getObject(objectKey, AuditDetailAccumulator.class);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addAuditDetail(String tableName, QRecordEntity entity, String message)
+   {
+      if(entity != null)
+      {
+         addAuditDetail(tableName, entity.toQRecord(), message);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addAuditDetail(String tableName, QRecord record, String message)
+   {
+      QTableMetaData table      = QContext.getQInstance().getTable(tableName);
+      Serializable   primaryKey = record.getValue(table.getPrimaryKeyField());
+      if(primaryKey == null)
+      {
+         LOG.info("Missing primary key in input record - audit detail message will not be recorded.", logPair("message", message));
+         return;
+      }
+
+      AuditSingleInput auditSingleInput = recordAuditInputMap.computeIfAbsent(new TableNameAndPrimaryKey(tableName, primaryKey), (key) -> new AuditSingleInput(table, record, header));
+      auditSingleInput.addDetail(message);
+   }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public Collection<AuditSingleInput> getAccumulatedAuditSingleInputs()
+   {
+      return (recordAuditInputMap.values());
+   }
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void clear()
+   {
+      recordAuditInputMap.clear();
+   }
+
+
+   private record TableNameAndPrimaryKey(String tableName, Serializable primaryKey) {}
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/audits/AuditSingleInput.java

@@ -41,7 +41,7 @@ import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
 /*******************************************************************************
  ** Input data to insert a single audit record (with optional child record)..
  *******************************************************************************/
-public class AuditSingleInput
+public class AuditSingleInput implements Serializable
 {
    private String  auditTableName;
    private String  auditUserName;

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/QueryOrGetInputInterface.java

@@ -60,6 +60,11 @@ public interface QueryOrGetInputInterface
    QBackendTransaction getTransaction();
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   String getTableName();
+
    /*******************************************************************************
     ** Setter for transaction
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/aggregate/GroupBy.java

@@ -24,6 +24,7 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.aggregate;
 
 import java.io.Serializable;
 import java.util.Objects;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 
 
@@ -38,6 +39,17 @@ public class GroupBy implements Serializable
 
 
 
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public GroupBy(QFieldMetaData field)
+   {
+      this.type = field.getType();
+      this.fieldName = field.getName();
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QFilterCriteria.java

@@ -27,6 +27,7 @@ import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.List;
+import java.util.Objects;
 import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.serialization.QFilterCriteriaDeserializer;
@@ -346,4 +347,37 @@ public class QFilterCriteria implements Serializable, Cloneable
       return (rs.toString());
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public boolean equals(Object o)
+   {
+      if(this == o)
+      {
+         return true;
+      }
+
+      if(o == null || getClass() != o.getClass())
+      {
+         return false;
+      }
+
+      QFilterCriteria that = (QFilterCriteria) o;
+      return Objects.equals(fieldName, that.fieldName) && operator == that.operator && Objects.equals(values, that.values) && Objects.equals(otherFieldName, that.otherFieldName);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public int hashCode()
+   {
+      return Objects.hash(fieldName, operator, values, otherFieldName);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QQueryFilter.java

@@ -27,6 +27,7 @@ import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Map;
+import java.util.Objects;
 import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
@@ -323,6 +324,18 @@ public class QQueryFilter implements Serializable, Cloneable
 
 
 
+   /*******************************************************************************
+    ** Fluent setter for adding a single subFilter
+    **
+    *******************************************************************************/
+   public QQueryFilter withSubFilter(QQueryFilter subFilter)
+   {
+      addSubFilter(subFilter);
+      return (this);
+   }
+
+
+
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -359,7 +372,7 @@ public class QQueryFilter implements Serializable, Cloneable
          rs.append(")");
 
          rs.append("OrderBy[");
-         for(QFilterOrderBy orderBy : orderBys)
+         for(QFilterOrderBy orderBy : CollectionUtils.nonNullList(orderBys))
          {
             rs.append(orderBy).append(",");
          }
@@ -467,4 +480,36 @@ public class QQueryFilter implements Serializable, Cloneable
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public boolean equals(Object o)
+   {
+      if(this == o)
+      {
+         return true;
+      }
+
+      if(o == null || getClass() != o.getClass())
+      {
+         return false;
+      }
+
+      QQueryFilter that = (QQueryFilter) o;
+      return Objects.equals(criteria, that.criteria) && Objects.equals(orderBys, that.orderBys) && booleanOperator == that.booleanOperator && Objects.equals(subFilters, that.subFilters) && Objects.equals(skip, that.skip) && Objects.equals(limit, that.limit);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public int hashCode()
+   {
+      return Objects.hash(criteria, orderBys, booleanOperator, subFilters, skip, limit);
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/query/QueryOutput.java

@@ -23,10 +23,12 @@ package com.kingsrook.qqq.backend.core.model.actions.tables.query;
 
 
 import java.io.Serializable;
+import java.util.ArrayList;
 import java.util.List;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.AbstractActionOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 
 
 /*******************************************************************************
@@ -89,4 +91,18 @@ public class QueryOutput extends AbstractActionOutput implements Serializable
       return storage.getRecords();
    }
 
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public <T extends QRecordEntity> List<T> getRecordEntities(Class<T> entityClass) throws QException
+   {
+      List<T> rs = new ArrayList<>();
+      for(QRecord record : storage.getRecords())
+      {
+         rs.add(QRecordEntity.fromQRecord(entityClass, record));
+      }
+      return (rs);
+   }
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/replace/ReplaceInput.java

@@ -39,6 +39,7 @@ public class ReplaceInput extends AbstractTableActionInput
    private UniqueKey           key;
    private List<QRecord>       records;
    private QQueryFilter        filter;
+   private boolean             performDeletes = true;
 
    private boolean omitDmlAudit = false;
 
@@ -207,4 +208,35 @@ public class ReplaceInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for performDeletes
+    *******************************************************************************/
+   public boolean getPerformDeletes()
+   {
+      return (this.performDeletes);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for performDeletes
+    *******************************************************************************/
+   public void setPerformDeletes(boolean performDeletes)
+   {
+      this.performDeletes = performDeletes;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for performDeletes
+    *******************************************************************************/
+   public ReplaceInput withPerformDeletes(boolean performDeletes)
+   {
+      this.performDeletes = performDeletes;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/tables/update/UpdateInput.java

@@ -51,8 +51,10 @@ public class UpdateInput extends AbstractTableActionInput
    ////////////////////////////////////////////////////////////////////////////////////////////
    private Boolean areAllValuesBeingUpdatedTheSame = null;
 
-   private boolean omitDmlAudit = false;
-   private String  auditContext = null;
+   private boolean omitTriggeringAutomations = false;
+   private boolean omitDmlAudit              = false;
+   private boolean omitModifyDateUpdate      = false;
+   private String  auditContext              = null;
 
 
 
@@ -321,4 +323,66 @@ public class UpdateInput extends AbstractTableActionInput
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for omitTriggeringAutomations
+    *******************************************************************************/
+   public boolean getOmitTriggeringAutomations()
+   {
+      return (this.omitTriggeringAutomations);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for omitTriggeringAutomations
+    *******************************************************************************/
+   public void setOmitTriggeringAutomations(boolean omitTriggeringAutomations)
+   {
+      this.omitTriggeringAutomations = omitTriggeringAutomations;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for omitTriggeringAutomations
+    *******************************************************************************/
+   public UpdateInput withOmitTriggeringAutomations(boolean omitTriggeringAutomations)
+   {
+      this.omitTriggeringAutomations = omitTriggeringAutomations;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for omitModifyDateUpdate
+    *******************************************************************************/
+   public boolean getOmitModifyDateUpdate()
+   {
+      return (this.omitModifyDateUpdate);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for omitModifyDateUpdate
+    *******************************************************************************/
+   public void setOmitModifyDateUpdate(boolean omitModifyDateUpdate)
+   {
+      this.omitModifyDateUpdate = omitModifyDateUpdate;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for omitModifyDateUpdate
+    *******************************************************************************/
+   public UpdateInput withOmitModifyDateUpdate(boolean omitModifyDateUpdate)
+   {
+      this.omitModifyDateUpdate = omitModifyDateUpdate;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/automation/TableTrigger.java

@@ -28,7 +28,7 @@ import com.kingsrook.qqq.backend.core.model.data.QField;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.TablesPossibleValueSourceMetaDataProvider;
-import com.kingsrook.qqq.backend.core.model.savedfilters.SavedFilter;
+import com.kingsrook.qqq.backend.core.model.savedviews.SavedView;
 import com.kingsrook.qqq.backend.core.model.scripts.Script;
 
 
@@ -51,7 +51,7 @@ public class TableTrigger extends QRecordEntity
    @QField(possibleValueSourceName = TablesPossibleValueSourceMetaDataProvider.NAME)
    private String tableName;
 
-   @QField(possibleValueSourceName = SavedFilter.TABLE_NAME)
+   @QField(possibleValueSourceName = SavedView.TABLE_NAME)
    private Integer filterId;
 
    @QField(possibleValueSourceName = Script.TABLE_NAME)

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/common/TimeZonePossibleValueSourceMetaDataProvider.java

@@ -0,0 +1,78 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.common;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.TimeZone;
+import java.util.function.Function;
+import java.util.function.Predicate;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PVSValueFormatAndFields;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
+import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class TimeZonePossibleValueSourceMetaDataProvider
+{
+   public static final String NAME = "timeZones";
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QPossibleValueSource produce()
+   {
+      return (produce(null, null));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public QPossibleValueSource produce(Predicate<String> filter, Function<String, String> labelMapper)
+   {
+      QPossibleValueSource possibleValueSource = new QPossibleValueSource()
+         .withName("timeZones")
+         .withType(QPossibleValueSourceType.ENUM)
+         .withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY);
+
+      List<QPossibleValue<?>> enumValues = new ArrayList<>();
+      for(String availableID : TimeZone.getAvailableIDs())
+      {
+         if(filter == null || filter.test(availableID))
+         {
+            String label = labelMapper == null ? availableID : labelMapper.apply(availableID);
+            enumValues.add(new QPossibleValue<>(availableID, label));
+         }
+      }
+
+      possibleValueSource.withEnumValues(enumValues);
+      return (possibleValueSource);
+   }
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/CompositeWidgetData.java

@@ -0,0 +1,220 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.AbstractBlockWidgetData;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseSlots;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseStyles;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseValues;
+
+
+/*******************************************************************************
+ ** Data used to render a Composite Widget - e.g., a collection of blocks
+ *******************************************************************************/
+public class CompositeWidgetData extends AbstractBlockWidgetData<CompositeWidgetData, BaseValues, BaseSlots, BaseStyles>
+{
+   private List<AbstractBlockWidgetData<?, ?, ?, ?>> blocks = new ArrayList<>();
+
+   private Map<String, Serializable> styleOverrides = new HashMap<>();
+
+   private Layout layout;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public enum Layout
+   {
+      /////////////////////////////////////////////////////////////
+      // note, these are used in QQQ FMD CompositeWidgetData.tsx //
+      /////////////////////////////////////////////////////////////
+      FLEX_ROW_WRAPPED,
+      FLEX_ROW_SPACE_BETWEEN,
+      TABLE_SUB_ROW_DETAILS,
+      BADGES_WRAPPER
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getBlockTypeName()
+   {
+      return "COMPOSITE";
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for blocks
+    **
+    *******************************************************************************/
+   public List<AbstractBlockWidgetData<?, ?, ?, ?>> getBlocks()
+   {
+      return blocks;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for blocks
+    **
+    *******************************************************************************/
+   public void setBlocks(List<AbstractBlockWidgetData<?, ?, ?, ?>> blocks)
+   {
+      this.blocks = blocks;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for blocks
+    **
+    *******************************************************************************/
+   public CompositeWidgetData withBlocks(List<AbstractBlockWidgetData<?, ?, ?, ?>> blocks)
+   {
+      this.blocks = blocks;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public CompositeWidgetData withBlock(AbstractBlockWidgetData<?, ?, ?, ?> block)
+   {
+      addBlock(block);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addBlock(AbstractBlockWidgetData<?, ?, ?, ?> block)
+   {
+      if(this.blocks == null)
+      {
+         this.blocks = new ArrayList<>();
+      }
+      this.blocks.add(block);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for styleOverrides
+    *******************************************************************************/
+   public Map<String, Serializable> getStyleOverrides()
+   {
+      return (this.styleOverrides);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for styleOverrides
+    *******************************************************************************/
+   public void setStyleOverrides(Map<String, Serializable> styleOverrides)
+   {
+      this.styleOverrides = styleOverrides;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for styleOverrides
+    *******************************************************************************/
+   public CompositeWidgetData withStyleOverrides(Map<String, Serializable> styleOverrides)
+   {
+      this.styleOverrides = styleOverrides;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public CompositeWidgetData withStyleOverride(String key, Serializable value)
+   {
+      addStyleOverride(key, value);
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addStyleOverride(String key, Serializable value)
+   {
+      if(this.styleOverrides == null)
+      {
+         this.styleOverrides = new HashMap<>();
+      }
+      this.styleOverrides.put(key, value);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for layout
+    *******************************************************************************/
+   public Layout getLayout()
+   {
+      return (this.layout);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for layout
+    *******************************************************************************/
+   public void setLayout(Layout layout)
+   {
+      this.layout = layout;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for layout
+    *******************************************************************************/
+   public CompositeWidgetData withLayout(Layout layout)
+   {
+      this.layout = layout;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/ParentWidgetData.java

@@ -35,6 +35,8 @@ public class ParentWidgetData extends QWidgetData
    private List<String> childWidgetNameList;
    private ParentWidgetMetaData.LayoutType layoutType = ParentWidgetMetaData.LayoutType.GRID;
 
+   private boolean isLabelPageTitle = false;
+
 
 
    /*******************************************************************************
@@ -121,4 +123,34 @@ public class ParentWidgetData extends QWidgetData
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for isLabelPageTitle
+    *******************************************************************************/
+   public boolean getIsLabelPageTitle()
+   {
+      return (this.isLabelPageTitle);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for isLabelPageTitle
+    *******************************************************************************/
+   public void setIsLabelPageTitle(boolean isLabelPageTitle)
+   {
+      this.isLabelPageTitle = isLabelPageTitle;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for isLabelPageTitle
+    *******************************************************************************/
+   public ParentWidgetData withIsLabelPageTitle(boolean isLabelPageTitle)
+   {
+      this.isLabelPageTitle = isLabelPageTitle;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/QWidgetData.java

@@ -22,6 +22,7 @@
 package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
 
 
+import java.io.Serializable;
 import java.util.List;
 import java.util.Map;
 
@@ -33,6 +34,7 @@ import java.util.Map;
 public abstract class QWidgetData
 {
    private String       label;
+   private String       sublabel;
    private String       footerHTML;
    private List<String> dropdownNameList;
    private List<String> dropdownLabelList;
@@ -47,6 +49,8 @@ public abstract class QWidgetData
    private List<List<Map<String, String>>> dropdownDataList;
    private String                          dropdownNeedsSelectedText;
 
+   private List<List<Serializable>> csvData;
+
 
 
    /*******************************************************************************
@@ -324,4 +328,65 @@ public abstract class QWidgetData
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for csvData
+    *******************************************************************************/
+   public List<List<Serializable>> getCsvData()
+   {
+      return (this.csvData);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for csvData
+    *******************************************************************************/
+   public void setCsvData(List<List<Serializable>> csvData)
+   {
+      this.csvData = csvData;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for csvData
+    *******************************************************************************/
+   public QWidgetData withCsvData(List<List<Serializable>> csvData)
+   {
+      this.csvData = csvData;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for sublabel
+    *******************************************************************************/
+   public String getSublabel()
+   {
+      return (this.sublabel);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for sublabel
+    *******************************************************************************/
+   public void setSublabel(String sublabel)
+   {
+      this.sublabel = sublabel;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for sublabel
+    *******************************************************************************/
+   public QWidgetData withSublabel(String sublabel)
+   {
+      this.sublabel = sublabel;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/StatisticsData.java

@@ -22,19 +22,24 @@
 package com.kingsrook.qqq.backend.core.model.dashboard.widgets;
 
 
+import java.io.Serializable;
+
+
 /*******************************************************************************
  ** Model containing datastructure expected by frontend statistics widget
  **
  *******************************************************************************/
 public class StatisticsData extends QWidgetData
 {
-   private Number  count;
-   private String  countFontSize;
-   private String  countURL;
-   private Number  percentageAmount;
-   private String  percentageLabel;
-   private boolean isCurrency     = false;
-   private boolean increaseIsGood = true;
+   private Serializable count;
+   private String       countFontSize;
+   private String       countURL;
+   private String       countContext;
+   private Number       percentageAmount;
+   private String       percentageLabel;
+   private String       percentageURL;
+   private boolean      isCurrency     = false;
+   private boolean      increaseIsGood = true;
 
 
 
@@ -50,7 +55,7 @@ public class StatisticsData extends QWidgetData
    /*******************************************************************************
     **
     *******************************************************************************/
-   public StatisticsData(Number count, Number percentageAmount, String percentageLabel)
+   public StatisticsData(Serializable count, Number percentageAmount, String percentageLabel)
    {
       this.count = count;
       this.percentageLabel = percentageLabel;
@@ -142,7 +147,7 @@ public class StatisticsData extends QWidgetData
     ** Getter for count
     **
     *******************************************************************************/
-   public Number getCount()
+   public Serializable getCount()
    {
       return count;
    }
@@ -153,7 +158,7 @@ public class StatisticsData extends QWidgetData
     ** Setter for count
     **
     *******************************************************************************/
-   public void setCount(Number count)
+   public void setCount(Serializable count)
    {
       this.count = count;
    }
@@ -164,7 +169,7 @@ public class StatisticsData extends QWidgetData
     ** Fluent setter for count
     **
     *******************************************************************************/
-   public StatisticsData withCount(Number count)
+   public StatisticsData withCount(Serializable count)
    {
       this.count = count;
       return (this);
@@ -306,4 +311,66 @@ public class StatisticsData extends QWidgetData
       return (this);
    }
 
+
+
+   /*******************************************************************************
+    ** Getter for countContext
+    *******************************************************************************/
+   public String getCountContext()
+   {
+      return (this.countContext);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for countContext
+    *******************************************************************************/
+   public void setCountContext(String countContext)
+   {
+      this.countContext = countContext;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for countContext
+    *******************************************************************************/
+   public StatisticsData withCountContext(String countContext)
+   {
+      this.countContext = countContext;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for percentageURL
+    *******************************************************************************/
+   public String getPercentageURL()
+   {
+      return (this.percentageURL);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for percentageURL
+    *******************************************************************************/
+   public void setPercentageURL(String percentageURL)
+   {
+      this.percentageURL = percentageURL;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for percentageURL
+    *******************************************************************************/
+   public StatisticsData withPercentageURL(String percentageURL)
+   {
+      this.percentageURL = percentageURL;
+      return (this);
+   }
+
 }

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/WidgetType.java

@@ -48,6 +48,7 @@ public enum WidgetType
    STEPPER("stepper"),
    TABLE("table"),
    USA_MAP("usaMap"),
+   COMPOSITE("composite"),
    DATA_BAG_VIEWER("dataBagViewer"),
    SCRIPT_VIEWER("scriptViewer");
 

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/AbstractBlockWidgetData.java

@@ -0,0 +1,419 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+import java.util.HashMap;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.QWidgetData;
+
+
+/*******************************************************************************
+ ** Base class for the data returned in rendering a block of a specific type.
+ **
+ ** The type parameters define the structure of the block's data, and should
+ ** generally be defined along with a sub-class of this class, in a block-specific
+ ** sub-package.
+ *******************************************************************************/
+public abstract class AbstractBlockWidgetData<
+   T extends AbstractBlockWidgetData<T, V, S, SX>,
+   V extends BlockValuesInterface,
+   S extends BlockSlotsInterface,
+   SX extends BlockStylesInterface> extends QWidgetData
+{
+   private String blockId;
+
+   private BlockTooltip tooltip;
+   private BlockLink    link;
+
+   private Map<S, BlockTooltip> tooltipMap;
+   private Map<S, BlockLink>    linkMap;
+
+   private V  values;
+   private SX styles;
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public final String getType()
+   {
+      return "block";
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public abstract String getBlockTypeName();
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public T withTooltip(S key, String value)
+   {
+      addTooltip(key, value);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addTooltip(S key, String value)
+   {
+      if(this.tooltipMap == null)
+      {
+         this.tooltipMap = new HashMap<>();
+      }
+      this.tooltipMap.put(key, new BlockTooltip().withTitle(value));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public T withTooltip(S key, BlockTooltip value)
+   {
+      addTooltip(key, value);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addTooltip(S key, BlockTooltip value)
+   {
+      if(this.tooltipMap == null)
+      {
+         this.tooltipMap = new HashMap<>();
+      }
+      this.tooltipMap.put(key, value);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tooltipMap
+    *******************************************************************************/
+   public Map<S, BlockTooltip> getTooltipMap()
+   {
+      return (this.tooltipMap);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tooltipMap
+    *******************************************************************************/
+   public void setTooltipMap(Map<S, BlockTooltip> tooltipMap)
+   {
+      this.tooltipMap = tooltipMap;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tooltipMap
+    *******************************************************************************/
+   public T withTooltipMap(Map<S, BlockTooltip> tooltipMap)
+   {
+      this.tooltipMap = tooltipMap;
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for tooltip
+    **
+    *******************************************************************************/
+   public BlockTooltip getTooltip()
+   {
+      return tooltip;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for tooltip
+    **
+    *******************************************************************************/
+   public void setTooltip(BlockTooltip tooltip)
+   {
+      this.tooltip = tooltip;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tooltip
+    **
+    *******************************************************************************/
+   public T withTooltip(String tooltip)
+   {
+      this.tooltip = new BlockTooltip(tooltip);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for tooltip
+    **
+    *******************************************************************************/
+   public T withTooltip(BlockTooltip tooltip)
+   {
+      this.tooltip = tooltip;
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public T withLink(S key, String value)
+   {
+      addLink(key, value);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addLink(S key, String value)
+   {
+      if(this.linkMap == null)
+      {
+         this.linkMap = new HashMap<>();
+      }
+      this.linkMap.put(key, new BlockLink(value));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public T withLink(S key, BlockLink value)
+   {
+      addLink(key, value);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   public void addLink(S key, BlockLink value)
+   {
+      if(this.linkMap == null)
+      {
+         this.linkMap = new HashMap<>();
+      }
+      this.linkMap.put(key, value);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for linkMap
+    *******************************************************************************/
+   public Map<S, BlockLink> getLinkMap()
+   {
+      return (this.linkMap);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for linkMap
+    *******************************************************************************/
+   public void setLinkMap(Map<S, BlockLink> linkMap)
+   {
+      this.linkMap = linkMap;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for linkMap
+    *******************************************************************************/
+   public T withLinkMap(Map<S, BlockLink> linkMap)
+   {
+      this.linkMap = linkMap;
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for link
+    **
+    *******************************************************************************/
+   public BlockLink getLink()
+   {
+      return link;
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for link
+    **
+    *******************************************************************************/
+   public void setLink(BlockLink link)
+   {
+      this.link = link;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for link
+    **
+    *******************************************************************************/
+   public T withLink(String link)
+   {
+      this.link = new BlockLink(link);
+      return (T) (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for link
+    **
+    *******************************************************************************/
+   public T withLink(BlockLink link)
+   {
+      this.link = link;
+      return (T) this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for values
+    *******************************************************************************/
+   public V getValues()
+   {
+      return (this.values);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for values
+    *******************************************************************************/
+   public void setValues(V values)
+   {
+      this.values = values;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for values
+    *******************************************************************************/
+   public T withValues(V values)
+   {
+      this.values = values;
+      return (T) this;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for styles
+    *******************************************************************************/
+   public SX getStyles()
+   {
+      return (this.styles);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for styles
+    *******************************************************************************/
+   public void setStyles(SX styles)
+   {
+      this.styles = styles;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for styles
+    *******************************************************************************/
+   public T withStyles(SX styles)
+   {
+      this.styles = styles;
+      return (T) this;
+   }
+
+
+   /*******************************************************************************
+    ** Getter for blockId
+    *******************************************************************************/
+   public String getBlockId()
+   {
+      return (this.blockId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for blockId
+    *******************************************************************************/
+   public void setBlockId(String blockId)
+   {
+      this.blockId = blockId;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for blockId
+    *******************************************************************************/
+   public T withBlockId(String blockId)
+   {
+      this.blockId = blockId;
+      return (T) this;
+   }
+
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/BlockLink.java

@@ -0,0 +1,86 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+/*******************************************************************************
+ ** A link used within a (widget) block.
+ **
+ ** Right now, just a href - but target is an obvious next-thing to add here.
+ *******************************************************************************/
+public class BlockLink
+{
+   private String href;
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public BlockLink()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public BlockLink(String href)
+   {
+      this.href = href;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for href
+    *******************************************************************************/
+   public String getHref()
+   {
+      return (this.href);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for href
+    *******************************************************************************/
+   public void setHref(String href)
+   {
+      this.href = href;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for href
+    *******************************************************************************/
+   public BlockLink withHref(String href)
+   {
+      this.href = href;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/BlockSlotsInterface.java

@@ -0,0 +1,32 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+/*******************************************************************************
+ ** marker-interface for classes (enums, actually) used to define the "slots"
+ ** within a widget-block that can have links or tooltips applied to them.
+ *******************************************************************************/
+public interface BlockSlotsInterface
+{
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/BlockStylesInterface.java

@@ -0,0 +1,32 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+/*******************************************************************************
+ ** Marker interface for classes that define the "styles" that can be customized
+ ** within a particular widget-block type.
+ *******************************************************************************/
+public interface BlockStylesInterface
+{
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/BlockTooltip.java

@@ -0,0 +1,122 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+/*******************************************************************************
+ ** A tooltip used within a (widget) block.
+ **
+ *******************************************************************************/
+public class BlockTooltip
+{
+   private String    title;
+   private Placement placement = Placement.BOTTOM;
+
+
+
+   public enum Placement
+   {BOTTOM, LEFT, RIGHT, TOP}
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public BlockTooltip()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Constructor
+    **
+    *******************************************************************************/
+   public BlockTooltip(String title)
+   {
+      this.title = title;
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for title
+    *******************************************************************************/
+   public String getTitle()
+   {
+      return (this.title);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for title
+    *******************************************************************************/
+   public void setTitle(String title)
+   {
+      this.title = title;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for title
+    *******************************************************************************/
+   public BlockTooltip withTitle(String title)
+   {
+      this.title = title;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for placement
+    *******************************************************************************/
+   public Placement getPlacement()
+   {
+      return (this.placement);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for placement
+    *******************************************************************************/
+   public void setPlacement(Placement placement)
+   {
+      this.placement = placement;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for placement
+    *******************************************************************************/
+   public BlockTooltip withPlacement(Placement placement)
+   {
+      this.placement = placement;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/BlockValuesInterface.java

@@ -0,0 +1,32 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks;
+
+
+/*******************************************************************************
+ ** Marker interface for classes that define the values that can be returned for
+ ** a particular widget-block type.
+ *******************************************************************************/
+public interface BlockValuesInterface
+{
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/base/BaseSlots.java

@@ -0,0 +1,33 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockSlotsInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public enum BaseSlots implements BlockSlotsInterface
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/base/BaseStyles.java

@@ -0,0 +1,33 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockStylesInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class BaseStyles implements BlockStylesInterface
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/base/BaseValues.java

@@ -0,0 +1,33 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockValuesInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class BaseValues implements BlockValuesInterface
+{
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/bignumberblock/BigNumberBlockData.java

@@ -0,0 +1,43 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.bignumberblock;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.AbstractBlockWidgetData;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class BigNumberBlockData extends AbstractBlockWidgetData<BigNumberBlockData, BigNumberValues, BigNumberSlots, BigNumberStyles>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getBlockTypeName()
+   {
+      return "BIG_NUMBER";
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/bignumberblock/BigNumberSlots.java

@@ -0,0 +1,36 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.bignumberblock;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockSlotsInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public enum BigNumberSlots implements BlockSlotsInterface
+{
+   HEADING,
+   NUMBER,
+   CONTEXT
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/bignumberblock/BigNumberStyles.java

@@ -0,0 +1,98 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.bignumberblock;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockStylesInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class BigNumberStyles implements BlockStylesInterface
+{
+   private String numberColor;
+   private String width;
+
+
+
+   /*******************************************************************************
+    ** Getter for numberColor
+    *******************************************************************************/
+   public String getNumberColor()
+   {
+      return (this.numberColor);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for numberColor
+    *******************************************************************************/
+   public void setNumberColor(String numberColor)
+   {
+      this.numberColor = numberColor;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for numberColor
+    *******************************************************************************/
+   public BigNumberStyles withNumberColor(String numberColor)
+   {
+      this.numberColor = numberColor;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for width
+    *******************************************************************************/
+   public String getWidth()
+   {
+      return (this.width);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for width
+    *******************************************************************************/
+   public void setWidth(String width)
+   {
+      this.width = width;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for width
+    *******************************************************************************/
+   public BigNumberStyles withWidth(String width)
+   {
+      this.width = width;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/bignumberblock/BigNumberValues.java

@@ -0,0 +1,131 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.bignumberblock;
+
+
+import java.io.Serializable;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockValuesInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class BigNumberValues implements BlockValuesInterface
+{
+   private String       heading;
+   private Serializable number;
+   private String       context;
+
+
+
+   /*******************************************************************************
+    ** Getter for heading
+    *******************************************************************************/
+   public String getHeading()
+   {
+      return (this.heading);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for heading
+    *******************************************************************************/
+   public void setHeading(String heading)
+   {
+      this.heading = heading;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for heading
+    *******************************************************************************/
+   public BigNumberValues withHeading(String heading)
+   {
+      this.heading = heading;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for number
+    *******************************************************************************/
+   public Serializable getNumber()
+   {
+      return (this.number);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for number
+    *******************************************************************************/
+   public void setNumber(Serializable number)
+   {
+      this.number = number;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for number
+    *******************************************************************************/
+   public BigNumberValues withNumber(Serializable number)
+   {
+      this.number = number;
+      return (this);
+   }
+
+
+
+   /*******************************************************************************
+    ** Getter for context
+    *******************************************************************************/
+   public String getContext()
+   {
+      return (this.context);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for context
+    *******************************************************************************/
+   public void setContext(String context)
+   {
+      this.context = context;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for context
+    *******************************************************************************/
+   public BigNumberValues withContext(String context)
+   {
+      this.context = context;
+      return (this);
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/divider/DividerBlockData.java

@@ -0,0 +1,46 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.divider;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.AbstractBlockWidgetData;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseSlots;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseStyles;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.base.BaseValues;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class DividerBlockData extends AbstractBlockWidgetData<DividerBlockData, BaseValues, BaseSlots, BaseStyles>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getBlockTypeName()
+   {
+      return "DIVIDER";
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/numbericonbadge/NumberIconBadgeBlockData.java

@@ -0,0 +1,43 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.numbericonbadge;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.AbstractBlockWidgetData;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class NumberIconBadgeBlockData extends AbstractBlockWidgetData<NumberIconBadgeBlockData, NumberIconBadgeValues, NumberIconBadgeSlots, NumberIconBadgeStyles>
+{
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Override
+   public String getBlockTypeName()
+   {
+      return "NUMBER_ICON_BADGE";
+   }
+
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/numbericonbadge/NumberIconBadgeSlots.java

@@ -0,0 +1,35 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.numbericonbadge;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockSlotsInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public enum NumberIconBadgeSlots implements BlockSlotsInterface
+{
+   NUMBER,
+   ICON
+}

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/dashboard/widgets/blocks/numbericonbadge/NumberIconBadgeStyles.java

@@ -0,0 +1,66 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2024.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+package com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.numbericonbadge;
+
+
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.blocks.BlockStylesInterface;
+
+
+/*******************************************************************************
+ **
+ *******************************************************************************/
+public class NumberIconBadgeStyles implements BlockStylesInterface
+{
+   private String color;
+
+
+
+   /*******************************************************************************
+    ** Getter for color
+    *******************************************************************************/
+   public String getColor()
+   {
+      return (this.color);
+   }
+
+
+
+   /*******************************************************************************
+    ** Setter for color
+    *******************************************************************************/
+   public void setColor(String color)
+   {
+      this.color = color;
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for color
+    *******************************************************************************/
+   public NumberIconBadgeStyles withColor(String color)
+   {
+      this.color = color;
+      return (this);
+   }
+
+}