Merge branch 'dev' into integration

docs/Introduction.adoc

@@ -91,7 +91,7 @@ And then having a bug in the check permission logic on the _Light Bulb Inventory
 No!
 
 All of the (really important, even though application developers hate doing it) aspects of security - you don't need to write ANY code for dealing with that.
-Just tell QQQ what Authentication provider you want to use (e.g., https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
+Just tell QQQ what Authentication provider you want to use (e.g., OAuth2 or https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
 QQQ just does it.
 
 ''''

docs/index.adoc

@@ -31,11 +31,9 @@ include::metaData/PermissionRules.adoc[leveloffset=+1]
 
 == Services
 
+include::misc/Javalin.adoc[leveloffset=+1]
 include::misc/ScheduledJobs.adoc[leveloffset=+1]
 
-=== Web server (Javalin)
-#todo#
-
 === API server (OpenAPI)
 #todo#
 

docs/misc/Javalin.adoc

@@ -0,0 +1,109 @@
+== QQQ Middleware:  Javalin web server
+include::../variables.adoc[]
+
+QQQ provides a standard implementation of a middleware layer - that is - code that exists between the
+QQQ backend and user interface.  This implementation is a web server built using the https://javalin.io/[Javalin framework],
+packaged and deployed in the `qqq-middleware-javalin` maven module
+
+The de facto way to create a QQQ application server is to write a class which uses an instance of one of the
+subclasses of `QApplicationJavalinServer`.
+
+For example, if your application metadata is defined in a directory of yaml files, your server class could be implemented as:
+
+[source,java]
+.ConfigFileBasedQQQApplication usage example
+----
+public static void main(String[] args)
+{
+   try
+   {
+      String path = "src/main/resources/metadata";
+      ConfigFilesBasedQQQApplication application = new ConfigFilesBasedQQQApplication(path);
+      QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(application);
+      javalinServer.start();
+   }
+   catch(Exception e)
+   {
+      LOG.error("Failed to start javalin server.  See stack trace for details.", e);
+   }
+}
+----
+
+A similar class exists if your metadata is produced by a package of Java MetaDataProducer objects:  `MetaDataProducerBasedQQQApplication`.
+
+=== QApplicationJavalinServer
+This class provides the bridge between your QQQ Application (e.g., your metadata) and the QQQ Middleware layer
+served by a Javalin web server.  It has several properties to control behaviors:
+
+* `Integer port` - (default `8000`) - port to use for serving HTTP.
+* `boolean serveFrontendMaterialDashboard` - (default `true`) whether to serve the javascript frontend provided
+in the maven artifact `qqq-frontend-material-dashboard`.
+* `boolean serveLegacyUnversionedMiddlewareAPI` - (default `true`) whether to serve a version the original implementation
+of the QQQ middleware, which current version of `qqq-frontend-material-dashboard` are compatible with.
+* `List<AbstractMiddlewareVersion> middlewareVersionList` - (default contains `MiddlewareVersionV1`) - list of
+newer, formally versioned implementations of the QQQ middleware interface to be served.
+* `Consumer<Javalin> javalinConfigurationCustomizer` - (default `null`) - optional hook to customize the
+javalin service object before it is started.
+* `List<QJavalinRouteProviderInterface> additionalRouteProviders` - (default `null`) - list of fully custom
+implementations of `QJavalinRouteProviderInterface`, to add additional endpoints to the javalin server.
+** _Note, you may first want to consider using JavalinRouteProviderMetaData instead - see below._
+* `QJavalinMetaData javalinMetaData` - (default `null`) - optional alternative place to define `JavalinMetaData` (vs.
+defining it in the `QInstance`). _Note that if it is set in both places, the one in the QApplicationJavalinServer
+is used._
+
+=== JavalinMetaData
+Certain behaviors of a QQQ Javalin server are configured in a declarative manner by adding a `QJavalinMetaData`
+object to the `supplementalMetaData` in your `QInstance` (or, as mentioned above, by setting it directly on the
+`QApplicationJavalinServer`):
+
+* `List<JavalinRouteProviderMetaData> routeProviders` - (default `null`) optional list of custom route providers to
+add to the Javalin server.  See below for details.
+* `String uploadedFileArchiveTableName` - (default `null`) - reference to a QQQ Table in your application instance,
+needed to support the Bulk Load process, as well as any other processes which need to accept an uploaded file
+as input.
+* `boolean loggerDisabled` - (default `false`)
+* `Function<QJavalinAccessLogger.LogEntry, Boolean> logFilter` - (default `null`)
+* `boolean queryWithoutLimitAllowed` - (default `false`)
+* `Integer queryWithoutLimitDefault` - (default `1000`)
+* `Level queryWithoutLimitLogLevel` - (default `INFO`)
+
+==== JavalinRouteProviderMetaData
+This type of metadata allows you to add additional http route providers to your Javalin instance, e.g., for
+serving static files or for running custom code from your application (in the form of QQQ Processes) to respond
+to HTTP requests.
+
+* `String hostedPath` - (required)
+* `String fileSystemPath` - (required for a static router)
+* `String processName` - required for a dynamic, process-based router.  Must be a process name within the QQQ Instance.
+See below for additional details
+* `List<String> methods` - required list of HTTP methods (verbs) that are served by the route provider
+* `QCodeReference routeAuthenticator - Optional reference to a class that implements `RouteAuthenticatorInterface`,
+to provide security authentication to all requests handled by the route provider.
+** A default implementation is provided as `SimpleRouteAuthenticator`, which requires that a user session be present
+to access paths served by the route provider.
+
+===== Process-based route provider processes
+If you define a `JavalinRouteProviderMetaData` with a `processName` (e.g., to serve dynamic HTTP responses from your javalin
+server), the process that you implement will be called to respond to any HTTP requests received by the javalin
+server which match the `hostedPath` and `methods` that are specified in the metadata.
+
+The QQQ javalin server will marshal request data from the javalin context into the process's payload, conforming to
+the shape of the `ProcessBasedRouterPayload` class.  Similarly, the http response will be built by taking values from
+the process's output/state conforming to the fields in that class.  As such, it is recommended to use a
+`ProcessBasedRouterPayload`  instance, as show in this example:
+
+[source,java]
+.Process-based router usage example (including ProcessBasedRouterPayload)
+----
+public class MyDynamicSiteProcessStep implements BackendStep
+{
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+   {
+      ProcessBasedRouterPayload payload = input.getProcessPayload(ProcessBasedRouterPayload.class);
+      String path = payload.getPath();
+      payload.setResponseString("You requested: " + path);
+      output.setProcessPayload(payload);
+   }
+}
+----

pom.xml

@@ -48,7 +48,7 @@
    </modules>
 
    <properties>
-      <revision>0.25.0-SNAPSHOT</revision>
+      <revision>0.26.0-SNAPSHOT</revision>
 
       <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
       <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/QProcessPayload.java

@@ -78,51 +78,13 @@ public class QProcessPayload
       try
       {
          List<QRecordEntityField> fieldList = getFieldList(this.getClass());
-         // originalRecordValues = new HashMap<>();
 
          for(QRecordEntityField qRecordEntityField : fieldList)
          {
             Serializable value      = processState.getValues().get(qRecordEntityField.getFieldName());
             Object       typedValue = qRecordEntityField.convertValueType(value);
             qRecordEntityField.getSetter().invoke(this, typedValue);
-            // originalRecordValues.put(qRecordEntityField.getFieldName(), value);
          }
-
-         // for(QRecordEntityAssociation qRecordEntityAssociation : getAssociationList(this.getClass()))
-         // {
-         //    List<QRecord> associatedRecords = qRecord.getAssociatedRecords().get(qRecordEntityAssociation.getAssociationAnnotation().name());
-         //    if(associatedRecords == null)
-         //    {
-         //       qRecordEntityAssociation.getSetter().invoke(this, (Object) null);
-         //    }
-         //    else
-         //    {
-         //       List<QRecordEntity> associatedEntityList = new ArrayList<>();
-         //       for(QRecord associatedRecord : CollectionUtils.nonNullList(associatedRecords))
-         //       {
-         //          associatedEntityList.add(QRecordEntity.fromQRecord(qRecordEntityAssociation.getAssociatedType(), associatedRecord));
-         //       }
-         //       qRecordEntityAssociation.getSetter().invoke(this, associatedEntityList);
-         //    }
-         // }
-
-         // for(QRecordEntityAssociation qRecordEntityAssociation : getAssociationList(this.getClass()))
-         // {
-         //    List<QRecord> associatedRecords = qRecord.getAssociatedRecords().get(qRecordEntityAssociation.getAssociationAnnotation().name());
-         //    if(associatedRecords == null)
-         //    {
-         //       qRecordEntityAssociation.getSetter().invoke(this, (Object) null);
-         //    }
-         //    else
-         //    {
-         //       List<QRecordEntity> associatedEntityList = new ArrayList<>();
-         //       for(QRecord associatedRecord : CollectionUtils.nonNullList(associatedRecords))
-         //       {
-         //          associatedEntityList.add(QRecordEntity.fromQRecord(qRecordEntityAssociation.getAssociatedType(), associatedRecord));
-         //       }
-         //       qRecordEntityAssociation.getSetter().invoke(this, associatedEntityList);
-         //    }
-         // }
       }
       catch(Exception e)
       {

qqq-backend-core/src/test/java/com/kingsrook/qqq/backend/core/model/metadata/fields/WhiteSpaceBehaviorTest.java

@@ -173,86 +173,6 @@ class WhiteSpaceBehaviorTest extends BaseTest
 
 
 
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Test
-   void testReads() throws QException
-   {
-      TestUtils.insertDefaultShapes(QContext.getQInstance());
-
-      List<QRecord> records = QueryAction.execute(TestUtils.TABLE_NAME_SHAPE, null);
-      assertEquals(Set.of("Triangle", "Square", "Circle"), records.stream().map(r -> r.getValueString("name")).collect(Collectors.toSet()));
-
-      QFieldMetaData field = QContext.getQInstance().getTable(TestUtils.TABLE_NAME_SHAPE).getField("name");
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
-
-      records = QueryAction.execute(TestUtils.TABLE_NAME_SHAPE, null);
-      assertEquals(Set.of("TRIANGLE", "SQUARE", "CIRCLE"), records.stream().map(r -> r.getValueString("name")).collect(Collectors.toSet()));
-
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
-      assertEquals("triangle", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, 1).getValueString("name"));
-
-      field.setBehaviors(Set.of(CaseChangeBehavior.NONE));
-      assertEquals("Triangle", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, 1).getValueString("name"));
-   }
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Test
-   void testWrites() throws QException
-   {
-      Integer id = 100;
-
-      QFieldMetaData field = QContext.getQInstance().getTable(TestUtils.TABLE_NAME_SHAPE).getField("name");
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
-      new InsertAction().execute(new InsertInput(TestUtils.TABLE_NAME_SHAPE).withRecord(new QRecord().withValue("id", id).withValue("name", "Octagon")));
-
-      //////////////////////////////////////////////////////////////////////////////////
-      // turn off the to-upper-case behavior, so we'll see what was actually inserted //
-      //////////////////////////////////////////////////////////////////////////////////
-      field.setBehaviors(Collections.emptySet());
-      assertEquals("OCTAGON", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, id).getValueString("name"));
-
-      ////////////////////////////////////////////
-      // change to toLowerCase and do an update //
-      ////////////////////////////////////////////
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
-      new UpdateAction().execute(new UpdateInput(TestUtils.TABLE_NAME_SHAPE).withRecord(new QRecord().withValue("id", id).withValue("name", "Octagon")));
-
-      ////////////////////////////////////////////////////////////////////////////////////
-      // turn off the to-lower-case behavior, so we'll see what was actually updated to //
-      ////////////////////////////////////////////////////////////////////////////////////
-      field.setBehaviors(Collections.emptySet());
-      assertEquals("octagon", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, id).getValueString("name"));
-   }
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Test
-   void testFilter()
-   {
-      QInstance      qInstance = QContext.getQInstance();
-      QTableMetaData table     = qInstance.getTable(TestUtils.TABLE_NAME_SHAPE);
-      QFieldMetaData field     = table.getField("name");
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
-      assertEquals("SQUARE", CaseChangeBehavior.TO_UPPER_CASE.applyToFilterCriteriaValue("square", qInstance, table, field));
-
-      field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
-      assertEquals("triangle", CaseChangeBehavior.TO_LOWER_CASE.applyToFilterCriteriaValue("Triangle", qInstance, table, field));
-
-      field.setBehaviors(Set.of(CaseChangeBehavior.NONE));
-      assertEquals("Circle", CaseChangeBehavior.NONE.applyToFilterCriteriaValue("Circle", qInstance, table, field));
-   }
-
-
-
    /*******************************************************************************
     **
     *******************************************************************************/
@@ -264,17 +184,17 @@ class WhiteSpaceBehaviorTest extends BaseTest
       ///////////////////////////////////////////
       // should be no errors on a string field //
       ///////////////////////////////////////////
-      assertTrue(CaseChangeBehavior.TO_UPPER_CASE.validateBehaviorConfiguration(table, table.getField("name")).isEmpty());
+      assertTrue(WhiteSpaceBehavior.TRIM.validateBehaviorConfiguration(table, table.getField("name")).isEmpty());
 
       //////////////////////////////////////////
       // should be an error on a number field //
       //////////////////////////////////////////
-      assertEquals(1, CaseChangeBehavior.TO_LOWER_CASE.validateBehaviorConfiguration(table, table.getField("id")).size());
+      assertEquals(1, WhiteSpaceBehavior.REMOVE_ALL_WHITESPACE.validateBehaviorConfiguration(table, table.getField("id")).size());
 
       /////////////////////////////////////////
       // NONE should be allowed on any field //
       /////////////////////////////////////////
-      assertTrue(CaseChangeBehavior.NONE.validateBehaviorConfiguration(table, table.getField("id")).isEmpty());
+      assertTrue(WhiteSpaceBehavior.NONE.validateBehaviorConfiguration(table, table.getField("id")).isEmpty());
    }
 
 }

qqq-dev-tools/CURRENT-SNAPSHOT-VERSION

@@ -1 +1 @@
-0.25.0
+0.26.0

qqq-middleware-javalin/src/main/java/com/kingsrook/qqq/middleware/javalin/QApplicationJavalinServer.java

@@ -106,10 +106,10 @@ public class QApplicationJavalinServer
    {
       QInstance qInstance = application.defineValidatedQInstance();
 
-      QJavalinMetaData qJavalinMetaData = QJavalinMetaData.of(qInstance);
+      QJavalinMetaData javalinMetaData = getJavalinMetaDataToUse(qInstance);
-      if(qJavalinMetaData != null)
+      if(javalinMetaData != null)
       {
-         addRouteProvidersFromMetaData(qJavalinMetaData);
+         addRouteProvidersFromMetaData(javalinMetaData);
       }
 
       service = Javalin.create(config ->
@@ -234,6 +234,28 @@ public class QApplicationJavalinServer
 
 
 
+   /***************************************************************************
+    **
+    ***************************************************************************/
+   private QJavalinMetaData getJavalinMetaDataToUse(QInstance qInstance)
+   {
+      if(this.javalinMetaData != null && QJavalinMetaData.of(qInstance) != null)
+      {
+         LOG.warn("JavalinMetaData is defined both in the QInstance and the QApplicationJavalinServer.  The one from the QInstance will be ignored - the one from the QJavalinApplicationServer will be used.");
+         return (this.javalinMetaData);
+      }
+      else if (this.javalinMetaData != null)
+      {
+         return (this.javalinMetaData);
+      }
+      else
+      {
+         return QJavalinMetaData.of(qInstance);
+      }
+   }
+
+
+
    /***************************************************************************
     ** initial tests with the SimpleFileSystemDirectoryRouter would sometimes
     ** have a Content-Type:text/html;charset=null !

qqq-sample-project/src/main/java/com/kingsrook/sampleapp/ConfigFileBasedSampleJavalinServer.java

@@ -22,11 +22,9 @@
 package com.kingsrook.sampleapp;
 
 
-import java.util.Arrays;
 import com.kingsrook.qqq.backend.core.instances.ConfigFilesBasedQQQApplication;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
-import com.kingsrook.sampleapp.metadata.SampleMetaDataProvider;
 
 
 /*******************************************************************************
@@ -35,7 +33,6 @@ import com.kingsrook.sampleapp.metadata.SampleMetaDataProvider;
 public class ConfigFileBasedSampleJavalinServer
 {
    private static final QLogger LOG = QLogger.getLogger(ConfigFileBasedSampleJavalinServer.class);
-   private final String path;
 
 
 
@@ -43,6 +40,8 @@ public class ConfigFileBasedSampleJavalinServer
     **
     *******************************************************************************/
    public static void main(String[] args)
+   {
+      try
       {
          String path = "src/main/resources/metadata";
          if(args.length > 0)
@@ -51,27 +50,9 @@ public class ConfigFileBasedSampleJavalinServer
             System.out.println("Using path from args [" + path + "]");
          }
 
-      new ConfigFileBasedSampleJavalinServer(path).start();
+         ConfigFilesBasedQQQApplication application   = new ConfigFilesBasedQQQApplication(path);
-   }
+         QApplicationJavalinServer      javalinServer = new QApplicationJavalinServer(application);
-
+         javalinServer.start();
-   /*******************************************************************************
-    ** Constructor
-    **
-    *******************************************************************************/
-   public ConfigFileBasedSampleJavalinServer(String path)
-   {
-      this.path = path;
-   }
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   public void start()
-   {
-      try
-      {
-         new QApplicationJavalinServer(new ConfigFilesBasedQQQApplication(path)).start();
       }
       catch(Exception e)
       {