QRunIO/qqq
2
0
Fork 0
mirror of https://github.com/Kingsrook/qqq.git synced 2025-07-19 05:30:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into integration

Browse Source
This commit is contained in:
Tim Chamberlain
2025-05-23 12:33:09 -05:00
parent 0328d94983 dfb584b367
commit 2366cbf474
9 changed files with 151 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/Introduction.adoc
View File

@ -91,7 +91,7 @@ And then having a bug in the check permission logic on the _Light Bulb Inventory
No!
All of the (really important, even though application developers hate doing it) aspects of security - you don't need to write ANY code for dealing with that.
Just tell QQQ what Authentication provider you want to use (e.g., https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
Just tell QQQ what Authentication provider you want to use (e.g., OAuth2 or https://auth0.com/[Auth0]), and - to paraphrase the old https://www.youtube.com/watch?v=YHzM4avGrKI[iMac ad] - there's no step 2.
QQQ just does it.
''''

4
docs/index.adoc
View File

@ -31,11 +31,9 @@ include::metaData/PermissionRules.adoc[leveloffset=+1]
== Services
include::misc/Javalin.adoc[leveloffset=+1]
include::misc/ScheduledJobs.adoc[leveloffset=+1]
=== Web server (Javalin)
#todo#
=== API server (OpenAPI)
#todo#

109
docs/misc/Javalin.adoc Normal file
View File

@ -0,0 +1,109 @@
== QQQ Middleware: Javalin web server
include::../variables.adoc[]
QQQ provides a standard implementation of a middleware layer - that is - code that exists between the
QQQ backend and user interface. This implementation is a web server built using the https://javalin.io/[Javalin framework],
packaged and deployed in the `qqq-middleware-javalin` maven module
The de facto way to create a QQQ application server is to write a class which uses an instance of one of the
subclasses of `QApplicationJavalinServer`.
For example, if your application metadata is defined in a directory of yaml files, your server class could be implemented as:
[source,java]
.ConfigFileBasedQQQApplication usage example
----
public static void main(String[] args)
{
try
{
String path = "src/main/resources/metadata";
ConfigFilesBasedQQQApplication application = new ConfigFilesBasedQQQApplication(path);
QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(application);
javalinServer.start();
}
catch(Exception e)
{
LOG.error("Failed to start javalin server. See stack trace for details.", e);
}
}
----
A similar class exists if your metadata is produced by a package of Java MetaDataProducer objects: `MetaDataProducerBasedQQQApplication`.
=== QApplicationJavalinServer
This class provides the bridge between your QQQ Application (e.g., your metadata) and the QQQ Middleware layer
served by a Javalin web server. It has several properties to control behaviors:
* `Integer port` - (default `8000`) - port to use for serving HTTP.
* `boolean serveFrontendMaterialDashboard` - (default `true`) whether to serve the javascript frontend provided
in the maven artifact `qqq-frontend-material-dashboard`.
* `boolean serveLegacyUnversionedMiddlewareAPI` - (default `true`) whether to serve a version the original implementation
of the QQQ middleware, which current version of `qqq-frontend-material-dashboard` are compatible with.
* `List<AbstractMiddlewareVersion> middlewareVersionList` - (default contains `MiddlewareVersionV1`) - list of
newer, formally versioned implementations of the QQQ middleware interface to be served.
* `Consumer<Javalin> javalinConfigurationCustomizer` - (default `null`) - optional hook to customize the
javalin service object before it is started.
* `List<QJavalinRouteProviderInterface> additionalRouteProviders` - (default `null`) - list of fully custom
implementations of `QJavalinRouteProviderInterface`, to add additional endpoints to the javalin server.
** _Note, you may first want to consider using JavalinRouteProviderMetaData instead - see below._
* `QJavalinMetaData javalinMetaData` - (default `null`) - optional alternative place to define `JavalinMetaData` (vs.
defining it in the `QInstance`). _Note that if it is set in both places, the one in the QApplicationJavalinServer
is used._
=== JavalinMetaData
Certain behaviors of a QQQ Javalin server are configured in a declarative manner by adding a `QJavalinMetaData`
object to the `supplementalMetaData` in your `QInstance` (or, as mentioned above, by setting it directly on the
`QApplicationJavalinServer`):
* `List<JavalinRouteProviderMetaData> routeProviders` - (default `null`) optional list of custom route providers to
add to the Javalin server. See below for details.
* `String uploadedFileArchiveTableName` - (default `null`) - reference to a QQQ Table in your application instance,
needed to support the Bulk Load process, as well as any other processes which need to accept an uploaded file
as input.
* `boolean loggerDisabled` - (default `false`)
* `Function<QJavalinAccessLogger.LogEntry, Boolean> logFilter` - (default `null`)
* `boolean queryWithoutLimitAllowed` - (default `false`)
* `Integer queryWithoutLimitDefault` - (default `1000`)
* `Level queryWithoutLimitLogLevel` - (default `INFO`)
==== JavalinRouteProviderMetaData
This type of metadata allows you to add additional http route providers to your Javalin instance, e.g., for
serving static files or for running custom code from your application (in the form of QQQ Processes) to respond
to HTTP requests.
* `String hostedPath` - (required)
* `String fileSystemPath` - (required for a static router)
* `String processName` - required for a dynamic, process-based router. Must be a process name within the QQQ Instance.
See below for additional details
* `List<String> methods` - required list of HTTP methods (verbs) that are served by the route provider
* `QCodeReference routeAuthenticator - Optional reference to a class that implements `RouteAuthenticatorInterface`,
to provide security authentication to all requests handled by the route provider.
** A default implementation is provided as `SimpleRouteAuthenticator`, which requires that a user session be present
to access paths served by the route provider.
===== Process-based route provider processes
If you define a `JavalinRouteProviderMetaData` with a `processName` (e.g., to serve dynamic HTTP responses from your javalin
server), the process that you implement will be called to respond to any HTTP requests received by the javalin
server which match the `hostedPath` and `methods` that are specified in the metadata.
The QQQ javalin server will marshal request data from the javalin context into the process's payload, conforming to
the shape of the `ProcessBasedRouterPayload` class. Similarly, the http response will be built by taking values from
the process's output/state conforming to the fields in that class. As such, it is recommended to use a
`ProcessBasedRouterPayload` instance, as show in this example:
[source,java]
.Process-based router usage example (including ProcessBasedRouterPayload)
----
public class MyDynamicSiteProcessStep implements BackendStep
{
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
ProcessBasedRouterPayload payload = input.getProcessPayload(ProcessBasedRouterPayload.class);
String path = payload.getPath();
payload.setResponseString("You requested: " + path);
output.setProcessPayload(payload);
}
}
----

2
pom.xml
View File

@ -48,7 +48,7 @@
</modules>
<properties>
<revision>0.25.0-SNAPSHOT</revision>
<revision>0.26.0-SNAPSHOT</revision>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

38
qqq-backend-core/src/main/java/com/kingsrook/qqq/backend/core/model/actions/processes/QProcessPayload.java
View File

@ -78,51 +78,13 @@ public class QProcessPayload
try
{
List<QRecordEntityField> fieldList = getFieldList(this.getClass());
// originalRecordValues = new HashMap<>();
for(QRecordEntityField qRecordEntityField : fieldList)
{
Serializable value = processState.getValues().get(qRecordEntityField.getFieldName());
Object typedValue = qRecordEntityField.convertValueType(value);
qRecordEntityField.getSetter().invoke(this, typedValue);
// originalRecordValues.put(qRecordEntityField.getFieldName(), value);
}
// for(QRecordEntityAssociation qRecordEntityAssociation : getAssociationList(this.getClass()))
// {
// List<QRecord> associatedRecords = qRecord.getAssociatedRecords().get(qRecordEntityAssociation.getAssociationAnnotation().name());
// if(associatedRecords == null)
// {
// qRecordEntityAssociation.getSetter().invoke(this, (Object) null);
// }
// else
// {
// List<QRecordEntity> associatedEntityList = new ArrayList<>();
// for(QRecord associatedRecord : CollectionUtils.nonNullList(associatedRecords))
// {
// associatedEntityList.add(QRecordEntity.fromQRecord(qRecordEntityAssociation.getAssociatedType(), associatedRecord));
// }
// qRecordEntityAssociation.getSetter().invoke(this, associatedEntityList);
// }
// }
// for(QRecordEntityAssociation qRecordEntityAssociation : getAssociationList(this.getClass()))
// {
// List<QRecord> associatedRecords = qRecord.getAssociatedRecords().get(qRecordEntityAssociation.getAssociationAnnotation().name());
// if(associatedRecords == null)
// {
// qRecordEntityAssociation.getSetter().invoke(this, (Object) null);
// }
// else
// {
// List<QRecordEntity> associatedEntityList = new ArrayList<>();
// for(QRecord associatedRecord : CollectionUtils.nonNullList(associatedRecords))
// {
// associatedEntityList.add(QRecordEntity.fromQRecord(qRecordEntityAssociation.getAssociatedType(), associatedRecord));
// }
// qRecordEntityAssociation.getSetter().invoke(this, associatedEntityList);
// }
// }
}
catch(Exception e)
{

86
qqq-backend-core/src/test/java/com/kingsrook/qqq/backend/core/model/metadata/fields/WhiteSpaceBehaviorTest.java
View File

@ -173,86 +173,6 @@ class WhiteSpaceBehaviorTest extends BaseTest
/*******************************************************************************
**
*******************************************************************************/
@Test
void testReads() throws QException
{
TestUtils.insertDefaultShapes(QContext.getQInstance());
List<QRecord> records = QueryAction.execute(TestUtils.TABLE_NAME_SHAPE, null);
assertEquals(Set.of("Triangle", "Square", "Circle"), records.stream().map(r -> r.getValueString("name")).collect(Collectors.toSet()));
QFieldMetaData field = QContext.getQInstance().getTable(TestUtils.TABLE_NAME_SHAPE).getField("name");
field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
records = QueryAction.execute(TestUtils.TABLE_NAME_SHAPE, null);
assertEquals(Set.of("TRIANGLE", "SQUARE", "CIRCLE"), records.stream().map(r -> r.getValueString("name")).collect(Collectors.toSet()));
field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
assertEquals("triangle", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, 1).getValueString("name"));
field.setBehaviors(Set.of(CaseChangeBehavior.NONE));
assertEquals("Triangle", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, 1).getValueString("name"));
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testWrites() throws QException
{
Integer id = 100;
QFieldMetaData field = QContext.getQInstance().getTable(TestUtils.TABLE_NAME_SHAPE).getField("name");
field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
new InsertAction().execute(new InsertInput(TestUtils.TABLE_NAME_SHAPE).withRecord(new QRecord().withValue("id", id).withValue("name", "Octagon")));
//////////////////////////////////////////////////////////////////////////////////
// turn off the to-upper-case behavior, so we'll see what was actually inserted //
//////////////////////////////////////////////////////////////////////////////////
field.setBehaviors(Collections.emptySet());
assertEquals("OCTAGON", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, id).getValueString("name"));
////////////////////////////////////////////
// change to toLowerCase and do an update //
////////////////////////////////////////////
field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
new UpdateAction().execute(new UpdateInput(TestUtils.TABLE_NAME_SHAPE).withRecord(new QRecord().withValue("id", id).withValue("name", "Octagon")));
////////////////////////////////////////////////////////////////////////////////////
// turn off the to-lower-case behavior, so we'll see what was actually updated to //
////////////////////////////////////////////////////////////////////////////////////
field.setBehaviors(Collections.emptySet());
assertEquals("octagon", GetAction.execute(TestUtils.TABLE_NAME_SHAPE, id).getValueString("name"));
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testFilter()
{
QInstance qInstance = QContext.getQInstance();
QTableMetaData table = qInstance.getTable(TestUtils.TABLE_NAME_SHAPE);
QFieldMetaData field = table.getField("name");
field.setBehaviors(Set.of(CaseChangeBehavior.TO_UPPER_CASE));
assertEquals("SQUARE", CaseChangeBehavior.TO_UPPER_CASE.applyToFilterCriteriaValue("square", qInstance, table, field));
field.setBehaviors(Set.of(CaseChangeBehavior.TO_LOWER_CASE));
assertEquals("triangle", CaseChangeBehavior.TO_LOWER_CASE.applyToFilterCriteriaValue("Triangle", qInstance, table, field));
field.setBehaviors(Set.of(CaseChangeBehavior.NONE));
assertEquals("Circle", CaseChangeBehavior.NONE.applyToFilterCriteriaValue("Circle", qInstance, table, field));
}
/*******************************************************************************
**
*******************************************************************************/
@ -264,17 +184,17 @@ class WhiteSpaceBehaviorTest extends BaseTest
///////////////////////////////////////////
// should be no errors on a string field //
///////////////////////////////////////////
assertTrue(CaseChangeBehavior.TO_UPPER_CASE.validateBehaviorConfiguration(table, table.getField("name")).isEmpty());
assertTrue(WhiteSpaceBehavior.TRIM.validateBehaviorConfiguration(table, table.getField("name")).isEmpty());
//////////////////////////////////////////
// should be an error on a number field //
//////////////////////////////////////////
assertEquals(1, CaseChangeBehavior.TO_LOWER_CASE.validateBehaviorConfiguration(table, table.getField("id")).size());
assertEquals(1, WhiteSpaceBehavior.REMOVE_ALL_WHITESPACE.validateBehaviorConfiguration(table, table.getField("id")).size());
/////////////////////////////////////////
// NONE should be allowed on any field //
/////////////////////////////////////////
assertTrue(CaseChangeBehavior.NONE.validateBehaviorConfiguration(table, table.getField("id")).isEmpty());
assertTrue(WhiteSpaceBehavior.NONE.validateBehaviorConfiguration(table, table.getField("id")).isEmpty());
}
}

2
qqq-dev-tools/CURRENT-SNAPSHOT-VERSION
View File

@ -1 +1 @@
0.25.0
0.26.0

28
qqq-middleware-javalin/src/main/java/com/kingsrook/qqq/middleware/javalin/QApplicationJavalinServer.java
View File

@ -106,10 +106,10 @@ public class QApplicationJavalinServer
{
QInstance qInstance = application.defineValidatedQInstance();
QJavalinMetaData qJavalinMetaData = QJavalinMetaData.of(qInstance);
if(qJavalinMetaData != null)
QJavalinMetaData javalinMetaData = getJavalinMetaDataToUse(qInstance);
if(javalinMetaData != null)
{
addRouteProvidersFromMetaData(qJavalinMetaData);
addRouteProvidersFromMetaData(javalinMetaData);
}
service = Javalin.create(config ->
@ -234,6 +234,28 @@ public class QApplicationJavalinServer
/***************************************************************************
**
***************************************************************************/
private QJavalinMetaData getJavalinMetaDataToUse(QInstance qInstance)
{
if(this.javalinMetaData != null && QJavalinMetaData.of(qInstance) != null)
{
LOG.warn("JavalinMetaData is defined both in the QInstance and the QApplicationJavalinServer. The one from the QInstance will be ignored - the one from the QJavalinApplicationServer will be used.");
return (this.javalinMetaData);
}
else if (this.javalinMetaData != null)
{
return (this.javalinMetaData);
}
else
{
return QJavalinMetaData.of(qInstance);
}
}
/***************************************************************************
** initial tests with the SimpleFileSystemDirectoryRouter would sometimes
** have a Content-Type:text/html;charset=null !

39
qqq-sample-project/src/main/java/com/kingsrook/sampleapp/ConfigFileBasedSampleJavalinServer.java
View File

@ -22,11 +22,9 @@
package com.kingsrook.sampleapp;
import java.util.Arrays;
import com.kingsrook.qqq.backend.core.instances.ConfigFilesBasedQQQApplication;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
import com.kingsrook.sampleapp.metadata.SampleMetaDataProvider;
/*******************************************************************************
@ -35,7 +33,6 @@ import com.kingsrook.sampleapp.metadata.SampleMetaDataProvider;
public class ConfigFileBasedSampleJavalinServer
{
private static final QLogger LOG = QLogger.getLogger(ConfigFileBasedSampleJavalinServer.class);
private final String path;
@ -43,35 +40,19 @@ public class ConfigFileBasedSampleJavalinServer
**
*******************************************************************************/
public static void main(String[] args)
{
String path = "src/main/resources/metadata";
if(args.length > 0)
{
path = args[0];
System.out.println("Using path from args [" + path + "]");
}
new ConfigFileBasedSampleJavalinServer(path).start();
}
/*******************************************************************************
** Constructor
**
*******************************************************************************/
public ConfigFileBasedSampleJavalinServer(String path)
{
this.path = path;
}
/*******************************************************************************
**
*******************************************************************************/
public void start()
{
try
{
new QApplicationJavalinServer(new ConfigFilesBasedQQQApplication(path)).start();
String path = "src/main/resources/metadata";
if(args.length > 0)
{
path = args[0];
System.out.println("Using path from args [" + path + "]");
}
ConfigFilesBasedQQQApplication application = new ConfigFilesBasedQQQApplication(path);
QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(application);
javalinServer.start();
}
catch(Exception e)
{