d25eb6ee48
Simplify file listing by replacing filters with requested paths
...
Refactor file listing mechanisms to replace the use of complex query filters with simpler, path-based requests. Updated module-specific implementations and removed unused filtering logic. Updated tests (zombie'ing some)
2025-02-20 11:41:29 -06:00
be4f3c68f0
Update expected error message
2025-02-19 20:17:56 -06:00
2502d102d9
Better version (i hope) of using ssh & sftp client objects
2025-02-19 20:17:32 -06:00
dcf7218abf
add basename field
2025-02-19 20:17:31 -06:00
bb1a43f11f
Initial checkin
2025-02-19 20:02:05 -06:00
e5bdf8cd5e
Move makeConnection to its own method (for use by test process); add postAction to try to close the things; add looking for 'path' criteria and adding it to readDir call
2025-02-19 20:01:30 -06:00
31a586f23e
Move stripLeadingSlash up to base class
2025-02-19 19:54:58 -06:00
91aa8faca2
Add baseNameFieldName
2025-02-19 19:54:47 -06:00
154c5442af
Add postAction(); move variants stuff to new BackendVariantsUtil; add baseName to ONE records; remove path criteria when filtering (assuming the listFiles method did it)
2025-02-19 19:54:37 -06:00
b93114a9ba
Initial add of sftp filesystem module
2025-02-14 20:26:44 -06:00
5a7199495d
Basic support for variants; more fields on ONE type file records (size, dates); apply skip, limit, filter, sort on listings/queries for ONE-type files; treat contents as heavy-field if so set; more try-catch (e.g., upon write file)
2025-02-14 20:24:10 -06:00
6ef0a89533
CE-1772: fix aws expecting content type if object metadata is given
2024-11-03 21:53:50 -06:00
ce50120234
CE-1772: s3 updates to allow content type specifications among other things
2024-11-03 21:34:50 -06:00
ab5af234af
Merged feature/checkstyle-updates into dev
2024-08-05 13:35:21 -05:00
840e1aada3
Applying checkstyle updates to test sources
2024-07-19 16:16:51 -05:00
912e40fe0b
Eliminated all warnings.
2024-07-19 16:16:27 -05:00
f9af2ba983
Remove all calls to actionInput.getInstance and getSesssion, in favor of the equivallent methods from QContext
2024-07-19 16:16:16 -05:00
65fe6a002c
Upgrade checkstyle; remove supressed indentation markers for new-style switches
2024-05-17 16:38:28 -05:00
9281d07e96
CE-1068: changes from review: allow multiple email address entry, fixed download urls, fixed localhost to use inbucket/filesystem
2024-04-24 17:11:48 -05:00
5e4305d1d5
CE-1068: checkpoint commit of SES support
2024-04-23 20:55:58 -05:00
a90f244c7f
Merge pull request #85 from Kingsrook/feature/backend-module-self-registration
...
Feature/backend module self registration
2024-04-18 09:07:30 -05:00
32021fc36e
Merged dev into feature/CE-881-create-basic-saved-reports
2024-04-12 19:55:15 -05:00
731d1e266a
Remove withBackendType(String), to encourage usage instead of with/set method that takes the module class, so the module will be init'ed and registered
2024-04-10 09:05:43 -05:00
1eae1d6dc4
Remove hard-coded registration of known backend-modules from QBackendModuleDispatcher, in favor of all modules self-registering in static-init block
2024-04-10 08:51:44 -05:00
3440d45060
Remove usages of log4j's placeholders-based log formatting calls (some of which were accidental and wrong anyway)
2024-04-09 18:52:43 -05:00
8da05b9ab9
CE-881 - Feedback from code review & continued QA testing
2024-04-04 08:56:13 -05:00
4dadff7fc2
CE-881 - Cleanups - string aggregates; json field names; excel sheet name cleansing; excel size limits; counts, etc
2024-04-02 15:43:59 -05:00
fb8257d34a
CE-881 - Updated to actually run in CI (e.g., w/ localstack)
2024-04-01 15:54:09 -05:00
1ff2bc60ae
CE-881 - Tests for storage action
2024-04-01 15:39:01 -05:00
6fffe3036c
CE-881 - Add new StorageAction to backends - e.g., for streaming data into a backend's data store. implemented for in-memory, filesystem, and s3
2024-04-01 12:52:00 -05:00
5f72978528
Refactor table customizers to follow a common interface (point being, so you can have 1 class instead of many for a table's closely-related actions)
2024-03-14 11:24:21 -05:00
52d7d0e8ae
Merge pull request #73 from Kingsrook/hotfix/file-importer
...
Increasing logging for s3 importer archive errors
2024-03-04 11:00:11 -06:00
557824c572
CE-781 Fix s3 delete to handle slashes it doesn't want
2024-02-21 10:48:41 -06:00
b506150842
CE-781 Updated logging in filesystem importer
2024-02-21 09:03:04 -06:00
c07d9a779a
Add record no to the import-record table's label
2024-02-20 21:48:00 -06:00
01d96cc8ae
Increasing logging for s3 importer archive errors
2024-02-20 18:24:48 -06:00
0dd26b8f31
CE-781 - eliminate instance validation errors when using FilesystemImporterStep FIELD_IMPORT_SECURITY_VALUE_SUPPLIERs
2024-01-23 19:12:41 -06:00
fb69c60e10
CE-781 - Add option to set secuirty key values in importFile & importRecord records dynamically through a QCodeReference to a Function
2024-01-23 14:44:10 -06:00
e7e93a6ab2
CE-781 - Rework api for adding automations (don't clobber if adding more than 1)
2024-01-23 14:41:43 -06:00
3ebc567299
CE-781 Fix archivePath as field on table; set maxRows 100 on child-widget; always archive files; allow security name/value; significant tests on importer step
2024-01-16 10:33:59 -06:00
dccbed87a7
CE-781 Add javadoc
2024-01-15 20:21:48 -06:00
f78d9a11b2
CE-781 javadoc syntax fix
2024-01-11 10:42:58 -06:00
624a723b54
CE-781 Initial checkin of filesystem importer meta-data template and process
2024-01-11 07:46:30 -06:00
b64883f34f
CE-781 rename beforeEach and afterEach (to help avoid overwriting in sub-classes)
2024-01-10 19:59:40 -06:00
4b1bdebe44
CE-781 Move backend type (name) up to public static final constant
2024-01-10 19:59:11 -06:00
92b052fe59
CE-773 avoid s3 list requests that start with / if backend & table have no basePaths
2023-12-29 19:12:34 -06:00
688e221f9a
CE-773 Fixing globs for local filesystem by using Files.walkFileTree. Refactored to share filter matching between s3 & local fs.
2023-12-29 08:20:38 -06:00
6e1ea5c8f1
CE-773 fix tables created in here, per new validationing!
2023-12-28 16:46:51 -06:00
872dec3177
CE-773 change fileNameFieldName and contentsFieldName to default as null - add validation to tableBackendDetails, specifically implemented in filesystem module
2023-12-28 16:38:40 -06:00
345d8022c1
CE-773 Feedback from code review
2023-12-28 10:33:23 -06:00
