Commit Graph

59 Commits

Author SHA1 Message Date
6f1e9413f6 Update for use-case of Get - listing a single file - to pass that file name in, to avoid listing huge directory when not needed 2025-04-08 13:35:08 -05:00
2fa829658f Merged feature/s3-table-set-content-type-on-insert into dev 2025-04-03 14:24:37 -05:00
94631585ee Update for s3 tables, to allow setting content-type in aws when inserting records (files) based on file name, hard-coded value, or another field.
this involved adding table & record params to writeFile method - a @Deprecated wrapper w/o those args is provided for backward compatibility
2025-04-01 15:50:16 -05:00
235cf9e16c Bugfix for s3 utils listObjectsInBucketMatchingGlob, for file names with chars that need URL Encoding (since we're using a pathMatcher class and file:/// URIs...) update test setup to have a file that triggered this error before the fix. 2025-04-01 11:09:35 -05:00
eae24e3eba Add method pemStringToDecodedBytes 2025-02-25 08:45:48 -06:00
21c4434831 Add support for public-key based authentication 2025-02-24 19:57:07 -06:00
cddc42db5b add testSimpleQueryForOneFile 2025-02-21 16:27:17 -06:00
d401cc9ae1 Implement and test DeleteAction functionality
- Unified `deleteFile` API across storage modules by removing unused `QInstance` parameter.
- Added implementations for S3, SFTP, and local filesystem deleteAction.
2025-02-20 14:29:08 -06:00
d25eb6ee48 Simplify file listing by replacing filters with requested paths
Refactor file listing mechanisms to replace the use of complex query filters with simpler, path-based requests. Updated module-specific implementations and removed unused filtering logic. Updated tests (zombie'ing some)
2025-02-20 11:41:29 -06:00
be4f3c68f0 Update expected error message 2025-02-19 20:17:56 -06:00
bb1a43f11f Initial checkin 2025-02-19 20:02:05 -06:00
e5bdf8cd5e Move makeConnection to its own method (for use by test process); add postAction to try to close the things; add looking for 'path' criteria and adding it to readDir call 2025-02-19 20:01:30 -06:00
b93114a9ba Initial add of sftp filesystem module 2025-02-14 20:26:44 -06:00
ce50120234 CE-1772: s3 updates to allow content type specifications among other things 2024-11-03 21:34:50 -06:00
ab5af234af Merged feature/checkstyle-updates into dev 2024-08-05 13:35:21 -05:00
840e1aada3 Applying checkstyle updates to test sources 2024-07-19 16:16:51 -05:00
912e40fe0b Eliminated all warnings. 2024-07-19 16:16:27 -05:00
a90f244c7f Merge pull request #85 from Kingsrook/feature/backend-module-self-registration
Feature/backend module self registration
2024-04-18 09:07:30 -05:00
731d1e266a Remove withBackendType(String), to encourage usage instead of with/set method that takes the module class, so the module will be init'ed and registered 2024-04-10 09:05:43 -05:00
4dadff7fc2 CE-881 - Cleanups - string aggregates; json field names; excel sheet name cleansing; excel size limits; counts, etc 2024-04-02 15:43:59 -05:00
fb8257d34a CE-881 - Updated to actually run in CI (e.g., w/ localstack) 2024-04-01 15:54:09 -05:00
1ff2bc60ae CE-881 - Tests for storage action 2024-04-01 15:39:01 -05:00
557824c572 CE-781 Fix s3 delete to handle slashes it doesn't want 2024-02-21 10:48:41 -06:00
0dd26b8f31 CE-781 - eliminate instance validation errors when using FilesystemImporterStep FIELD_IMPORT_SECURITY_VALUE_SUPPLIERs 2024-01-23 19:12:41 -06:00
fb69c60e10 CE-781 - Add option to set secuirty key values in importFile & importRecord records dynamically through a QCodeReference to a Function 2024-01-23 14:44:10 -06:00
3ebc567299 CE-781 Fix archivePath as field on table; set maxRows 100 on child-widget; always archive files; allow security name/value; significant tests on importer step 2024-01-16 10:33:59 -06:00
624a723b54 CE-781 Initial checkin of filesystem importer meta-data template and process 2024-01-11 07:46:30 -06:00
b64883f34f CE-781 rename beforeEach and afterEach (to help avoid overwriting in sub-classes) 2024-01-10 19:59:40 -06:00
92b052fe59 CE-773 avoid s3 list requests that start with / if backend & table have no basePaths 2023-12-29 19:12:34 -06:00
688e221f9a CE-773 Fixing globs for local filesystem by using Files.walkFileTree. Refactored to share filter matching between s3 & local fs. 2023-12-29 08:20:38 -06:00
6e1ea5c8f1 CE-773 fix tables created in here, per new validationing! 2023-12-28 16:46:51 -06:00
872dec3177 CE-773 change fileNameFieldName and contentsFieldName to default as null - add validation to tableBackendDetails, specifically implemented in filesystem module 2023-12-28 16:38:40 -06:00
345d8022c1 CE-773 Feedback from code review 2023-12-28 10:33:23 -06:00
940080bc86 CE-773 update to support listing/filtering filesystem tables with Cardinality.ONE (single-record per-file) 2023-12-27 16:10:45 -06:00
9e66bc0ab9 Try turning s3 tests w/ localstack back on (w/ new orb) 2023-11-24 08:30:46 -06:00
107086094a Temp disable while localstack is failing... 2023-09-28 15:31:18 -05:00
4f896dde97 Temp disable while localstack is failing... 2023-09-28 15:20:27 -05:00
2efc732530 Fixing for CI 2023-06-16 09:55:06 -05:00
1ed51e0a35 Disabling these tests - poorly written, and no longer viable as a concept 2023-06-16 09:36:15 -05:00
c08856a92c Merge branch 'feature/CTLE-433-cart-rover-now-extensiv-integration' into integration/sprint-26 2023-06-06 11:14:58 -05:00
e66b649699 CTLE-433: fixed more tests due to new data 2023-05-10 16:35:00 -05:00
036b7dc115 Refactor to get rid of Usage parameter in QCodeReference 2023-05-05 16:59:52 -05:00
2d81f24887 updated copyrights 2023-04-20 12:27:08 -05:00
6629015fbf Pin localstack to versiosn 1.4, which seems to NOT be borken. 2023-03-27 14:42:52 -05:00
f28af62c5e Update insert action to do pre-step - e.g., to prime amazon s3 client 2023-02-21 22:30:49 -06:00
d2e7b794f4 moving QLogger package 2023-01-18 12:11:40 -06:00
178078282c Switch to use QLogger everywhere 2023-01-17 10:44:45 -06:00
d3fa1df56f Implementation of QContext everywhere, instead of passing QInstance and QSession in all ActionInputs 2023-01-15 19:41:23 -06:00
7fae3e2329 Add table-based authentication module; update javalin to support Authentication: Basic header; Move authentication classes 2022-12-28 17:00:08 -06:00
1d1461deea Adding filesystem writing - used by javalin to store uploaded files; done async, via new base class for actions 2022-11-17 19:59:29 -06:00