(nifi) branch support/nifi-1.x updated (ce56117e43 -> 9e30d739f7)
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a change to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git from ce56117e43 NIFI-12441 Added No Tracking Strategy to ListS3 new 6bbae6c335 NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20 new 91582a06d0 NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11 new 9e30d739f7 NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml| 2 +- .../nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/pom.xml | 2 +- nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml| 2 +- nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-)
(nifi) 02/03: NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git commit 91582a06d0f921b263f07ddff024009a5eca4ff6 Author: mr1716 AuthorDate: Thu Feb 1 14:07:14 2024 -0500 NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11 This closes #8344 Signed-off-by: David Handermann (cherry picked from commit f70db0d23577d48dad9dd1e67820fd7b8e3019fd) --- .../nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml | 2 +- nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml| 2 +- nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml index a763715305..ad428b2aec 100644 --- a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml +++ b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml @@ -94,7 +94,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml b/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml index d765f37b24..14493a658f 100644 --- a/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml +++ b/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml @@ -429,7 +429,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test diff --git a/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml b/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml index 6b1b3e2ca5..0dd8eebe4e 100644 --- a/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml +++ b/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml @@ -441,7 +441,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test
(nifi) 01/03: NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git commit 6bbae6c3357bdbdc0f0ef8b11b9ab792de77ba8d Author: mr1716 AuthorDate: Thu Feb 1 14:11:52 2024 -0500 NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20 This closes #8345 Signed-off-by: David Handermann (cherry picked from commit 6d55e772c1bd37cd27cd21bbd013f19702f26943) --- nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml b/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml index 1fac89ad5f..1050fb957d 100644 --- a/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml +++ b/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml @@ -25,7 +25,7 @@ nifi-email-processors jar -5.5.18 +5.5.20 5.2.5
(nifi) 03/03: NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git commit 9e30d739f74562c5f4f5f112c9ad9309de84e6e3 Author: mr1716 AuthorDate: Thu Feb 1 13:59:14 2024 -0500 NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0 This closes #8343 Signed-off-by: David Handermann (cherry picked from commit 00e5b454ab061bce6a23743d7974607d77c7a538) --- nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml b/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml index 47cd49461f..a189212a7a 100644 --- a/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml +++ b/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml @@ -46,7 +46,7 @@ language governing permissions and limitations under the License. --> com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml index 35b1aee851..9e3be54474 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml @@ -241,7 +241,7 @@ com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani diff --git a/nifi-nar-bundles/nifi-standard-bundle/pom.xml b/nifi-nar-bundles/nifi-standard-bundle/pom.xml index 818aad2173..e341234100 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-standard-bundle/pom.xml @@ -141,7 +141,7 @@ com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani
(nifi) 04/04: NIFI-12724 Upgraded Kubernetes Client from 6.9.2 to 6.10.0
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git commit 2950b4376747e223a94370bcf3860e54a071ff27 Author: mr1716 AuthorDate: Thu Feb 1 13:43:32 2024 -0500 NIFI-12724 Upgraded Kubernetes Client from 6.9.2 to 6.10.0 This closes #8340 Signed-off-by: David Handermann --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index b70cef54a7..77e6a1bbb2 100644 --- a/pom.xml +++ b/pom.xml @@ -111,7 +111,7 @@ 1.12.637 2.23.3 2.10.1 - 6.9.2 + 6.10.0 1.9.10 4.12.0 3.7.0
(nifi) 03/04: NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git commit 00e5b454ab061bce6a23743d7974607d77c7a538 Author: mr1716 AuthorDate: Thu Feb 1 13:59:14 2024 -0500 NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0 This closes #8343 Signed-off-by: David Handermann --- nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml b/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml index 8bc2d2663a..f517f03a7d 100644 --- a/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml +++ b/nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml @@ -49,7 +49,7 @@ language governing permissions and limitations under the License. --> com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani diff --git a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml index 3d0b0e8a2f..e514b21466 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml +++ b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml @@ -248,7 +248,7 @@ com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani diff --git a/nifi-nar-bundles/nifi-standard-bundle/pom.xml b/nifi-nar-bundles/nifi-standard-bundle/pom.xml index 1fe9c2431f..f697c976b7 100644 --- a/nifi-nar-bundles/nifi-standard-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-standard-bundle/pom.xml @@ -113,7 +113,7 @@ com.aayushatharva.brotli4j brotli4j -1.13.0 +1.16.0 org.tukaani
(nifi) branch main updated (e16286ef40 -> 2950b43767)
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git from e16286ef40 NIFI-12707 Allow LookupRecord to operate on multiple child-records new 6d55e772c1 NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20 new f70db0d235 NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11 new 00e5b454ab NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0 new 2950b43767 NIFI-12724 Upgraded Kubernetes Client from 6.9.2 to 6.10.0 The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: nifi-nar-bundles/nifi-compress-bundle/nifi-compress-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml| 2 +- .../nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml | 2 +- nifi-nar-bundles/nifi-standard-bundle/pom.xml | 2 +- nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml| 2 +- nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml | 2 +- pom.xml | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-)
(nifi) 02/04: NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git commit f70db0d23577d48dad9dd1e67820fd7b8e3019fd Author: mr1716 AuthorDate: Thu Feb 1 14:07:14 2024 -0500 NIFI-12729 Upgraded unboundid-ldapsdk from 6.0.10 to 6.0.11 This closes #8344 Signed-off-by: David Handermann --- .../nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml | 2 +- nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml| 2 +- nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml index e1a3bbb65b..abe589e1d1 100644 --- a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml +++ b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers/pom.xml @@ -94,7 +94,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml b/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml index 035e7caa3f..d3ba5ee370 100644 --- a/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml +++ b/nifi-registry/nifi-registry-core/nifi-registry-framework/pom.xml @@ -405,7 +405,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test diff --git a/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml b/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml index a63d0ff0a6..b288df9e6f 100644 --- a/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml +++ b/nifi-registry/nifi-registry-core/nifi-registry-web-api/pom.xml @@ -309,7 +309,7 @@ com.unboundid unboundid-ldapsdk -6.0.10 +6.0.11 test
(nifi) 01/04: NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20
This is an automated email from the ASF dual-hosted git repository. exceptionfactory pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git commit 6d55e772c1bd37cd27cd21bbd013f19702f26943 Author: mr1716 AuthorDate: Thu Feb 1 14:11:52 2024 -0500 NIFI-12730 Upgraded Spring Integration from 5.5.18 to 5.5.20 This closes #8345 Signed-off-by: David Handermann --- nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml b/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml index 86b0f7e4ae..ddb94f1b2b 100644 --- a/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml +++ b/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml @@ -25,7 +25,7 @@ nifi-email-processors jar -5.5.18 +5.5.20 5.2.5
(nifi) branch main updated: NIFI-12707 Allow LookupRecord to operate on multiple child-records
This is an automated email from the ASF dual-hosted git repository.
exceptionfactory pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new e16286ef40 NIFI-12707 Allow LookupRecord to operate on multiple
child-records
e16286ef40 is described below
commit e16286ef406f156b6836817261551d78a8447968
Author: Mark Payne
AuthorDate: Wed Jan 31 10:18:47 2024 -0500
NIFI-12707 Allow LookupRecord to operate on multiple child-records
- In order to accommodate this, also needed to improve DataTypeUtils so
that it knows that Record A is wider than Record B if Record A contains all
fields of Record B and more.
- Removed unit tests and resources that are overly complex and no longer
applicable
- Fixed issue in unit test based on different line endings between
operating systems
This closes #8331
Signed-off-by: David Handermann
---
.../serialization/record/util/DataTypeUtils.java | 72 +++--
.../serialization/record/TestDataTypeUtils.java| 107
.../java/org/apache/nifi/util/MockFlowFile.java| 19 +-
.../nifi/processors/standard/LookupRecord.java | 190 -
.../nifi/processors/standard/TestLookupRecord.java | 297 +
.../nifi/json/TestJsonTreeRowRecordReader.java | 173 +---
.../nifi/yaml/TestYamlTreeRowRecordReader.java | 144 --
...-merged-embedded-arrays-and-single-records.json | 40 ---
...-merged-embedded-arrays-and-single-records.yaml | 19 --
9 files changed, 600 insertions(+), 461 deletions(-)
diff --git
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
index cebf33428b..000435e410 100644
---
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
+++
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
@@ -1722,6 +1722,10 @@ public class DataTypeUtils {
if (otherArrayType.getElementType() == null) {
return Optional.of(thisDataType);
} else {
+final Optional widerElementType =
getWiderType(thisArrayType.getElementType(), otherArrayType.getElementType());
+if (widerElementType.isPresent()) {
+return
Optional.of(RecordFieldType.ARRAY.getArrayDataType(widerElementType.get()));
+}
return Optional.empty();
}
}
@@ -1792,37 +1796,61 @@ public class DataTypeUtils {
return Optional.of(thisDataType);
}
break;
+case RECORD:
+if (otherFieldType != RecordFieldType.RECORD) {
+return Optional.empty();
+}
+
+final RecordDataType thisRecordDataType = (RecordDataType)
thisDataType;
+final RecordDataType otherRecordDataType = (RecordDataType)
otherDataType;
+return getWiderRecordType(thisRecordDataType,
otherRecordDataType);
}
return Optional.empty();
}
-private static boolean isDecimalType(final RecordFieldType fieldType) {
-switch (fieldType) {
-case FLOAT:
-case DOUBLE:
-case DECIMAL:
-return true;
-default:
-return false;
+private static Optional getWiderRecordType(final RecordDataType
thisRecordDataType, final RecordDataType otherRecordDataType) {
+final RecordSchema thisSchema = thisRecordDataType.getChildSchema();
+final RecordSchema otherSchema = otherRecordDataType.getChildSchema();
+
+if (thisSchema == null && otherSchema != null) {
+return Optional.of(otherRecordDataType);
+} else if (thisSchema != null && otherSchema == null) {
+return Optional.of(thisRecordDataType);
+} else if (thisSchema == null && otherSchema == null) {
+return Optional.empty();
+}
+
+final Set thisFields = new
HashSet<>(thisSchema.getFields());
+final Set otherFields = new
HashSet<>(otherSchema.getFields());
+
+if (thisFields.containsAll(otherFields)) {
+return Optional.of(thisRecordDataType);
+}
+
+if (otherFields.containsAll(thisFields)) {
+return Optional.of(otherRecordDataType);
}
+
+return Optional.empty();
+}
+
+private static boolean isDecimalType(final RecordFieldType fieldType) {
+return switch (fieldType) {
+case FLOAT, DOUBLE, DECIMAL -> true;
+default -> false;
+};
}
private static int getIntegerTypeValue(final RecordFieldType fieldType) {
-
(nifi) branch support/nifi-1.x updated: NIFI-12441 Added No Tracking Strategy to ListS3
This is an automated email from the ASF dual-hosted git repository.
turcsanyi pushed a commit to branch support/nifi-1.x
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/support/nifi-1.x by this push:
new ce56117e43 NIFI-12441 Added No Tracking Strategy to ListS3
ce56117e43 is described below
commit ce56117e43c7cdce37ca76d54550a2b45acb508a
Author: Juldrixx
AuthorDate: Sat Dec 23 04:38:03 2023 -0600
NIFI-12441 Added No Tracking Strategy to ListS3
This closes #8088
Signed-off-by: David Handermann
(cherry picked from commit 9a919339090b26b0b39321758b8f8074597320ca)
---
.../org/apache/nifi/processors/aws/s3/ListS3.java | 110 ++---
.../apache/nifi/processors/aws/s3/TestListS3.java | 59 +++
2 files changed, 157 insertions(+), 12 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/ListS3.java
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/ListS3.java
index 4be64b0953..431778f1ef 100644
---
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/ListS3.java
+++
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/ListS3.java
@@ -144,12 +144,17 @@ public class ListS3 extends AbstractS3Processor
implements VerifiableProcessor {
" However an additional DistributedMapCache controller service is
required and more JVM heap memory is used." +
" For more information on how the 'Entity Tracking Time Window'
property works, see the description.");
+public static final AllowableValue NO_TRACKING = new
AllowableValue("none", "No Tracking",
+"This strategy lists all entities without any tracking. The same
entities will be listed each time" +
+" this processor is scheduled. It is recommended to change
the default run schedule value." +
+" Any property that relates to the persisting state will
be ignored.");
+
public static final PropertyDescriptor LISTING_STRATEGY = new Builder()
.name("listing-strategy")
.displayName("Listing Strategy")
.description("Specify how to determine new/updated entities. See each
strategy descriptions for detail.")
.required(true)
-.allowableValues(BY_TIMESTAMPS, BY_ENTITIES)
+.allowableValues(BY_TIMESTAMPS, BY_ENTITIES, NO_TRACKING)
.defaultValue(BY_TIMESTAMPS.getValue())
.build();
@@ -345,7 +350,7 @@ public class ListS3 extends AbstractS3Processor implements
VerifiableProcessor {
try {
listedEntityTracker.clearListedEntities();
} catch (IOException e) {
-throw new RuntimeException("Failed to reset previously listed
entities due to " + e, e);
+throw new RuntimeException("Failed to reset previously listed
entities", e);
}
}
resetEntityTrackingState = false;
@@ -465,11 +470,92 @@ public class ListS3 extends AbstractS3Processor
implements VerifiableProcessor {
listByTrackingTimestamps(context, session);
} else if (BY_ENTITIES.equals(listingStrategy)) {
listByTrackingEntities(context, session);
+} else if (NO_TRACKING.equals(listingStrategy)) {
+listNoTracking(context, session);
} else {
throw new ProcessException("Unknown listing strategy: " +
listingStrategy);
}
}
+private void listNoTracking(ProcessContext context, ProcessSession
session) {
+final AmazonS3 client = getClient(context);
+
+S3BucketLister bucketLister = getS3BucketLister(context, client);
+
+final long startNanos = System.nanoTime();
+final long minAgeMilliseconds =
context.getProperty(MIN_AGE).asTimePeriod(TimeUnit.MILLISECONDS);
+final Long maxAgeMilliseconds = context.getProperty(MAX_AGE) != null ?
context.getProperty(MAX_AGE).asTimePeriod(TimeUnit.MILLISECONDS) : null;
+final long listingTimestamp = System.currentTimeMillis();
+
+final String bucket =
context.getProperty(BUCKET).evaluateAttributeExpressions().getValue();
+final int batchSize = context.getProperty(BATCH_SIZE).asInteger();
+
+int listCount = 0;
+int totalListCount = 0;
+
+getLogger().trace("Start listing, listingTimestamp={}",
listingTimestamp);
+
+final S3ObjectWriter writer;
+final RecordSetWriterFactory writerFactory =
context.getProperty(RECORD_WRITER).asControllerService(RecordSetWriterFactory.class);
+if (writerFactory == null) {
+writer = new AttributeObjectWriter(session);
+} else {
+writer = new RecordObjectWriter(session, writerFactory,
getLogger(),
