[nifi] branch main updated: NIFI-7572: Force ScriptedTransformRecord to recompile the script when started
This is an automated email from the ASF dual-hosted git repository.
alopresto pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 6488db1 NIFI-7572: Force ScriptedTransformRecord to recompile the
script when started
6488db1 is described below
commit 6488db13762fd42ce6c51f574e2e47ac29006ab5
Author: Matthew Burgess
AuthorDate: Thu Aug 6 10:54:41 2020 -0400
NIFI-7572: Force ScriptedTransformRecord to recompile the script when
started
This closes #4458.
Signed-off-by: Andy LoPresto
---
.../processors/script/ScriptedTransformRecord.java | 2 +
.../script/TestScriptedTransformRecord.java| 60 ++
2 files changed, 62 insertions(+)
diff --git
a/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
b/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
index d14187e..0adf82c 100644
---
a/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
+++
b/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
@@ -183,6 +183,8 @@ public class ScriptedTransformRecord extends
AbstractProcessor implements Search
scriptToRun = IOUtils.toString(scriptStream,
Charset.defaultCharset());
}
}
+// Always compile when first run
+compiledScriptRef.set(null);
}
diff --git
a/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/java/org/apache/nifi/processors/script/TestScriptedTransformRecord.java
b/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/java/org/apache/nifi/processors/script/TestScriptedTransformRecord.java
index 3b15cf3..08676c8 100644
---
a/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/java/org/apache/nifi/processors/script/TestScriptedTransformRecord.java
+++
b/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/java/org/apache/nifi/processors/script/TestScriptedTransformRecord.java
@@ -333,6 +333,66 @@ public class TestScriptedTransformRecord {
assertEquals("Unknown Author",
outputRecords.get(1).getAsRecord("book", bookSchema).getValue("author"));
}
+@Test
+public void testRecompileJythonScript() throws InitializationException {
+final RecordSchema schema = createSimpleNumberSchema();
+setup(schema);
+
+testRunner.setProperty(ScriptedTransformRecord.LANGUAGE, "python");
+testRunner.setProperty(ScriptingComponentUtils.SCRIPT_BODY, "_ =
record");
+
+final Map num1 = new HashMap<>();
+num1.put("num", 1);
+final Map num2 = new HashMap<>();
+num2.put("num", 2);
+final Map num3 = new HashMap<>();
+num3.put("num", 3);
+
+recordReader.addRecord(new MapRecord(schema, num1));
+recordReader.addRecord(new MapRecord(schema, num2));
+recordReader.addRecord(new MapRecord(schema, num3));
+
+testRunner.enqueue(new byte[0]);
+
+testRunner.run();
+
testRunner.assertAllFlowFilesTransferred(ScriptedTransformRecord.REL_SUCCESS,
1);
+
+MockFlowFile out =
testRunner.getFlowFilesForRelationship(ScriptedTransformRecord.REL_SUCCESS).get(0);
+out.assertAttributeEquals("record.count", "3");
+assertEquals(3, testRunner.getCounterValue("Records
Transformed").intValue());
+assertEquals(0, testRunner.getCounterValue("Records
Dropped").intValue());
+
+List recordsWritten = recordWriter.getRecordsWritten();
+assertEquals(3, recordsWritten.size());
+
+for (int i = 0; i < 3; i++) {
+assertEquals(i + 1,
recordsWritten.get(i).getAsInt("num").intValue());
+}
+
+testRunner.clearTransferState();
+// reset the writer
+testRunner.removeControllerService(recordWriter);
+recordWriter = new ArrayListRecordWriter(schema);
+testRunner.addControllerService("record-writer", recordWriter);
+testRunner.enableControllerService(recordWriter);
+
+testRunner.setProperty(ScriptingComponentUtils.SCRIPT_BODY,
"record.setValue(\"num\", 5)\n_ = record");
+
+testRunner.enqueue(new byte[0]);
+testRunner.run();
+
testRunner.assertAllFlowFilesTransferred(ScriptedTransformRecord.REL_SUCCESS,
1);
+
+out =
testRunner.getFlowFilesForRelationship(ScriptedTransformRecord.REL_SUCCESS).get(0);
+out.assertAttributeEquals("record.count", "3");
+
+recordsWritten = recordWriter.getRecordsWritten();
+assertEquals(3, recordsWritten.size());
+
+for (int i = 0; i <
[nifi-registry] branch main updated: NIFIREG-408 Initial commit adding TenantsClient and PoliciesClient
This is an automated email from the ASF dual-hosted git repository.
kdoran pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-registry.git
The following commit(s) were added to refs/heads/main by this push:
new 4dde23b NIFIREG-408 Initial commit adding TenantsClient and
PoliciesClient
4dde23b is described below
commit 4dde23b664fe2595723a90c32632cfe8e285dda8
Author: Bryan Bende
AuthorDate: Mon Jul 27 13:42:22 2020 -0400
NIFIREG-408 Initial commit adding TenantsClient and PoliciesClient
NIFIREG-408 Separate tenants IT into separate tests for users and groups
This closes #292.
Signed-off-by: Kevin Doran
---
.../nifi/registry/client/NiFiRegistryClient.java | 54 +++
.../nifi/registry/client/PoliciesClient.java | 62
.../apache/nifi/registry/client/TenantsClient.java | 166 +
.../client/impl/AbstractCRUDJerseyClient.java | 111 ++
.../client/impl/JerseyNiFiRegistryClient.java | 24 +++
.../registry/client/impl/JerseyPoliciesClient.java | 65
.../registry/client/impl/JerseyTenantsClient.java | 113 ++
.../web/api/SecureNiFiRegistryClientIT.java| 149 ++
8 files changed, 712 insertions(+), 32 deletions(-)
diff --git
a/nifi-registry-core/nifi-registry-client/src/main/java/org/apache/nifi/registry/client/NiFiRegistryClient.java
b/nifi-registry-core/nifi-registry-client/src/main/java/org/apache/nifi/registry/client/NiFiRegistryClient.java
index e141136..a7497a1 100644
---
a/nifi-registry-core/nifi-registry-client/src/main/java/org/apache/nifi/registry/client/NiFiRegistryClient.java
+++
b/nifi-registry-core/nifi-registry-client/src/main/java/org/apache/nifi/registry/client/NiFiRegistryClient.java
@@ -33,6 +33,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
BucketClient getBucketClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with flows
*/
@@ -43,6 +45,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
FlowClient getFlowClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with flows/snapshots
*/
@@ -53,6 +57,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
FlowSnapshotClient getFlowSnapshotClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with bucket items
*/
@@ -63,6 +69,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
ItemsClient getItemsClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for obtaining information about the current user
*/
@@ -73,6 +81,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
UserClient getUserClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with extension bundles
*/
@@ -83,6 +93,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
BundleClient getBundleClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with extension bundle versions
*/
@@ -93,6 +105,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
BundleVersionClient getBundleVersionClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with the extension repository
*/
@@ -103,6 +117,8 @@ public interface NiFiRegistryClient extends Closeable {
*/
ExtensionRepoClient getExtensionRepoClient(String ... proxiedEntity);
+
//---
+
/**
* @return the client for interacting with extensions
*/
@@ -113,6 +129,44 @@ public interface NiFiRegistryClient extends Closeable {
*/
ExtensionClient getExtensionClient(String ... proxiedEntity);
+
//---
+
+/**
+ * Returns client for interacting with tenants.
+ *
+ * @return the client for interacting with tenants
+ */
+TenantsClient getTenantsClient();
+
+/**
+ * Returns client for interacting with tenants.
+
[nifi] branch main updated: NIFI-7708: Add Azure SDK for Java (and transitive dependencies) to LICENCE and NOTICE files
This is an automated email from the ASF dual-hosted git repository. jfrazee pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/main by this push: new 5cc7c04 NIFI-7708: Add Azure SDK for Java (and transitive dependencies) to LICENCE and NOTICE files 5cc7c04 is described below commit 5cc7c0485aedfac7ad731aba5fcfbec67e8806f2 Author: Peter Gyori AuthorDate: Tue Aug 4 16:51:25 2020 +0200 NIFI-7708: Add Azure SDK for Java (and transitive dependencies) to LICENCE and NOTICE files This closes #4452 Signed-off-by: Joey Frazee --- nifi-assembly/LICENSE | 115 +- nifi-assembly/NOTICE | 105 + .../src/main/resources/META-INF/LICENSE| 162 .../src/main/resources/META-INF/NOTICE | 169 - 4 files changed, 549 insertions(+), 2 deletions(-) diff --git a/nifi-assembly/LICENSE b/nifi-assembly/LICENSE index 85e4f1d..96872ae 100644 --- a/nifi-assembly/LICENSE +++ b/nifi-assembly/LICENSE @@ -1610,6 +1610,34 @@ information can be found here: http://www.adobe.com/devnet/xmp/library/eula-xmp- OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +This product bundles 'NanoHttpd Core 2.3.1' which is available under a 3-Clause BSD license. +Copyright (c) 2012 - 2016, nanohttpd + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the nanohttpd nor the names of its contributors + may be used to endorse or promote products derived from this software without + specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, +INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +OF THE POSSIBILITY OF SUCH DAMAGE. + This product bundles 'Jsoup' which is available under "The MIT license". More information can be found here: http://jsoup.org/license @@ -2869,6 +2897,27 @@ The binary distribution of this product bundles the 'ASM' library which is avail ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +The binary distribution of this product bundles the 'Stax2 API 4.2' library which is available under the Simplified BSD License. + +Copyright (c) 2008 FasterXML LLC i...@fasterxml.com + +Redistribution and use in source and binary forms, with or without modification, are permitted +provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this list of conditions and the +following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in +the documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED +WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR +OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + The binary distribution of this product bundles 'A
[nifi] branch main updated: NIFI-7706, NIFI-5702: Allow NiFi to keep FlowFiles if their queue is unknown. This way, if a Flow is inadvertently removed, updated, etc., and NiFi is restarted, the data w
This is an automated email from the ASF dual-hosted git repository.
bbende pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new eca7f15 NIFI-7706, NIFI-5702: Allow NiFi to keep FlowFiles if their
queue is unknown. This way, if a Flow is inadvertently removed, updated, etc.,
and NiFi is restarted, the data will not be dropped by default. The old
mechanism of dropping data is exposed via a property
eca7f15 is described below
commit eca7f153d0b23d39f14e5c65d1f96c7f663b2bbe
Author: Mark Payne
AuthorDate: Wed Aug 5 15:47:05 2020 -0400
NIFI-7706, NIFI-5702: Allow NiFi to keep FlowFiles if their queue is
unknown. This way, if a Flow is inadvertently removed, updated, etc., and NiFi
is restarted, the data will not be dropped by default. The old mechanism of
dropping data is exposed via a property
This closes #4454.
Signed-off-by: Bryan Bende
---
.../controller/repository/FlowFileRepository.java | 10 ++
.../apache/nifi/controller/flow/FlowManager.java | 5 +
.../org/apache/nifi/controller/FlowController.java | 2 +-
.../nifi/controller/flow/StandardFlowManager.java | 42 +
.../repository/WriteAheadFlowFileRepository.java | 105 -
.../bootstrap/tasks/ContentRepositoryScanTask.java | 13 +++
.../nifi-framework/nifi-resources/pom.xml | 1 +
.../src/main/resources/conf/nifi.properties| 1 +
nifi-system-tests/nifi-system-test-suite/pom.xml | 7 ++
.../nifi/tests/system/AggregateNiFiInstance.java | 17 +++-
.../apache/nifi/tests/system/NiFiClientUtil.java | 18
.../org/apache/nifi/tests/system/NiFiSystemIT.java | 8 +-
.../SpawnedStandaloneNiFiInstanceFactory.java | 4 +
.../system/restart/FlowFileRestorationIT.java | 102
.../resources/conf/clustered/node1/bootstrap.conf | 2 +-
.../resources/conf/clustered/node2/bootstrap.conf | 2 +-
.../cli/impl/client/nifi/ConnectionClient.java | 3 +
.../client/nifi/impl/JerseyConnectionClient.java | 24 +
18 files changed, 331 insertions(+), 35 deletions(-)
diff --git
a/nifi-framework-api/src/main/java/org/apache/nifi/controller/repository/FlowFileRepository.java
b/nifi-framework-api/src/main/java/org/apache/nifi/controller/repository/FlowFileRepository.java
index 0a7bad7..67165ee 100644
---
a/nifi-framework-api/src/main/java/org/apache/nifi/controller/repository/FlowFileRepository.java
+++
b/nifi-framework-api/src/main/java/org/apache/nifi/controller/repository/FlowFileRepository.java
@@ -23,6 +23,7 @@ import
org.apache.nifi.controller.repository.claim.ResourceClaimManager;
import java.io.Closeable;
import java.io.IOException;
import java.util.Collection;
+import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -178,4 +179,13 @@ public interface FlowFileRepository extends Closeable {
throws IOException {
return null;
}
+
+/**
+ * Returns the set of Resource Claims that are referenced by FlowFiles
that have been "orphaned" because they belong to FlowFile Queues/Connections
+ * that did not exist in the flow when NiFi started
+ * @return the set of orphaned Resource Claims
+ */
+default Set findOrphanedResourceClaims() {
+return Collections.emptySet();
+}
}
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/flow/FlowManager.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/flow/FlowManager.java
index ba24705..c4503c4 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/flow/FlowManager.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/controller/flow/FlowManager.java
@@ -328,6 +328,11 @@ public interface FlowManager {
ParameterContextManager getParameterContextManager();
/**
+ * @return the number of each type of component (Processor, Controller
Service, Process Group, Funnel, Input Port, Output Port, Reporting Task, Remote
Process Group)
+ */
+Map getComponentCounts();
+
+/**
* Purges all components from the flow, including:
*
* Process Groups (and all components within it)
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
index 97b4c13..500cb15 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java
+++
b/nifi
[nifi] branch main updated: [NIFI-7705] update fronted deps make variable and parameters tables position relative
This is an automated email from the ASF dual-hosted git repository.
rfellows pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 6596fb1 [NIFI-7705] update fronted deps make variable and parameters
tables position relative
6596fb1 is described below
commit 6596fb1f8724d6163db3c4f899a57049a8355274
Author: Scott Aslan
AuthorDate: Wed Aug 5 11:15:05 2020 -0400
[NIFI-7705] update fronted deps
make variable and parameters tables position relative
Signed-off-by: Rob Fellows
This closes #4449
---
.../nifi-framework/nifi-web/nifi-web-ui/pom.xml| 4 +-
.../src/main/frontend/package-lock.json| 130 ++---
.../nifi-web-ui/src/main/frontend/package.json | 4 +-
.../nifi-web-ui/src/main/webapp/css/dialog.css | 1 +
.../webapp/css/new-parameter-context-dialog.css| 1 +
5 files changed, 71 insertions(+), 69 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
index 2d9d7d6..d5c41b9 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
@@ -23,7 +23,7 @@
nifi-web-ui
war
-v8.10.0
+v12.7.0
true
true
${project.build.directory}/tmp
@@ -389,7 +389,7 @@
npm
---cache-min Infinity install
+--silent ci
${frontend.working.dir}
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/frontend/package-lock.json
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/frontend/package-lock.json
index bb1f954..a59b046 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/frontend/package-lock.json
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/frontend/package-lock.json
@@ -82,9 +82,9 @@
"resolved": "https://registry.npmjs.org/chalk/-/chalk-0.4.0.tgz";,
"integrity": "sha1-UZmj3c0MHv4jvAjBsCewYXbgxk8=",
"requires": {
-"ansi-styles": "1.0.0",
-"has-color": "0.1.7",
-"strip-ansi": "0.1.1"
+"ansi-styles": "~1.0.0",
+"has-color": "~0.1.0",
+"strip-ansi": "~0.1.0"
}
},
"commander": {
@@ -144,11 +144,11 @@
"resolved": "https://registry.npmjs.org/d3-brush/-/d3-brush-1.0.4.tgz";,
"integrity": "sha1-AMLyOAGfJPbAoZSibUGhUw/+e8Q=",
"requires": {
-"d3-dispatch": "1.0.3",
-"d3-drag": "1.2.1",
-"d3-interpolate": "1.1.6",
-"d3-selection": "1.3.0",
-"d3-transition": "1.1.1"
+"d3-dispatch": "1",
+"d3-drag": "1",
+"d3-interpolate": "1",
+"d3-selection": "1",
+"d3-transition": "1"
}
},
"d3-chord": {
@@ -156,8 +156,8 @@
"resolved": "https://registry.npmjs.org/d3-chord/-/d3-chord-1.0.4.tgz";,
"integrity": "sha1-fexPC6iG9xP+ERxF92NBT290yiw=",
"requires": {
-"d3-array": "1.2.1",
-"d3-path": "1.0.5"
+"d3-array": "1",
+"d3-path": "1"
}
},
"d3-collection": {
@@ -180,8 +180,8 @@
"resolved": "https://registry.npmjs.org/d3-drag/-/d3-drag-1.2.1.tgz";,
"integrity":
"sha512-Cg8/K2rTtzxzrb0fmnYOUeZHvwa4PHzwXOLZZPwtEs2SKLLKLXeYwZKBB+DlOxUvFmarOnmt//cU4+3US2lyyQ==",
"requires": {
-"d3-dispatch": "1.0.3",
-"d3-selection": "1.3.0"
+"d3-dispatch": "1",
+"d3-selection": "1"
}
},
"d3-dsv": {
@@ -189,9 +189,9 @@
"resolved": "https://registry.npmjs.org/d3-dsv/-/d3-dsv-1.0.8.tgz";,
"integrity":
"sha512-IVCJpQ+YGe3qu6odkPQI0KPqfxkhbP/oM1XhhE/DFiYmcXKfCRub4KXyiuehV1d4drjWVXHUWx4gHqhdZb6n/A==",
"requires": {
-"commander": "2.20.3",
-"iconv-lite": "0.4.24",
-"rw": "1.3.3"
+"commander": "2",
+"iconv-lite": "0.4",
+"rw": "1"
}
},
"d3-ease": {
@@ -204,10 +204,10 @@
"resolved": "https://registry.npmjs.org/d3-force/-/d3-force-1.1.0.tgz";,
"integrity":
"sha512-2HVQz3/VCQs0QeRNZTYb7GxoUCeb6bOzMp/cGcLa87awY9ZsPvXOGeZm0iaGBjXic6I1ysKwMn+g+5jSAdzwcg==",
"requires": {
-"d3-collection": "1.0.4",
-"d3-dispatch": "1.0.3",
-"d3-quadtree": "1.0.3",
-"d3-timer": "1.0.7"
+"d3-collection": "1",
+"d3-dispatch": "1",
+"d3-quadtree": "1",
+"d3-timer": "1"
}
},
"d3-format": {
@@ -220,7
[nifi-minifi-cpp] branch main updated: MINIFICPP-1261 - Mark ScopeGuard class deprecated
This is an automated email from the ASF dual-hosted git repository.
aboda pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 422fecc MINIFICPP-1261 - Mark ScopeGuard class deprecated
422fecc is described below
commit 422fecc85da4786a0980d48bf89de5641c2ee743
Author: Adam Hunyadi
AuthorDate: Fri Jul 24 11:34:45 2020 +0200
MINIFICPP-1261 - Mark ScopeGuard class deprecated
MINIFICPP-1261 - Replace internal usages of ScopeGuard with gsl::finally
Signed-off-by: Arpad Boda
This closes #849
---
extensions/librdkafka/PublishKafka.cpp| 1 -
extensions/opc/src/opc.cpp| 7 ---
.../processors/SourceInitiatedSubscriptionListener.cpp| 9 +
extensions/rocksdb-repos/FlowFileRepository.cpp | 1 -
extensions/sftp/client/SFTPClient.cpp | 10 +-
extensions/sftp/processors/ListSFTP.cpp | 1 -
extensions/sftp/processors/PutSFTP.cpp| 1 -
extensions/sftp/processors/SFTPProcessorBase.cpp | 1 -
extensions/standard-processors/tests/unit/HashContentTest.cpp | 1 -
extensions/windows-event-log/Bookmark.cpp | 1 -
.../windows-event-log/CollectorInitiatedSubscription.cpp | 11 ++-
extensions/windows-event-log/ConsumeWindowsEventLog.cpp | 9 +
extensions/windows-event-log/wel/WindowsEventLog.cpp | 5 +++--
libminifi/include/utils/ScopeGuard.h | 6 +++---
libminifi/src/io/tls/TLSSocket.cpp| 1 -
libminifi/src/utils/OsUtils.cpp | 4 ++--
16 files changed, 33 insertions(+), 36 deletions(-)
diff --git a/extensions/librdkafka/PublishKafka.cpp
b/extensions/librdkafka/PublishKafka.cpp
index ae8f339..a6a3182 100644
--- a/extensions/librdkafka/PublishKafka.cpp
+++ b/extensions/librdkafka/PublishKafka.cpp
@@ -29,7 +29,6 @@
#include "utils/TimeUtil.h"
#include "utils/StringUtils.h"
-#include "utils/ScopeGuard.h"
#include "utils/GeneralUtils.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
diff --git a/extensions/opc/src/opc.cpp b/extensions/opc/src/opc.cpp
index 2baa8aa..8f8d136 100644
--- a/extensions/opc/src/opc.cpp
+++ b/extensions/opc/src/opc.cpp
@@ -19,11 +19,12 @@
#include "opc.h"
//MiNiFi includes
-#include "utils/ScopeGuard.h"
#include "utils/StringUtils.h"
#include "logging/Logger.h"
#include "Exception.h"
+#include "utils/gsl.h"
+
//Standard includes
#include
#include
@@ -302,7 +303,7 @@ void Client::traverse(UA_NodeId nodeId,
std::function cb,
UA_BrowseResponse bResp = UA_Client_Service_browse(client_, bReq);
- utils::ScopeGuard guard([&bResp]() {
+ const auto guard = gsl::finally([&bResp]() {
UA_BrowseResponse_deleteMembers(&bResp);
});
@@ -363,7 +364,7 @@ UA_StatusCode
Client::translateBrowsePathsToNodeIdsRequest(const std::string& pa
UA_TranslateBrowsePathsToNodeIdsResponse response =
UA_Client_Service_translateBrowsePathsToNodeIds(client_, request);
- utils::ScopeGuard guard([&browsePath]() {
+ const auto guard = gsl::finally([&browsePath]() {
UA_BrowsePath_deleteMembers(&browsePath);
});
diff --git
a/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
b/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
index d285b95..367e374 100644
--- a/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
+++ b/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
@@ -51,9 +51,10 @@ extern "C" {
#include "io/StreamFactory.h"
#include "ResourceClaim.h"
#include "utils/StringUtils.h"
-#include "utils/ScopeGuard.h"
#include "utils/file/FileUtils.h"
+#include "utils/gsl.h"
+
#define XML_NS_CUSTOM_SUBSCRIPTION
"http://schemas.microsoft.com/wbem/wsman/1/subscription";
#define XML_NS_CUSTOM_AUTHENTICATION
"http://schemas.microsoft.com/wbem/wsman/1/authentication";
#define XML_NS_CUSTOM_POLICY "http://schemas.xmlsoap.org/ws/2002/12/policy";
@@ -391,7 +392,7 @@ void
SourceInitiatedSubscriptionListener::Handler::sendResponse(struct mg_connec
}
bool
SourceInitiatedSubscriptionListener::Handler::handleSubscriptionManager(struct
mg_connection* conn, const std::string& endpoint, WsXmlDocH request) {
- utils::ScopeGuard request_guard([&]() {
+ const auto request_guard = gsl::finally([&]() {
ws_xml_destroy_doc(request);
});
@@ -406,7 +407,7 @@ bool
SourceInitiatedSubscriptionListener::Handler::handleSubscriptionManager(str
// Create reponse envelope from request
WsXmlDocH response = wsman_create_response_envelope(request, nullptr);
- utils::ScopeGuard response_guard([&]() {
+ const auto response_guard = gsl::finally([&]() {
ws_xml_destroy_doc(response);
})
[nifi-minifi-cpp] branch main updated: MINIFICPP-1261 - Mark ScopeGuard class deprecated
This is an automated email from the ASF dual-hosted git repository.
aboda pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 422fecc MINIFICPP-1261 - Mark ScopeGuard class deprecated
422fecc is described below
commit 422fecc85da4786a0980d48bf89de5641c2ee743
Author: Adam Hunyadi
AuthorDate: Fri Jul 24 11:34:45 2020 +0200
MINIFICPP-1261 - Mark ScopeGuard class deprecated
MINIFICPP-1261 - Replace internal usages of ScopeGuard with gsl::finally
Signed-off-by: Arpad Boda
This closes #849
---
extensions/librdkafka/PublishKafka.cpp| 1 -
extensions/opc/src/opc.cpp| 7 ---
.../processors/SourceInitiatedSubscriptionListener.cpp| 9 +
extensions/rocksdb-repos/FlowFileRepository.cpp | 1 -
extensions/sftp/client/SFTPClient.cpp | 10 +-
extensions/sftp/processors/ListSFTP.cpp | 1 -
extensions/sftp/processors/PutSFTP.cpp| 1 -
extensions/sftp/processors/SFTPProcessorBase.cpp | 1 -
extensions/standard-processors/tests/unit/HashContentTest.cpp | 1 -
extensions/windows-event-log/Bookmark.cpp | 1 -
.../windows-event-log/CollectorInitiatedSubscription.cpp | 11 ++-
extensions/windows-event-log/ConsumeWindowsEventLog.cpp | 9 +
extensions/windows-event-log/wel/WindowsEventLog.cpp | 5 +++--
libminifi/include/utils/ScopeGuard.h | 6 +++---
libminifi/src/io/tls/TLSSocket.cpp| 1 -
libminifi/src/utils/OsUtils.cpp | 4 ++--
16 files changed, 33 insertions(+), 36 deletions(-)
diff --git a/extensions/librdkafka/PublishKafka.cpp
b/extensions/librdkafka/PublishKafka.cpp
index ae8f339..a6a3182 100644
--- a/extensions/librdkafka/PublishKafka.cpp
+++ b/extensions/librdkafka/PublishKafka.cpp
@@ -29,7 +29,6 @@
#include "utils/TimeUtil.h"
#include "utils/StringUtils.h"
-#include "utils/ScopeGuard.h"
#include "utils/GeneralUtils.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
diff --git a/extensions/opc/src/opc.cpp b/extensions/opc/src/opc.cpp
index 2baa8aa..8f8d136 100644
--- a/extensions/opc/src/opc.cpp
+++ b/extensions/opc/src/opc.cpp
@@ -19,11 +19,12 @@
#include "opc.h"
//MiNiFi includes
-#include "utils/ScopeGuard.h"
#include "utils/StringUtils.h"
#include "logging/Logger.h"
#include "Exception.h"
+#include "utils/gsl.h"
+
//Standard includes
#include
#include
@@ -302,7 +303,7 @@ void Client::traverse(UA_NodeId nodeId,
std::function cb,
UA_BrowseResponse bResp = UA_Client_Service_browse(client_, bReq);
- utils::ScopeGuard guard([&bResp]() {
+ const auto guard = gsl::finally([&bResp]() {
UA_BrowseResponse_deleteMembers(&bResp);
});
@@ -363,7 +364,7 @@ UA_StatusCode
Client::translateBrowsePathsToNodeIdsRequest(const std::string& pa
UA_TranslateBrowsePathsToNodeIdsResponse response =
UA_Client_Service_translateBrowsePathsToNodeIds(client_, request);
- utils::ScopeGuard guard([&browsePath]() {
+ const auto guard = gsl::finally([&browsePath]() {
UA_BrowsePath_deleteMembers(&browsePath);
});
diff --git
a/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
b/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
index d285b95..367e374 100644
--- a/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
+++ b/extensions/openwsman/processors/SourceInitiatedSubscriptionListener.cpp
@@ -51,9 +51,10 @@ extern "C" {
#include "io/StreamFactory.h"
#include "ResourceClaim.h"
#include "utils/StringUtils.h"
-#include "utils/ScopeGuard.h"
#include "utils/file/FileUtils.h"
+#include "utils/gsl.h"
+
#define XML_NS_CUSTOM_SUBSCRIPTION
"http://schemas.microsoft.com/wbem/wsman/1/subscription";
#define XML_NS_CUSTOM_AUTHENTICATION
"http://schemas.microsoft.com/wbem/wsman/1/authentication";
#define XML_NS_CUSTOM_POLICY "http://schemas.xmlsoap.org/ws/2002/12/policy";
@@ -391,7 +392,7 @@ void
SourceInitiatedSubscriptionListener::Handler::sendResponse(struct mg_connec
}
bool
SourceInitiatedSubscriptionListener::Handler::handleSubscriptionManager(struct
mg_connection* conn, const std::string& endpoint, WsXmlDocH request) {
- utils::ScopeGuard request_guard([&]() {
+ const auto request_guard = gsl::finally([&]() {
ws_xml_destroy_doc(request);
});
@@ -406,7 +407,7 @@ bool
SourceInitiatedSubscriptionListener::Handler::handleSubscriptionManager(str
// Create reponse envelope from request
WsXmlDocH response = wsman_create_response_envelope(request, nullptr);
- utils::ScopeGuard response_guard([&]() {
+ const auto response_guard = gsl::finally([&]() {
ws_xml_destroy_doc(response);
})
[nifi-minifi-cpp] branch main updated: MINIFICPP-1313 Do a multifile lookup in onSchedule
This is an automated email from the ASF dual-hosted git repository.
aboda pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 41fc5ae MINIFICPP-1313 Do a multifile lookup in onSchedule
41fc5ae is described below
commit 41fc5aec2becee1a799b7c69cd8717423e7e6286
Author: Ferenc Gerlits
AuthorDate: Mon Aug 3 18:19:38 2020 +0200
MINIFICPP-1313 Do a multifile lookup in onSchedule
Signed-off-by: Arpad Boda
This closes #859
---
.../standard-processors/processors/TailFile.cpp | 18 --
extensions/standard-processors/processors/TailFile.h | 2 ++
.../standard-processors/tests/unit/TailFileTests.cpp | 20 +++-
3 files changed, 29 insertions(+), 11 deletions(-)
diff --git a/extensions/standard-processors/processors/TailFile.cpp
b/extensions/standard-processors/processors/TailFile.cpp
index 426667e..c5d2a1a 100644
--- a/extensions/standard-processors/processors/TailFile.cpp
+++ b/extensions/standard-processors/processors/TailFile.cpp
@@ -366,7 +366,9 @@ void TailFile::onSchedule(const
std::shared_ptr &context,
context->getProperty(LookupFrequency.getName(), lookup_frequency_);
-// in multiple mode, we check for new/removed files in every onTrigger
+recoverState(context);
+
+doMultifileLookup();
} else {
tail_mode_ = Mode::SINGLE;
@@ -378,13 +380,13 @@ void TailFile::onSchedule(const
std::shared_ptr &context,
} else {
throw minifi::Exception(ExceptionType::PROCESSOR_EXCEPTION, "File to
tail must be a fully qualified file");
}
+
+recoverState(context);
}
std::string rolling_filename_pattern_glob;
context->getProperty(RollingFilenamePattern.getName(),
rolling_filename_pattern_glob);
rolling_filename_pattern_ =
utils::file::PathUtils::globToRegex(rolling_filename_pattern_glob);
-
- recoverState(context);
}
void TailFile::parseStateFileLine(char *buf, std::map
&state) const {
@@ -633,9 +635,7 @@ void TailFile::onTrigger(const
std::shared_ptr &, const st
if (tail_mode_ == Mode::MULTIPLE) {
if (last_multifile_lookup_ + lookup_frequency_ <
std::chrono::steady_clock::now()) {
logger_->log_debug("Lookup frequency %" PRId64 " ms have elapsed, doing
new multifile lookup", int64_t{lookup_frequency_.count()});
- checkForRemovedFiles();
- checkForNewFiles();
- last_multifile_lookup_ = std::chrono::steady_clock::now();
+ doMultifileLookup();
} else {
logger_->log_trace("Skipping multifile lookup");
}
@@ -745,6 +745,12 @@ void TailFile::updateStateAttributes(TailState &state,
uint64_t size, uint64_t c
state.checksum_ = checksum;
}
+void TailFile::doMultifileLookup() {
+ checkForRemovedFiles();
+ checkForNewFiles();
+ last_multifile_lookup_ = std::chrono::steady_clock::now();
+}
+
void TailFile::checkForRemovedFiles() {
std::vector file_names_to_remove;
diff --git a/extensions/standard-processors/processors/TailFile.h
b/extensions/standard-processors/processors/TailFile.h
index 080505c..c673601 100644
--- a/extensions/standard-processors/processors/TailFile.h
+++ b/extensions/standard-processors/processors/TailFile.h
@@ -168,6 +168,8 @@ class TailFile : public core::Processor {
bool getStateFromLegacyStateFile(const
std::shared_ptr& context,
std::map
&new_tail_states) const;
+ void doMultifileLookup();
+
void checkForRemovedFiles();
void checkForNewFiles();
diff --git a/extensions/standard-processors/tests/unit/TailFileTests.cpp
b/extensions/standard-processors/tests/unit/TailFileTests.cpp
index 4bd803b..7d0e2b7 100644
--- a/extensions/standard-processors/tests/unit/TailFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/TailFileTests.cpp
@@ -1499,9 +1499,9 @@ TEST_CASE("TailFile interprets the lookup frequency
property correctly", "[multi
auto log_attribute = plan->addProcessor("LogAttribute", "Log",
core::Relationship("success", "description"), true);
plan->setProperty(log_attribute,
processors::LogAttribute::FlowFilesToLog.getName(), "0");
- testController.runSession(plan, true);
-
SECTION("Lookup frequency not set => defaults to 10 minutes") {
+testController.runSession(plan, true);
+
std::shared_ptr tail_file_processor =
std::dynamic_pointer_cast(tail_file);
REQUIRE(tail_file_processor);
REQUIRE(tail_file_processor->getLookupFrequency() ==
std::chrono::minutes{10});
@@ -1509,8 +1509,10 @@ TEST_CASE("TailFile interprets the lookup frequency
property correctly", "[multi
SECTION("Lookup frequency set to zero => new files are picked up
immediately") {
plan->setProperty(tail_file,
processors::TailFile::LookupFrequency.getName(), "0 sec");
+testController.runSession(plan, true);
+REQUIRE(LogTestController::getInstance().contains("Logged 1 flow files"));
-plan->reset(true);
+
[nifi-minifi-cpp] branch main updated (656fca6 -> 215acba)
This is an automated email from the ASF dual-hosted git repository. aboda pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git. from 656fca6 MINIFICPP-1203 - Replace linter reported tabs with two spaces add 215acba MINIFICPP-1314 Filter queries by opcode No new revisions were added by this update. Summary of changes: extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp | 7 +++ 1 file changed, 7 insertions(+)
[nifi-minifi-cpp] branch main updated (2af8cc8 -> 656fca6)
This is an automated email from the ASF dual-hosted git repository. aboda pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git. from 2af8cc8 MINIFICPP-1318 migrate PersistenceTests to /var/tmp add 656fca6 MINIFICPP-1203 - Replace linter reported tabs with two spaces No new revisions were added by this update. Summary of changes: extensions/coap/COAPLoader.h | 2 +- extensions/expression-language/Expression.cpp | 18 +-- extensions/gps/GetGPS.h| 45 +++--- extensions/http-curl/HTTPCurlLoader.h | 2 +- extensions/http-curl/sitetosite/HTTPTransaction.h | 4 +- extensions/http-curl/tests/HTTPHandlers.h | 6 +- extensions/jni/JVMCreator.h| 4 +- extensions/jni/jvm/JVMLoader.h | 4 +- extensions/libarchive/ManipulateArchive.h | 64 extensions/mqtt/processors/PublishMQTT.cpp | 2 +- extensions/opencv/OpenCVLoader.h | 2 +- extensions/sftp/processors/ListSFTP.cpp| 4 +- extensions/sql/data/MaxCollector.h | 2 +- extensions/sql/processors/ExecuteSQL.cpp | 4 +- .../windows-event-log/ConsumeWindowsEventLog.h | 6 +- extensions/windows-event-log/TailEventLog.cpp | 124 +++ extensions/windows-event-log/TailEventLog.h| 167 ++--- .../windows-event-log/wel/MetadataWalker.cpp | 4 +- extensions/windows-event-log/wel/MetadataWalker.h | 2 +- .../windows-event-log/wel/WindowsEventLog.cpp | 2 +- extensions/windows-event-log/wel/WindowsEventLog.h | 2 +- main/MainHelper.h | 4 +- nanofi/include/cxx/Plan.h | 2 +- nanofi/src/api/nanofi.cpp | 2 +- nanofi/src/core/cstream.c | 2 +- 25 files changed, 226 insertions(+), 254 deletions(-)
