[GitHub] [nifi] jfrazee commented on pull request #5486: NIFI-9338: Add Azure Blob processors using Azure Blob Storage client …
jfrazee commented on pull request #5486: URL: https://github.com/apache/nifi/pull/5486#issuecomment-963807528 @turcsanyip Thanks for the updates. I tested several scenarios and it looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
levilentz commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745274751
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
##
@@ -119,7 +121,8 @@
.description("Comma-separated list of paths to files and/or
directories which contain modules containing custom transformations (that are
not included on NiFi's classpath).")
.required(false)
.identifiesExternalResource(ResourceCardinality.MULTIPLE,
ResourceType.FILE, ResourceType.DIRECTORY)
-.expressionLanguageSupported(ExpressionLanguageScope.NONE)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
Review comment:
I went through both processors and they should function in the same
manner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
levilentz commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745274634
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/test/java/org/apache/nifi/processors/jolt/record/TestJoltTransformRecord.java
##
@@ -582,6 +582,35 @@ public void
testTransformInputCustomTransformationIgnored() throws IOException {
new String(transformed.toByteArray()));
}
+@Test
+public void testExpressionLanguageJarFile() throws IOException {
+generateTestData(1, null);
+final String outputSchemaText = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestJoltTransformRecord/defaultrOutputSchema.avsc")));
+runner.setProperty(writer, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY,
SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
+runner.setProperty(writer, SchemaAccessUtils.SCHEMA_TEXT,
outputSchemaText);
+runner.setProperty(writer, "Pretty Print JSON", "true");
+runner.enableControllerService(writer);
+final String customJarPath =
"src/test/resources/TestJoltTransformRecord/TestCustomJoltTransform.jar";
+final String spec = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestJoltTransformRecord/defaultrSpec.json")));
+final String customJoltTransform = "TestCustomJoltTransform";
+runner.setVariable("CUSTOM_JAR", customJarPath);
+runner.setProperty(JoltTransformRecord.JOLT_SPEC, "${JOLT_SPEC}");
+runner.setProperty(JoltTransformRecord.MODULES, "${CUSTOM_JAR}");
+runner.setProperty(JoltTransformRecord.JOLT_TRANSFORM,
JoltTransformRecord.DEFAULTR);
Review comment:
Good catch!
I did try to have it also load the jar in the test suite, however, I cant
seem to get it to modify the class path within that context (the test suite),
so I just have it assert that it was valid. Please let me know if you have a
better way of effectuating that unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
levilentz commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745273975
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -234,7 +235,7 @@
protected Collection customValidate(ValidationContext
validationContext) {
final List results = new
ArrayList<>(super.customValidate(validationContext));
final String transform =
validationContext.getProperty(JOLT_TRANSFORM).getValue();
-final String customTransform =
validationContext.getProperty(CUSTOM_CLASS).getValue();
+final String customTransform =
validationContext.getProperty(CUSTOM_CLASS).evaluateAttributeExpressions().getValue();
Review comment:
Updated logic for variable registry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
levilentz commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745273641
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -151,7 +152,7 @@
.displayName("Custom Transformation Class Name")
.description("Fully Qualified Class Name for Custom
Transformation")
.required(false)
-.expressionLanguageSupported(ExpressionLanguageScope.NONE)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
Added dependsOn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
levilentz commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745273174
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -160,7 +161,7 @@
.displayName("Custom Module Directory")
.description("Comma-separated list of paths to files and/or
directories which contain modules containing custom transformations (that are
not included on NiFi's classpath).")
.required(false)
-
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
This was a mistake and now resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5493: NIFI-8806 - Refactoring ListenTCP to use netty.
exceptionfactory commented on a change in pull request #5493:
URL: https://github.com/apache/nifi/pull/5493#discussion_r745232775
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListenTCP.java
##
@@ -87,122 +96,200 @@
.defaultValue(ClientAuth.REQUIRED.name())
.build();
+// Deprecated
public static final PropertyDescriptor MAX_RECV_THREAD_POOL_SIZE = new
PropertyDescriptor.Builder()
.name("max-receiving-threads")
.displayName("Max Number of Receiving Message Handler Threads")
.description(
-"The maximum number of threads might be available for
handling receiving messages ready all the time. " +
-"Cannot be bigger than the \"Max Number of TCP
Connections\". " +
-"If not set, the value of \"Max Number of TCP
Connections\" will be used.")
+"This property is deprecated and no longer used.")
.addValidator(StandardValidators.createLongValidator(1, 65535,
true))
.required(false)
.build();
+// Deprecated
protected static final PropertyDescriptor POOL_RECV_BUFFERS = new
PropertyDescriptor.Builder()
.name("pool-receive-buffers")
.displayName("Pool Receive Buffers")
.description(
-"When turned on, the processor uses pre-populated pool of
buffers when receiving messages. " +
-"This is prepared during initialisation of the processor.
" +
-"With high value of Max Number of TCP Connections and
Receive Buffer Size this strategy might allocate significant amount of memory!
" +
-"When turned off, the byte buffers will be created on
demand and be destroyed after use.")
-.required(true)
+"This property is deprecated and no longer used.")
+.required(false)
.defaultValue("True")
.allowableValues("True", "False")
.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
.build();
-@Override
-protected List getAdditionalProperties() {
-return Arrays.asList(
-MAX_CONNECTIONS,
-MAX_RECV_THREAD_POOL_SIZE,
-POOL_RECV_BUFFERS,
-SSL_CONTEXT_SERVICE,
-CLIENT_AUTH
-);
-}
+public static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("Messages received successfully will be sent out this
relationship.")
+.build();
+
+protected List descriptors;
+protected Set relationships;
+protected volatile int port;
+protected volatile BlockingQueue events;
+protected volatile BlockingQueue errorEvents;
+protected volatile EventServer eventServer;
+protected volatile byte[] messageDemarcatorBytes;
+protected volatile EventBatcher eventBatcher;
@Override
-protected Collection customValidate(final
ValidationContext validationContext) {
-final List results = new ArrayList<>();
+protected void init(final ProcessorInitializationContext context) {
+final List descriptors = new ArrayList<>();
+descriptors.add(ListenerProperties.PORT);
+descriptors.add(ListenerProperties.RECV_BUFFER_SIZE);
+descriptors.add(ListenerProperties.MAX_MESSAGE_QUEUE_SIZE);
+// Deprecated
+descriptors.add(ListenerProperties.MAX_SOCKET_BUFFER_SIZE);
+descriptors.add(ListenerProperties.CHARSET);
+descriptors.add(ListenerProperties.MAX_CONNECTIONS);
+descriptors.add(ListenerProperties.MAX_BATCH_SIZE);
+descriptors.add(ListenerProperties.MESSAGE_DELIMITER);
+// Deprecated
+descriptors.add(MAX_RECV_THREAD_POOL_SIZE);
+// Deprecated
+descriptors.add(POOL_RECV_BUFFERS);
+descriptors.add(ListenerProperties.NETWORK_INTF_NAME);
Review comment:
This property should be moved to the first position so retain the order
from the current version.
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListenTCP.java
##
@@ -87,122 +96,200 @@
.defaultValue(ClientAuth.REQUIRED.name())
.build();
+// Deprecated
public static final PropertyDescriptor MAX_RECV_THREAD_POOL_SIZE = new
PropertyDescriptor.Builder()
.name("max-receiving-threads")
.displayName("Max Number of Receiving Message Handler Threads")
.description(
-"The maximum number of threads might be available for
handling receiving messages ready all the time. " +
-"Cannot be bigger than the \"Max Number of TCP
Connections\". " +
-"If
[jira] [Assigned] (NIFI-9194) Upsert for Oracle12+
[ https://issues.apache.org/jira/browse/NIFI-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-9194: -- Assignee: (was: Matt Burgess) > Upsert for Oracle12+ > > > Key: NIFI-9194 > URL: https://issues.apache.org/jira/browse/NIFI-9194 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework > Environment: All OS >Reporter: ROBERTO SANTOS >Priority: Major > Labels: features > Fix For: 1.16.0 > > Original Estimate: 168h > Time Spent: 1h > Remaining Estimate: 23h 10m > > I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and > its Unit tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9194) Upsert for Oracle12+
[ https://issues.apache.org/jira/browse/NIFI-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440847#comment-17440847 ] ASF subversion and git services commented on NIFI-9194: --- Commit e906c1d70fcea31a4a797541d23fb95108790165 in nifi's branch refs/heads/main from Roberto Santos [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e906c1d ] NIFI-9194: Upsert for Oracle12+ Fixes pr #5366. Fixes pr #5366. Replace tabchars fot whitespaces. Fixes pr #5366. Replaced tabchars for whitespaces. Removed unnecessary comments. Signed-off-by: Matthew Burgess This closes #5366 > Upsert for Oracle12+ > > > Key: NIFI-9194 > URL: https://issues.apache.org/jira/browse/NIFI-9194 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework > Environment: All OS >Reporter: ROBERTO SANTOS >Priority: Major > Labels: features > Fix For: 1.16.0 > > Original Estimate: 168h > Time Spent: 1h > Remaining Estimate: 23h 10m > > I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and > its Unit tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9194) Upsert for Oracle12+
[ https://issues.apache.org/jira/browse/NIFI-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9194: --- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upsert for Oracle12+ > > > Key: NIFI-9194 > URL: https://issues.apache.org/jira/browse/NIFI-9194 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework > Environment: All OS >Reporter: ROBERTO SANTOS >Priority: Major > Labels: features > Fix For: 1.16.0 > > Original Estimate: 168h > Time Spent: 1h > Remaining Estimate: 23h 10m > > I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and > its Unit tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (NIFI-9194) Upsert for Oracle12+
[ https://issues.apache.org/jira/browse/NIFI-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-9194: -- Assignee: Matt Burgess > Upsert for Oracle12+ > > > Key: NIFI-9194 > URL: https://issues.apache.org/jira/browse/NIFI-9194 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework > Environment: All OS >Reporter: ROBERTO SANTOS >Assignee: Matt Burgess >Priority: Major > Labels: features > Fix For: 1.16.0 > > Original Estimate: 168h > Time Spent: 1h > Remaining Estimate: 23h 10m > > I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and > its Unit tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 closed pull request #5366: NIFI-9194: Upsert for Oracle12+
mattyb149 closed pull request #5366: URL: https://github.com/apache/nifi/pull/5366 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5366: NIFI-9194: Upsert for Oracle12+
mattyb149 commented on pull request #5366: URL: https://github.com/apache/nifi/pull/5366#issuecomment-963737436 +1 LGTM, tried with a live Oracle 12 instance, verified the UPSERT capability is working as expected. Thanks for the new feature! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9194) Upsert for Oracle12+
[ https://issues.apache.org/jira/browse/NIFI-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9194: --- Description: I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and its Unit tests. (was: I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and it’s Unity tests.) > Upsert for Oracle12+ > > > Key: NIFI-9194 > URL: https://issues.apache.org/jira/browse/NIFI-9194 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework > Environment: All OS >Reporter: ROBERTO SANTOS >Priority: Major > Labels: features > Original Estimate: 168h > Time Spent: 40m > Remaining Estimate: 23.5h > > I have implemented Upsert capability for Oracle12+ in PutDatabaseRecord and > its Unit tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-8272) Deleted/renamed metrics are still being shown in the REST API Prometheus endpoint
[ https://issues.apache.org/jira/browse/NIFI-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440842#comment-17440842 ] ASF subversion and git services commented on NIFI-8272: --- Commit f410c8df0a2b05cdf078e3d3f6b086ac7ebed5f9 in nifi's branch refs/heads/main from noblenumbat360 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f410c8d ] NIFI-8272 Delete stale metrics from REST API Prometheus endpoint. Added test tag to the nifi-web-api pom.xml and corrected imports. Signed-off-by: Matthew Burgess This closes #5447 > Deleted/renamed metrics are still being shown in the REST API Prometheus > endpoint > - > > Key: NIFI-8272 > URL: https://issues.apache.org/jira/browse/NIFI-8272 > Project: Apache NiFi > Issue Type: Bug >Reporter: Noble Numbat >Assignee: Noble Numbat >Priority: Minor > Labels: REST_API, metrics, prometheus > Fix For: 1.16.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Processors, connections, funnels, etc that have been deleted/renamed from a > flow are still being shown in the REST API Prometheus metrics endpoint at > their last reported value before they were deleted/renamed. > I checked the PrometheusReportingTask and deleted items are not displayed in > the metrics output as I would expect is the correct behaviour. > The bug can be replicated with a flow as simple as a single processor by > deleting the processor and observing the metrics still included on subsequent > calls to the endpoint. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (NIFI-8272) Deleted/renamed metrics are still being shown in the REST API Prometheus endpoint
[ https://issues.apache.org/jira/browse/NIFI-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-8272. Fix Version/s: 1.16.0 Resolution: Fixed > Deleted/renamed metrics are still being shown in the REST API Prometheus > endpoint > - > > Key: NIFI-8272 > URL: https://issues.apache.org/jira/browse/NIFI-8272 > Project: Apache NiFi > Issue Type: Bug >Reporter: Noble Numbat >Assignee: Noble Numbat >Priority: Minor > Labels: REST_API, metrics, prometheus > Fix For: 1.16.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Processors, connections, funnels, etc that have been deleted/renamed from a > flow are still being shown in the REST API Prometheus metrics endpoint at > their last reported value before they were deleted/renamed. > I checked the PrometheusReportingTask and deleted items are not displayed in > the metrics output as I would expect is the correct behaviour. > The bug can be replicated with a flow as simple as a single processor by > deleting the processor and observing the metrics still included on subsequent > calls to the endpoint. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 closed pull request #5447: NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
mattyb149 closed pull request #5447: URL: https://github.com/apache/nifi/pull/5447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5447: NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
mattyb149 commented on pull request #5447: URL: https://github.com/apache/nifi/pull/5447#issuecomment-963729464 +1 LGTM, GHA passed and I tried on a live NiFi instance, verified the removed processor's metrics no longer showed up in the Prometheus scrape results. Thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1201: MINIFICPP-1668 Remove request parameters from Azure primary uri
szaszm closed pull request #1201: URL: https://github.com/apache/nifi-minifi-cpp/pull/1201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5444: NIFI-9286: Add expression language to JOLT processors and fixes the custom module implementation to use custom jars in the proces
mattyb149 commented on a change in pull request #5444:
URL: https://github.com/apache/nifi/pull/5444#discussion_r745094543
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -151,7 +152,7 @@
.displayName("Custom Transformation Class Name")
.description("Fully Qualified Class Name for Custom
Transformation")
.required(false)
-.expressionLanguageSupported(ExpressionLanguageScope.NONE)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
This property is ignored unless the transformation type is CUSTOMR
right? If so, it should have a `.dependsOn()` call in the builder, same goes
for Custom Module Directory below.
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/test/java/org/apache/nifi/processors/jolt/record/TestJoltTransformRecord.java
##
@@ -582,6 +582,35 @@ public void
testTransformInputCustomTransformationIgnored() throws IOException {
new String(transformed.toByteArray()));
}
+@Test
+public void testExpressionLanguageJarFile() throws IOException {
+generateTestData(1, null);
+final String outputSchemaText = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestJoltTransformRecord/defaultrOutputSchema.avsc")));
+runner.setProperty(writer, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY,
SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
+runner.setProperty(writer, SchemaAccessUtils.SCHEMA_TEXT,
outputSchemaText);
+runner.setProperty(writer, "Pretty Print JSON", "true");
+runner.enableControllerService(writer);
+final String customJarPath =
"src/test/resources/TestJoltTransformRecord/TestCustomJoltTransform.jar";
+final String spec = new
String(Files.readAllBytes(Paths.get("src/test/resources/TestJoltTransformRecord/defaultrSpec.json")));
+final String customJoltTransform = "TestCustomJoltTransform";
+runner.setVariable("CUSTOM_JAR", customJarPath);
+runner.setProperty(JoltTransformRecord.JOLT_SPEC, "${JOLT_SPEC}");
+runner.setProperty(JoltTransformRecord.MODULES, "${CUSTOM_JAR}");
+runner.setProperty(JoltTransformRecord.JOLT_TRANSFORM,
JoltTransformRecord.DEFAULTR);
Review comment:
This should be `CUSTOMR` not `DEFAULTR` right? That should also
illustrate that the processor should be invalid before running, as
customValidate() should (currently) fail since the `JOLT_SPEC` attribute is not
available at the time of validation.
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
##
@@ -119,7 +121,8 @@
.description("Comma-separated list of paths to files and/or
directories which contain modules containing custom transformations (that are
not included on NiFi's classpath).")
.required(false)
.identifiesExternalResource(ResourceCardinality.MULTIPLE,
ResourceType.FILE, ResourceType.DIRECTORY)
-.expressionLanguageSupported(ExpressionLanguageScope.NONE)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
Review comment:
See my comment above, I would think these would be set to the same value
for both JOLT processors
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -160,7 +161,7 @@
.displayName("Custom Module Directory")
.description("Comma-separated list of paths to files and/or
directories which contain modules containing custom transformations (that are
not included on NiFi's classpath).")
.required(false)
-
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
Review comment:
Your comment on the main page made me think this was going to be
returned to VARIABLE_REGISTRY?
##
File path:
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java
##
@@ -234,7 +235,7 @@
protected Collection customValidate(ValidationContext
validationContext) {
final List results = new
ArrayList<>(super.customValidate(validationContext));
final String transform =
validationContext.getProperty(JOLT_TRANSFORM).getValue();
-final String customTransform =
validationContext.getProperty(CUSTOM_CLASS).getValue();
+final String customTransform =
validationContext.getProperty(CUSTOM_CLASS).evaluateAttributeExpressions().getValue();
Review
[GitHub] [nifi] turcsanyip commented on a change in pull request #5482: NIFI-9334 Add support for upsert in 'PutMongoRecord'.
turcsanyip commented on a change in pull request #5482:
URL: https://github.com/apache/nifi/pull/5482#discussion_r744988569
##
File path:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoRecord.java
##
@@ -57,28 +70,77 @@
"a configured MongoDB collection. This processor does not support
updates, deletes or upserts. The number of documents to insert at a time is
controlled " +
Review comment:
Could you please update the documentation?
It is not correct any more: _"This processor does not support updates,
deletes or upserts."_
##
File path:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoRecord.java
##
@@ -190,4 +303,49 @@ private List convertArrays(Object[] input) {
return retVal;
}
+
+private Bson[] buildFilters(Map>
updateKeyFieldPathToFieldChain, Document readyToUpsert) {
+Bson[] filters = updateKeyFieldPathToFieldChain.entrySet()
+.stream()
+.map(updateKeyFieldPath__fieldChain -> {
+String fieldPath = updateKeyFieldPath__fieldChain.getKey();
+List fieldChain =
updateKeyFieldPath__fieldChain.getValue();
+
+Object value = readyToUpsert;
+String previousField = null;
+for (String field : fieldChain) {
+if (!(value instanceof Map)) {
+throw new ProcessException("field '" + previousField +
"' (from field expression '" + fieldPath + "') is not an embedded document");
+}
+
+value = ((Map) value).get(field);
+
+if (value == null) {
+throw new ProcessException("field '" + field + "'
(from field expression '" + fieldPath + "') has no value");
+}
+
+previousField = field;
+}
+
+Bson filter = Filters.eq(fieldPath, value);
+return filter;
+})
+.collect(Collectors.toList())
+.toArray(new Bson[0]);
+
+return filters;
+}
+
+private boolean updateModeIs(String updateValueToMatch, ProcessContext
context, FlowFile flowFile) {
Review comment:
`updateModeToMatch` ?
##
File path:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoRecord.java
##
@@ -190,4 +303,49 @@ private List convertArrays(Object[] input) {
return retVal;
}
+
+private Bson[] buildFilters(Map>
updateKeyFieldPathToFieldChain, Document readyToUpsert) {
+Bson[] filters = updateKeyFieldPathToFieldChain.entrySet()
+.stream()
+.map(updateKeyFieldPath__fieldChain -> {
+String fieldPath = updateKeyFieldPath__fieldChain.getKey();
+List fieldChain =
updateKeyFieldPath__fieldChain.getValue();
+
+Object value = readyToUpsert;
+String previousField = null;
+for (String field : fieldChain) {
+if (!(value instanceof Map)) {
+throw new ProcessException("field '" + previousField +
"' (from field expression '" + fieldPath + "') is not an embedded document");
+}
+
+value = ((Map) value).get(field);
+
+if (value == null) {
+throw new ProcessException("field '" + field + "'
(from field expression '" + fieldPath + "') has no value");
+}
+
+previousField = field;
+}
+
+Bson filter = Filters.eq(fieldPath, value);
+return filter;
+})
+.collect(Collectors.toList())
+.toArray(new Bson[0]);
Review comment:
`.toArray(Bson[]::new)` could be used instead of collect + toArray.
##
File path:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/PutMongoRecord.java
##
@@ -135,15 +222,41 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
for (String name : schema.getFieldNames()) {
document.put(name, contentMap.get(name));
}
-inserts.add(convertArrays(document));
-if (inserts.size() == ceiling) {
-collection.insertMany(inserts);
-added += inserts.size();
-inserts = new ArrayList<>();
+Document readyToUpsert = convertArrays(document);
+
+WriteModel writeModel;
+if (context.getProperty(UPDATE_KEY_FIELDS).isSet()) {
+Bson[] filters =
buildFilters(updateKeyFieldPathToFieldChain, readyToUpsert);
+
+
[jira] [Updated] (NIFI-9185) Add Avro logical type to SelectHive3QL processor
[ https://issues.apache.org/jira/browse/NIFI-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-9185: --- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add Avro logical type to SelectHive3QL processor > > > Key: NIFI-9185 > URL: https://issues.apache.org/jira/browse/NIFI-9185 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 1.16.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Add a similar property as ExecuteSQL and default it to false, add logical > type support for SelectHive3QL processor. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9185) Add Avro logical type to SelectHive3QL processor
[ https://issues.apache.org/jira/browse/NIFI-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440689#comment-17440689 ] ASF subversion and git services commented on NIFI-9185: --- Commit 8491486181df6bd346b5953b1665d77113d72f5e in nifi's branch refs/heads/main from Timea Barna [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8491486 ] NIFI-9185 Add Avro logical type to SelectHive3QL processor Modifying unit test to avoid systemdefault timezone usuage NIFI-9185 Applying review recommendations removing duplicate dependency from pom.xml Signed-off-by: Matthew Burgess This closes #5358 > Add Avro logical type to SelectHive3QL processor > > > Key: NIFI-9185 > URL: https://issues.apache.org/jira/browse/NIFI-9185 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 1.16.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Add a similar property as ExecuteSQL and default it to false, add logical > type support for SelectHive3QL processor. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 closed pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
mattyb149 closed pull request #5358: URL: https://github.com/apache/nifi/pull/5358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
mattyb149 commented on pull request #5358: URL: https://github.com/apache/nifi/pull/5358#issuecomment-963488393 +1 LGTM, tested with an HDP 3 Hive and verified the expected behavior. Thanks for the improvement/fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-9171) Proxy Configuration for NiFi embedded in iframe
[ https://issues.apache.org/jira/browse/NIFI-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-9171. Resolution: Information Provided [~sukhwinder13] If you have additional question related to this issue, please feel free to follow up. > Proxy Configuration for NiFi embedded in iframe > --- > > Key: NIFI-9171 > URL: https://issues.apache.org/jira/browse/NIFI-9171 > Project: Apache NiFi > Issue Type: Task > Components: Configuration Management, Security >Affects Versions: 1.13.1 > Environment: Ubuntu >Reporter: sukhwinder kaur >Assignee: David Handermann >Priority: Major > Labels: security > > Hi > While accessing the Nifi URL under iframe getting following error > Refused to frame 'https://localhost:9443/' because an ancestor violates the > following Content Security Policy directive: "frame-ancestors 'self'". > I have setup Nifi with certificate. > > can you please help with proxy pass file for nifi ? > I have tried : > https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/html/administration-guide.html#how-to-install-and-start-nifi > !https://i.imgur.com/Qhghsdy.png! > > > > Not working. > > THanks -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5476: NIFI-9259: Adding GeohashRecord Processor
exceptionfactory commented on a change in pull request #5476:
URL: https://github.com/apache/nifi/pull/5476#discussion_r744967480
##
File path: nifi-nar-bundles/nifi-geohash-bundle/nifi-geohash-processors/pom.xml
##
@@ -0,0 +1,98 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-geohash-bundle
+1.15.0-SNAPSHOT
+
+
+nifi-geohash-processors
+jar
+
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-utils
+1.15.0-SNAPSHOT
+
+
+org.apache.nifi
+nifi-mock
+1.15.0-SNAPSHOT
+test
+
Review comment:
For clearer organization, it would be helpful to move all of the `test`
scope dependencies after the standard compile dependencies.
##
File path:
nifi-nar-bundles/nifi-geohash-bundle/nifi-geohash-processors/src/test/java/org/apache/nifi/processors/geohash/GeohashRecordTest.java
##
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.geohash;
+
+import org.apache.avro.Schema;
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.avro.AvroTypeUtil;
+import org.apache.nifi.controller.ControllerService;
+import org.apache.nifi.json.JsonRecordSetWriter;
+import org.apache.nifi.json.JsonTreeReader;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.schema.access.SchemaAccessUtils;
+import org.apache.nifi.serialization.record.MockSchemaRegistry;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.codehaus.jackson.map.ObjectMapper;
+
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class GeohashRecordTest {
+
+private final TestRunner runner =
TestRunners.newTestRunner(GeohashRecord.class);
Review comment:
To avoid unexpected behavior, recommend removing the `final` keyword and
moving the `newTestRunner()` assignment into the `setUp()` method.
##
File path:
nifi-nar-bundles/nifi-geohash-bundle/nifi-geohash-processors/src/main/java/org/apache/nifi/processors/geohash/GeohashRecord.java
##
@@ -0,0 +1,449 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.geohash;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import
[GitHub] [nifi] pvillard31 commented on pull request #5486: NIFI-9338: Add Azure Blob processors using Azure Blob Storage client …
pvillard31 commented on pull request #5486: URL: https://github.com/apache/nifi/pull/5486#issuecomment-963444682 Hey @jfrazee - I played with the processors added by this pull request and it was working as expected. Are you fine with the changes made by @turcsanyip following your initial review? Did you get the chance to make more tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9334) Add support for upsert in PutMongoRecord.
[ https://issues.apache.org/jira/browse/NIFI-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-9334: -- Status: Patch Available (was: Open) > Add support for upsert in PutMongoRecord. > - > > Key: NIFI-9334 > URL: https://issues.apache.org/jira/browse/NIFI-9334 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Tamas Palfy >Assignee: Tamas Palfy >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > PutMongoRecord currently supports only inserts. > We could add an optional property through which one could list the fields > that would pose as keys when upserts are needed. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] tpalfy commented on a change in pull request #5381: NIFI-9206: Add RemoveRecordField processor and implement the ability …
tpalfy commented on a change in pull request #5381:
URL: https://github.com/apache/nifi/pull/5381#discussion_r744962782
##
File path:
nifi-commons/nifi-record-path/src/main/java/org/apache/nifi/record/path/RecordFieldRemover.java
##
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.record.path;
+
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordFieldRemovalPath;
+import org.apache.nifi.serialization.record.RecordSchema;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+public class RecordFieldRemover {
+private Record record;
+
+public RecordFieldRemover(Record record) {
+this.record = record;
+}
+
+public Record remove(String path) {
+RecordPath recordPath = new RecordPath(path);
+RecordPathResult recordPathResult =
org.apache.nifi.record.path.RecordPath.compile(recordPath.toString()).evaluate(record);
+List selectedFields =
recordPathResult.getSelectedFields().collect(Collectors.toList());
+
+if (recordPath.isAppliedToAllElementsInCollection()) {
+removeAllElementsFromCollection(selectedFields);
+} else {
+selectedFields.forEach(field -> field.remove());
+}
+
+if (recordPath.pathRemovalRequiresSchemaModification()) {
+modifySchema(selectedFields);
+}
+
+record.regenerateSchema();
+return record;
+}
+
+private void removeAllElementsFromCollection(List
selectedFields) {
+if (!selectedFields.isEmpty()) {
+Optional parentOptional =
selectedFields.get(0).getParent();
+if (parentOptional.isPresent()) {
+FieldValue parent = parentOptional.get();
+parent.removeContent();
+}
+}
+}
+
+private void modifySchema(List selectedFields) {
+List concretePaths =
getConcretePaths(selectedFields);
+removePathsFromSchema(concretePaths);
+}
+
+private List getConcretePaths(List
selectedFields) {
+List paths = new
ArrayList<>(selectedFields.size());
+for (FieldValue field : selectedFields) {
+RecordFieldRemovalPath path = new RecordFieldRemovalPath();
+addToPathIfNotRoot(field, path);
+
+Optional parentOptional = field.getParent();
+while (parentOptional.isPresent()) {
+FieldValue parent = parentOptional.get();
+addToPathIfNotRoot(parent, path);
+parentOptional = parent.getParent();
+}
+
+paths.add(path);
+}
+return paths;
+}
+
+private void removePathsFromSchema(List paths) {
+for (RecordFieldRemovalPath path : paths) {
+RecordSchema schema = record.getSchema();
+schema.removePath(path);
+}
+}
+
+private void addToPathIfNotRoot(FieldValue field, RecordFieldRemovalPath
path) {
+if (field.getParent().isPresent()) {
+path.add(field.getField().getFieldName());
+}
+}
+
+private static class RecordPath {
+private String recordPath;
+
+public RecordPath(final String recordPath) {
+this.recordPath = preprocessRecordPath(recordPath);
+}
+
+public boolean isAppliedToAllElementsInCollection() {
+return recordPath.endsWith("[*]") ||
recordPath.endsWith("[0..-1]");
+}
+
+@Override
+public String toString() {
+return recordPath;
+}
+
+private String preprocessRecordPath(final String recordPath) {
+if (recordPath.endsWith("]")) {
+return unifyRecordPathEnd(recordPath);
+}
+return recordPath;
+}
+
+private String unifyRecordPathEnd(final String recordPath) {
+String lastSquareBracketsOperator =
getLastSquareBracketsOperator(recordPath);
+if (lastSquareBracketsOperator.equals("[*]")) {
+return recordPath.substring(0, recordPath.lastIndexOf('[')) +
"[*]";
+} else
[jira] [Commented] (NIFI-9344) Conduct 1.15 Release
[ https://issues.apache.org/jira/browse/NIFI-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440629#comment-17440629 ] ASF subversion and git services commented on NIFI-9344: --- Commit 1894843 from Joe Witt in branch 'site/trunk' [ https://svn.apache.org/r1894843 ] NIFI-9344 > Conduct 1.15 Release > > > Key: NIFI-9344 > URL: https://issues.apache.org/jira/browse/NIFI-9344 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9368) Fix the ordering of the log entries on the processor level bulletin pop-up
[ https://issues.apache.org/jira/browse/NIFI-9368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-9368: -- Status: Patch Available (was: Open) > Fix the ordering of the log entries on the processor level bulletin pop-up > -- > > Key: NIFI-9368 > URL: https://issues.apache.org/jira/browse/NIFI-9368 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.15.0 >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The order of the log entries on the processor level bulletin pop-up was > originally descending (latest log first). It has been changed to ascending > due to a side effect of a change in NIFI-9309. > Restore the former behaviour. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] turcsanyip opened a new pull request #5512: NIFI-9368: Fixed the ordering of the log entries on the processor lev…
turcsanyip opened a new pull request #5512: URL: https://github.com/apache/nifi/pull/5512 …el bulletin pop-up https://issues.apache.org/jira/browse/NIFI-9368 In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `main`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9344) Conduct 1.15 Release
[ https://issues.apache.org/jira/browse/NIFI-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440568#comment-17440568 ] ASF subversion and git services commented on NIFI-9344: --- Commit 1894840 from Joe Witt in branch 'site/trunk' [ https://svn.apache.org/r1894840 ] NIFI-9344 fixing links > Conduct 1.15 Release > > > Key: NIFI-9344 > URL: https://issues.apache.org/jira/browse/NIFI-9344 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] mattyb149 commented on pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
mattyb149 commented on pull request #5358: URL: https://github.com/apache/nifi/pull/5358#issuecomment-963297407 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-9369) JoltTransformJSON removes unicode escape sequence after output
Önder Ezdi created NIFI-9369:
Summary: JoltTransformJSON removes unicode escape sequence after
output
Key: NIFI-9369
URL: https://issues.apache.org/jira/browse/NIFI-9369
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Environment: Linux 64-bit, Ubuntu distro, default OS encoding is UTF-8
Reporter: Önder Ezdi
Fix For: 1.16.0
Problem:
If a JSON comes with unicode escape sequence, processor accepts it and after
transformation, exports it with native characters instead unicode sequence.
Later InvokeHTTP requests to any server breaks json due character problem and
server won't accept if standard encoding configurations applied.
Behavior:
JoltTransformJSON processor input:
{code:java}
{"value":
"\u011f\u00fc\u015f\u0131\u00f6\u00e7\u011e\u00dc\u015e\u0130\u00d6\u00c7"}{code}
JoltTransformJSON processor output:
{code:java}
{"value": "ğüşıöçĞÜŞİÖÇ"}{code}
Expected:
If its not a bug, "Encode Output" or "No encoding" output options should be
supplied as in dropdown selection or something like that on UI.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
exceptionfactory commented on a change in pull request #5358:
URL: https://github.com/apache/nifi/pull/5358#discussion_r744824629
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java
##
@@ -298,14 +318,32 @@ public static Schema createSchema(final ResultSet rs,
String recordName, boolean
// Did not find direct suitable type, need to be clarified
case DECIMAL:
case NUMERIC:
-
builder.name(columnName).type().unionOf().nullBuilder().endNull().and().stringType().endUnion().noDefault();
+if (useLogicalTypes) {
+final int precision =
getPrecision(meta.getPrecision(i));
+final int scale = getScale(meta.getScale(i));
+
builder.name(columnName).type().unionOf().nullBuilder().endNull().and()
+.type(LogicalTypes.decimal(precision,
scale).addToSchema(Schema.create(Schema.Type.BYTES))).endUnion().noDefault();
Review comment:
Thanks for the explanation @timeabarna, that makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] timeabarna commented on a change in pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
timeabarna commented on a change in pull request #5358:
URL: https://github.com/apache/nifi/pull/5358#discussion_r744804777
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java
##
@@ -298,14 +318,32 @@ public static Schema createSchema(final ResultSet rs,
String recordName, boolean
// Did not find direct suitable type, need to be clarified
case DECIMAL:
case NUMERIC:
-
builder.name(columnName).type().unionOf().nullBuilder().endNull().and().stringType().endUnion().noDefault();
+if (useLogicalTypes) {
+final int precision =
getPrecision(meta.getPrecision(i));
+final int scale = getScale(meta.getScale(i));
+
builder.name(columnName).type().unionOf().nullBuilder().endNull().and()
+.type(LogicalTypes.decimal(precision,
scale).addToSchema(Schema.create(Schema.Type.BYTES))).endUnion().noDefault();
Review comment:
Thanks @exceptionfactory. Avro logical types have well defined Avro
object formats for example a date is an int type with logicalType date where
int is the number of days since EPOCH time. For decimal it is either byte or
fixed type with logicalType decimal and precision and scale must be provided as
well.
{
"type": "bytes",
"logicalType": "decimal",
"precision": 4,
"scale": 2
}
So the byte type is coming from the Avro specification:
https://avro.apache.org/docs/1.8.2/spec.html#Decimal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] timeabarna commented on a change in pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
timeabarna commented on a change in pull request #5358: URL: https://github.com/apache/nifi/pull/5358#discussion_r744797325 ## File path: nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/pom.xml ## @@ -74,6 +74,10 @@ org.apache.nifi nifi-kerberos-user-service-api + +org.apache.nifi +nifi-kerberos-user-service-api Review comment: Thanks @exceptionfactory for spotting this, removed duplicate dependency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1337) no matching conversion for static_cast from 'const org::apache::nifi::minifi::core::PropertyValue' to 'std::__1::chrono::duration
[ https://issues.apache.org/jira/browse/MINIFICPP-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1337: Affects Version/s: (was: 1.0.0) > no matching conversion for static_cast from 'const > org::apache::nifi::minifi::core::PropertyValue' to > 'std::__1::chrono::duration >' > > > Key: MINIFICPP-1337 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1337 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: $ cmake --version > cmake version 3.16.3 > $ clang --version > clang version 10.0.0-4ubuntu1 > Target: x86_64-pc-linux-gnu > Thread model: posix >Reporter: Ivan Serdyuk >Priority: Critical > Attachments: TailFile_build_error[1].log > > > I was compiling MiNiFi using Clang 10.0.0-4ubuntu1 release. > I did like this: > $ cmake -DENABLE_COAP=ON -DASAN_BUILD=ON -DSKIP_TESTS=ON -DUSE_SHARED_LIBS=ON > -DPORTABLE=ON -DBUILD_ROCKSDB=ON -DBUILD_IDENTIFIER= > -DCMAKE_BUILD_TYPE=MinSizeRel -DFAIL_ON_WARNINGS= -DCMAKE_C_COMPILER=clang > -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="-stdlib=libc++" .. > And (eventually) got this: > [ 47%] Building CXX object > extensions/standard-processors/CMakeFiles/minifi-standard-processors.dir/processors/TailFile.cpp.o > In file included from > /home/ubuntu/minifi_cpp/extensions/standard-processors/processors/TailFile.cpp:40: > In file included from > /home/ubuntu/minifi_cpp/extensions/standard-processors/processors/TailFile.h:30: > In file included from > /home/ubuntu/minifi_cpp/extensions/standard-processors/../../libminifi/include/core/Processor.h:39: > /home/ubuntu/minifi_cpp/extensions/standard-processors/../../libminifi/include/core/ConfigurableComponent.h:230:13: > error: no matching conversion for static_cast from 'const > org::apache::nifi::minifi::core::PropertyValue' to > 'std::__1::chrono::duration >' > value = static_cast(item.getValue()); > ^~~ > /home/ubuntu/minifi_cpp/extensions/standard-processors/../../libminifi/include/core/ProcessorNode.h:71:30: > note: in instantiation of function template specialization > 'org::apache::nifi::minifi::core::ConfigurableComponent::getProperty long, std::__1::ratio<1, 1000> > >' requested here > return processor_cast->getProperty(name, value); > ^ > /home/ubuntu/minifi_cpp/extensions/standard-processors/../../libminifi/include/core/ProcessContext.h:329:29: > note: in instantiation of function template specialization > 'org::apache::nifi::minifi::core::ProcessorNode::getProperty long, std::__1::ratio<1, 1000> > >' requested here > return processor_node_->getProperty::type>(name, > value); > ^ > /home/ubuntu/minifi_cpp/extensions/standard-processors/../../libminifi/include/core/ProcessContext.h:102:12: > note: in instantiation of function template specialization > 'org::apache::nifi::minifi::core::ProcessContext::getPropertyImp long, std::__1::ratio<1, 1000> > >' requested here > return getPropertyImp::type>(name, value); > ^ > /home/ubuntu/minifi_cpp/extensions/standard-processors/processors/TailFile.cpp:367:14: > note: in instantiation of function template specialization > 'org::apache::nifi::minifi::core::ProcessContext::getProperty long, std::__1::ratio<1, 1000> > >' requested here > context->getProperty(LookupFrequency.getName(), lookup_frequency_); > ^ > /usr/lib/llvm-10/bin/../include/c++/v1/chrono:1021:28: note: candidate > constructor (the implicit copy constructor) not viable: no known conversion > from 'const org::apache::nifi::minifi::core::PropertyValue' to 'const > std::__1::chrono::duration >' for 1st > argument > class _LIBCPP_TEMPLATE_VIS duration > ^ > /usr/lib/llvm-10/bin/../include/c++/v1/chrono:1021:28: note: candidate > constructor (the implicit move constructor) not viable: no known conversion > from 'const org::apache::nifi::minifi::core::PropertyValue' to > 'std::__1::chrono::duration >' for 1st > argument > /usr/lib/llvm-10/bin/../include/c++/v1/chrono:1073:18: note: candidate > template ignored: requirement > 'is_convertible long>::value' was not satisfied [with _Rep2 = > org::apache::nifi::minifi::core::PropertyValue] > explicit duration(const _Rep2& __r, > ^ > /usr/lib/llvm-10/bin/../include/c++/v1/chrono:1085:9: note: candidate > template ignored: could not match 'duration type-parameter-0-1>' against 'const > org::apache::nifi::minifi::core::PropertyValue' > duration(const duration<_Rep2, _Period2>& __d, > ^ > /usr/lib/llvm-10/bin/../include/c++/v1/chrono:1066:9: note: candidate > constructor not viable: requires 0 arguments, but 1 was provided > duration() = default; > ^ > 1 error generated. > make[2]: *** >
[jira] [Commented] (NIFI-9304) Improve the GCP Pub/Sub processors to support Pub/Sub Lite
[ https://issues.apache.org/jira/browse/NIFI-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440514#comment-17440514 ] ASF subversion and git services commented on NIFI-9304: --- Commit 37c0527a72971051fab48224c7e8800e41e0bb62 in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=37c0527 ] NIFI-9304 - Adding PublishGCPubSubLite and ConsumeGCPubSubLite processors Signed-off-by: Joe Gresock This closes #5460. > Improve the GCP Pub/Sub processors to support Pub/Sub Lite > -- > > Key: NIFI-9304 > URL: https://issues.apache.org/jira/browse/NIFI-9304 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Adding two new processors to interact with Google Cloud Platform Pub/Sub Lite > service > https://github.com/apache/nifi/pull/5460 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (NIFI-9304) Improve the GCP Pub/Sub processors to support Pub/Sub Lite
[ https://issues.apache.org/jira/browse/NIFI-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Gresock updated NIFI-9304: -- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Improve the GCP Pub/Sub processors to support Pub/Sub Lite > -- > > Key: NIFI-9304 > URL: https://issues.apache.org/jira/browse/NIFI-9304 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.16.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Adding two new processors to interact with Google Cloud Platform Pub/Sub Lite > service > https://github.com/apache/nifi/pull/5460 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] asfgit closed pull request #5460: NIFI-9304 - Adding PublishGCPubSubLite and ConsumeGCPubSubLite processors
asfgit closed pull request #5460: URL: https://github.com/apache/nifi/pull/5460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9350) Add NiFi Registry NarProvider
[ https://issues.apache.org/jira/browse/NIFI-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Gresock updated NIFI-9350: -- Fix Version/s: 1.16.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add NiFi Registry NarProvider > - > > Key: NIFI-9350 > URL: https://issues.apache.org/jira/browse/NIFI-9350 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Fix For: 1.16.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > With the introduction of the NarProvider concept, we can provide an > implementation that retrieves NARs from NiFi Registry. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9350) Add NiFi Registry NarProvider
[ https://issues.apache.org/jira/browse/NIFI-9350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440485#comment-17440485 ] ASF subversion and git services commented on NIFI-9350: --- Commit c96809012b51156872c0446473953f0cf8548f93 in nifi's branch refs/heads/main from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c968090 ] NIFI-9350 Add NiFi Registry NarProvider implementation Signed-off-by: Joe Gresock This closes #5497. > Add NiFi Registry NarProvider > - > > Key: NIFI-9350 > URL: https://issues.apache.org/jira/browse/NIFI-9350 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > With the introduction of the NarProvider concept, we can provide an > implementation that retrieves NARs from NiFi Registry. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] asfgit closed pull request #5497: NIFI-9350 Add NiFi Registry NarProvider implementation
asfgit closed pull request #5497: URL: https://github.com/apache/nifi/pull/5497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1643) Support using Managed Identity credentials for Azure processors
[ https://issues.apache.org/jira/browse/MINIFICPP-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi resolved MINIFICPP-1643. -- Resolution: Fixed > Support using Managed Identity credentials for Azure processors > --- > > Key: MINIFICPP-1643 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1643 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Time Spent: 6h 20m > Remaining Estimate: 0h > > Currently PutAzureBlobStorage and AzureStorageCredentialService only support > using key or SAS token to access Azure storage, but we should support Managed > Identity token credentials as well for Azure environment. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5511: NIFI-9367 Update README.md
exceptionfactory commented on a change in pull request #5511: URL: https://github.com/apache/nifi/pull/5511#discussion_r744739089 ## File path: nifi-docker/dockerhub/README.md ## @@ -77,6 +77,14 @@ The minimum to run a NiFi instance is as follows: This will provide a running instance, exposing the instance UI to the host system on at port 8443, viewable at `https://localhost:8443/nifi`. +It will generate credentials, which could be found in the docker logs after the startup phase (about a minute): Review comment: Recommend adjusting the wording as follows, similar to the main project README: ```suggestion The default configuration generates a random username and password on startup. NiFi writes the generated credentials to the application log. The following command can be used to find the generated credentials on operating systems with grep installed: ``` ## File path: nifi-docker/dockerhub/README.md ## @@ -86,7 +94,7 @@ Environment variables can be used to set the NiFi communication ports and hostna -e NIFI_WEB_HTTPS_PORT='9443' \ apache/nifi:latest -Single User Authentication credentials can be specified using environment variables as follows: +The esiest way to run NiFi is probably with specifying Single User Authentication credentials using environment variables as follows: Review comment: Recommend reverting this change as the existing wording is already clear. ## File path: nifi-docker/dockerhub/README.md ## @@ -77,6 +77,14 @@ The minimum to run a NiFi instance is as follows: This will provide a running instance, exposing the instance UI to the host system on at port 8443, viewable at `https://localhost:8443/nifi`. +It will generate credentials, which could be found in the docker logs after the startup phase (about a minute): + +docker logs nifi | grep Generated + +the credentials will then be displayed such as + +Generated Username [5ae5ed8e-b695-4589-bfb0-aee8989a08b3] +Generated Password [YJZD3kP8SFzF+SbRmHlcTBifiZTvS6bb] Review comment: ```suggestion Generated Username [USERNAME] Generated Password [PASSWORD] ``` ## File path: nifi-docker/dockerhub/README.md ## @@ -95,6 +103,7 @@ Single User Authentication credentials can be specified using environment variab -e SINGLE_USER_CREDENTIALS_PASSWORD=ctsBtRBKHRAx69EqUghvvgEvjnaLjFEB \ apache/nifi:latest +Please note that the password must be 12 chars minimmum, or a new one will be generated (see above to find it in the logs) Review comment: The wording should be adjusted to correct the spelling of `minimum`, and spelling out `characters` would be helpful. ```suggestion Please note that the password must be 12 characters minimum, otherwise NiFi will generate a random username and password. ``` ## File path: nifi-docker/dockerhub/README.md ## @@ -77,6 +77,14 @@ The minimum to run a NiFi instance is as follows: This will provide a running instance, exposing the instance UI to the host system on at port 8443, viewable at `https://localhost:8443/nifi`. +It will generate credentials, which could be found in the docker logs after the startup phase (about a minute): + +docker logs nifi | grep Generated + +the credentials will then be displayed such as Review comment: Following the project README: ```suggestion NiFi logs the generated credentials as follows: ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-9366) nifi-persistent-provenance-repository tests create unwanted provenance_repository directory in filesystem
[
https://issues.apache.org/jira/browse/NIFI-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-9366:
---
Fix Version/s: 1.16.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> nifi-persistent-provenance-repository tests create unwanted
> provenance_repository directory in filesystem
> -
>
> Key: NIFI-9366
> URL: https://issues.apache.org/jira/browse/NIFI-9366
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Trivial
> Fix For: 1.16.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Executing the tests within the {{nifi-persistent-provenance-repository}}
> results in an unwanted directory being created in the filesystem that could
> accidentally be committed to git:
> *
> {{nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/provenance_repository/}}
> This is caused by the Groovy-based test
> {{EncryptedWriteAheadProvenanceRepositoryTest#testShouldRegisterAndGetEvents}},
> which doesn't override the default {{nifi.properties}} value for
> {{nifi.provenance.repository.directory.}} of {{provenance_repository}} - this
> results in the directory being created as indicated above. It would be better
> to locate the directory either in {{target/}} (therefore ignored by default
> in Git commits as part of the Maven build) or a Temp directory as done in
> {{#testWriteAheadProvenanceRepositoryShouldRegisterAndRetrieveEvents}} within
> the same Test class.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9366) nifi-persistent-provenance-repository tests create unwanted provenance_repository directory in filesystem
[
https://issues.apache.org/jira/browse/NIFI-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440467#comment-17440467
]
ASF subversion and git services commented on NIFI-9366:
---
Commit f86fe0d61ae0310a191e14f7521ef9e3e4d0727c in nifi's branch
refs/heads/main from Chris Sampson
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f86fe0d ]
NIFI-9366 Create and remove provenance repository directory in test methods
- Updated nifi-persistent-provenance-repository tests
This closes #5510
Signed-off-by: David Handermann
> nifi-persistent-provenance-repository tests create unwanted
> provenance_repository directory in filesystem
> -
>
> Key: NIFI-9366
> URL: https://issues.apache.org/jira/browse/NIFI-9366
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.0
>Reporter: Chris Sampson
>Assignee: Chris Sampson
>Priority: Trivial
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Executing the tests within the {{nifi-persistent-provenance-repository}}
> results in an unwanted directory being created in the filesystem that could
> accidentally be committed to git:
> *
> {{nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/provenance_repository/}}
> This is caused by the Groovy-based test
> {{EncryptedWriteAheadProvenanceRepositoryTest#testShouldRegisterAndGetEvents}},
> which doesn't override the default {{nifi.properties}} value for
> {{nifi.provenance.repository.directory.}} of {{provenance_repository}} - this
> results in the directory being created as indicated above. It would be better
> to locate the directory either in {{target/}} (therefore ignored by default
> in Git commits as part of the Maven build) or a Temp directory as done in
> {{#testWriteAheadProvenanceRepositoryShouldRegisterAndRetrieveEvents}} within
> the same Test class.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [nifi] exceptionfactory closed pull request #5510: NIFI-9366 prevent unwanted provenance_repository directory being crea…
exceptionfactory closed pull request #5510: URL: https://github.com/apache/nifi/pull/5510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-9344) Conduct 1.15 Release
[ https://issues.apache.org/jira/browse/NIFI-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440460#comment-17440460 ] ASF subversion and git services commented on NIFI-9344: --- Commit 7fdc07cccdc0e23d4986557a9314e36859704a3b in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7fdc07c ] NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 > Conduct 1.15 Release > > > Key: NIFI-9344 > URL: https://issues.apache.org/jira/browse/NIFI-9344 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9344) Conduct 1.15 Release
[ https://issues.apache.org/jira/browse/NIFI-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440461#comment-17440461 ] ASF subversion and git services commented on NIFI-9344: --- Commit c0558959523cdf8ed28f4ff42594133948bd67f0 in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c055895 ] NIFI-9344-RC3 prepare for next development iteration > Conduct 1.15 Release > > > Key: NIFI-9344 > URL: https://issues.apache.org/jira/browse/NIFI-9344 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (NIFI-9344) Conduct 1.15 Release
[ https://issues.apache.org/jira/browse/NIFI-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440462#comment-17440462 ] ASF subversion and git services commented on NIFI-9344: --- Commit 9e845d87ff04e8fac38154418d3b73b39ea31ac5 in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9e845d8 ] Merge branch 'NIFI-9344-RC3' > Conduct 1.15 Release > > > Key: NIFI-9344 > URL: https://issues.apache.org/jira/browse/NIFI-9344 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5358: NIFI-9185 Add Avro logical type to SelectHive3QL processor
exceptionfactory commented on a change in pull request #5358:
URL: https://github.com/apache/nifi/pull/5358#discussion_r744716834
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/nifi/util/hive/HiveJdbcCommon.java
##
@@ -298,14 +318,32 @@ public static Schema createSchema(final ResultSet rs,
String recordName, boolean
// Did not find direct suitable type, need to be clarified
case DECIMAL:
case NUMERIC:
-
builder.name(columnName).type().unionOf().nullBuilder().endNull().and().stringType().endUnion().noDefault();
+if (useLogicalTypes) {
+final int precision =
getPrecision(meta.getPrecision(i));
+final int scale = getScale(meta.getScale(i));
+
builder.name(columnName).type().unionOf().nullBuilder().endNull().and()
+.type(LogicalTypes.decimal(precision,
scale).addToSchema(Schema.create(Schema.Type.BYTES))).endUnion().noDefault();
Review comment:
Is there a particular reason for using the `Schema.Type.BYTES` reference
here as opposed to the union with `stringType`?
##
File path: nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/pom.xml
##
@@ -74,6 +74,10 @@
org.apache.nifi
nifi-kerberos-user-service-api
+
+org.apache.nifi
+nifi-kerberos-user-service-api
Review comment:
Is there a reason for introducing this dependency declaration? It
appears to to duplicate the dependency directly above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-9368) Fix the ordering of the log entries on the processor level bulletin pop-up
Peter Turcsanyi created NIFI-9368: - Summary: Fix the ordering of the log entries on the processor level bulletin pop-up Key: NIFI-9368 URL: https://issues.apache.org/jira/browse/NIFI-9368 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.15.0 Reporter: Peter Turcsanyi Assignee: Peter Turcsanyi The order of the log entries on the processor level bulletin pop-up was originally descending (latest log first). It has been changed to ascending due to a side effect of a change in NIFI-9309. Restore the former behaviour. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (MINIFICPP-1674) fix azure dependencies
[ https://issues.apache.org/jira/browse/MINIFICPP-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1674: Resolution: Fixed Status: Resolved (was: Patch Available) > fix azure dependencies > -- > > Key: MINIFICPP-1674 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1674 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.10.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Fix For: 0.11.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > clang 7, debian buster: > [5/129] Building CXX object > sdk/core/azure-core/CMakeFiles/azure-core.dir/src/http/curl/curl.cpp.o > FAILED: sdk/core/azure-core/CMakeFiles/azure-core.dir/src/http/curl/curl.cpp.o > /usr/bin/clang++ -DBUILD_CURL_HTTP_TRANSPORT_ADAPTER > -Isdk/core/azure-core/inc -isystem > /home/szaszm/nifi-minifi-cpp/build/thirdparty/libressl-install/include > -isystem /home/szaszm/nifi-minifi-cpp/build/thirdparty/curl-install/include > -O3 -pipe -march=native -Wall -Wextra -std=c++2a -w -O2 -g -DNDEBUG -fPIC > -fno-operator-names -Wold-style-cast -Xclang -Wall -Wextra -pedantic > -Wdocumentation -Wdocumentation-unknown-command -Wcast-qual -std=gnu++14 -MD > -MT sdk/core/azure-co > re/CMakeFiles/azure-core.dir/src/http/curl/curl.cpp.o -MF > sdk/core/azure-core/CMakeFiles/azure-core.dir/src/http/curl/curl.cpp.o.d -o > sdk/core/azure-core/CMakeFiles/azure-core.dir/src/http/curl/curl.cpp.o -c > sdk/core/azure-core/src/http/curl/curl.cpp > In file included from sdk/core/azure-core/src/http/curl/curl.cpp:12: > In file included from > sdk/core/azure-core/src/http/curl/curl_connection_pool_private.hpp:15: > sdk/core/azure-core/src/http/curl/curl_connection_private.hpp:23:10: fatal > error: 'curl/curl.h' file not found > #include > ^ > 1 error generated. > clang 13, gentoo: > [6/39] Building CXX object > sdk/storage/azure-storage-common/CMakeFiles/azure-storage-common.dir/src/xml_wrapper.cpp.o > FAILED: > sdk/storage/azure-storage-common/CMakeFiles/azure-storage-common.dir/src/xml_wrapper.cpp.o > /usr/lib/llvm/13/bin/clang++ -DBUILD_CURL_HTTP_TRANSPORT_ADAPTER > -I/home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/azure-sdk-cpp-src/sdk/storage/azure-storage-common/inc > > -I/home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/libxml2-install/include/libxml2 > > -I/home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/azure-sdk-cpp-src/sdk/core/azure-core/inc > -isystem > /home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/curl-install/include > -isyst > em > /home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/libressl-install/include > -O3 -pipe -march=native -Wall -Wextra -std=c++20 -w -O2 -g -DNDEBUG -fPIC > -fno-operator-names -Wold-style-cast -Xclang -Wall -Wextra -pedantic > -Wdocumentation -Wdocumentation-unknown-command -Wcast-qual -pthread > -std=gnu++14 -MD -MT > sdk/storage/azure-storage-common/CMakeFiles/azure-storage-common.dir/src/xml_wrapper.cpp.o > -MF sdk/storage/azure-storage-common/CMakeFiles/azure-storage-co > mmon.dir/src/xml_wrapper.cpp.o.d -o > sdk/storage/azure-storage-common/CMakeFiles/azure-storage-common.dir/src/xml_wrapper.cpp.o > -c > /home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/azure-sdk-cpp-src/sdk/storage/azure-storage-common/src/xml_wrapper.cpp > /home/szaszm/nifi-minifi-cpp-2/build-clang-ninja/thirdparty/azure-sdk-cpp-src/sdk/storage/azure-storage-common/src/xml_wrapper.cpp:22:10: > fatal error: 'libxml/xmlreader.h' file not found > #include > ^~~~ > 1 error generated. > > > Repeated invocation of the build tool works around the issue if multiple > threads are used. The headers are not yet downloaded with ExternalProject by > the time we're trying to build the azure libs, but they might be finished on > another thread after the error. > > CMake add_dependencies should be added on the targets that depend on these > headers, or the dependencies should be converted to FetchContent, which does > download during the cmake configure phase, not during build. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [nifi-minifi-cpp] adamdebreceni closed pull request #1207: MINIFICPP-1674 fix azure-sdk target name, use install dir
adamdebreceni closed pull request #1207: URL: https://github.com/apache/nifi-minifi-cpp/pull/1207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni closed pull request #1210: MINIFICPP-1681 DefragmentText should trigger when empty if Maximum Bu…
adamdebreceni closed pull request #1210: URL: https://github.com/apache/nifi-minifi-cpp/pull/1210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
