[jira] [Created] (NIFI-8668) ConsumeAzureEventHub NiFi processors need to support storage SAS token authentication
Timea Barna created NIFI-8668: - Summary: ConsumeAzureEventHub NiFi processors need to support storage SAS token authentication Key: NIFI-8668 URL: https://issues.apache.org/jira/browse/NIFI-8668 Project: Apache NiFi Issue Type: Improvement Reporter: Timea Barna Assignee: Timea Barna ConsumeAzureEventHub processor does not support SAS token authentication for the 'Storage Account' configuration section. The processor uses it to save topic offsets in an ADLS storage account. The only way is to provide a storage account secret key at the moment. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] chihhanyu opened a new pull request #5135: GCP BigQuery processors support using designate project resource for ingestion
chihhanyu opened a new pull request #5135:
URL: https://github.com/apache/nifi/pull/5135
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Please refer to JIRA ticket for details:
https://issues.apache.org/jira/browse/NIFI-8611
This change is already used in our production environment which can properly
support using different source project and ingestion project in the property of
GCP processors.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5098: NIFI-8248 Modify PutAzureDataLakeStorage processor to use temp file
exceptionfactory commented on a change in pull request #5098: URL: https://github.com/apache/nifi/pull/5098#discussion_r646977533 ## File path: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/pom.xml ## @@ -78,6 +78,11 @@ + +com.azure +azure-core-http-netty +1.7.0 Review comment: Is there a reason for using this version as opposed to the more recent version `1.9.2`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5098: NIFI-8248 Modify PutAzureDataLakeStorage processor to use temp file
exceptionfactory commented on a change in pull request #5098:
URL: https://github.com/apache/nifi/pull/5098#discussion_r646976688
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java
##
@@ -129,6 +131,16 @@
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
.build();
+public static final PropertyDescriptor TEMP_FILE_PREFIX = new
PropertyDescriptor.Builder()
+.name("azure-temp-file-prefix")
+.displayName("Temp File Prefix for Azure")
Review comment:
Thanks for pointing out the other filter properties @jfrazee. I agree
with you, it sounds like it would be cleaner to set a default value for the
File Filter property to exclude temporary files as opposed to introducing a new
property.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8667) Deadlock when enabling Controller Service and all Referencing Components
[
https://issues.apache.org/jira/browse/NIFI-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8667:
-
Status: Patch Available (was: Open)
> Deadlock when enabling Controller Service and all Referencing Components
>
>
> Key: NIFI-8667
> URL: https://issues.apache.org/jira/browse/NIFI-8667
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.14.0
>
>
> I have a flow configured with an AvroSchemaRegistry. I have a JSON Reader and
> a JSON Writer both referencing it. I then have a few processors referencing
> both the reader and the writer.
> When I click Enable on the AvroSchemaRegistry and choose to enable all
> referencing components, I encountered a deadlock. The threads involved were:
> {quote}
> "Timer-Driven Process Thread-9" Id=80 WAITING on
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@32a7e726 **
> DEADLOCKED THREAD **
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.getState(StandardControllerServiceNode.java:363)
> at
> org.apache.nifi.controller.AbstractComponentNode.validateReferencedControllerServices(AbstractComponentNode.java:719)
> at
> org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:611)
> at
> org.apache.nifi.controller.StandardProcessorNode.computeValidationErrors(StandardProcessorNode.java:1054)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:574)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:588)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.enable(ServiceStateTransition.java:83)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:460)
> - waiting on java.util.concurrent.atomic.AtomicBoolean@126cb602
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Number of Locked Synchronizers: 2
> - java.util.concurrent.ThreadPoolExecutor$Worker@6492d91f
> -
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc
> "Timer-Driven Process Thread-8" Id=78 WAITING on
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc **
> DEADLOCKED THREAD **
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.getState(StandardControllerServiceNode.java:363)
> at
[GitHub] [nifi] markap14 opened a new pull request #5134: NIFI-8667: When marking a Controller Service as enabled, ensure that …
markap14 opened a new pull request #5134:
URL: https://github.com/apache/nifi/pull/5134
…we release the write lock before calling validation methods of referencing
components. Otherwise, we can encounter a deadlock.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] bgeisberger commented on pull request #4783: NiFi-8152: Fix NPE in RedisDistributedMapCacheClientService…
bgeisberger commented on pull request #4783: URL: https://github.com/apache/nifi/pull/4783#issuecomment-856291614 @MikeThomsen Sorry, I worded that poorly. What I meant there was that the NPEs were introduced with the changes of NIFI-8502 and are fixed with this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8667) Deadlock when enabling Controller Service and all Referencing Components
[
https://issues.apache.org/jira/browse/NIFI-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358887#comment-17358887
]
Mark Payne commented on NIFI-8667:
--
It appears that this was caused by NIFI-8212, so it has not yet been released
in any official releases.
> Deadlock when enabling Controller Service and all Referencing Components
>
>
> Key: NIFI-8667
> URL: https://issues.apache.org/jira/browse/NIFI-8667
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.14.0
>
>
> I have a flow configured with an AvroSchemaRegistry. I have a JSON Reader and
> a JSON Writer both referencing it. I then have a few processors referencing
> both the reader and the writer.
> When I click Enable on the AvroSchemaRegistry and choose to enable all
> referencing components, I encountered a deadlock. The threads involved were:
> {quote}
> "Timer-Driven Process Thread-9" Id=80 WAITING on
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@32a7e726 **
> DEADLOCKED THREAD **
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.getState(StandardControllerServiceNode.java:363)
> at
> org.apache.nifi.controller.AbstractComponentNode.validateReferencedControllerServices(AbstractComponentNode.java:719)
> at
> org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:611)
> at
> org.apache.nifi.controller.StandardProcessorNode.computeValidationErrors(StandardProcessorNode.java:1054)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:574)
> at
> org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:588)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.enable(ServiceStateTransition.java:83)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:460)
> - waiting on java.util.concurrent.atomic.AtomicBoolean@126cb602
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Number of Locked Synchronizers: 2
> - java.util.concurrent.ThreadPoolExecutor$Worker@6492d91f
> -
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc
> "Timer-Driven Process Thread-8" Id=78 WAITING on
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc **
> DEADLOCKED THREAD **
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
> at
> org.
[jira] [Created] (NIFI-8667) Deadlock when enabling Controller Service and all Referencing Components
Mark Payne created NIFI-8667:
Summary: Deadlock when enabling Controller Service and all
Referencing Components
Key: NIFI-8667
URL: https://issues.apache.org/jira/browse/NIFI-8667
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.14.0
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 1.14.0
I have a flow configured with an AvroSchemaRegistry. I have a JSON Reader and a
JSON Writer both referencing it. I then have a few processors referencing both
the reader and the writer.
When I click Enable on the AvroSchemaRegistry and choose to enable all
referencing components, I encountered a deadlock. The threads involved were:
{quote}
"Timer-Driven Process Thread-9" Id=80 WAITING on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@32a7e726 **
DEADLOCKED THREAD **
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at
org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
at
org.apache.nifi.controller.service.StandardControllerServiceNode.getState(StandardControllerServiceNode.java:363)
at
org.apache.nifi.controller.AbstractComponentNode.validateReferencedControllerServices(AbstractComponentNode.java:719)
at
org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:611)
at
org.apache.nifi.controller.StandardProcessorNode.computeValidationErrors(StandardProcessorNode.java:1054)
at
org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:574)
at
org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:588)
at
org.apache.nifi.controller.service.ServiceStateTransition.enable(ServiceStateTransition.java:83)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:460)
- waiting on java.util.concurrent.atomic.AtomicBoolean@126cb602
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Number of Locked Synchronizers: 2
- java.util.concurrent.ThreadPoolExecutor$Worker@6492d91f
- java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc
"Timer-Driven Process Thread-8" Id=78 WAITING on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@b052adc **
DEADLOCKED THREAD **
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at
org.apache.nifi.controller.service.ServiceStateTransition.getState(ServiceStateTransition.java:125)
at
org.apache.nifi.controller.service.StandardControllerServiceNode.getState(StandardControllerServiceNode.java:363)
at
org.apache.nifi.controller.AbstractComponentNode.validateReferencedControllerServices(AbstractComponentNode.java:719)
at
org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:611)
at
org.apache.nifi.controller.StandardProcessorNode.computeValidationErrors(StandardProcessorNode.java:1054)
at
org.apache.nifi.controller.AbstractC
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5130: NIFI-8639: Add custom headers and dynamic URLs to ConnectWebsocket
exceptionfactory commented on a change in pull request #5130:
URL: https://github.com/apache/nifi/pull/5130#discussion_r646963775
##
File path:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/pom.xml
##
@@ -52,5 +52,17 @@
1.14.0-SNAPSHOT
test
+
+org.apache.nifi
+nifi-websocket-services-jetty
+1.14.0-SNAPSHOT
+test
+
+
+org.apache.nifi
+nifi-websocket-services-jetty
+1.14.0-SNAPSHOT
+test
+
Review comment:
This dependency declaration appears to be duplicated.
##
File path:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/AbstractWebSocketGatewayProcessor.java
##
@@ -165,27 +177,36 @@ private void deregister() {
}
@Override
-public final void onTrigger(final ProcessContext context, final
ProcessSessionFactory sessionFactory) throws ProcessException {
+public final void onTrigger(final ProcessContext context, final
ProcessSessionFactory sessionFactory) {
if (processSessionFactory == null) {
processSessionFactory = sessionFactory;
}
if (!isProcessorRegisteredToService()) {
try {
-registerProcessorToService(context, webSocketService ->
onWebSocketServiceReady(webSocketService));
-} catch (IOException|WebSocketConfigurationException e) {
+registerProcessorToService(context, webSocketService ->
onWebSocketServiceReady(webSocketService, context));
+} catch (IOException | WebSocketConfigurationException e) {
// Deregister processor if it failed so that it can retry next
onTrigger.
deregister();
context.yield();
throw new ProcessException("Failed to register processor to
WebSocket service due to: " + e, e);
}
+
+} else {
+try {
+onWebSocketServiceReady(webSocketService, context);
+} catch (IOException e) {
+deregister();
+context.yield();
+throw new ProcessException("Failed to renew session and
connect to WebSocket service due to: " + e, e);
Review comment:
Should the message include `e.getMessage()` instead of implicitly
calling `e.toString()`?
```suggestion
throw new ProcessException("Failed to renew session and
connect to WebSocket service due to: " + e.getMessage(), e);
```
##
File path:
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketClient.java
##
@@ -293,35 +303,37 @@ private void connect(final String clientId, String
sessionId) throws IOException
try {
router = routers.getRouterOrFail(clientId);
} catch (WebSocketConfigurationException e) {
-throw new IllegalStateException("Failed to get router due to:
" + e, e);
+throw new IllegalStateException("Failed to get router due to:
" + e, e);
Review comment:
Should this message call `e.getMessage()`?
```suggestion
throw new IllegalStateException("Failed to get router due
to: " + e.getMessage(), e);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5122: NIFI-8499 - encrypted flow repository swap file implementation
exceptionfactory commented on a change in pull request #5122:
URL: https://github.com/apache/nifi/pull/5122#discussion_r646959897
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/EncryptedFileSystemSwapManager.java
##
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.controller;
+
+import org.apache.nifi.controller.repository.FlowFileSwapManager;
+import org.apache.nifi.security.kms.CryptoUtils;
+import org.apache.nifi.security.kms.EncryptionException;
+import org.apache.nifi.security.kms.KeyProvider;
+import org.apache.nifi.security.repository.RepositoryEncryptorUtils;
+import
org.apache.nifi.security.repository.config.FlowFileRepositoryEncryptionConfiguration;
+import org.apache.nifi.util.NiFiProperties;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.crypto.Cipher;
+import javax.crypto.CipherInputStream;
+import javax.crypto.CipherOutputStream;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.GCMParameterSpec;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.security.GeneralSecurityException;
+import java.security.SecureRandom;
+
+/**
+ *
+ * An implementation of {@link FlowFileSwapManager} that swaps FlowFiles
+ * to/from local disk. The swap file is encrypted using AES/GCM, using the
+ * encryption key defined in nifi.properties for the FlowFile repository.
+ *
+ */
+@SuppressWarnings("unused") // instantiation via reflection in product
Review comment:
Is this annotation still necessary now that the other constructors are
not included?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] stan-antyufeev-8451 edited a comment on pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
stan-antyufeev-8451 edited a comment on pull request #5119: URL: https://github.com/apache/nifi/pull/5119#issuecomment-856259088 @jfrazee I tested locally and it works for me. I need to promote this fix to our Nifi cluster. Appreciated your work, guys. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] stan-antyufeev-8451 commented on pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
stan-antyufeev-8451 commented on pull request #5119: URL: https://github.com/apache/nifi/pull/5119#issuecomment-856259088 @jfrazee I tested locally and it works for me. I need to promote this fix to our Nifi cluster. Appreciated you work, guys. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] stan-antyufeev-8451 commented on a change in pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
stan-antyufeev-8451 commented on a change in pull request #5119:
URL: https://github.com/apache/nifi/pull/5119#discussion_r646867817
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/test/java/org/apache/nifi/services/azure/storage/TestADLSCredentialsControllerService.java
##
@@ -327,6 +327,28 @@ public void testGetCredentialsDetailsWithSasToken() throws
Exception {
assertNull(actual.getServicePrincipalClientSecret());
}
+@Test
+public void testGetCredentialsDetailsWithSasTokenUsingEL() throws
Exception {
+// GIVEN
+configureAccountName();
+configureSasTokenUsingEL();
+
+runner.enableControllerService(credentialsService);
+
+// WHEN
+ADLSCredentialsDetails actual =
credentialsService.getCredentialsDetails(new HashMap<>());
+
+// THEN
Review comment:
Hi
Can we merge this change to proceed further? I need this feature to test
on our one of the solution built on Nifi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
turcsanyip commented on pull request #5119: URL: https://github.com/apache/nifi/pull/5119#issuecomment-856240840 @jfrazee Added Additional Details page to explain the security considerations. Feel free to suggest more items that could be worth mentioning. Also changed the descriptions of some properties because the security concerns regarding EL were displayed for some properties that do not support EL at all. @exceptionfactory Removed the comments from the test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8659) JoltTransformRecord should support transformation of one record to multiple output records
[
https://issues.apache.org/jira/browse/NIFI-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358817#comment-17358817
]
ASF subversion and git services commented on NIFI-8659:
---
Commit 659fda10f3b7ed3d398c2ab8c1cb505880924594 in nifi's branch
refs/heads/main from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=659fda1 ]
NIFI-8659: Support multiple output records in JoltTransformRecord (#5126)
NIFI-8659: Support multiple output records in JoltTransformRecord
> JoltTransformRecord should support transformation of one record to multiple
> output records
> --
>
> Key: NIFI-8659
> URL: https://issues.apache.org/jira/browse/NIFI-8659
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently the JOLT transformation in JoltTransformRecord is applied to each
> record individually, and a single record is expected and written out. However
> there are some cases where multiple output records are desired for each input
> record, such as "hoisting" a nested array of records up to the top-level
> (without a top-level key which would make it a single record). For example:
> Input record:
> {"x": [{"a": 1, "b":2}, {"a": 3, "b": 4}, {"a": 5, "b": 6}]}
> Desired output (3 records):
> [
> {"a": 1, "b":2},
> {"a": 3, "b": 4},
> {"a": 5, "b": 6}
> ]
> This would currently still require the schema of the array elements to be
> identical, as the first transformed record's schema will be used to write all
> the output records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NIFI-8659) JoltTransformRecord should support transformation of one record to multiple output records
[
https://issues.apache.org/jira/browse/NIFI-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-8659.
--
Fix Version/s: 1.14.0
Resolution: Fixed
> JoltTransformRecord should support transformation of one record to multiple
> output records
> --
>
> Key: NIFI-8659
> URL: https://issues.apache.org/jira/browse/NIFI-8659
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently the JOLT transformation in JoltTransformRecord is applied to each
> record individually, and a single record is expected and written out. However
> there are some cases where multiple output records are desired for each input
> record, such as "hoisting" a nested array of records up to the top-level
> (without a top-level key which would make it a single record). For example:
> Input record:
> {"x": [{"a": 1, "b":2}, {"a": 3, "b": 4}, {"a": 5, "b": 6}]}
> Desired output (3 records):
> [
> {"a": 1, "b":2},
> {"a": 3, "b": 4},
> {"a": 5, "b": 6}
> ]
> This would currently still require the schema of the array elements to be
> identical, as the first transformed record's schema will be used to write all
> the output records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8659) JoltTransformRecord should support transformation of one record to multiple output records
[
https://issues.apache.org/jira/browse/NIFI-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358816#comment-17358816
]
ASF subversion and git services commented on NIFI-8659:
---
Commit 659fda10f3b7ed3d398c2ab8c1cb505880924594 in nifi's branch
refs/heads/main from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=659fda1 ]
NIFI-8659: Support multiple output records in JoltTransformRecord (#5126)
NIFI-8659: Support multiple output records in JoltTransformRecord
> JoltTransformRecord should support transformation of one record to multiple
> output records
> --
>
> Key: NIFI-8659
> URL: https://issues.apache.org/jira/browse/NIFI-8659
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently the JOLT transformation in JoltTransformRecord is applied to each
> record individually, and a single record is expected and written out. However
> there are some cases where multiple output records are desired for each input
> record, such as "hoisting" a nested array of records up to the top-level
> (without a top-level key which would make it a single record). For example:
> Input record:
> {"x": [{"a": 1, "b":2}, {"a": 3, "b": 4}, {"a": 5, "b": 6}]}
> Desired output (3 records):
> [
> {"a": 1, "b":2},
> {"a": 3, "b": 4},
> {"a": 5, "b": 6}
> ]
> This would currently still require the schema of the array elements to be
> identical, as the first transformed record's schema will be used to write all
> the output records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markap14 commented on pull request #5126: NIFI-8659: Support multiple output records in JoltTransformRecord
markap14 commented on pull request #5126: URL: https://github.com/apache/nifi/pull/5126#issuecomment-856200178 Thanks @mattyb149 all looks good to me now. Have tested and all works as expected. +1 merged to main! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5126: NIFI-8659: Support multiple output records in JoltTransformRecord
markap14 merged pull request #5126: URL: https://github.com/apache/nifi/pull/5126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
jfrazee commented on pull request #5119: URL: https://github.com/apache/nifi/pull/5119#issuecomment-856191204 @stan-antyufeev-8451 Are you able to or have you given if a test? I'm happy to merge it if so. I can't imagine any issue given the change, but I try to not merge without trying stuff out and hadn't had a chance to. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] stan-antyufeev-8451 commented on a change in pull request #5119: NIFI-8656: Support expression language for SAS Token in the ADLS Gen2…
stan-antyufeev-8451 commented on a change in pull request #5119:
URL: https://github.com/apache/nifi/pull/5119#discussion_r646867817
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/test/java/org/apache/nifi/services/azure/storage/TestADLSCredentialsControllerService.java
##
@@ -327,6 +327,28 @@ public void testGetCredentialsDetailsWithSasToken() throws
Exception {
assertNull(actual.getServicePrincipalClientSecret());
}
+@Test
+public void testGetCredentialsDetailsWithSasTokenUsingEL() throws
Exception {
+// GIVEN
+configureAccountName();
+configureSasTokenUsingEL();
+
+runner.enableControllerService(credentialsService);
+
+// WHEN
+ADLSCredentialsDetails actual =
credentialsService.getCredentialsDetails(new HashMap<>());
+
+// THEN
Review comment:
Hi
Can we merge this change to proceed further? I need this feature to test
our on one of the solution built on Nifi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8061) Update Jackson Datablind Version
[
https://issues.apache.org/jira/browse/NIFI-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358778#comment-17358778
]
Tiago Oliveira commented on NIFI-8061:
--
Could you in a generic way explain how to do it? Do I need to recompile all of
the default NARs of Nifi?
> Update Jackson Datablind Version
>
>
> Key: NIFI-8061
> URL: https://issues.apache.org/jira/browse/NIFI-8061
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.11.0, 1.12.0, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.13.0,
> 1.12.1
>Reporter: David Borncamp
>Priority: Major
> Labels: dependancy, dependencies
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Upgrade {{com.fasterxml.jackson.core:jackson-databind}} in NiFi's maven poms
> to version 2.9.10.6. This resolves [CVE
> 2020-25649|http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649]
> [http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649] with this [issue
> fixed|https://issues.apache.org/jira/secure/[https://github.com/FasterXML/jackson-databind/issues/2826](https://github.com/FasterXML/jackson-databind/issues/2826]
> [https://github.com/FasterXML/jackson-databind/issues/2826] .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8662) Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
[
https://issues.apache.org/jira/browse/NIFI-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358769#comment-17358769
]
Andrew Mcdonald commented on NIFI-8662:
---
That's correct, my region is not in the standard Region list and it ends in
'gov'.
> Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
> -
>
> Key: NIFI-8662
> URL: https://issues.apache.org/jira/browse/NIFI-8662
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> The AWS client library cannot parse the region from custom endpoint URLs
> properly.
> NIFI-5456 fixed this issue via passing the region configured on the
> processor to {{AmazonWebServiceClient.setEndpoint()}} directly (no parsing
> needed in the client library, neither in NiFi).
> NIFI-5893 implemented the fix in another way: parsing the region from the
> endpoint URL on the NiFi side. It is not clear for me what special use case
> it wanted to solve but a regular VPCE endpoint does not work with it now.
> Endpoint URL:
> {{https://vpce-*-.sqs.us-west-2.vpce.amazonaws.com}}
> Error:
> {code:java}
> 2021-06-04 18:25:57,101 ERROR [Timer-Driven Process Thread-5]
> o.apache.nifi.processors.aws.sqs.PutSQS
> PutSQS[id=c4714170-c2cb-39e9-a36c-c43e4604f64a] Failed to send messages to
> Amazon SQS due to com.amazonaws.services.sqs.model.AmazonSQSException:
> Credential should be scoped to a valid region, not 'us-east-1'. (Service:
> AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> 63ea72ad-a856-5eca-8c00-2b99da238d07)
> {code}
> It seems the "sqs" part does not match the regex used for parsing:
> {{^(?:.+[vpce-][a-z0-9-]+\.)?([a-z0-9-]+)$}}.
> However, the endpoint properly works with NIFI-5456 only.
> To support both fixes, I will implement the following logic:
> - use the parse method from NIFI-5893 first and use that region if the parse
> successful (no change here)
> - if the parsing fails, then fall back to NIFI-5456 (that is using the
> configured region) instead of the hard coded "us-east-1"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8610: - Fix Version/s: 1.14.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8662) Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
[
https://issues.apache.org/jira/browse/NIFI-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358757#comment-17358757
]
Peter Turcsanyi commented on NIFI-8662:
---
[~amcdonald] Thank you for the feedback!
So your url does not end with {{.vpce.amazonaws.com}} and for this reason the
regex solution introduced in NIFI-5893 does not apply here
([https://github.com/apache/nifi/blob/f07e17ba74da2759213d52431e00f9d0ba5d39a5/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java#L321-L326]).
The AWS client library parses your url properly.
If I understand it correctly, your region is not available in the standard
Region list of the processor, is it? That's why it needs to be retrieved from
the url.
> Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
> -
>
> Key: NIFI-8662
> URL: https://issues.apache.org/jira/browse/NIFI-8662
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> The AWS client library cannot parse the region from custom endpoint URLs
> properly.
> NIFI-5456 fixed this issue via passing the region configured on the
> processor to {{AmazonWebServiceClient.setEndpoint()}} directly (no parsing
> needed in the client library, neither in NiFi).
> NIFI-5893 implemented the fix in another way: parsing the region from the
> endpoint URL on the NiFi side. It is not clear for me what special use case
> it wanted to solve but a regular VPCE endpoint does not work with it now.
> Endpoint URL:
> {{https://vpce-*-.sqs.us-west-2.vpce.amazonaws.com}}
> Error:
> {code:java}
> 2021-06-04 18:25:57,101 ERROR [Timer-Driven Process Thread-5]
> o.apache.nifi.processors.aws.sqs.PutSQS
> PutSQS[id=c4714170-c2cb-39e9-a36c-c43e4604f64a] Failed to send messages to
> Amazon SQS due to com.amazonaws.services.sqs.model.AmazonSQSException:
> Credential should be scoped to a valid region, not 'us-east-1'. (Service:
> AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> 63ea72ad-a856-5eca-8c00-2b99da238d07)
> {code}
> It seems the "sqs" part does not match the regex used for parsing:
> {{^(?:.+[vpce-][a-z0-9-]+\.)?([a-z0-9-]+)$}}.
> However, the endpoint properly works with NIFI-5456 only.
> To support both fixes, I will implement the following logic:
> - use the parse method from NIFI-5893 first and use that region if the parse
> successful (no change here)
> - if the parsing fails, then fall back to NIFI-5456 (that is using the
> configured region) instead of the hard coded "us-east-1"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] pgyori commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
pgyori commented on a change in pull request #5116: URL: https://github.com/apache/nifi/pull/5116#discussion_r646786993 ## File path: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/GenericScriptRunner.java ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.script.impl; + +import javax.script.Bindings; +import javax.script.ScriptEngine; +import javax.script.ScriptException; + +/** + * This class offers methods to perform Javascript-specific operations during the script runner lifecycle. Review comment: This is generic, not Javascript-specific. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pgyori commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
pgyori commented on a change in pull request #5116:
URL: https://github.com/apache/nifi/pull/5116#discussion_r646664759
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
##
@@ -223,14 +195,15 @@ public void onTrigger(ProcessContext context,
ProcessSessionFactory sessionFacto
scriptingComponentHelper.createResources();
Review comment:
This is not strictly relevant to this modification, but I found that if
we have 1 flowfile in the input queue and we start the processor with e.g. 3
concurrent threads, then onTrigger will be fully executed 3 times, and only 1
of the executions will have a flowfile to process. Usually this is avoided by
putting this in onTrigger:
`FlowFile incomingFlowFile = session.get();
if (incomingFlowFile == null) {
return;
}`
However, we cannot call session.get() in onTrigger in this case since it is
the script's task to get the flowfile. I did not find any means to check in the
beginning of onTrigger if the incoming queue is empty to avoid fully executing
it in such a case. Do you know any solution to this?
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
##
@@ -253,31 +226,16 @@ public void onTrigger(ProcessContext context,
ProcessSessionFactory sessionFacto
}
}
-scriptEngine.setBindings(bindings, ScriptContext.ENGINE_SCOPE);
-
-// Execute any engine-specific configuration before the script
is evaluated
-ScriptEngineConfigurator configurator =
-
scriptingComponentHelper.scriptEngineConfiguratorMap.get(scriptingComponentHelper.getScriptEngineName().toLowerCase());
-
-// Evaluate the script with the configurator (if it exists) or
the engine
-if (configurator != null) {
-configurator.init(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-configurator.eval(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-} else {
-scriptEngine.eval(scriptToRun);
-}
+scriptRunner.run(bindings);
// Commit this session for the user. This plus the outermost
catch statement mimics the behavior
// of AbstractProcessor. This class doesn't extend
AbstractProcessor in order to share a base
// class with InvokeScriptedProcessor
session.commitAsync();
+scriptingComponentHelper.scriptRunnerQ.offer(scriptRunner);
Review comment:
If this line is not in a finally block, then I think we might lose a
ScriptRunner if something throws a non-ScriptException Throwable in this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8625: --- Status: Patch Available (was: In Progress) > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Assignee: Matt Burgess >Priority: Critical > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > Time Spent: 1h 40m > Remaining Estimate: 0h > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
mattyb149 commented on a change in pull request #5116:
URL: https://github.com/apache/nifi/pull/5116#discussion_r646735130
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
##
@@ -253,31 +226,16 @@ public void onTrigger(ProcessContext context,
ProcessSessionFactory sessionFacto
}
}
-scriptEngine.setBindings(bindings, ScriptContext.ENGINE_SCOPE);
-
-// Execute any engine-specific configuration before the script
is evaluated
-ScriptEngineConfigurator configurator =
-
scriptingComponentHelper.scriptEngineConfiguratorMap.get(scriptingComponentHelper.getScriptEngineName().toLowerCase());
-
-// Evaluate the script with the configurator (if it exists) or
the engine
-if (configurator != null) {
-configurator.init(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-configurator.eval(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-} else {
-scriptEngine.eval(scriptToRun);
-}
+scriptRunner.run(bindings);
// Commit this session for the user. This plus the outermost
catch statement mimics the behavior
// of AbstractProcessor. This class doesn't extend
AbstractProcessor in order to share a base
// class with InvokeScriptedProcessor
session.commitAsync();
+scriptingComponentHelper.scriptRunnerQ.offer(scriptRunner);
} catch (ScriptException e) {
-// Reset the configurator on error, this can indicate to the
configurator to recompile the script on next init()
-ScriptEngineConfigurator configurator =
-
scriptingComponentHelper.scriptEngineConfiguratorMap.get(scriptingComponentHelper.getScriptEngineName().toLowerCase());
-if (configurator != null) {
-configurator.reset();
-}
+// Create a new ScriptRunner to replace the one that caused an
exception
+scriptingComponentHelper.setupScriptRunners(1, scriptToRun,
getLogger());
Review comment:
The successful tasks offer their runner back to the queue, the
unsuccessful ones should replace the runner with a new one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
mattyb149 commented on a change in pull request #5116:
URL: https://github.com/apache/nifi/pull/5116#discussion_r646734198
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/rules/handlers/script/ScriptedActionHandler.java
##
@@ -190,53 +184,63 @@ public void onEnabled(final ConfigurationContext context)
{
super.onEnabled(context);
// Call an non-interface method onEnabled(context), to allow a
scripted ActionHandler the chance to set up as necessary
-final Invocable invocable = (Invocable) scriptEngine;
-if (configurationContext != null) {
-try {
-// Get the actual object from the script engine, versus the
proxy stored in ActionHandler. The object may have additional methods,
-// where ActionHandler is a proxied interface
-final Object obj = scriptEngine.get("actionHandler");
-if (obj != null) {
-try {
-invocable.invokeMethod(obj, "onEnabled", context);
-} catch (final NoSuchMethodException nsme) {
-if (getLogger().isDebugEnabled()) {
-getLogger().debug("Configured script ActionHandler
does not contain an onEnabled() method.");
+if (scriptRunner != null) {
+final ScriptEngine scriptEngine = scriptRunner.getScriptEngine();
+final Invocable invocable = (Invocable) scriptEngine;
+if (configurationContext != null) {
+try {
+// Get the actual object from the script engine, versus
the proxy stored in ActionHandler. The object may have additional methods,
+// where ActionHandler is a proxied interface
+final Object obj =
scriptRunner.getScriptEngine().get("actionHandler");
+if (obj != null) {
+try {
+invocable.invokeMethod(obj, "onEnabled", context);
+} catch (final NoSuchMethodException nsme) {
+if (getLogger().isDebugEnabled()) {
+getLogger().debug("Configured script
ActionHandler does not contain an onEnabled() method.");
+}
}
+} else {
+throw new ScriptException("No ActionHandler was
defined by the script.");
}
-} else {
-throw new ScriptException("No ActionHandler was defined by
the script.");
+} catch (ScriptException se) {
+throw new ProcessException("Error executing
onEnabled(context) method", se);
}
-} catch (ScriptException se) {
-throw new ProcessException("Error executing onEnabled(context)
method", se);
}
+} else {
+throw new ProcessException("Error creating ScriptRunner");
}
Review comment:
Looks like a number of scripted components use this same pattern (they
have to get the Object from the script engine instead of using a proxied object
that only lets you invoke the interface method). I think we should refactor
those under a separate improvement Jira
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1581) PROJECT_VERSION with leading zeros are invalid
[
https://issues.apache.org/jira/browse/MINIFICPP-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1581:
--
Fix Version/s: 0.11.0
> PROJECT_VERSION with leading zeros are invalid
> --
>
> Key: MINIFICPP-1581
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1581
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Martin Zink
>Assignee: Martin Zink
>Priority: Major
> Fix For: 0.11.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Previously the project version was defined by setting the
> PPROJECT_VERSION_\{MAJOR,MINOR,PATCH,TWEAK} variables separatly.
> Now it is defined by the
> project(nifi-minifi-cpp VERSION ) call
> This changed the behaviour if the project version has leading zeros.
> Previously a project version 1.07.06 was valid, but now it will be 1.7.6 due
> to a cmake policy.
> [https://cmake.org/cmake/help/git-stage/policy/CMP0096.html]
>
> We should change the cmake files so it can handle project versions with
> leading zeros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1585) ConsumeKafka tests do not compile on Windows
[
https://issues.apache.org/jira/browse/MINIFICPP-1585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits resolved MINIFICPP-1585.
---
Resolution: Fixed
Fixed by https://github.com/apache/nifi-minifi-cpp/pull/1102
> ConsumeKafka tests do not compile on Windows
>
>
> Key: MINIFICPP-1585
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1585
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Affects Versions: 0.10.0
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Major
> Fix For: 0.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Also add the {{/K}} flag to at least one of the Windows CI builds to make it
> easier to catch similar issues in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1583) Build failure in windows ci jobs
[ https://issues.apache.org/jira/browse/MINIFICPP-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1583: -- Fix Version/s: 0.10.0 > Build failure in windows ci jobs > > > Key: MINIFICPP-1583 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1583 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Fix For: 0.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > [https://github.com/apache/nifi-minifi-cpp/actions/runs/905916989] > This is due to unreported conflicts with > [https://github.com/apache/nifi-minifi-cpp/commit/05ae25c1aae280767206b641f8c35d485aa85245] > and > [https://github.com/apache/nifi-minifi-cpp/commit/d3b149c047d627f7ae6bbc7df638673e7e62408c] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1457) Implement InputRequirements
[
https://issues.apache.org/jira/browse/MINIFICPP-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1457:
--
Fix Version/s: 0.10.0
> Implement InputRequirements
> ---
>
> Key: MINIFICPP-1457
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1457
> Project: Apache NiFi MiNiFi C++
> Issue Type: New Feature
>Reporter: Arpad Boda
>Assignee: Ádám Markovics
>Priority: Major
> Fix For: 0.10.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> InputRequirements already exist in NiFi.
> Processors can have the following "annotations":
> {code}
> enum Requirement {
> /**
> * This value is used to indicate that the Processor requires input
> from other Processors
> * in order to run. As a result, the Processor will not be valid if
> it does not have any
> * incoming connections.
> */
> INPUT_REQUIRED,
> /**
> * This value is used to indicate that the Processor will consume
> data from an incoming
> * connection but does not require an incoming connection in order to
> perform its task.
> * If the {@link InputRequirement} annotation is not present, this is
> the default value
> * that is used.
> */
> INPUT_ALLOWED,
> /**
> * This value is used to indicate that the Processor is a "Source
> Processor" and does
> * not accept incoming connections. Because the Processor does not
> pull FlowFiles from
> * an incoming connection, it can be very confusing for users who
> create incoming connections
> * to the Processor. As a result, this value can be used in order to
> clarify that incoming
> * connections will not be used. This prevents the user from even
> creating such a connection.
> */
> INPUT_FORBIDDEN;
> }
> {code}
> Should be used both in yaml config validation and population of C2 manifest
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1582) Build failure in centos ci job
[
https://issues.apache.org/jira/browse/MINIFICPP-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1582:
--
Fix Version/s: 0.10.0
> Build failure in centos ci job
> --
>
> Key: MINIFICPP-1582
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1582
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Martin Zink
>Priority: Major
> Fix For: 0.10.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
>
> [https://github.com/apache/nifi-minifi-cpp/runs/2744828310?check_suite_focus=true]
> {code:java}
> #10 8.861 /bin/sh: cmake3: command not found
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1568) PerformanceDataMonitorTests transiently fail
[ https://issues.apache.org/jira/browse/MINIFICPP-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1568: -- Fix Version/s: 0.10.0 > PerformanceDataMonitorTests transiently fail > > > Key: MINIFICPP-1568 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1568 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gabor Gyimesi >Assignee: Gabor Gyimesi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.10.0 > > Attachments: PerformanceDataMonitorTests_failure_win_vs2017.log > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1577) Separate build and test steps in ubuntu-20.04 ci job
[ https://issues.apache.org/jira/browse/MINIFICPP-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1577: -- Fix Version/s: 0.10.0 > Separate build and test steps in ubuntu-20.04 ci job > > > Key: MINIFICPP-1577 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1577 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Trivial > Fix For: 0.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In [https://github.com/apache/nifi-minifi-cpp/pull/1069] the build and test > steps have been separated, but ubuntu-20.04 was left out. > It has a test phase, but the build step also runs the tests. Which is > redundant and makes the ci output confusing. (test failure shows up as build > failure) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1578) Update versions in the docker-based package builders
[
https://issues.apache.org/jira/browse/MINIFICPP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1578:
--
Fix Version/s: (was: 0.11.0)
0.10.0
> Update versions in the docker-based package builders
>
>
> Key: MINIFICPP-1578
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1578
> Project: Apache NiFi MiNiFi C++
> Issue Type: Task
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.10.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The docker-based {{make debian}} etc targets use old versions of the Centos,
> Debian, Fedora and Ubuntu distros; update them to current versions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-987) Error handling in GetFile has multiple issues
[
https://issues.apache.org/jira/browse/MINIFICPP-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-987:
-
Fix Version/s: (was: 0.11.0)
0.10.0
> Error handling in GetFile has multiple issues
> -
>
> Key: MINIFICPP-987
> URL: https://issues.apache.org/jira/browse/MINIFICPP-987
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Arpad Boda
>Assignee: Ferenc Gerlits
>Priority: Major
> Fix For: 0.10.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> In case GetFile is set to delete the transferred files, the following check
> is made:
> {code:java}
> access(fullName.c_str(), W_OK) != 0{code}
> This doesn't really help as the containing folder should be checked on Unix
> file systems. Not to mention ACL... On Windows, there is no reliable way to
> predict the result of the delete operation.
> In session::import, we try to delete the file, but the return value of the
> function is ignored:
> {code:java}
> if (!keepSource)
> std::remove(source.c_str());{code}
> Which means that in case of failure at delete, the file is going to be
> transmitted endlessly in every ontrigger call.
> Although adding error handling here (throwing an exception) wouldn't really
> help as exceptions are handled per execution, not per file. Which means any
> error (a file couldn't be read for eg.) in the given directory would prevent
> the processor to transmit the rest.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1571) fix nanofi warnings -Wstringop-truncation (GCC 10+)
[
https://issues.apache.org/jira/browse/MINIFICPP-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1571:
--
Fix Version/s: (was: 0.11.0)
0.10.0
> fix nanofi warnings -Wstringop-truncation (GCC 10+)
> ---
>
> Key: MINIFICPP-1571
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1571
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Trivial
> Fix For: 0.10.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{{color:#00}In file included from
> {color}{color:#00}/usr/include/string.h:519{color}{color:#00}, {color}
> from {color:#00}../thirdparty/ut/uthash.h:29{color}{color:#00},
> {color}
> from {color:#00}../nanofi/include/api/ecu.h:24{color}{color:#00},
> {color}
> from {color:#00}../nanofi/src/api/ecu.c:1{color}{color:#00}: {color}
> In function ‘{color:#00}strncpy{color}{color:#00}’, {color}
> inlined from ‘{color:#00}get_properties.part.0{color}{color:#00}’ at
> {color}{color:#00}../nanofi/src/api/ecu.c:407:5{color}{color:#00}:
> {color}
> {color:#00}/usr/include/bits/string_fortified.h:95:10:{color}
> {color:#ff54ff}warning:
> {color}{color:#00}‘{color}{color:#00}__builtin_strncpy{color}{color:#00}’
> output truncated before terminating nul copying as many bytes from a string
> as its length
> [{color}{color:#ff54ff}-Wstringop-truncation{color}{color:#00}] {color}
> 95 \| return {color:#ff54ff}__builtin___strncpy_chk (__dest, __src,
> __len,{color}
> \|
> {color:#ff54ff}^~{color}
> 96 \| {color:#ff54ff} __glibc_objsize
> (__dest)){color}{color:#00}; {color}
> \|
> {color:#ff54ff}~{color}
> {color:#00}../nanofi/src/api/ecu.c:{color}{color:#00} In function
> ‘{color}{color:#00}get_properties.part.0{color}{color:#00}’: {color}
> {color:#00}../nanofi/src/api/ecu.c:405:15:{color} {color:#54}note:
> {color}{color:#00}length computed here {color}
> 405 \| int len = {color:#54}strlen(file_path){color}{color:#00};
> {color}
> \| {color:#54}^{color}
> In file included from
> {color:#00}/usr/include/string.h:519{color}{color:#00}, {color}
> from {color:#00}../thirdparty/ut/uthash.h:29{color}{color:#00},
> {color}
> from {color:#00}../nanofi/include/api/ecu.h:24{color}{color:#00},
> {color}
> from {color:#00}../nanofi/src/api/ecu.c:1{color}{color:#00}: {color}
> In function ‘{color:#00}strncpy{color}{color:#00}’, {color}
> inlined from ‘{color:#00}on_trigger_tailfilechunk{color}{color:#00}’
> at {color}{color:#00}../nanofi/src/api/ecu.c:277:9{color}{color:#00}:
> {color}
> {color:#00}/usr/include/bits/string_fortified.h:95:10:{color}
> {color:#ff54ff}warning:
> {color}{color:#00}‘{color}{color:#00}__builtin_strncpy{color}{color:#00}’
> output truncated before terminating nul copying as many bytes from a string
> as its length
> [{color}{color:#ff54ff}-Wstringop-truncation{color}{color:#00}] {color}
> 95 \| return {color:#ff54ff}__builtin___strncpy_chk (__dest, __src,
> __len,{color}
> \|
> {color:#ff54ff}^~{color}
> 96 \| {color:#ff54ff} __glibc_objsize
> (__dest)){color}{color:#00}; {color}
> \|
> {color:#ff54ff}~{color}
> {color:#00}../nanofi/src/api/ecu.c:{color}{color:#00} In function
> ‘{color}{color:#00}on_trigger_tailfilechunk{color}{color:#00}’:
> {color}
> {color:#00}../nanofi/src/api/ecu.c:275:19:{color} {color:#54}note:
> {color}{color:#00}length computed here {color}
> 275 \| int len =
> {color:#54}strlen(file_path){color}{color:#00}; {color}
> \| {color:#54}^{color}
> }}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1570) Azure SDK transient cmake patch failure
[ https://issues.apache.org/jira/browse/MINIFICPP-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1570: -- Fix Version/s: (was: 0.11.0) 0.10.0 > Azure SDK transient cmake patch failure > --- > > Key: MINIFICPP-1570 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1570 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.9.0 >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.10.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1572) Fix -Wmaybe-uninitialized warnings
[ https://issues.apache.org/jira/browse/MINIFICPP-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1572: -- Fix Version/s: (was: 0.11.0) 0.10.0 > Fix -Wmaybe-uninitialized warnings > -- > > Key: MINIFICPP-1572 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1572 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gabor Gyimesi >Assignee: Gabor Gyimesi >Priority: Trivial > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1569) Windows jobs do not run tests in CI
[ https://issues.apache.org/jira/browse/MINIFICPP-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1569: -- Fix Version/s: (was: 0.11.0) 0.10.0 > Windows jobs do not run tests in CI > --- > > Key: MINIFICPP-1569 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1569 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gabor Gyimesi >Assignee: Gabor Gyimesi >Priority: Major > Fix For: 0.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Since MINIFICPP-1084 was merged to main tests are skipped in the Windows > builds. The original intent was to not run them as part of the build phase > only later in the test running phase, but unfortunately the build was also > removed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1562) Failing CI due to corrupted cache on Windows-2019
[
https://issues.apache.org/jira/browse/MINIFICPP-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-1562:
--
Fix Version/s: (was: 0.11.0)
0.10.0
> Failing CI due to corrupted cache on Windows-2019
> -
>
> Key: MINIFICPP-1562
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1562
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Martin Zink
>Assignee: Gabor Gyimesi
>Priority: Major
> Fix For: 0.10.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The windows-vs2019 CI is failing due to corrupted clcache.
> Unfortunetly github doesnt have clean way to clear the cache.
> {code:java}
> GitHub will remove any cache entries that have not been accessed in over 7
> days. There is no limit on the number of caches you can store, but the total
> size of all caches in a repository is limited to 5 GB. If you exceed this
> limit, GitHub will save your cache but will begin evicting caches until the
> total size is less than 5 GB.
> {code}
> But since the cache is used I dont think github will clear it.
> A quick and dirty fix is: if we were to change the key in the .github/ci.yml
> file, it would fix the issue.
> [https://github.community/t/how-to-clear-cache-in-github-actions/129038/3]
> If this issue comes up again maybe we could implement a workflow that clears
> the caches, with
> [https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8061) Update Jackson Datablind Version
[
https://issues.apache.org/jira/browse/NIFI-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358671#comment-17358671
]
David Borncamp commented on NIFI-8061:
--
Not that I know of. I have not been working on it, but it should be a fairly
easy change.
> Update Jackson Datablind Version
>
>
> Key: NIFI-8061
> URL: https://issues.apache.org/jira/browse/NIFI-8061
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.11.0, 1.12.0, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.13.0,
> 1.12.1
>Reporter: David Borncamp
>Priority: Major
> Labels: dependancy, dependencies
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Upgrade {{com.fasterxml.jackson.core:jackson-databind}} in NiFi's maven poms
> to version 2.9.10.6. This resolves [CVE
> 2020-25649|http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649]
> [http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649] with this [issue
> fixed|https://issues.apache.org/jira/secure/[https://github.com/FasterXML/jackson-databind/issues/2826](https://github.com/FasterXML/jackson-databind/issues/2826]
> [https://github.com/FasterXML/jackson-databind/issues/2826] .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1102: MINIFICPP-1585 Rename enum values in ConsumeKafkaTests::PublishEvent
arpadboda closed pull request #1102: URL: https://github.com/apache/nifi-minifi-cpp/pull/1102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1101: MINIFICPP-1557 Fix CPU usage percentage calculation and extend verification
lordgamez commented on a change in pull request #1101:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1101#discussion_r646695046
##
File path: extensions/http-curl/tests/HTTPHandlers.h
##
@@ -422,11 +423,15 @@ class HeartbeatHandler : public ServerAwareHandler {
void verify(struct mg_connection *conn) {
auto post_data = readPayload(conn);
+if (!isServerRunning()) {
+ return;
+}
if (!IsNullOrEmpty(post_data)) {
rapidjson::Document root;
- rapidjson::ParseResult ok = root.Parse(post_data.data(),
post_data.size());
- assert(ok);
- (void)ok; // unused in release builds
+ rapidjson::ParseResult result = root.Parse(post_data.data(),
post_data.size());
+ if (!result) {
+throw std::runtime_error("JSON parse error: " +
std::string(rapidjson::GetParseError_En(result.Code())) + "\n JSON data: " +
post_data);
Review comment:
Updated in a199f6907d09938d30af9fe21b65ffc0e5116cb2
##
File path: libminifi/src/utils/ProcessCpuUsageTracker.cpp
##
@@ -126,7 +127,7 @@ double
ProcessCpuUsageTracker::getProcessCpuUsageBetweenLastTwoQueries() const {
uint64_t sys_cpu_times_diff = sys_cpu_times_ - previous_sys_cpu_times_;
uint64_t user_cpu_times_diff = user_cpu_times_ - previous_user_cpu_times_;
double percent = static_cast(sys_cpu_times_diff +
user_cpu_times_diff) / static_cast(cpu_times_diff);
- percent = percent / std::thread::hardware_concurrency();
+ percent = percent / (std::max)(static_cast(1),
std::thread::hardware_concurrency());
Review comment:
Updated in a199f6907d09938d30af9fe21b65ffc0e5116cb2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (NIFI-8666) Parameters cannot be referenced from Expression Language if they have a space in the name
[
https://issues.apache.org/jira/browse/NIFI-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358657#comment-17358657
]
Mark Payne edited comment on NIFI-8666 at 6/7/21, 3:19 PM:
---
Unfortunately I cannot seem to get Jira to display the description properly as
the curly braces appear to cause it trouble. The intent there is not to have
all of the newly within but to be more of the syntax:
DOLLAR OPEN_CURLY POUND OPEN_CURLY MyParameter CLOSE_CURLY :toUpper()
CLOSE_CURLY...
was (Author: markap14):
Unfortunately I cannot seem to get Jira to display the description properly as
the curly braces appear to cause it trouble. The intent there is not to have
all of the newly within but to be more of the syntax:
DOLLAR OPEN_CURLY POUND {MyParameter}:toUpper() CLOSE_CURLY...
> Parameters cannot be referenced from Expression Language if they have a space
> in the name
> -
>
> Key: NIFI-8666
> URL: https://issues.apache.org/jira/browse/NIFI-8666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parameter names are allowed to have spaces in them, but any parameter with a
> space cannot be referenced from within Expression Language. For example, we
> cannot use:
> ${#{My Parameter}:toUpper()} or ${ #{'My Parameter'}:toUpper()}
> We should be able to reference the parameter using the single ticks in order
> to quote the parameter name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8666) Parameters cannot be referenced from Expression Language if they have a space in the name
[
https://issues.apache.org/jira/browse/NIFI-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358657#comment-17358657
]
Mark Payne commented on NIFI-8666:
--
Unfortunately I cannot seem to get Jira to display the description properly as
the curly braces appear to cause it trouble. The intent there is not to have
all of the newly within but to be more of the syntax:
DOLLAR OPEN_CURLY POUND {MyParameter}:toUpper() CLOSE_CURLY...
> Parameters cannot be referenced from Expression Language if they have a space
> in the name
> -
>
> Key: NIFI-8666
> URL: https://issues.apache.org/jira/browse/NIFI-8666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parameter names are allowed to have spaces in them, but any parameter with a
> space cannot be referenced from within Expression Language. For example, we
> cannot use:
> ${#{My Parameter}:toUpper()} or ${ #{'My Parameter'}:toUpper()}
> We should be able to reference the parameter using the single ticks in order
> to quote the parameter name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8666) Parameters cannot be referenced from Expression Language if they have a space in the name
[
https://issues.apache.org/jira/browse/NIFI-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8666:
-
Description:
Parameter names are allowed to have spaces in them, but any parameter with a
space cannot be referenced from within Expression Language. For example, we
cannot use:
${#{My Parameter}:toUpper()} or ${ #{'My Parameter'}:toUpper()}
We should be able to reference the parameter using the single ticks in order to
quote the parameter name.
was:
Parameter names are allowed to have spaces in them, but any parameter with a
space cannot be referenced from within Expression Language. For example, we
cannot use:
{\{ ${#{My Parameter}:toUpper()} }} or \{{ ${#{'My Parameter'}:toUpper()} }}
We should be able to reference the parameter using the single ticks in order to
quote the parameter name.
> Parameters cannot be referenced from Expression Language if they have a space
> in the name
> -
>
> Key: NIFI-8666
> URL: https://issues.apache.org/jira/browse/NIFI-8666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parameter names are allowed to have spaces in them, but any parameter with a
> space cannot be referenced from within Expression Language. For example, we
> cannot use:
> ${#{My Parameter}:toUpper()} or ${ #{'My Parameter'}:toUpper()}
> We should be able to reference the parameter using the single ticks in order
> to quote the parameter name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8666) Parameters cannot be referenced from Expression Language if they have a space in the name
[
https://issues.apache.org/jira/browse/NIFI-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8666:
-
Fix Version/s: 1.14.0
Status: Patch Available (was: Open)
> Parameters cannot be referenced from Expression Language if they have a space
> in the name
> -
>
> Key: NIFI-8666
> URL: https://issues.apache.org/jira/browse/NIFI-8666
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parameter names are allowed to have spaces in them, but any parameter with a
> space cannot be referenced from within Expression Language. For example, we
> cannot use:
> {\{ ${#{My Parameter}:toUpper()} }} or \{{ ${#{'My Parameter'}:toUpper()} }}
> We should be able to reference the parameter using the single ticks in order
> to quote the parameter name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #5133: NIFI-8666: Allow users to escape parameter names in Expression Langua…
markap14 opened a new pull request #5133:
URL: https://github.com/apache/nifi/pull/5133
…ge using quotes.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
arpadboda closed pull request #1098: URL: https://github.com/apache/nifi-minifi-cpp/pull/1098 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pgyori commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
pgyori commented on a change in pull request #5116:
URL: https://github.com/apache/nifi/pull/5116#discussion_r646653878
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java
##
@@ -253,31 +226,16 @@ public void onTrigger(ProcessContext context,
ProcessSessionFactory sessionFacto
}
}
-scriptEngine.setBindings(bindings, ScriptContext.ENGINE_SCOPE);
-
-// Execute any engine-specific configuration before the script
is evaluated
-ScriptEngineConfigurator configurator =
-
scriptingComponentHelper.scriptEngineConfiguratorMap.get(scriptingComponentHelper.getScriptEngineName().toLowerCase());
-
-// Evaluate the script with the configurator (if it exists) or
the engine
-if (configurator != null) {
-configurator.init(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-configurator.eval(scriptEngine, scriptToRun,
scriptingComponentHelper.getModules());
-} else {
-scriptEngine.eval(scriptToRun);
-}
+scriptRunner.run(bindings);
// Commit this session for the user. This plus the outermost
catch statement mimics the behavior
// of AbstractProcessor. This class doesn't extend
AbstractProcessor in order to share a base
// class with InvokeScriptedProcessor
session.commitAsync();
+scriptingComponentHelper.scriptRunnerQ.offer(scriptRunner);
} catch (ScriptException e) {
-// Reset the configurator on error, this can indicate to the
configurator to recompile the script on next init()
-ScriptEngineConfigurator configurator =
-
scriptingComponentHelper.scriptEngineConfiguratorMap.get(scriptingComponentHelper.getScriptEngineName().toLowerCase());
-if (configurator != null) {
-configurator.reset();
-}
+// Create a new ScriptRunner to replace the one that caused an
exception
+scriptingComponentHelper.setupScriptRunners(1, scriptToRun,
getLogger());
Review comment:
That makes perfect sense. However, ExecuteScript has 1 instance of
ScriptingComponentHelper. If the ExecuteScript processor is set to use 3
concurrent tasks, then 3 ScriptRunner instances get stored in the
scriptingComponentHelper.scriptRunnerQ queue. If one ScriptRunner causes an
exception, the scriptingComponentHelper.scriptRunnerQ queue gets replaced with
a new queue with 1 element in it. Or am I missing something here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request #1103: MINIFICPP-1567 enable linter checks in extensions (part 3)
martinzink opened a new pull request #1103: URL: https://github.com/apache/nifi-minifi-cpp/pull/1103 enabled linter in libarchive enabled linter in mqtt enabled linter in opc enabled linter in opencv enabled linter in openwsman enabled linter in pcap Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5116: NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
mattyb149 commented on a change in pull request #5116:
URL: https://github.com/apache/nifi/pull/5116#discussion_r646650078
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/record/sink/script/ScriptedRecordSink.java
##
@@ -217,26 +211,31 @@ public void onEnabled(final ConfigurationContext context)
{
super.onEnabled(context);
// Call an non-interface method onEnabled(context), to allow a
scripted RecordSinkService the chance to set up as necessary
-final Invocable invocable = (Invocable) scriptEngine;
-if (configurationContext != null) {
-try {
-// Get the actual object from the script engine, versus the
proxy stored in RecordSinkService. The object may have additional methods,
-// where RecordSinkService is a proxied interface
-final Object obj = scriptEngine.get("recordSink");
-if (obj != null) {
-try {
-invocable.invokeMethod(obj, "onEnabled", context);
-} catch (final NoSuchMethodException nsme) {
-if (getLogger().isDebugEnabled()) {
-getLogger().debug("Configured script
RecordSinkService does not contain an onEnabled() method.");
+if (scriptRunner != null) {
+final ScriptEngine scriptEngine = scriptRunner.getScriptEngine();
+final Invocable invocable = (Invocable) scriptEngine;
Review comment:
I decided to update the `createResources()` method to take a boolean
(defaulting to true) to indicate whether the supported script engines are
required to be Invocable. I'll change ExecuteScript to call it with false, and
the rest will only allow a choice of an Invocable engine, so the individual
checks should be unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-8666) Parameters cannot be referenced from Expression Language if they have a space in the name
Mark Payne created NIFI-8666:
Summary: Parameters cannot be referenced from Expression Language
if they have a space in the name
Key: NIFI-8666
URL: https://issues.apache.org/jira/browse/NIFI-8666
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Parameter names are allowed to have spaces in them, but any parameter with a
space cannot be referenced from within Expression Language. For example, we
cannot use:
{\{ ${#{My Parameter}:toUpper()} }} or \{{ ${#{'My Parameter'}:toUpper()} }}
We should be able to reference the parameter using the single ticks in order to
quote the parameter name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] rhavermans opened a new pull request #5132: Nifi 8664 upgrade json-path 2.4.0 -> 2.6.0
rhavermans opened a new pull request #5132:
URL: https://github.com/apache/nifi/pull/5132
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Upgrade json-path from 2.4.0 to 2.6.0. This enables the new features of
json-path like aggregations in evaluateJsonPath processor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #1102: MINIFICPP-1585 Rename enum values in ConsumeKafkaTests::PublishEvent
fgerlits opened a new pull request #1102: URL: https://github.com/apache/nifi-minifi-cpp/pull/1102 https://issues.apache.org/jira/browse/MINIFICPP-1584 `TRANSACTION_COMMIT` is defined in `winnt.h` as a macro, so the build fails on Windows when `librdkafka` is enabled (using the `/K` flag). I have renamed it, and a few other enum values for consistency. Also added /K to the VS2019 Windows CI job, to make sure bugs like this get caught in the CI stage in the future. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MINIFICPP-1587) The windows-event-log extension does not compile if there is a space in the path
[
https://issues.apache.org/jira/browse/MINIFICPP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358628#comment-17358628
]
Ferenc Gerlits commented on MINIFICPP-1587:
---
As part of this task, it would be good to add some {{.gitignore}} rules so git
does not list these generated files as new:
{noformat}
Untracked files:
(use "git add ..." to include in what will be committed)
extensions/windows-event-log/tests/custom-provider/unit-test-provider.cs
extensions/windows-event-log/tests/custom-provider/unit-test-provider.dll
extensions/windows-event-log/tests/custom-provider/unit-test-provider.h
extensions/windows-event-log/tests/custom-provider/unit-test-provider.man
extensions/windows-event-log/tests/custom-provider/unit-test-provider.rc
extensions/windows-event-log/tests/custom-provider/unit-test-provider.res
extensions/windows-event-log/tests/custom-provider/unit-test-providerTEMP.BIN
{noformat}
> The windows-event-log extension does not compile if there is a space in the
> path
>
>
> Key: MINIFICPP-1587
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1587
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.11.0
>
>
> If you check out the MiNiFi code into a directory which contains a space
> (e.g. {{C:\Users\Jane Doe\src\minifi}}), then the {{windows-event-log}}
> extension fails to compile with an error at
> {{CWELCustomProviderTests.cpp:37}}, with the message "error C1083: Cannot
> open include file: 'custom-provider/unit-test-provider.h': No such file or
> directory".
> This is because the {{generate-and-register.bat}} script in
> {{extensions/windows-event-log/tests/CMakeLists.txt}} fails, with an error
> similar to
> {noformat}
> Doe\src\minifi\extensions\windows-event-log\tests\custom-provider"/unit-test-provider.man"
> was unexpected at this time.
> {noformat}
> I suspect more quoting or escaping is needed in {{generate-and-register.bat}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-4613) Provenance search results are no longer display total numbered count
[ https://issues.apache.org/jira/browse/NIFI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358625#comment-17358625 ] Andrew Atwood commented on NIFI-4613: - I'm going to try to make it optional, with default being current behavior of stopping query when limit is reached. Also, I believe NIFI-2065 is when this behavior was introduced. > Provenance search results are no longer display total numbered count > > > Key: NIFI-4613 > URL: https://issues.apache.org/jira/browse/NIFI-4613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Juan C. Sequeiros >Assignee: Andrew Atwood >Priority: Minor > > At least until 0.7 when doing a provenance search and it returns more than 1K > results, on the results page it would say displaying 1,000 of < total number > of results > > Ex: 1,000 of 500,000 > Now it returns if more than 1,000: > Displaying 1,000 of 1,000 + > We lose the quick indication of how many results it found. > It loses ability to quickly compare searches. > I could certainly narrow my searches but before the ability to at least know > a numbered result was a quick way to troubleshoot or get stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-4613) Provenance search results are no longer display total numbered count
[ https://issues.apache.org/jira/browse/NIFI-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Atwood reassigned NIFI-4613: --- Assignee: Andrew Atwood > Provenance search results are no longer display total numbered count > > > Key: NIFI-4613 > URL: https://issues.apache.org/jira/browse/NIFI-4613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Juan C. Sequeiros >Assignee: Andrew Atwood >Priority: Minor > > At least until 0.7 when doing a provenance search and it returns more than 1K > results, on the results page it would say displaying 1,000 of < total number > of results > > Ex: 1,000 of 500,000 > Now it returns if more than 1,000: > Displaying 1,000 of 1,000 + > We lose the quick indication of how many results it found. > It loses ability to quickly compare searches. > I could certainly narrow my searches but before the ability to at least know > a numbered result was a quick way to troubleshoot or get stats. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8665) Allow ControlRate to have a cluster scope
Mark Bean created NIFI-8665: --- Summary: Allow ControlRate to have a cluster scope Key: NIFI-8665 URL: https://issues.apache.org/jira/browse/NIFI-8665 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.13.2 Reporter: Mark Bean Assignee: Mark Bean ControlRate processor should have a scope property to allow the rate to be limited across a cluster. This allows a maximum rate regardless of the number of nodes that may be dynamically added/removed from a cluster. If node-specific, the throughput of the cluster is dependent on the number of nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] arpadboda closed pull request #5081: [NIFI-8610] Do not reuse generic records in convertAvroToORC
arpadboda closed pull request #5081: URL: https://github.com/apache/nifi/pull/5081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] arpadboda commented on pull request #5081: [NIFI-8610] Do not reuse generic records in convertAvroToORC
arpadboda commented on pull request #5081: URL: https://github.com/apache/nifi/pull/5081#issuecomment-855930523 Merged in f07e17ba74da2759213d52431e00f9d0ba5d39a5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8061) Update Jackson Datablind Version
[
https://issues.apache.org/jira/browse/NIFI-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358599#comment-17358599
]
Tiago Alexandre Soares de Oliveira commented on NIFI-8061:
--
Hi [~dborncamp]
Any news on this issue?
> Update Jackson Datablind Version
>
>
> Key: NIFI-8061
> URL: https://issues.apache.org/jira/browse/NIFI-8061
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.11.0, 1.12.0, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.13.0,
> 1.12.1
>Reporter: David Borncamp
>Priority: Major
> Labels: dependancy, dependencies
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Upgrade {{com.fasterxml.jackson.core:jackson-databind}} in NiFi's maven poms
> to version 2.9.10.6. This resolves [CVE
> 2020-25649|http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649]
> [http://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-25649] with this [issue
> fixed|https://issues.apache.org/jira/secure/[https://github.com/FasterXML/jackson-databind/issues/2826](https://github.com/FasterXML/jackson-databind/issues/2826]
> [https://github.com/FasterXML/jackson-databind/issues/2826] .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358598#comment-17358598 ] Arpad Boda commented on NIFI-8610: -- [~pgyori]: good spot, thanks! Sure, gonna close in a min. > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1101: MINIFICPP-1557 Fix CPU usage percentage calculation and extend verification
szaszm commented on a change in pull request #1101:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1101#discussion_r646582374
##
File path: libminifi/src/utils/ProcessCpuUsageTracker.cpp
##
@@ -126,7 +127,7 @@ double
ProcessCpuUsageTracker::getProcessCpuUsageBetweenLastTwoQueries() const {
uint64_t sys_cpu_times_diff = sys_cpu_times_ - previous_sys_cpu_times_;
uint64_t user_cpu_times_diff = user_cpu_times_ - previous_user_cpu_times_;
double percent = static_cast(sys_cpu_times_diff +
user_cpu_times_diff) / static_cast(cpu_times_diff);
- percent = percent / std::thread::hardware_concurrency();
+ percent = percent / (std::max)(static_cast(1),
std::thread::hardware_concurrency());
Review comment:
`uint32_t{1}` is shorter and avoids the cast.
##
File path: extensions/http-curl/tests/HTTPHandlers.h
##
@@ -422,11 +423,15 @@ class HeartbeatHandler : public ServerAwareHandler {
void verify(struct mg_connection *conn) {
auto post_data = readPayload(conn);
+if (!isServerRunning()) {
+ return;
+}
if (!IsNullOrEmpty(post_data)) {
rapidjson::Document root;
- rapidjson::ParseResult ok = root.Parse(post_data.data(),
post_data.size());
- assert(ok);
- (void)ok; // unused in release builds
+ rapidjson::ParseResult result = root.Parse(post_data.data(),
post_data.size());
+ if (!result) {
+throw std::runtime_error("JSON parse error: " +
std::string(rapidjson::GetParseError_En(result.Code())) + "\n JSON data: " +
post_data);
Review comment:
`fmt::format` or `StringUtils::join_pack` would be nicer IMO.
`fmt::format` allows nice format string. If you don't like that,
`StringUtils::join_pack` can avoid intermediate allocation and deallocation of
temporaries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8662) Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
[
https://issues.apache.org/jira/browse/NIFI-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358589#comment-17358589
]
Andrew Mcdonald commented on NIFI-8662:
---
The aws url for the vpc I'm working in has the form
https://sqs.\{region}.\{something1}.\{something2}.\{something3}.gov
;}
> Failed to parse AWS region from VPCE endpoint URL in AbstractAWSProcessor
> -
>
> Key: NIFI-8662
> URL: https://issues.apache.org/jira/browse/NIFI-8662
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Peter Turcsanyi
>Assignee: Peter Turcsanyi
>Priority: Major
>
> The AWS client library cannot parse the region from custom endpoint URLs
> properly.
> NIFI-5456 fixed this issue via passing the region configured on the
> processor to {{AmazonWebServiceClient.setEndpoint()}} directly (no parsing
> needed in the client library, neither in NiFi).
> NIFI-5893 implemented the fix in another way: parsing the region from the
> endpoint URL on the NiFi side. It is not clear for me what special use case
> it wanted to solve but a regular VPCE endpoint does not work with it now.
> Endpoint URL:
> {{https://vpce-*-.sqs.us-west-2.vpce.amazonaws.com}}
> Error:
> {code:java}
> 2021-06-04 18:25:57,101 ERROR [Timer-Driven Process Thread-5]
> o.apache.nifi.processors.aws.sqs.PutSQS
> PutSQS[id=c4714170-c2cb-39e9-a36c-c43e4604f64a] Failed to send messages to
> Amazon SQS due to com.amazonaws.services.sqs.model.AmazonSQSException:
> Credential should be scoped to a valid region, not 'us-east-1'. (Service:
> AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> 63ea72ad-a856-5eca-8c00-2b99da238d07)
> {code}
> It seems the "sqs" part does not match the regex used for parsing:
> {{^(?:.+[vpce-][a-z0-9-]+\.)?([a-z0-9-]+)$}}.
> However, the endpoint properly works with NIFI-5456 only.
> To support both fixes, I will implement the following logic:
> - use the parse method from NIFI-5893 first and use that region if the parse
> successful (no change here)
> - if the parsing fails, then fall back to NIFI-5456 (that is using the
> configured region) instead of the hard coded "us-east-1"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #1101: MINIFICPP-1557 Fix CPU usage percentage calculation and extend verification
lordgamez opened a new pull request #1101: URL: https://github.com/apache/nifi-minifi-cpp/pull/1101 `std::thread::hardware_concurrency` can return with 0 if the value is not well defined or not computable, which can cause zero division in the calculation of the CPU usage. This happened quite often in github's virtual environments, that resulted in the heartbeat payload JSON to be truncated, and the invalid JSON caused the test to fail transiently. The test is also updated to include more information of the parsing failure and to avoid any timing issues when the server is shut down. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358588#comment-17358588 ] Peter Gyori commented on NIFI-8610: --- I just realized that it has been merged. I got confused by the PR still showing as "open". [~aboda] thank you for merging! Can you please set the PR status to closed? I suppose it shows open because the commit message says "This closes #1081" and the PR number is 5081. > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1099: MINIFICPP-1582 fixing build failure in centos ci job
arpadboda closed pull request #1099: URL: https://github.com/apache/nifi-minifi-cpp/pull/1099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1100: MINIFICPP-1583 windows build fix
arpadboda closed pull request #1100: URL: https://github.com/apache/nifi-minifi-cpp/pull/1100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1587) The windows-event-log extension does not compile if there is a space in the path
Ferenc Gerlits created MINIFICPP-1587:
-
Summary: The windows-event-log extension does not compile if there
is a space in the path
Key: MINIFICPP-1587
URL: https://issues.apache.org/jira/browse/MINIFICPP-1587
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Ferenc Gerlits
Fix For: 0.11.0
If you check out the MiNiFi code into a directory which contains a space (e.g.
{{C:\Users\Jane Doe\src\minifi}}), then the {{windows-event-log}} extension
fails to compile with an error at {{CWELCustomProviderTests.cpp:37}}, with the
message "error C1083: Cannot open include file:
'custom-provider/unit-test-provider.h': No such file or directory".
This is because the {{generate-and-register.bat}} script in
{{extensions/windows-event-log/tests/CMakeLists.txt}} fails, with an error
similar to
{noformat}
Doe\src\minifi\extensions\windows-event-log\tests\custom-provider"/unit-test-provider.man"
was unexpected at this time.
{noformat}
I suspect more quoting or escaping is needed in {{generate-and-register.bat}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] gresockj opened a new pull request #5131: NIFI-8651: Refactor Sensitive Properties Providers for extension
gresockj opened a new pull request #5131:
URL: https://github.com/apache/nifi/pull/5131
Description of PR
Refactors NiFiProperties and NiFiRegistryProperties:
* NiFiProperties and NiFiRegistryProperties now implement a common
interface, `ApplicationProperties`.
* ProtectedNiFiProperties and ProtectedNiFiRegistryProperties now implement
a common interface, `ProtectedApplicationProperties`.
* ApplicationPropertiesProtector now handles the majority of the protection
logic for ProtectedNiFiProperties and ProtectedNiFiRegistryProperties, so that
sensitive property handling will be uniform between NiFi and NiFi Registry
Refactors SensitivePropertyProviderFactory:
* There is now only one SensitivePropertyProviderFactory, which can be
configured with a master key and a new BootstrapProperties object, which
represents the bootstrap.conf file. All configuration for future
SensitivePropertiesProviders (SPPs) is intended to go into bootstrap.conf.
* The single StandardSensitivePropertyProviderFactory implementation is
intended to select the appropriate SPP based on a new
SensitivePropertyProtectionScheme enum.
* Each SPP implementation can choose to use the master key and/or the
BootstrapProperties provided in the SPP Factory
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [x] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8664) Upgrade json-path (jayway) dependency
[ https://issues.apache.org/jira/browse/NIFI-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramon Havermans updated NIFI-8664: -- Description: Upgrading to json-path 2.6.0 will allow new features like aggregation. Pull request will follow. See [https://github.com/json-path/JsonPath] for details. was: Upgrading to json-path 2.6.0 will allow new features like aggregation. Pull request will follow. > Upgrade json-path (jayway) dependency > - > > Key: NIFI-8664 > URL: https://issues.apache.org/jira/browse/NIFI-8664 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.13.2 >Reporter: Ramon Havermans >Priority: Minor > > Upgrading to json-path 2.6.0 will allow new features like aggregation. > > Pull request will follow. > > See [https://github.com/json-path/JsonPath] for details. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8664) Upgrade json-path (jayway) dependency
[ https://issues.apache.org/jira/browse/NIFI-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramon Havermans updated NIFI-8664: -- Description: Upgrading to json-path 2.6.0 will allow new features like aggregation. Pull request will follow. was: Upgrading to json-path 2.5.0 will allow new features like aggregation. Pull request will follow. > Upgrade json-path (jayway) dependency > - > > Key: NIFI-8664 > URL: https://issues.apache.org/jira/browse/NIFI-8664 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.13.2 >Reporter: Ramon Havermans >Priority: Minor > > Upgrading to json-path 2.6.0 will allow new features like aggregation. > > Pull request will follow. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358548#comment-17358548 ] Peter Gyori commented on NIFI-8610: --- I can't merge unfortunately, because I'm not a committer. But I'll ask around. > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] martinzink commented on a change in pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
martinzink commented on a change in pull request #1098: URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#discussion_r646466932 ## File path: docker/debian/Dockerfile ## @@ -28,7 +28,9 @@ ENV MINIFI_BASE_DIR /opt/minifi ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ -RUN apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 cmake +RUN echo "deb http://deb.debian.org/debian buster-backports main" | tee -a /etc/apt/sources.list \ +&& apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 libtool \ Review comment: updated in https://github.com/apache/nifi-minifi-cpp/pull/1098/commits/ca4eaa48a97ac2da56937811589de8d3c8a02161 ## File path: CMakeLists.txt ## @@ -17,7 +17,8 @@ # under the License. # -cmake_minimum_required(VERSION 3.11) +cmake_minimum_required(VERSION 3.16) Review comment: updated in https://github.com/apache/nifi-minifi-cpp/pull/1098/commits/b20daf04ede9c0d02c1934b75a00da6254cc2e1f -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
fgerlits commented on a change in pull request #1098: URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#discussion_r646448149 ## File path: docker/debian/Dockerfile ## @@ -28,7 +28,9 @@ ENV MINIFI_BASE_DIR /opt/minifi ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ -RUN apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 cmake +RUN echo "deb http://deb.debian.org/debian buster-backports main" | tee -a /etc/apt/sources.list \ +&& apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 libtool \ Review comment: Java 8 was not available in Centos 8, so I changed it to Java 11, but forgot to update the JAVA_HOME line. I would just update that to Java 11, too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a change in pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
martinzink commented on a change in pull request #1098: URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#discussion_r646444127 ## File path: docker/debian/Dockerfile ## @@ -28,7 +28,9 @@ ENV MINIFI_BASE_DIR /opt/minifi ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ -RUN apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 cmake +RUN echo "deb http://deb.debian.org/debian buster-backports main" | tee -a /etc/apt/sources.list \ +&& apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 libtool \ Review comment: Not sure. I just modified the cmake source. It was changed in #1093 , and we are still using and installing java 8 in ubuntu for example. @fgerlits what should be here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358482#comment-17358482 ] ASF subversion and git services commented on NIFI-8610: --- Commit f07e17ba74da2759213d52431e00f9d0ba5d39a5 in nifi's branch refs/heads/main from Dominik Przybysz [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f07e17b ] [NIFI-8610] Do not reuse generic records in convertAvroToORC [NIFI-8610] Simplify decimal test for convertAvroToORC Signed-off-by: Arpad Boda This closes #1081 > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358483#comment-17358483 ] ASF subversion and git services commented on NIFI-8610: --- Commit f07e17ba74da2759213d52431e00f9d0ba5d39a5 in nifi's branch refs/heads/main from Dominik Przybysz [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f07e17b ] [NIFI-8610] Do not reuse generic records in convertAvroToORC [NIFI-8610] Simplify decimal test for convertAvroToORC Signed-off-by: Arpad Boda This closes #1081 > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
szaszm commented on a change in pull request #1098: URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#discussion_r646416848 ## File path: docker/debian/Dockerfile ## @@ -28,7 +28,9 @@ ENV MINIFI_BASE_DIR /opt/minifi ENV MINIFI_HOME $MINIFI_BASE_DIR/nifi-minifi-cpp-$MINIFI_VERSION ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ -RUN apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 cmake +RUN echo "deb http://deb.debian.org/debian buster-backports main" | tee -a /etc/apt/sources.list \ +&& apt-get update && apt-get install -y openjdk-11-jdk openjdk-11-source libpython3.7-dev sudo git maven libarchive13 libtool \ Review comment: Why are we installing openjdk-11 while setting JAVA_HOME to openjdk-8 three lines above? ## File path: CMakeLists.txt ## @@ -17,7 +17,8 @@ # under the License. # -cmake_minimum_required(VERSION 3.11) +cmake_minimum_required(VERSION 3.16) Review comment: When raising cmake or other requirements, it needs to be documented in the README. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358454#comment-17358454 ] Dominik Przybysz commented on NIFI-8610: [~pgyori] This task is already assigned to me. Can you merge my PR? > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8610) Invalid bigdecimal values in ORC when multiple avro records provided
[ https://issues.apache.org/jira/browse/NIFI-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dominik Przybysz reassigned NIFI-8610: -- Assignee: Dominik Przybysz > Invalid bigdecimal values in ORC when multiple avro records provided > > > Key: NIFI-8610 > URL: https://issues.apache.org/jira/browse/NIFI-8610 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: Dominik Przybysz >Assignee: Dominik Przybysz >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Processor: convertAvroToORC > Input: flow file contains a lot of avro records with decimal values as bytes > with logical type decimal > Output: ORC file with different values for decimal values > Reason: AvroReader used in processor reuses previous generic record when > fetching next one > Solution: Don't reuse generic records when reading records from flow file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8664) Upgrade json-path (jayway) dependency
Ramon Havermans created NIFI-8664: - Summary: Upgrade json-path (jayway) dependency Key: NIFI-8664 URL: https://issues.apache.org/jira/browse/NIFI-8664 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.13.2 Reporter: Ramon Havermans Upgrading to json-path 2.5.0 will allow new features like aggregation. Pull request will follow. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] martinzink commented on pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
martinzink commented on pull request #1098:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#issuecomment-855721065
> I thought you were going to replace occurrences of
`${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}`
with `${PROJECT_VERSION}`, which would solve the original issue without forcing
an upgrade to CMake 3.16. Did that plan fail?
Yeah, that didnt fail. I just found out that all of our targets use newer
cmake anyways.
Except for debian but were using this backported (3.16 version) prior to
#1093 as well.
I still think that we could use the PROJECT_VERSION variable exclusively but
not sure if leaving these separate variables inconsistent is a good idea in the
long term (maybe some tool uses them?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a change in pull request #5111: NIFI-8645: Disable LongRunningTaskMonitor by default
turcsanyip commented on a change in pull request #5111: URL: https://github.com/apache/nifi/pull/5111#discussion_r646356521 ## File path: nifi-docs/src/main/asciidoc/administration-guide.adoc ## @@ -3732,6 +3732,21 @@ These properties determine the behavior of the internal NiFi predictive analytic | +[[runtime_monitoring_properties]] +=== Runtime Monitoring Properties + +Long-Running Task Monitor periodically checks the NiFi processor executor threads and produces warning logs and bulletin messages for those that have been running for a longer period of time. +It can be used to detect possibly stuck / hanging processor tasks. +Please note the performance impact of the task monitor: it creates a thread dump for every run that may affect the normal flow execution. +The Long-Running Task Monitor can be disabled via defining no values for its properties, and it is disabled by default. Review comment: Good idea, I added a note on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a change in pull request #5111: NIFI-8645: Disable LongRunningTaskMonitor by default
turcsanyip commented on a change in pull request #5111: URL: https://github.com/apache/nifi/pull/5111#discussion_r646356067 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java ## @@ -311,7 +311,7 @@ private final StandardFlowManager flowManager; private final RepositoryContextFactory repositoryContextFactory; private final RingBufferGarbageCollectionLog gcLog; -private final FlowEngine longRunningTaskMonitorThreadPool; +private final Optional longRunningTaskMonitorThreadPool; Review comment: I added some example values in the error error message. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a change in pull request #5111: NIFI-8645: Disable LongRunningTaskMonitor by default
turcsanyip commented on a change in pull request #5111:
URL: https://github.com/apache/nifi/pull/5111#discussion_r646355685
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties
##
@@ -321,5 +321,5 @@
nifi.analytics.connection.model.score.name=${nifi.analytics.connection.model.sco
nifi.analytics.connection.model.score.threshold=${nifi.analytics.connection.model.score.threshold}
# runtime monitoring properties
-nifi.monitor.long.running.task.schedule=${nifi.monitor.long.running.task.schedule}
-nifi.monitor.long.running.task.threshold=${nifi.monitor.long.running.task.threshold}
+nifi.monitor.long.running.task.schedule=
Review comment:
There is no single measurement unit that must be used but the property
can be configured like other time period type properties: 60 sec, 60 secs, 60
seconds, 1 min, etc.
It is mentioned in the Admin Guide that these are time period values with an
example value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (MINIFICPP-1586) `make linter` gives errors when it is run in an unpacked release tarball
Ferenc Gerlits created MINIFICPP-1586:
-
Summary: `make linter` gives errors when it is run in an unpacked
release tarball
Key: MINIFICPP-1586
URL: https://issues.apache.org/jira/browse/MINIFICPP-1586
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Ferenc Gerlits
Assignee: Amina Dinari
Fix For: 0.11.0
Steps to reproduce:
# download a release source tarball, eg.
[https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/0.10.0/nifi-minifi-cpp-0.10.0-source.tar.gz]
# unpack it
# {{cd}} into the unpacked directory, and run {{mkdir build && cd build &&
cmake .. && make linter}}
Expected result: "Built target linter", no errors.
Actual result: lots of linter errors saying "#ifndef header guard has wrong
style".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8663) Unstable behavior with content archiving enabled
[ https://issues.apache.org/jira/browse/NIFI-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Pellmann updated NIFI-8663: Environment: Default standard NiFi from official download without changes. nifi.content.repository.archive.enabled=true nifi.content.repository.archive.max.usage.percentage=50% Specifically, I have a disk that is 80% full when NiFi starts. For the test I used a Mac Book Pro 13” i7, 2019 with SSD. The file limits (launchctl limit maxfiles) are: filemaxfiles 65536 65536. openjdk 11.0.6 was: Default standard NiFi from official download without changes. nifi.content.repository.archive.enabled=true nifi.content.repository.archive.max.usage.percentage=50% Specifically, I have a disk that is 80% full when NiFi starts. For the test I used a Mac Book Pro 13” i7, 2019 with SSD. The file limits (launchctl limit maxfiles) are: filemaxfiles 65536 65536. > Unstable behavior with content archiving enabled > > > Key: NIFI-8663 > URL: https://issues.apache.org/jira/browse/NIFI-8663 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.12.1, 1.13.2 > Environment: Default standard NiFi from official download without > changes. > nifi.content.repository.archive.enabled=true > nifi.content.repository.archive.max.usage.percentage=50% > Specifically, I have a disk that is 80% full when NiFi starts. For the test I > used a Mac Book Pro 13” i7, 2019 with SSD. The file limits (launchctl limit > maxfiles) are: filemaxfiles 65536 65536. > openjdk 11.0.6 >Reporter: Marc Pellmann >Priority: Major > Attachments: content_archiving_bug.xml > > > If content archiving is enabled when NiFi is started > (nifi.content.repository.archive.enabled=true) and the threshold above which > deletion occurs is less than the actual free space available on disk, > unexpected behavior will occur. > I have attached a simple flow (content_archiving_bug.xml) that is accessed > via a HandleHttpRequest. In one of the branches then very many FlowFiles are > created (DuplicateFlowFile). (This is to simulate a complex flow in > production to show the behavior there). > If you call this branch via the browser a few (5-10) times, this leads to the > fact that either *no more FlowFiles are processed in the flow* or that this > happens only extremely slowly. > In addition, the HandleHttpRequest then takes no more requests from the queue > and reports queue full back (to show this better, the queue is configured > smaller). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8663) Unstable behavior with content archiving enabled
Marc Pellmann created NIFI-8663: --- Summary: Unstable behavior with content archiving enabled Key: NIFI-8663 URL: https://issues.apache.org/jira/browse/NIFI-8663 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.13.2, 1.12.1 Environment: Default standard NiFi from official download without changes. nifi.content.repository.archive.enabled=true nifi.content.repository.archive.max.usage.percentage=50% Specifically, I have a disk that is 80% full when NiFi starts. For the test I used a Mac Book Pro 13” i7, 2019 with SSD. The file limits (launchctl limit maxfiles) are: filemaxfiles 65536 65536. Reporter: Marc Pellmann Attachments: content_archiving_bug.xml If content archiving is enabled when NiFi is started (nifi.content.repository.archive.enabled=true) and the threshold above which deletion occurs is less than the actual free space available on disk, unexpected behavior will occur. I have attached a simple flow (content_archiving_bug.xml) that is accessed via a HandleHttpRequest. In one of the branches then very many FlowFiles are created (DuplicateFlowFile). (This is to simulate a complex flow in production to show the behavior there). If you call this branch via the browser a few (5-10) times, this leads to the fact that either *no more FlowFiles are processed in the flow* or that this happens only extremely slowly. In addition, the HandleHttpRequest then takes no more requests from the queue and reports queue full back (to show this better, the queue is configured smaller). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1585) ConsumeKafka tests do not compile on Windows
Ferenc Gerlits created MINIFICPP-1585:
-
Summary: ConsumeKafka tests do not compile on Windows
Key: MINIFICPP-1585
URL: https://issues.apache.org/jira/browse/MINIFICPP-1585
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Ferenc Gerlits
Assignee: Ferenc Gerlits
Fix For: 0.10.0
Also add the {{/K}} flag to at least one of the Windows CI builds to make it
easier to catch similar issues in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] fgerlits commented on pull request #1098: MINIFICPP-1581: set cmake policy CMP0096 to NEW to preserve the leadi…
fgerlits commented on pull request #1098:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1098#issuecomment-855657097
I thought you were going to replace occurrences of
`${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}`
with `${PROJECT_VERSION}`, which would solve the original issue without forcing
an upgrade to CMake 3.16. Did that plan fail?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
